Nginx and W3 Total Cache

With the increasing popularity of the lightweight nginx server, the latest developer build of W3 Total Cache, includes support for nginx. As nginx does not have directory specify files (e.g. .htaccess), the settings are intended to be added to one’s nginx.conf file.

Before preceding, I simply cannot pass up an opportunity to extol the virtues of this plugin – its optimizations are multifaceted, and the end results are beyond compare – many thanks to the developer, W3 EDGE for all the hard work developing such an outstanding performance plugin.

The following nginx configuration is generated by W3 Total Cache, with a few modifications.

Note: the setup is as follows:
nginx (0.8.53) frontend with apache (2.2.16) backend
W3 Total cache (0.9.1.4b) with page cache, browser cache, minify, and url rewriting (and using a CDN, but that doesn’t affect the nginx rules)

server {
	server_name  www.example.com example.com;
	access_log  /var/www/html/example.com/logs/nginx_access.log;
	error_log  /var/www/html/example.com/logs/nginx_error.log debug;
	root   /var/www/html/example.com/public_html/;

	# BEGIN W3TC Page Cache core
		set $w3tc_rewrite 1;
	if ($request_method = POST) {
		set $w3tc_rewrite 0;
	}
	if ($query_string != "") {
		set $w3tc_rewrite 0;
	}
	if ($request_uri !~ \/$) {
		set $w3tc_rewrite 0;
	}
	set $w3tc_rewrite2 1;
	if ($request_uri ~* "(\/wp-admin\/|\/xmlrpc.php|\/wp-(app|cron|login|register|mail)\.php|wp-.*\.php|index\.php)") {
		set $w3tc_rewrite2 0;
	}
	if ($request_uri ~* "(wp\-comments\-popup\.php|wp\-links\-opml\.php|wp\-locations\.php)") {
		set $w3tc_rewrite2 1;
	}
	if ($w3tc_rewrite2 != 1) {
		set $w3tc_rewrite 0;
	}
	if ($http_cookie ~* "(comment_author|wp\-postpass|wordpress_\[a\-f0\-9\]\+|wordpress_logged_in)") {
		set $w3tc_rewrite 0;
	}
	set $w3tc_ua "";
	if ($http_user_agent ~* "(2\.0\ mmp|240x320|alcatel|amoi|asus|au\-mic|audiovox|avantgo|benq|bird|blackberry|blazer|cdm|cellphone|danger|ddipocket|docomo|dopod|elaine/3\.0|ericsson|eudoraweb|fly|haier|hiptop|hp\.ipaq|htc|huawei|i\-mobile|iemobile|j\-phone|kddi|konka|kwc|kyocera/wx310k|lenovo|lg|lg/u990|lge\ vx|midp|midp\-2\.0|mmef20|mmp|mobilephone|mot\-v|motorola|netfront|newgen|newt|nintendo\ ds|nintendo\ wii|nitro|nokia|novarra|o2|openweb|opera\ mobi|opera\.mobi|palm|panasonic|pantech|pdxgw|pg|philips|phone|playstation\ portable|portalmmm|ppc|proxinet|psp|pt|qtek|sagem|samsung|sanyo|sch|sec|sendo|sgh|sharp|sharp\-tq\-gx10|small|smartphone|softbank|sonyericsson|sph|symbian|symbian\ os|symbianos|toshiba|treo|ts21i\-10|up\.browser|up\.link|uts|vertu|vodafone|wap|willcome|windows\ ce|windows\.ce|winwap|xda|zte)") {
		set $w3tc_ua _low;
	}
	if ($http_user_agent ~* "(acer\ s100|android|archos5|blackberry9500|blackberry9530|blackberry9550|cupcake|docomo\ ht\-03a|dream|htc\ hero|htc\ magic|htc_dream|htc_magic|incognito|ipad|iphone|ipod|lg\-gw620|liquid\ build|maemo|mot\-mb200|mot\-mb300|nexus\ one|opera\ mini|samsung\-s8000|series60.*webkit|series60/5\.0|sonyericssone10|sonyericssonu20|sonyericssonx10|t\-mobile\ mytouch\ 3g|t\-mobile\ opal|tattoo|webmate|webos)") {
		set $w3tc_ua _high;
	}
	set $w3tc_ref "";
	set $w3tc_ssl "";
	if ($scheme = https) {
		set $w3tc_ssl _ssl;
	}
	set $w3tc_enc "";
	if ($http_accept_encoding ~ gzip) {
		set $w3tc_enc .gzip;
	}
	#set $w3tc_rewrite 1;
	if (!-f "/var/www/html/example.com/public_html/wp-content/w3tc/pgcache/$request_uri/_index$w3tc_ua$w3tc_ref$w3tc_ssl.html$w3tc_enc") {
		set $w3tc_rewrite 0;
	}
	if ($w3tc_rewrite = 1) {
		rewrite (.*) "/wp-content/w3tc/pgcache/$1/_index$w3tc_ua$w3tc_ref$w3tc_ssl.html$w3tc_enc" last;
	}
	# END W3TC Page Cache core

	# BEGIN W3TC Page Cache cache
	location ~ /wp-content/w3tc/pgcache.*html$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Cached)";
		add_header Vary "Accept-Encoding, Cookie";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 7d;
	}
	location ~ /wp-content/w3tc/pgcache.*gzip$ {
		gzip off;
		types {}
		default_type text/html;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Cached, Gzipped)";
		add_header Vary "Accept-Encoding, Cookie";
		add_header Content-Encoding gzip;
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 7d;
	}
	# END W3TC Page Cache cache

	# BEGIN W3TC Browser Cache - Part 1
	gzip on;
	gzip_types text/css application/x-javascript text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon;
	# END W3TC Browser Cache - Part 1

	# BEGIN W3TC Minify core
	set $w3tc_enc "";
	if ($http_accept_encoding ~ gzip) {
		set $w3tc_enc .gzip;
	}
	if (-f $request_filename$w3tc_enc) {
		rewrite (.*) $1$w3tc_enc break;
	}
	rewrite ^/wp-content/w3tc/min/([a-f0-9]+)\/(.+)\.(include(\-(footer|body))?(-nb)?)\.[0-9]+\.(css|js)$ /wp-content/w3tc/min/index.php?tt=$1&gg=$2&g=$3&t=$7 last;
	# END W3TC Minify core

	# BEGIN W3TC Minify cache
	location ~ /wp-content/w3tc/min.*\.js$ {
		types {}
		default_type application/x-javascript;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify)";
		add_header Vary "Accept-Encoding";
	}
	location ~ /wp-content/w3tc/min.*\.css$ {
		types {}
		default_type text/css;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify)";
		add_header Vary "Accept-Encoding";
	}
	location ~ /wp-content/w3tc/min.*js\.gzip$ {
		gzip off;
		types {}
		default_type application/x-javascript;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify, Gzipped)";
		add_header Vary "Accept-Encoding";
		add_header Content-Encoding gzip;
	}
	location ~ /wp-content/w3tc/min.*css\.gzip$ {
		gzip off;
		types {}
		default_type text/css;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify, Gzipped)";
		add_header Vary "Accept-Encoding";
		add_header Content-Encoding gzip;
	}
	# END W3TC Minify cache

	# BEGIN W3TC Browser Cache - Part 2

	location ~ \.(css|js)$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 7d;
	}
	location ~ \.(html|htm|rtf|rtx|svg|svgz|txt|xsd|xsl|xml)$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 1d;
	}
	location ~ \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|swf|tar|tif|tiff|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip)$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 30d;
	}

	# END W3TC Browser Cache - Part 2

	try_files $uri $uri/ /index.php?q=$request_uri;

	location ~ \.php$ {
		access_log off;
		proxy_pass   http://127.0.0.1:8080;
	}

	location ~ /\.ht {
		deny all;
	}
}

The following changes were made from the default configuration in order for things to work:

  • Change paths from absolute to relative (with respect to document root) – the only path(s) that remain absolute are those checking whether or not a file exists.
  • Expires headers added based on personal preference.
  • X-Powered-By header modified to show action performed.
  • Some additional Cache-Control and Pragma headers added.
  • The browser cache block was split into two.
    • The reason being that if the browser cache block precedes any of the other blocks that serve gzipped files, the gzip extension is matched and the Content-type is over-ridden. The solution is to either remove the gzip extension from the browser cache list; or to move the processing of the browser cache extensions after all the gzipped files have been processed.
  • Finally, the last few lines, pass any files not handled by W3 Total Cache onto WordPress, and proxy PHP scripts to apache running in the background. The last stanza denies access to .htaccess files that are still used by apache.
  • The simplest proof of this setup was to turn apache off – if the page hadn’t been previously loaded (and no cached copy was present), a 502-Bad gateway error was returned; if however a cached copy existed, the page loaded without any necessity for apache or mysql, exactly as it should.

One other change that I think might prove beneficial is to prevent the minify code from redirecting root (/). That is to say, replacing:

if (-f $request_filename$w3tc_enc) {
	rewrite (.*) $1$w3tc_enc break;
}

with:

set $min_rewrite 1;
if ($request_uri ~ ^\/$){
	set $min_rewrite 0;
}

if (!-f $request_filename$w3tc_enc) {
	set $min_rewrite 0;
}

if ($min_rewrite = 1){
	rewrite (.*) $1$w3tc_enc break;
}

The reason for this suggestion is that WordPress redirects index.php back to root, which, while functional, will result in a ‘logical’ redirect loop (i.e. nginx redirects / to index.php, WordPress (apache) redirects index.php back to /). Since two separate entities do the redirects, there isn’t actually a loop, but from the perspective of the request, a redirect is issued pointing to itself (which by definition is a loop). (This behaviour was brought to my attention by the fact that my Pingdom monitor for this site kept notifying me of downtime due to an “Infinite redirection loop”).

In order for W3 Total Cache to treat the server as nginx instead of apache (in this case, apache is running PHP), I modified wp-content/plugins/w3-total-cache/inc/define.php, such that:
function w3_is_apache() always returned false (line: ~410) and
function w3_is_nginx() always returned true (line: ~420).

For interest sake, a quick run of apachebench (ab) shows processing times averaging 10ms for a cached page:

# ab -n 100 -c 10 http://www.thatsgeeky.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.thatsgeeky.com (be patient).....done

Server Software:        nginx/0.8.53
Server Hostname:        www.thatsgeeky.com
Server Port:            80

Document Path:          /
Document Length:        37139 bytes

Concurrency Level:      10
Time taken for tests:   0.145 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      3784101 bytes
HTML transferred:       3739964 bytes
Requests per second:    688.83 [#/sec] (mean)
Time per request:       14.517 [ms] (mean)
Time per request:       1.452 [ms] (mean, across all concurrent requests)
Transfer rate:          25455.22 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    2   0.9      2       5
Processing:     9   12   1.3     12      15
Waiting:        1    3   0.7      3       5
Total:         10   14   1.1     14      17

Percentage of the requests served within a certain time (ms)
  50%     14
  66%     14
  75%     15
  80%     15
  90%     15
  95%     16
  98%     17
  99%     17
 100%     17 (longest request)

There remain some issues to be resolved with the setup of W3 Total Cache (minify paths for instance), but the parts relating to nginx are (finally) working as they should.

By cyberx86

Just a random guy who dabbles with assorted technologies yet works in a completely unrelated field.

9 comments

  1. Would this configuration work with WordPress when you have Network mode activated and what modifications would you suggest to have it working properly for blogs installed as sub-domains and using “WordPress MU Domain Mapping” plugin?

    Thanks in advance!

    1. If you have been able to get W3TC to work with apache on your setup, the configuration above should be an almost drop in replacement for the W3TC rules – however, you will need to port the network specific (multisite) .htaccess rules to nginx. I unfortunately, do not have any firsthand experience with recent versions of WordPress MU/Network Mode. I believe that a few months ago W3TC was not fully supported on multisite setups – I think that each sub-site had to be configured individually – but that might have changed by now. I do know however, that even a few months ago, people had had success setting it up, but I don’t know the specifics. Sorry I can’t provide more guidance on the matter.

  2. I was waiting for nginx integration!!!! Just one (silly) question, where to get developement releases of w3 total cache?

    1. I think both have their uses – with nginx’s proxy_cache it might be harder to expire a page when it is updated, whereas w3tc just overwrites the file on disk. W3TC also stores a gzipped copy, so the file can be served without recompressing each time (not sure if proxy_cache does that – using nginx’s gzip_static module). Having actual files also means you can use a CDN for them if desired. I think both would accomplish the same task, but I don’t think there is a compelling reason to use proxy_cache (it might generate the pages faster). To be quite honest, proxy_cache hadn’t even crossed my mind until your comment – I will have to look into it a bit more – thanks.

Leave a Reply to cyberx86 Cancel reply

Your email address will not be published. Required fields are marked *