Successfully hosting Drupal on nginx on Dreamhost - a Dreamhost Drupal Hosting Adventure

Date: Fri Apr 25 2014 nginx »»»» Dreamhost »»»» Drupal Performance »»»» Drupal
Among the standard performance recommendations for Drupal is to switch to the nginx web server.  Because of nginx's design it's much more performant than Apache, supposedly.  I don't know enough myself about nginx to say why it's better, other than having an understanding that nginx has an event-oriented architecture that's cleaner than Apache's.  Every so often I get on a kick of trying to use nginx based on reading some blog post or tutorial, and a couple weeks ago I came across another one that inspired me to give nginx another try (see link at bottom of this post).  And along the way I came up with a solution to the issue that's always plagued me over using nginx with Drupal.  Hopefully the result will be reliable enough so that I'll stay with nginx.

Fortunately Dreamhost has a control panel option on Dreamhost VPS's to switch between different web servers.  If you have a Dreamhost VPS you can choose between using Apache, nginx, lighthttpd OR to have no web server at all.  I'm not sure why you'd be renting a Dreamhost VPS that has no VPS but it's an option that some may want, and they'll let you do so if you want.  In any case it's easy enough to use the control panel and switch to nginx.  The "Drupal Hosting Adventure" blog post (link at the bottom) I started from assumes you have a bare Ubuntu server and takes you through all the steps of setting up nginx from scratch.  Fortunately as a Dreamhost VPS user you can skip over some of the steps.

Unfortunately just switching to nginx ins't going to give you a working Drupal setup.  So let's take this by the steps ..

Dreamhost provides us with an nginx configuration that supports multiple domains etc.  This way their control panel can still be used to host domains on a VPS configured for nginx, just as on a VPS configured for Apache.  I'm sure their intention is that we have the same control panel experience irregardless of the web server being used.

Basically there's a whole bunch of nginx configuration we don't have to do.

Dreamhost has documented their setup on the support wiki at:  http://wiki.dreamhost.com/Nginx
Their nginx setup configures an FastCGI/PHP process group for each user on the VPS that has domains its hosting.  I try to have one user per VPS to host domains, and hence my VPS has one cluster of processes.  If you have more than one user you'll have more than one cluster of processes.

This is what I mean:
[psNNNNN]$ ps auxf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
...
root 7661 0.0 0.1 80520 4636 ? Ss 01:10 0:00 nginx: master process /dh/nginx/bin/nginx-be -c /dh/nginx/servers/httpd-ps42313/nginx.conf -p /dh/nginx/servers/httpd-ps42313/var/
dhapache 7662 0.0 0.2 82164 6832 ? S 01:10 0:19 \_ nginx: worker process
dhapache 7663 0.0 0.2 82164 6936 ? S 01:10 0:18 \_ nginx: worker process
11931648 7665 0.0 0.3 138024 9796 ? Ss 01:10 0:00 /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 26535 1.5 3.0 200316 80340 ? S 17:22 0:55 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28003 1.6 3.0 200528 80784 ? S 17:23 0:56 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28347 1.4 2.7 193284 73288 ? S 17:23 0:48 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28581 1.5 3.3 207232 87276 ? S 17:24 0:53 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28749 1.5 3.4 211672 91656 ? S 17:24 0:52 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 29523 1.6 2.9 197508 77408 ? R 17:24 0:54 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30294 1.5 3.1 203600 83248 ? S 17:25 0:51 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30823 1.5 2.9 197564 77472 ? S 17:25 0:52 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30876 1.5 2.9 198676 78564 ? S 17:25 0:49 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30952 1.5 2.8 196496 76084 ? S 17:25 0:50 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30972 1.5 2.6 190212 70912 ? S 17:25 0:51 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31465 1.6 2.8 196136 76180 ? S 17:26 0:52 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31655 1.5 2.8 193820 74368 ? S 17:26 0:51 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31777 1.5 3.0 199040 79104 ? S 17:26 0:49 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31918 1.6 3.3 208132 88012 ? S 17:26 0:51 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 32176 1.5 2.6 189148 69792 ? S 17:27 0:48 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 32590 1.4 2.9 197028 76544 ? S 17:27 0:46 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 32653 1.5 3.1 202848 83236 ? S 17:27 0:50 \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock

This is something to be aware of and you may want to enforce only one user per VPS with domains, so that it limits the number of PHP processes.

The next thing to be aware of is the nginx configuration.  They provide a configuration they think will work for most people and it looks good enough to me that you probably don't want to touch it.  However what you WILL want to do is create a per-domain configuration to handle Drupal.  Or, for that matter, any application that normally works with Apache htaccess files will need some special work to run under nginx.

This issue is endemic among web applications, that they get "clean URL's" by dint of some rewrite magic in a .htaccess file that uses mod_rewrite rules to hide the ugly URL's the application developers blessed us with.

In my opinion this is a horribly broken scenario, and that web applications should work with clean URL's at all time because user experience matters.  But there are many web applications out there that use horrid URL's and hide the ugliness behind ugly mod_rewrite rulesets.

Fortunately Drupal's ugly URL's aren't so ugly … Drupal's URL's are in the form http://example.com/index.php?q=path/to/node … which makes the rewrite rules not terribly hard to create.  But Drupal, like so many other web applications, does their URL rewriting with mod_rewrite and we have to replicate this in an nginx configuration file.

Dreamhost wants us to install per-domain nginx configuration in file(s) of this name pattern:

/home/YOURUSER/nginx/YOURDOMAIN.COM/*

Here is the configuration I'm using this minute.  It's adapted and simplified from the Drupal Hosting Adventure blog post (link below) because Dreamhost already does for us many things and we can get away with just this:

[psNNNNN]$ cat ~/nginx/davidherron.com/drupal.conf
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
expires max;
log_not_found off;
}

location / {
try_files $uri $uri/ @drupal;
}

location @drupal {
rewrite ^/(.*)$ /index.php?q=$1 last;
}

This is pretty straightforward.  The first block sets a long expires value in the header if its a static file.  The second block tries the requested URI to see if its a file in the file system, if it is NOT one then it executes the "@drupal" rule.  The @drupal rule is what does the rewrite to change over to Drupal's particular URL format.  And, well, that's about it.  The Dreamhost Wiki has some other rules you might think about, but this is enough to get you going.

Now let's talk about how to make this reliable, because reliability is not covered at all on the Dreamhost wiki page.  What always hung me up on nginx was that it worked great for a few hours, nice, fast, wonderful, but then a day later I'd get an email from someone saying "why does the website say 504 Bad Gateway"?  Ugh.  It happened every time I tried nginx, after awhile it would get into this mode where all it does is give this ugly error page.  Unfortunately I didn't know enough about nginx to figure it out, and worst there wasn't any obvious (to me) explanation of what to do.
This time I did some yahoogling and found that this is an endemic problem with using not only PHP under nginx, but other platforms like Ruby.  (see links below)  What the Bad Gateway error tells us is that the language platform (FastCGI/PHP in this case) has crashed.  Because nginx doesn't know how to restart FastCGI all it knows is that the service isn't responding and all it can do is throw up this ugly Bad Gateway page.

Interestingly the discussion on the dreamhost forum says Dreamhost's support knows about this, but the Dreamhost Wiki doesn't say anything about this.  The pages linked below makes it clear this is a "known problem" with known solutions, namely to "monitor" FastCGI/PHP and restart it upon crashes.

What "monitor" means is to regularly inspect whether FastCGI/PHP is still running, so that it can be restarted when needed.  This is where I can't yet provide good advice, because I haven't fully fleshed out this part of the story.

I'm working with the Dreamhost support on the reliability issue.  They know about it, but aren't documenting it on their support Wiki.  Hurm.  Anyway, they have a shell script they can install.   It can detect when FastCGI processes aren't there, and restart the FastCGI/PHP processes when needed.  Talk to Dreamhost support about this.  If FastCGI/PHP isn't running the script does this:

/etc/init.d/nginx startphp 

This is a special function in the nginx init script they provide, and essentially it starts the FastCGI process group as shown above.

Their script is working well so far - except for one day when there were FastCGI/PHP processes running, but my website was non-responsive.  After awhile of trying different things I ended up rebooting the VPS (clearing up the problem) and then later I wrote this script as a first step towards detecting that condition.

[psNNNNN]$ cat check.sh
PHPINFO=`curl -f http://SUBDOMAIN.DOMAIN.com 2>/dev/null | grep phpinfo | wc -l`
if [ "$PHPINFO" -lt 1 ]; then
echo "NEED TO REBOOT - nginx nonfunctioning"
fi

PHPVERSION=`curl -f http://SUBDOMAIN.DOMAIN.com/phpinfo.php 2>/dev/null | grep "PHP Version"`
if [ x"$PHPVERSION" = "x" ]; then
echo "NEED TO REBOOT - PHP nonfunctioning"
fi
What I've done is - for each VPS I rent from Dreamhost - assign a dummy domain to the VPS to make it easier to access each of my VPS's.  Basically my dummy domain is more rememberable than the "psNNNNN" name which Dreamhost assigned the VPS.  What assigning a dummy domain to the VPS does is create a little website hosted on that VPS.  That little dummy website is something we can use to check whether both nginx and FastCGI/PHP are running, which is what this script does.

The first step
curl -f http://SUBDOMAIN.DOMAIN.com
Is going to cause nginx to print a directory listing of the dummy website.  Because I have a script named "phpinfo.php" in that directory, the PHPINFO variable will end up with a count indicating whether or not "phpinfo" is in the output and telling us whether the directory listing is printed telling us whether nginx is running.

The next step
curl -f http://SUBDOMAIN.DOMAIN.com/phpinfo.php
Well, this is going to run phpinfo.php (which is just a phpinfo() function invocation) and print out the results.  All the script does is grep out some text that will be in the output, "PHP Version", and if that is present it indicates that PHP is running.

I set this up to be run using cron and have verified that if there's no output cron won't send me an email, but if there is output then cron sends me an email.  Hence I have an email-based warning that FastCGI or nginx is not running.  The script is suggesting when REBOOT is desirable and could be modified to go ahead and do the reboot.

This script might be enough monitoring to implement the required reliability to keep my website running.  I'm still testing it, and after a couple days it has yet to detect a problem.
In any case I hope this has been helpful.

http://cweagans.net/blog/2011/10/26/drupal-hosting-adventure
http://hostingfu.com/article/keeping-your-php-fastcgi-processes-alive
http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:nginx:daemontools_spawnfcgi?&#comment_56be08e074b4b3b746eaad8a86a768ce
http://discussion.dreamhost.com/archive/index.php/thread-128870.html
http://www.ruby-forum.com/topic/1058059