Using nginx as a load-balancing proxy to a mongrel cluster on Dreamhost

Posted by Trey Mon, 18 Feb 2008 18:27:00 GMT

There’s been a big push recently for Dreamhost to support rails better or for rails to be easier to support. Finally, those efforts are starting to pay off and Dreamhost is beginning to respond.

Now, with a few quick clicks of a button in Dreamhost’s panel (if you are using their Private Server service), you can have your rails app running under mongrel and being proxied through apache.

Not bad. In fact, it’s the easiest rails deployment I’ve seen barring the “we’ll set it up for you” kind. However, there’s still one glaring problem. Note the singularity of the word ‘mongrel’. That’s right, you get one, I repeat, one mongrel for your app. If you wasnt to perform the standard mongrel deployment of setting up a mongrel cluster and balancing the load between all of those mongrels, you’re back to being all on your own. What kind of rails app only needs one mongrel anyway?

I would love to migrate all of my rails apps over to my Dreamhost private server, but first I have to figure out how to balance the load between my many mongrels. Since, Dreamhost doesn’t let you touch your apache config, you can’t do the standard apache mod-proxy to mongrel cluster approach. You need another step. Apache to blank to mongrel clusters.

After investigating Pen, Pound, and Balance, all good options for the right circumstances, I decided I should get my hands dirty with that new server from mother Russia, nginx, that I’ve been hearing so much about.

Since nginx isn’t officially supported on Dreamhost, you’be got to compile and install you’re own. This actually turns out to be a good thing in this case, as there is a patched version of nginx that supports fair balancing instead of the round-robin approach used by the official stable branch.

Installing nginx

First, if you haven’t installed software on you Dreamhost account before, create a src directory in your home folder to store any source code you will download.

$ mkdir src

Now, nginx requires a few libraries to be present in order to work. From the nginx English wiki:

Requirements
  • gzip module requires zlib library
  • rewrite module requires pcre library
  • ssl support requires openssl library

We’ll be using all of these. Luckily, though, Dreamhost already has the openssl and zlib libraries installed. We just need to grab the pcre library. The latest version is 7.6.

Note: this might actually be installed on Dreamhost somewhere, but I was unable to find it.

Change into your source directory and download the library.

$ cd src
$ curl -O http://superb-east.dl.sourceforge.net/sourceforge/pcre/pcre-7.6.tar.bz2
$ tar -xjvf pcre-7.6.tar.bz2

Now, we aren’t actually going to install the library. Instead, we’re just going to tell nginx where the library is when we compile it, so it can grab the necessary chunks and get on with it’s business.

Next, let’s download that patched nginx so we can have fair balancing. If you for some reason don’t want fair balancing, you can get substitute the url below with this one: http://sysoev.ru/nginx/nginx-0.5.35.tar.gz

Still in the ~/src directory:

$ curl -O http://www.12spokes.com/nginx.git-upstream_fair-0.6.tar
$ tar -xvf nginx.git-upstream_fair-0.6.tar 
$ cd nginx.git
$ ./configure --with-pcre=/home/<your username>/src/pcre-7.6/ --prefix=/home/<your username>/local/nginx --with-http_ssl_module
$ make
$ make install

Note: I couldn’t figure out how to curl or wget this tarball from the git repository, so I downloaded it and put it in my directory for now. If anyone can figure out how to get it form the command-line, please let me know in the comments. If you want to get the file straight from the source, the url for the repo is http://git.localdomain.pl/?p=nginx.git;a=tree;h=upstream_fair-0.6;hb=upstream_fair-0.6

Now you have nginx installed on Dreamhost. You just need to set yourself up with a config file and you’ll be ready to start balancing those mongrels (fairly even). My config file is basically a combination of the default config file found in /home//local/nginx/conf and Ezra’s optimized config

Set up your mongrel clusters

If you don’t already, have some mongrels running, set some up:

$ cd /directory/of/your/rails/app
$ mongrel_rails cluster::configure -e production -p 8000 -N 3 -c /directory/of/your/rails/app -a 127.0.0.1

Then fire ‘em up.

$ mongrel_rails cluster:start

If you set everything up correctly, you should now have three mongrels running on ports 8000, 8001, and 8002. You’ll want to wrap this last command into a startup script, so you can feed it to cron and have monit monitor your mongrels, but that’s beyond the scope of this tutorial.

Configureing nginx

Now, in your nginx.conf file set up the cluster

upstream mongrel_yourapp {
    fair;
    server 127.0.0.1:8000;
    server 127.0.0.1:8001;
    server 127.0.0.1:8002;
}

Another thing you’ll need to change is the port that nginx listens to. Most of the examples on the web are using nginx to replace apache. Since this isn’t an option on Dreamhost, we’ll need to have nginx running on a different port. I use 8080.

server {
    # port to listen on. Can also be set to an IP:PORT
    listen       8080;

    ...

Starting nginx

Now, test your config file with the -t option and if everything is peachy, fire it up.

$ /home/<your username>/local/nginx/sbin/nginx -t
    2008/02/18 09:52:25 [info] 11891#0: the configuration file /home/<your username>/local/nginx/conf/nginx.conf syntax is ok
    2008/02/18 09:52:25 [info] 11891#0: the configuration file /home/<your username>/local/nginx/conf/nginx.conf was tested successfully
$ /home/<your username>/local/nginx/sbin/nginx

At this point, you have nginx running on port 8080. It then proxies to your mongrels running on port 8000, 8001, and 8002, keeping everything nice and even. We just need to tell Dreamhost to have apache proxy all requests to our application’s domain to nginx instead of trying to serve them itself.

Configure Dreamhost’s Proxy

Go to Domains->Mongrel and Proxy. Under ‘Set Up A Proxy Server Port’, choose the url for your application (leaving the path blank if your app serves the entire domain) and enter 8080, the port for nginx, into the ‘Port Number to Proxy’. Click the ‘Add Proxy Server’ and wait. After the standard Dreamhost 10 minutes to do anything passes, you should be able to go to your main url and see your application served up through all those little proxy tubes we just finished putting in place.

Voila.

The initial tests that I’ve performed shows a slight hit by having to still go through apache, but nothing too terrible. There’s probably a lot of other things we can do to optimize our applications before we get to that one. After I get some more benchmarking done, I’ll post my results.

IE conditional comment helper for Rails 1

Posted by tieg Thu, 05 Apr 2007 00:10:00 GMT

These days, now that we have IE7, it’s common for those of us developing for the web to use conditional comments instead of CSS hacks to make sure our sites look good in different versions of Internet Explorer. (Or to pass anything else exclusively to IE)

I find myself using them on a regular basis, so here’s a little helper you can throw in your ./helpers/application.helper.rb to Rubify some of that commentage:

  # info at http://msdn.microsoft.com/workshop/author/dhtml/overview/ccomment_ovw.asp
  def conditional_comments(options={}, &proc)
    inverse    = options[:inverse] || ''
    comparison = options[:comparison] ? " #{options[:comparison]}" : ''
    version    = options[:version] || 5
    concat("<!--[if#{comparison} #{inverse}IE #{version}]>\n#{yield}\n<![endif]-->", proc.binding)
  end

Just put it in an ERb evaluation block and feed it a block of output:

<% conditional_comments :comparison => 'gte', version => 5.0 do
      stylesheet_link_tag 'layout_ie', 'typography_ie'
   end %>

Output:

<!--[if gte IE 5.0]>
<link href="/stylesheets/layout_ie.css" media="screen" rel="Stylesheet" type="text/css" />
<link href="/stylesheets/typography_ie.css" media="screen" rel="Stylesheet" type="text/css" />
<![endif]-->

It might be a little too much abstraction or processing for your tastes, but I was just a little sick of seeing the comments in every header. Yatta!