Recently, we have deployed into a test production our latest Django project. Choosing which deployment to use it’s not easy, as there are a lot of different tools for the job, but as we expect some load on the system, we have been spending some time in getting a good deployment that will allow us to be confident on the grow of the system.

After some research, we decided to use CherryPy and Cherokee.

Why CherryPy?

• It’s pure Python, and easily integrated with Django. You can do it by yourself (it’s not very difficult), or you can take the easy way and use the django-cpserver tool. Anyway, we probably will end up customizing django-server to suit your needs, as there are some things that seems to be lacking (like logging support)
• It’s fast! Also it’s mature and stable. All three are must-have characteristics for any web server.

Why Cherokee?

• Its admin system. It’s really easy to use (instead of the usual .conf file, which can be a headache) and also quite safe, as it will only be active when a command is called, with a one-time-only password. It’s a great feature!
• It allows us to make our deployment. It’s described bellow. Basically, we are using it as a HTTP reverse proxy that will balance the load between several CherryPy instances. It will also serve our static files.
• Low memory footprint.
• Cherokee is able to manage all the CherryPy processes, so you don have to worry about launching and controlling them. You don’t have to daemonize the CherryPy processes also, making all the process much easier.
• It’s blazingly fast! It’s specially good serving static files. The speed is in the same league than ngix.

Not everything is great. We have also some problems, already solved or that will be solved in the future.

• As I said, django-cpserver doesn’t allow you to specify logging parameters to CherryPy. We will have to tweak it, although this is not difficult.
• We have a problem with this configuration with CherryPy 3.1 Ocasionally, when you ask for a page in Chrome or IE (but not in Firefox), you’ll get a blank page and a 408 error (request timeout). This will indicate that the connection between the client and the server has been lost, and it’s happening only when the option Keep Alive in the Cherokee server is activated. Asking for the page again will reload the data, as a new connection will be open. Apparently, this was due a bug in CherryPy that has been corrected in version 3.2 (which is now on Release Candidate state). Version 3.2.0RC1 seems to work perfectly fine.

Our deployment configuration is described in the following diagram.

Architecture

The static data is directly served by Cherokee, while the Django application is supported by CherryPy. There are several processes of Django-CherryPy working, one of each core the server has, to increase the response avoiding problems with the GIL working over multiple cores, as well as making the architecture much scalable. Cherokee is conecting to the CherryPy instances over TCP/IP, so it gives quite flexibility. It’s also possible to use a Unix domain socket path, which could be even (a little) faster. As Cherokee allow to define an Information Source as a local interpreter command, you just write an script creating a new CherryPy instance, and Cherokee will manage to set up the processes and to kill everything when you stop the server. Cool.

We have also used memcached to increase the response of Django. It’s quite easy to use. Just install memcached on the machine, python-memcached to allow Python access it, and with a few configuration touches on your settings.py file, you’re ready. At the moment, as our views won’t change often, our approach is to use the @cache_page decorator, to cache completely the view. We’re not caching the complete site as that will not cache everything that gets URL parameters, which we’ve been using for generic_views.

All the architecture is highly scalable. Right now everything is on the same machine, but can be spread on different machines. Each process (Cherokee, CherryPy instances and Database) can be separated without much effort. Also, more instances of Django+CherryPy can be created. The only part that can be difficult to duplicate is the Cherokee server, but it’s difficult it will come to be the bottleneck of the system, unless the system grows REALLY big.

The system seems to be really fast and we’re having great testing results for the moment. In a couple of weeks the system is supposed to be public, and we expect quite a lot of hits, so performance it’s quite important, but, for the moment, we are quite satisfied with our deployment.

PD: For a tutorial on configuring Cherokee and CherryPy, you could check this great post.