Development & Design Blog

In this article (the spiritual successor of the mod_wsgi article), I’m going to walk through the installation and configuration of a TurboGears 2.0 production environment, on top of Apache and mod_wsgi.

Step 1: Prereq’s

I assume you have Apache 2/mod_wsgi/virtualenv set up as we did in the previous post and an already functional TG2.0 app that works with the paster serve server.

I also assume that your python version is 2.4, but the same instructions should work for 2.3-2.5, just by replacing the string “python2.4″ with (e.g.) “python2.5″ wherever it appears below.

The example paths I will be using are as follows (keep in mind that they’ll all need to be readable by your Apache user [e.g. "www" or "nobody"]):

base application directory:              /home/myuser/myapp/
application deployment config:           /home/myuser/myapp/myapp/config/deployment.ini
application wsgi script:                 /home/myuser/myapp/wsgi-config/wsgi-deployment.py
base wsgi python environment:            /wherever/you/keep/shared/libraries/python-wsgi-baseline 
application specific python environment: /wherever/you/keep/shared/libraries/python-myapp
base wsgi python egg cache:              /wherever/you/keep/shared/libraries/python-egg-cache

Make sure the user you’re using to install with has access to gcc. Some of the packages required by TG2 require you to compile things. If you’re on CentOS, you’ll want to see the post on gcc permissions.

You can test this by trying to run gcc. If you get an error like the following, you’ll need to update your the permissions.

$ gcc
bash: /usr/bin/gcc: Permission denied

Step 2: Set up a Virtual Environment

First we create the application specific python virtualenv environment.

$ cd /wherever/you/keep/shared/libraries
$ source python-wsgi-baseline/bin/activate
$ virtualenv python-myapp
$ deactivate
$ source python-myapp/bin/activate

Now you have created and activated the application specific environment, install turbogears to it!

Step 3: Install Python Packages

(myapp-python)$ easy_install -i http://www.turbogears.org/2.0/downloads/current/index tg.devtools

Now install your tg app like usual: perhaps you just check it out of git/svn and make sure the required packages are installed, perhaps you use the setup.py script.

Sometimes TG’s dependency resolution isn’t perfect. On one of my old installations, before installing tg.devtools, I had to make sure that new enough versions of repoze.who and webob were installed:

$ easy_install "repoze.who"
$ easy_install "WebOb>=0.9.6.1"

As far as I know, however, the latest release should work fine out-of-the-box.

Step 4: Test the Configuration with Paster

Now make sure you’ve got your deployment.ini set up correctly for the server (database user/password is the most immediate thing that will likely change from development.ini).

You can test your installation by running the server with paster.

$ cd /home/myuser/myapp
$ paste serve myapp/config/deployment.ini

If this works fine, you’re ready to set up the WSGI interface through Apache.

Step 5: Setup Apache-writable Cache Directories

First, we’ll create a “data” directory for session data and things associated with the deployment instance. This directory must be both readable and writable by your Apache user.

In this example, we’ll make it read/writable by group, and add the Apache user’s default group to the filer’s ownership. apachegroup is probably “www” or “nobody” on your system.

$ cd /home/myuser/myapp
$ mkdir myapp/config/data
$ sudo chown myuser:apachegroup myapp/config/data
$ sudo chmod g+rw myapp/config/data

Next, we’ll create the egg cache: this is necessary because the Apache user shouldn’t have write access to the shared python scripts, but will need to cache copies of them like any python application.

We’ll create the folder

$ mkdir /wherever/you/keep/shared/libraries/python-egg-cache
$ sudo chown apacheuser:apachegroup /wherever/you/keep/shared/libraries/python-egg-cache

Next, we’ll create the wsgi script for the application.

Step 6: Create WSGI Script

$ cd /home/myuser/myapp
$ mkdir wsgi-config
$ vim wsgi-config/wsgi-deployment.py

import os, sys, site
 
os.environ['PYTHON_EGG_CACHE'] = '/wherever/you/keep/shared/libraries/python-wsgi-egg-cache'
site.addsitedir('/wherever/you/keep/shared/libraries/python-myapp/lib/python2.4/site-packages')
 
sys.path.insert(0, '/home/myuser/myapp')
sys.path.insert(1, '/wherever/you/keep/shared/libraries/python-myapp/lib/python2.4')
 
from paste.deploy import loadapp
 
application = loadapp('config:/home/myuser/myapp/myapp/config/deployment.ini')

Step 7: Update Apache Config

I assume that your Apache setup is configured as it would be with cpanel/whm. Following the directions here, for std or ssl (I’ll assume std) connections, create the file: /usr/local/apache/conf/userdata/[ssl or std]/[apache version 1 or 2]/<user>/<domain>/<something>.conf

In this case we’ll create:

$ sudo vim /usr/local/apache/conf/userdata/std/2/myuser/myapp.com/wsgi.conf

Contents:

# You can tweak this next line to your heart's content based on the mod_wsgi documentation, but this should work for now.
WSGIDaemonProcess myuser threads=1 display-name=%{GROUP}
WSGIProcessGroup myuser
 
# we'll make the root directory of the domain call the wsgi script
WSGIScriptAlias / /home/myuser/myapp/wsgi-scripts/wsgi-deployment.py
 
# make the wsgi script accessible
<Directory /home/myuser/myapp/wsgi-scripts>;
	Order allow,deny
	Allow from all
</Directory>
 
# We'll need to create exceptions for all static content.
# Aliases to serve general project media
Alias /styles /home/myuser/myapp/myapp/public/styles
Alias /admin /home/myuser/myapp/myapp/public/admin
Alias /images /home/myuser/myapp/myapp/public/images
Alias /scripts /home/myuser/myapp/myapp/public/scripts
 
# Make all the static directories accessible
<Directory /home/myuser/myapp/myapp/public/*>
	Order allow,deny
	Allow from all
</Directory>

Now, verify that the script can be read by Apache properly:

$ sudo /scripts/verify_vhost_includes --show-test-output

If you get output like:

Testing /usr/local/apache/conf/userdata/std/2/myuser/myapp.com/wsgi.conf...FAILED
No changes made without --commit flag
[TEST RESULTS]
Syntax error on line 2 of /usr/local/apache/conf/userdata/std/2/myuser/myapp.com/wsgi.conf:
Invalid command 'WSGIDaemonProcess', perhaps misspelled or defined by a module not included in the server configuration
[/TEST RESULTS]

This seems to indicate that the WSGI module hasn’t been properly loaded. All I can tell you is that when I ignored this warning, everything worked fine anyways.

Now rebuild the httpd.conf to add include statements for your wsgi.conf:

$ sudo /usr/local/cpanel/bin/build_apache_conf

That’s it! Your Apache config should now include the WSGI configuration to load your application’s WSGI script, and the WSGI script is set up to use your deployment.ini config.

Done!

Restart Apache and check out your site!

Possible Gotchas:

1. CENTOS 4: CentOS 4 ships with libexpat 1.95.7. If you are using Python >= 2.4, that ships with libexpat 1.95.8. These two versions of libexpat are not compatible with each other.
   If your site gives a 500 error, and the Apache logs show a message like:

   [notice] child pid 3238 exit signal Segmentation fault (11)

   Then you probably have a libexpat version mismatch.

   Apache is generally dynamically linked to libexpat, while python is statically linked, so just updating your system version of libexpat to 1.95.8 should solve the problem.

   Check out Graham’s documentation for more details.

2. Python 2.4:
   Python 2.5 introduced built-in support for hashlib. Python 2.4 has an installable hashlib module that provides essentially the same functionality. There are a few inconsistencies, though.

   If your site returns a 500 error and you have an error message in your Apache logs that looks like:

   File '/wherever/you/keep/shared/libraries/python-myapp/lib/python2.4/site-packages/Beaker-1.3-py2.4.egg/beaker/session.py', line 40 in value_decode
     sig = hmac.new(self.secret, val[40:], sha1).hexdigest()
   File '/usr/local/lib/python2.4/hmac.py', line 107 in new
     return HMAC(key, msg, digestmod)
   File '/usr/local/lib/python2.4/hmac.py', line 42 in __init__
     self.outer = digestmod.new()
   AttributeError: 'builtin_function_or_method' object has no attribute 'new'

   You’ve run into one of those inconsistencies. The simplest way to fix it, is to replace the hmac.py in your python2.4 installation with hmac.py from a python2.5 installation. You should be able to do this inside your python-wsgi-baseline virtualenv folder. [6]

3. File Permissions:
   If you didn’t set up permissions for your data directory or python-egg-cache directory properly, you’ll get error messages like:

     File "/home/simple/library/python-wsgi-baseline/lib/python2.4/os.py", line 156, in makedirs
       makedirs(head, mode)
     File "/home/simple/library/python-wsgi-baseline/lib/python2.4/os.py", line 159, in makedirs
       mkdir(name, mode)
   OSError: [Errno 13] Permission denied: '/home/mediaple/mediaplex/mediaplex/config/data'

   Or:

   ExtractionError: Can't extract file(s) to egg cache
    
   The following error occurred while trying to extract file(s) to the Python egg cache:
    
   [Errno 13] Permission denied: '/var/www/.python-eggs

4. Other gotchas: see Graham Dumpleton’s list of Issues.

REFERENCES:

1. How to Install TurboGears 2
2. mod_wsgi integration with Pylons
3. mod_wsgi issues with libexpat
4. High Availability TG2 via Paster, Apache+modWSGI and/or nginx on the TurboGears mailing list
5. CPanel docs on Apache’s Custom Directives
6. AttributeError in beaker in tg2.0b2 python2.4 on the TurboGears Trac
7. potential issues with mod_wsgi