Development & Design Blog

Intro

If you haven’t used virtualenv before, it’s essentially a quick way of setting up customized Python environments. When one virtual environment is active, new packages are installed only to that environment, rather than your default Python site-packages directory, so you can easily avoid cluttering up your base Python installation with temporary, unstable, or otherwise unsavoury packages. In addition to this, you can set up a virtualenv with the --no-site-packages option. This will disable access to your default Python site-packages entirely. This makes it very easy to ensure the correct version of any particular package is in use. It is especially handy for development.

# to install virtualenv:
$ sudo easy_install virtualenv
 
# or, on macports:
$ sudo port -v install py25-virtualenv

The Problem

Using MacPorts, creating virtualenv environments with --no-site-packages enabled will fail when Python 2.5 applications try to access libraries like hashlib (including sha and md5).

This is because, under MacPorts, the Python 2.5 core libraries are separated from the less core libraries like hashlib. These are installed separately.

# e.g.
$ sudo port install -v py25-hashlib

Under normal circumstances this works just fine—hashlib is installed to /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/ like other user extensions. The problem comes when you want to run Python without the site-packages.

# like with
$ python -S
# or with
$ virtualenv --no-site-packages

This is because many python applications expect hashlib, et al., to be part of the core python libraries, and installed at /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/. This is generally a safe expectation, but one that the MacPorts guys evidently didn’t consider when doing the MacPorts 1.7 packages.

The Solution

The workaround, as presented in the google groups link is to copy all of the .../site-packages/*.so files to .../lib-dynload/.

$ cp /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/*.so  \
     /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/

The google groups posting says to symlink the files, but I found that I had to copy them to make it work.

References:

1. Split python25 into python25-core and modules on MacPorts Trac
2. py25-virtualenv – 1.3.2 – –no-site-packages broken due to non-standard macports module install on MacPorts Trac
3. Optional Python stdlib modules should install into lib-dynload and not site-packages on MacPorts Trac
4. Shouldn’t Python stdlib C extensions be in lib-dynload rather than site-packages? on the macports-users mailing list
5. virtualenv issue on an OS X system using MacPorts on the python-virtualenv mailing list