by
tundish
2012 November 24
Saturday morning

Python applications are increasingly deployed in virtual environments, where they are isolated from the packaged dependencies of the Operating System. The price for that? Those dependencies must be satisfied by your deployment system instead.

Requirement

You want to supply your Python package along with all its dependencies.

Preparation

You first need to collect those dependencies together.

  1. Generate a requirements file from your working virtual environment (py2.7 in this example):

    $ ./py2.7/bin/pip freeze -l > requirements.txt
    
  2. Create a new virtual environment to catch the downloads:

    $ virtualenv --distribute --no-site-packages pkg-cache
    $ mkdir pkg-cache/download
    
  3. Recreate your working environment, catching the downloaded packages this time:

    $ ./pkg-cache/bin/pip install --download=pkg-cache/download -r requirements.txt
    
  4. Check they all arrived. Add virtualenv-1.N.N.tar.gz if it's not there:

    $ ls -l pkg-cache/download/
    amqplib-1.0.2.tgz
    .
    .
    .
    lxml-2.3.5.tar.gz
    .
    .
    virtualenv-1.8.2.tar.gz
    
  5. It's unusual for production targets to have a compiler toolchain. So you have to build all compiled packages on a similar architecture first.:

    $ cd pkg-cache/download/
    $ tar -xzvf lxml-2.3.5.tar.gz
    $ cd lxml-2.3.5
    $ python setup.py bdist_egg
    $ cp dist/lxml-2.3.5-py2.7-linux-x86_64.egg ../
    $ cd -; rm -rf lxml-2.3.5.tar.gz lxml-2.3.5
    

Distribution

  1. Zip up all the packages
  2. scp the whole lot to the target
  3. Unpack them

Deployment

  1. Untar the virtualenv package and create a virtual environment (called py2.7 here):

    $ tar -xzvf virtualenv-1.8.2.tar.gz
    $ python2.7 ./virtualenv-1.8.2/virtualenv.py --distribute py2.7
    
  2. Install the supplied dependencies. Egg files should be installed first with easy_install like this:

    $ ./py2.7/bin/easy_install lxml-2.3.5-py2.7-linux-x86_64.egg
    
  3. Install the package from its source distribution like this:

    $ ./py2.7/bin/pip install --no-index --find-links=file:///`pwd` <package-name>
    

    This will automatically install all the remaining dependencies.

We know because we do. We do because we can.