by
tundish
2013 February 13
Wednesday evening

High hopes

Every one of us hopes that his work is a success; that his contribution is welcome, and that his software has users. But it's rarely we wonder what success might mean for that project itself, and what pains might come from growth.

All projects go through stages of growth; from the first proof of concept to a delivered product; then possibly to a comprehensive set of solutions for an entire problem domain.

In the first of four tutorials, I'm going to show you how to package your Python program in the simplest way possible. And then explain why you probably shouldn't do that.

With only a little more effort, you can design your package for scale. This will allow others to contribute without creating complexity. At the same time, you won't lose control over the code you care about.

Inspyration

We'll begin with the simplest useful project I can think of; a message of the day (MOTD) application. It gives you a random quote or proverb to provide perspective to your working day.

We will build this application into a Python distribution which can be installed by anyone. The user will get a program he can use from the command line. He'll also be able to import the MOTD library for use in his own programs.

We'll start from scratch so you can see how a simple distribution is made.

A simple Python distribution

Start by creating a directory called ppfsp1 after the title of this article. Inside that directory, place four empty files as follows:

ppfsp1/
├── inspyration.py
├── MANIFEST.in
├── README.txt
└── setup.py

We can produce a distributable module by populating just three of these files. Now let's fill them in, beginning with the module which does the work.

inspyration.py

Every good Python script begins with two functional comments. The first tells us what interpreter to use, the second specifies the multicharacter encoding of the text.

#!/usr/bin/env python3
# encoding: UTF-8

Every module should have a __doc__ variable containing its docstring.

__doc__ = """A program which prints a message-of-the-day"""

This is to be a command line script, so in a moment we'll need the sys module. We will be picking a phrase at random. There's a function for this in the random module, so we import that too.

import random
import sys

Next we'll define a sequence of our favourite phrases.

data = [
    "Many a slip twixt cup and lip.",
    "The gates of Hell are barred from the inside.",
    "When money comes in the door, friendship leaves by the window.",
    "Those whom the gods wish to destroy, they first make mad."
    "A journey of a thousand miles begins with a single step.",
    "Better to light a candle than curse the darkness.",
    "Early to bed and early to rise makes a man healthy, wealthy and wise.",
    "Fine words butter no parsnips.",
    "Nothing succeeds like success.",
    "To travel hopefully is a better thing than to arrive."
]

Here's the function our module provides to select one of the phrases:

def pick():
    return random.choice(data)

When the module is invoked as a script by itself, we kick it off like this:

if __name__ == "__main__":
    print(pick())
    sys.exit(0)

README.txt

This is the place to keep the documentation for a small project. It's the first thing the user reads after he downloads the source. It should say:

  • What it does
  • How to contact the author

If you wish, you can format this information in reStructuredText. Soon, we'll find out why that's useful.

Here's what I'm putting in the README.txt for inspyration:

:author:    D Haynes
:contact:   tundish@thuswise.org

`Inspyration` was written to illustrate some aspects of packaging Python
programs.

setup.py

setup.py is another Python file, but it's not really a place for any of our code. Think of it as a registration form for our project, which happens to have the syntax of Python.

It should have a shebang line, an encoding declaration, and the necessary imports.

#!/usr/bin/env python
# encoding: UTF-8

from distutils.core import setup
import os.path

We need to include documentation for our project. If we supply reStructuredText, it will be rendered as a web page in PyPI. Common practice is to suck in our README.txt file for that purpose. Nobody is ever going to import our setup.py file as if it were a module, but I can't think of a reason why we shouldn't call this description __doc__ anyway.

__doc__ = open(os.path.join(os.path.dirname(__file__), "README.txt"),
               "r").read()

Now for the setup invocation. This is mandatory. Your results vary enormously depending on what arguments you provide. Consider this scrap here as standard boilerplate.

setup(
    name="inspyration",
    version="0.01",
    description="A simple MOTD program to illustrate packaging techniques",
    author="D Haynes",
    author_email="tundish@thuswise.org",
    url="http://pypi.python.org/pypi/inspyration",
    long_description=__doc__,
    classifiers=[
        "Operating System :: OS Independent",
        "Programming Language :: Python :: 3",
        "License :: CC0 1.0 Universal (CC0 1.0) Public Domain Dedication"
    ],
    py_modules=["inspyration"],
    scripts=["inspyration.py"]
)

Installation

We are now going to check we can install the code.

Create a virtual environment

When testing packages, you should install them in a separate environment so that they don't get mixed up with your real programs. We use virtualenv here to 'clone' our current Python 3 environment to a local directory:

$ virtualenv -p python3 --system-site-packages py3

Install the package

This is the exciting part; pretending to be a user, installing inspyration for the first time with this command:

$ ./py3/bin/python setup.py install

Testing

Even such a simple program can give a lot of joy. Let's lean back and enjoy the show for a moment:

watch ./py3/bin/inspyration.py

Oops.

If you followed this far, then it won't have been long before you saw this:

Those whom the gods wish to destroy, they first make mad.A journey of a thousand miles begins with a single step.

Reflection

It seems I missed out a comma in the data, and concatenated two phrases which should have been separate. Silly me.

It was such a simple piece of code too. Perhaps we should just fix it and ship it.

But then, every time I add to my list of phrases, I run the risk of the same thing happening again. And as the list gets longer, it will be harder to spot them by eye.

This is the moment when we realise we should have written a unit test. And with a heavy heart, we issue the stern command:

$ ./py3/bin/pip uninstall inspyration

Next time, I'll show you how to change the structure of your project so that it can contain tests as well as code.

We know because we do. We do because we can.