by
tundish
2013 February 23
Saturday morning

Recap

In the first part of this series, our simple Python project contained only a single script. We thought we were clever enough to write this little program without any bugs, but we were wrong.

Releasing untested code is like serving up sausages on sticks. It may seem like what people want at the time, but it doesn't address the customer's long term needs.

Today, our project will evolve from kids' party food to a decent, healthy snack. Something more like a Duck Wrap. It will look nicer from the outside, and will have better stuff in the middle.

Turn a module into a package

Copy your files from the previous tutorial to a directory called ppfsp2. Your project should look like this:

ppfsp2/
├── inspyration.py
├── MANIFEST.in
├── README.txt
└── setup.py

This is how we left our project last time. It contained a single Python module, and the bits and pieces we needed to make a PyPI distribution.

What we want instead is a container, not just for our code, but for unit tests as well. This is what a Python package does; it provides a parent for modules.

So make a directory called inspyration. Add a file to that directory called __init__.py. This is what makes inspyration now a Python package instead of simply a folder. It has this one line only:

__version__ = "0.02"

Now move inspyration.py to be main.py inside that directory:

ppfsp2/
├── inspyration
│   ├── __init__.py
│   └── main.py
├── MANIFEST.in
├── README.txt
└── setup.py

We will need to change setup.py too. That's so our source distribution contains the right files.

We need to change the setup.py in three places. First to import our package from the source directory:

import inspyration

This allows us to define our package version without repeating ourselves. Change the version argument to setup to look like this:

version=inspyration.__version__,

At the end of the setup invocation, we are going to declare that inspyration is a package, and no longer a module.

Take out this line:

py_modules=["inspyration"],

... and replace it with this:

packages=["inspyration"],

With these structural changes done, we can set about improving the behaviour of our app.

  • We'll make the program easier to call from the command line
  • Refactor the code a little
  • Add a test to catch our known bug.

Finally, we'll fix the bug so that our project can ship.

Add a console entry point

Now there's no longer an inspyration.py, how do we invoke the program?

Python packages have a feature called entry points. They specify what parts of the packages are publicly available for use. They can be used in many ways, but to begin with we will define a console entry point so our user can invoke inspyration like any other command line program, eg: date or cal.

This is a design decision which has a slight down-side. It creates an installation dependency beyond the standard distutils module. Change the import line in setup.py so that it now reads:

from setuptools import setup

In the past, opinion was divided on whether setuptools should be considered essential. However, if you are going to use Python packaging to best advantage it is a necessary ingredient.

We will now remove the distutils script declaration, and replace it with the newer setuptools equivalent. Remove this line from setup.py:

scripts=["inspyration.py"]

... and paste in these instead:

entry_points={
    "console_scripts": [
        "inspyration = inspyration.main:run"
    ],
}

Separate the data from the code

We are close to writing a test for our package. Before we do, there is a simple bit of refactoring to move our list of phrases into a separate module.

Remove the declaration of the data from main.py and put it in a separate file called content.py.

We'll take this opportunity to make main.py look more like a proper Python command line program. Here's a full listing:

#!/usr/bin/env python3
# encoding: UTF-8

import argparse
import random
import sys

from inspyration.content import data


def pick():
    return random.choice(data)


def main(args):
    print(pick())
    return 0


def parser():
    rv = argparse.ArgumentParser()
    return rv


def run():
    p = parser()
    args = p.parse_args()
    rv = main(args)
    sys.exit(rv)

if __name__ == "__main__":
    run()

Add a test module

Now for some code to test our content.py module. Best practice for a module of unit tests is for it to take the name of the code module it relates to, but to prefix that name with test_.

So, make a new file inside the inspyration directory called test_content.py, and paste into it this code:

#!/usr/bin/env python3
# encoding: UTF-8

import re
import unittest

from inspyration.content import data


class ContentTest(unittest.TestCase):

    def test_each_quote_starts_with_uppercase(self):
        self.assertTrue(all(i[0].isupper() for i in data))

    def test_each_quote_ends_in_full_stop(self):
        self.assertTrue(all(i.endswith(".") for i in data))

    def test_each_internal_full_stop_precedes_a_space(self):
        testRE = re.compile("\.\S")
        found = [testRE.search(i) for i in data]
        self.assertFalse(any(found),
                         [i.string for i in found if i is not None])

Take a second to read this; we have created a unit test module for inspyration.content which has three tests assembled under a test class. The class applies rules for the proper formatting of inspirational phrases.

The final shape of your project should look like this:

ppfsp2/
├── inspyration
│   ├── __init__.py
│   ├── content.py
│   ├── main.py
│   └── test_content.py
├── MANIFEST.in
├── README.txt
└── setup.py

Well done for getting this far! I hope you remember how to make a virtualenv from the last tutorial, because now it's time to test our packaging and distribution:

$ virtualenv -p python3 --system-site-packages py3

Installation

Issue this command to install inspyration into the virtualenv:

$ ./py3/bin/python setup.py install

Pretending to be a curious user, we can discover the version of our package at run-time:

$ ./py3/bin/python
Python 3.2.3 (default, Oct 19 2012, 20:10:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import inspyration
>>> inspyration.__version__
'0.02'

Testing

Here is the payoff; we will run our tests to check the code has no bugs. Invoke the unittest module from the virtualenv like this:

$ ./py3/bin/python -m unittest inspyration.test_content
F..
======================================================================
FAIL: test_each_internal_full_stop_precedes_a_space
(inspyration.test_content.ContentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "inspyration/test_content.py", line 21, in
test_each_internal_full_stop_precedes_a_space
    [i.string for i in found if i is not None])
AssertionError: True is not false :
['Those whom the gods wish to destroy, they first make mad.A journey of a thousand miles begins with a single step.']

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

... and there's our problem. Fix the bug by adding a comma to delimit every string in content.py.

We find ourselves running tests frequently; we want a simple way to do it without much typing. There is a test discovery feature to unittest which is very useful:

$ ./py3/bin/python -m unittest discover

That will run all the tests to be found in our project's source tree. We should also check that a newly installed package behaves itself too:

$ ./py3/bin/pip uninstall inspyration
$ ./py3/bin/python setup.py install
$ ./py3/bin/python -m unittest inspyration.test_content

... either way, when you run the tests again, you should see this comforting message:

...
----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

Release

We've done a responsible job here; our package has tests and they pass. You can confidently share the code now. Make a source distribution with this command:

$ ./py3/bin/python setup.py sdist

In the dist directory you will see a file inspyration-0.02.tar.gz. This is a Python source distribution of the highest quality, accepted the world over as proof you are a Python developer.

For many projects, that's as hard as it gets. But next time, we'll go a little further. We'll look at what happens when people like your code.

In fact, they'll like it so much they'll want to change it.

We know because we do. We do because we can.