vendredi 2 octobre 2015

Now I'm using py.test and you should too

You all know that Python is battery included. In particular it is shipped with unittest module, which is a test framework inspired by JUnit. Globally, it's a great module, to which nice features have been added in the latest version of the language:

  • New assertions
  • Mocks
  • Test discovery

However, it has a annoying drawbacks: it's verbosity and the fact it's not PEP8 compliant (or let's say not Pythonic). Even JUnit got better with Java 1.5 and the usage of annotation, freeing the programmer to extend a base TestCase class to name the testing methods test_something.

If you're tired of this or if you just want to spent less effort on testing, give a try to py.test.

Name a function test_something and write you assertions using assert Python instruction. Then simply run:

py.test

Which will discover and run all the tests in your project directory structure. The results are then displayed in a detail way (configurable, of course).

Though everything is not so bright, I enjoy using it on a daily basis. Here is my return of experience.

Good points:

  • It's very concise: no subclassing (nor even class) needed, no self.assertWeirdStuff either. This boilerplate avoidance is a countable gain of time. It's by itself a reason to switch.
  • It runs also your unittest-style tests and your doctests, therefore you can take your time to migrate your testing base.
  • Output is configurable and is very helpful. For instance, Using -vv switch provides a detailed diff which I used lately to compare string blocks. It can also help to integrate with CI platform.
  • You have lots of plugins available, for instance to write test cases in Gherkin or implement property based testing.

Not so good points:

  • It is not included in the standard distribution, so you've to install it through pip. Not so long ago, you also had to install pip. For Python beginners, it is not so simple (believe me).
  • Forget what you know about setup and teardown method. This features are available in py.test through @pytest.fixture decorator. The thing is, if implementing setup method through fixture is straightforward, teardown is a bit weird when you're not used to. See bellow for the explanation.
  • Lots lots lots of options. It is not an absolute bad thing, you'll say. But all I want to do is testing!

So you see the bad points are not so bad ^_^

Here is the trick to introduce teardown behaviour, extracted from py.test official documentation:

import smtplib
import pytest

@pytest.fixture(scope="module")
def smtp(request):
    smtp = smtplib.SMTP("merlinux.eu")
    def fin():
        print ("teardown smtp")
        smtp.close()
    request.addfinalizer(fin)
    return smtp # provide the fixture value

Explanations:

  • scope parameter allows to chose if the fixture should be run for each testing function, each test class or once for the whole test modules.
  • request parameter in the fixture function allows you to access your testing session. The teardown function should be added to the request with the addfinalizer, that accepts a callable.

To conclude, use py.test. It is really easy to put tests in place and it's highly versatile. And you won't lose your time writing boilerplate.