Pytest

pytest Documentation
Release 2.1.0
holger krekel et alii
July 09, 2011
CONTENTS
Getting started basics 1.1 Welcome to pytest! . . . . . . . 1.2 Installation and Getting Started 1.3 Usage and Invocations . . . . . 1.4 Good Integration Practises . . . 1.5 Project examples . . . . . . . . 1.6 Some Issues and Questions . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
3 3 4 7 10 12 13 17 17 26 31 33 39 40 43 43 45 47 50 54 56 57 58 61 62 65 68 68 69 70 71 71 72 72 72 73
Usages and Examples 2.1 Demo of Python failure reports with py.test . . . 2.2 basic patterns and examples . . . . . . . . . . . 2.3 mysetup pattern: application specic test xtures 2.4 parametrizing tests . . . . . . . . . . . . . . . . 2.5 Changing standard (Python) test discovery . . . 2.6 Working with non-python tests . . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
py.test reference documentation 3.1 pytest builtin helpers . . . . . . . . . . . . . . . . . . 3.2 basic test conguration . . . . . . . . . . . . . . . . . 3.3 The writing and reporting of assertions in tests . . . . 3.4 Injecting objects into test functions (funcargs) . . . . . 3.5 extended xUnit style setup xtures . . . . . . . . . . . 3.6 Capturing of the stdout/stderr output . . . . . . . . . . 3.7 monkeypatching/mocking modules and environments 3.8 xdist: pytest distributed testing plugin . . . . . . . . . 3.9 temporary directories and les . . . . . . . . . . . . . 3.10 skip and xfail: dealing with tests that can not succeed . 3.11 mark test functions with attributes . . . . . . . . . . . 3.12 asserting deprecation and other warnings. . . . . . . . 3.13 unittest.TestCase support . . . . . . . . . . . . . . . . 3.14 Running test written for nose . . . . . . . . . . . . . 3.15 doctest integration for modules and test les . . . . . Working with plugins and conftest les 4.1 conftest.py: local per-directory plugins . 4.2 Installing External Plugins / Searching . 4.3 Writing a plugin by looking at examples 4.4 Making your plugin installable by others 4.5 Plugin discovery order at tool startup . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
4.6 4.7 4.8 4.9 5 6
Requiring/Loading plugins in a test module or conftest le Accessing another plugin by name . . . . . . . . . . . . . Finding out which plugins are active . . . . . . . . . . . . deactivate / unregister a plugin by name . . . . . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
73 73 73 74 75 77 77 77 77 78 78 79 83 83 83 85 85 85 87 87 88 88 89 90 93 93 93 94 94 95 96 96 96 98 99 99 100 101 101 103 103 103 103 104 104 104 105 105
py.test default plugin reference py.test hook reference 6.1 hook specication and validation . . . . . . . . . . 6.2 initialisation, command line and conguration hooks 6.3 generic runtest hooks . . . . . . . . . . . . . . . 6.4 collection hooks . . . . . . . . . . . . . . . . . . . 6.5 reporting hooks . . . . . . . . . . . . . . . . . . . . Reference of important objects involved in hooks Talks and Tutorials 8.1 tutorial examples and blog postings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 conference talks and tutorials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback and contribute to py.test 9.1 Contact channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Working from version control or a tarball . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
7 8
10 Release announcements 10.1 py.test 2.1.0: perfected assertions and bug xes . . . . . . . . . . . 10.2 py.test 2.0.3: bug xes and speed ups . . . . . . . . . . . . . . . . 10.3 py.test 2.0.2: bug xes, improved xfail/skip expressions, speedups . 10.4 py.test 2.0.1: bug xes . . . . . . . . . . . . . . . . . . . . . . . . 10.5 py.test 2.0.0: asserts++, unittest++, reporting++, cong++, docs++ 11 Changelog history 11.1 Changes between 2.0.3 and 2.1.0.DEV 11.2 Changes between 2.0.2 and 2.0.3 . . . 11.3 Changes between 2.0.1 and 2.0.2 . . . 11.4 Changes between 2.0.0 and 2.0.1 . . . 11.5 Changes between 1.3.4 and 2.0.0 . . . 11.6 Changes between 1.3.3 and 1.3.4 . . . 11.7 Changes between 1.3.2 and 1.3.3 . . . 11.8 Changes between 1.3.1 and 1.3.2 . . . 11.9 Changes between 1.3.0 and 1.3.1 . . . 11.10 Changes between 1.2.1 and 1.3.0 . . . 11.11 Changes between 1.2.1 and 1.2.0 . . . 11.12 Changes between 1.2 and 1.1.1 . . . . 11.13 Changes between 1.1.1 and 1.1.0 . . . 11.14 Changes between 1.1.0 and 1.0.2 . . . 11.15 Changes between 1.0.1 and 1.0.2 . . . 11.16 Changes between 1.0.0 and 1.0.1 . . . 11.17 Changes between 1.0.0b9 and 1.0.0 . . 11.18 Changes between 1.0.0b8 and 1.0.0b9 . 11.19 Changes between 1.0.0b7 and 1.0.0b8 . 11.20 Changes between 1.0.0b3 and 1.0.0b7 . 11.21 Changes between 1.0.0b1 and 1.0.0b3 . 11.22 Changes between 0.9.2 and 1.0.0b1 . . 11.23 Changes between 0.9.1 and 0.9.2 . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
ii
11.24 Changes between 0.9.0 and 0.9.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 12 New pytest names in 2.0 (at is better than nested) 13 example: specifying and selecting acceptance tests 14 example: decorating a funcarg in a test module Python Module Index Index 107 109 111 113 115
iii
iv
pytest Documentation, Release 2.1.0
Download latest version as PDF
CONTENTS
CONTENTS
CHAPTER
ONE
GETTING STARTED BASICS

1.1 Welcome to pytest!
a mature full-featured testing tool runs on Posix/Windows, Python 2.4-3.2, PyPy and Jython comprehensive online and PDF documentation continuously tested on many Python interpreters used in many projects and organisations, in test suites ranging from 10 to 10s of thousands of tests comes with many tested examples supports good integration practises provides no-boilerplate testing makes it easy to get started, rened usage options assert with the assert statement helpful traceback and failing assertion reporting allows print debugging and the capturing of standard output during test execution supports PEP 8 compliant coding styles in tests supports functional testing and complex test setups advanced skip and xfail: dealing with tests that can not succeed generic marking and test selection can distribute tests to multiple CPUs through xdist plugin can continuously re-run failing tests many builtin helpers exible Conventions for Python test discovery unique dependency injection through funcargs parametrized test functions integrates many common testing methods
can integrate nose, unittest.py and doctest.py style tests, including running testcases made for Django and trial supports extended xUnit style setup supports domain-specic Working with non-python tests supports the generation of testing coverage reports Javascript unit- and functional testing extensive plugin and customization system all collection, reporting, running aspects are delegated to hook functions customizations can be per-directory, per-project or per PyPI released plugins it is easy to add command line options or do other kind of add-ons and customizations.
1.2 Installation and Getting Started

Pythons: Python 2.4-3.2, Jython, PyPy Platforms: Unix/Posix and Windows PyPI package name: pytest documentation as PDF: download
1.2.1 Installation
Installation options:
easy_install -U pytest pip install -U pytest # or
To check your installation has installed the correct version:

$ py.test --version This is py.test version 2.0.3, imported from /home/hpk/p/pytest/pytest.pyc setuptools registered plugins: pytest-xdist-1.6.dev3 at /home/hpk/p/pytest-xdist/xdist/plugin.pyc pytest-incremental-0.1.0 at /home/hpk/venv/0/lib/python2.6/site-packages/pytest_incremental.pyc
If you get an error checkout Known Installation issues.
1.2.2 Our rst test run

Lets create a rst test le with a simple test function:
# content of test_sample.py def func(x): return x + 1 def test_answer(): assert func(3) == 5
Thats it. You can execute the test function now:
Chapter 1. Getting started basics
$ py.test =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 1 items test_sample.py F ================================= FAILURES ================================= _______________________________ test_answer ________________________________ def test_answer(): assert func(3) == 5 assert 4 == 5 + where 4 = func(3)
> E E
test_sample.py:5: AssertionError ========================= 1 failed in 0.02 seconds =========================
py.test found the test_answer function by following standard test discovery rules, basically detecting the test_ prexes. We got a failure report because our little func(3) call did not return 5. Note: You can simply use the assert statement for asserting test expectations. pytests Advanced assertion introspection will intelligently report intermediate values of the assert expression freeing you from the need to learn the many names of JUnit legacy methods.
1.2.3 Asserting that a certain exception is raised

If you want to assert that some code raises an exception you can use the raises helper:
# content of test_sysexit.py import pytest def f(): raise SystemExit(1) def test_mytest(): with pytest.raises(SystemExit): f()
Running it with, this time in quiet reporting mode:

$ py.test -q test_sysexit.py collecting ... collected 1 items . 1 passed in 0.01 seconds
1.2.4 Grouping multiple tests in a class

Once you start to have more than a few tests it often makes sense to group tests logically, in classes and modules. Lets write a class containing two tests:
# content of test_class.py class TestClass: def test_one(self): x = "this"
1.2. Installation and Getting Started
assert h in x def test_two(self): x = "hello" assert hasattr(x, check)
The two tests are found because of the standard Conventions for Python test discovery. There is no need to subclass anything. We can simply run the module by passing its lename:
$ py.test -q test_class.py collecting ... collected 2 items .F ================================= FAILURES ================================= ____________________________ TestClass.test_two ____________________________ self = <test_class.TestClass instance at 0x142c320> def test_two(self): x = "hello" assert hasattr(x, check) assert False + where False = hasattr(hello, check)
> E E
test_class.py:8: AssertionError 1 failed, 1 passed in 0.03 seconds
The rst test passed, the second failed. Again we can easily see the intermediate values used in the assertion, helping us to understand the reason for the failure.
1.2.5 Going functional: requesting a unique temporary directory

For functional tests one often needs to create some les and pass them to application objects. py.test provides the versatile Dependency injection through function arguments which allows to request arbitrary resources, for example a unique temporary directory:
# content of test_tmpdir.py def test_needsfiles(tmpdir): print tmpdir assert 0
We list the name tmpdir in the test function signature and py.test will lookup and call a factory to create the resource before performing the test function call. Lets just run it:
$ py.test -q test_tmpdir.py collecting ... collected 1 items F ================================= FAILURES ================================= _____________________________ test_needsfiles ______________________________ tmpdir = local(/tmp/pytest-10/test_needsfiles0) def test_needsfiles(tmpdir): print tmpdir assert 0 assert 0
> E
test_tmpdir.py:3: AssertionError
----------------------------- Captured stdout -----------------------------/tmp/pytest-10/test_needsfiles0 1 failed in 0.13 seconds
Before the test runs, a unique-per-test-invocation temporary directory was created. More info at temporary directories and les. You can nd out what kind of builtin Dependency injection through function arguments exist by typing:
py.test --funcargs # shows builtin and custom function arguments
1.2.6 where to go next

Here are a few suggestions where to go next: calling pytest through python -m pytest for command line invocation examples good practises for virtualenv, test layout, genscript support py.test reference documentation for documentation and examples on using py.test Working with plugins and conftest les managing and writing plugins
1.2.7 Known Installation issues

easy_install or pip not found? Consult distribute docs to install the easy_install tool on your machine. You may also use the older setuptools project but it lacks bug xes and does not work on Python3. If you use Python2 you may also install pip. py.test not found on Windows despite installation? Windows: If easy_install or py.test are not found you need to add the Python script path to your PATH, see here: Python for Windows. You may alternatively use an ActivePython install which does this for you automatically. Jython2.5.1 on Windows XP: Jython does not create command line launchers so py.test will not work correctly. You may install py.test on CPython and type py.test --genscript=mytest and then use jython mytest to run py.test for your tests to run with Jython. Usages and Examples for more complex examples
1.3 Usage and Invocations

1.3.1 calling pytest through python -m pytest
New in version 2.0. If you use Python-2.5 or later you can invoke testing through the Python interpreter from the command line:
python -m pytest [...]
This is equivalent to invoking the command line script py.test [...] directly.
1.3. Usage and Invocations
1.3.2 Getting help on version, option names, environment variables

py.test --version # shows where pytest was imported from py.test --funcargs # show available builtin function arguments py.test -h | --help # show help on command line and config file options
1.3.3 Stopping after the rst (or N) failures

To stop the testing process after the rst (N) failures:
py.test -x py.test -maxfail=2 # stop after first failure # stop after two failures
1.3.4 specifying tests / selecting tests

Several test run options:
py.test test_mod.py py.test somepath py.test -k string # run tests in module # run all tests below path # only run tests whose names contain a string
Import pkg and use its lesystem location to nd and run tests:
py.test --pyargs pkg # run all tests found below directory of pypkg
1.3.5 Modifying Python traceback printing

Examples for modifying traceback printing:
py.test --showlocals # show local variables in tracebacks py.test -l # show local variables (shortcut) py.test py.test py.test py.test --tb=long --tb=native --tb=short --tb=line # # # # the default informative traceback formatting the Python standard library formatting a shorter traceback format only one line per failure
1.3.6 Dropping to PDB (Python Debugger) on failures

Python comes with a builtin Python debugger called PDB. py.test allows to drop into the PDB prompt via a command line option:
py.test --pdb
This will invoke the Python debugger on every failure. Often you might only want to do this for the rst failing test to understand a certain failure situation:
py.test -x --pdb # drop to PDB on first failure, then end test session py.test --pdb --maxfail=3 # drop to PDB for the first three failures
1.3.7 Setting a breakpoint / aka set_trace()

If you want to set a breakpoint and enter the pdb.set_trace() you can use a helper:
import pytest def test_function(): ... pytest.set_trace()
# invoke PDB debugger and tracing
In previous versions you could only enter PDB tracing if you disable capturing on the command line via py.test -s.
1.3.8 creating JUnitXML format les

To create result les which can be read by Hudson or other Continuous integration servers, use this invocation:
py.test --junitxml=path
to create an XML le at path.
1.3.9 creating resultlog format les

To create plain-text machine-readable result les you can issue:
py.test --resultlog=path
and look at the content at the path location. Such les are used e.g. by the PyPy-test web page to show test results over several revisions.
1.3.10 send test report to pocoo pastebin service

Creating a URL for each test failure:
py.test --pastebin=failed
This will submit test run information to a remote Paste service and provide a URL for each failure. You may select tests as usual or add for example -x if you only want to send one particular failure. Creating a URL for a whole test session log:
py.test --pastebin=all
Currently only pasting to the http://paste.pocoo.org service is implemented.
1.3.11 calling pytest from Python code

New in version 2.0. You can invoke py.test from Python code directly:
pytest.main()
this acts as if you would call py.test from the command line. It will not raise SystemExit but return the exitcode instead. You can pass in options and arguments:
pytest.main([x, mytestdir])
1.3. Usage and Invocations
or pass in a string:
pytest.main("-x mytestdir")
You can specify additional plugins to pytest.main:

# content of myinvoke.py import pytest class MyPlugin: def pytest_addoption(self, parser): raise pytest.UsageError("hi from our plugin") pytest.main(plugins=[MyPlugin()])
Running it will exit quickly:

$ python myinvoke.py ERROR: hi from our plugin
1.4 Good Integration Practises

1.4.1 Work with virtual environments
We recommend to use virtualenv environments and use easy_install (or pip) for installing your application dependencies as well as the pytest package itself. This way you will get a much more reproducible environment. A good tool to help you automate test runs against multiple dependency congurations or Python interpreters is tox.
1.4.2 Use tox and Continuous Integration servers

If you frequently relase code to the public you may want to look into tox, the virtualenv test automation tool and its pytest support. The basic idea is to generate a JUnitXML le through the --junitxml=PATH option and have a continuous integration server like Jenkins pick it up and generate reports.
1.4.3 Create a py.test standalone Script

If you are a maintainer or application developer and want others to easily run tests you can generate a completely standalone py.test script:
py.test --genscript=runtests.py
generates a runtests.py script which is a fully functional basic py.test script, running unchanged under Python2 and Python3. You can tell people to download the script and then e.g. run it like this:
python runtests.py
1.4.4 Integrating with distutils / python setup.py test

You can easily integrate test runs into your distutils or setuptools based project. Use the genscript method to generate a standalone py.test script:
py.test --genscript=runtests.py
10
and make this script part of your distribution and then add this to your setup.py le:
from distutils.core import setup, Command # you can also import from setuptools class PyTest(Command): user_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): import sys,subprocess errno = subprocess.call([sys.executable, runtest.py]) raise SystemExit(errno) setup( #..., cmdclass = {test: PyTest}, #..., )
If you now type:

python setup.py test
this will execute your tests using runtest.py. As this is a standalone version of py.test no prior installation whatsoever is required for calling the test command. You can also pass additional arguments to the subprocess-calls such as your test directory or other options.
1.4.5 Conventions for Python test discovery

py.test implements the following standard test discovery: collection starts from the initial command line arguments which may be directories, lenames or test ids. recurse into directories, unless they match norecursedirs test_*.py or *_test.py les, imported by their package name. Test prexed test classes (without an __init__ method) test_ prexed test functions or methods are test items For examples of how to cnd cusotmize your test discovery Changing standard (Python) test discovery. py.test additionally discovers tests using the standard unittest.TestCase subclassing technique.
1.4.6 Choosing a test layout / import rules

py.test supports common test layouts: inlining test directories into your application package, useful if you want to keep (unit) tests and actually tested code close together:
mypkg/ __init__.py appmodule.py ... test/
1.4. Good Integration Practises
11
test_app.py ...
putting tests into an extra directory outside your actual application code, useful if you have many functional tests or want to keep tests separate from actual application code:
mypkg/ __init__.py appmodule.py tests/ test_app.py ...
You can always run your tests by pointing to it:

py.test tests/test_app.py py.test mypkg/test/test_app.py py.test mypkg py.test ... # # # # for for run run external test dirs inlined test dirs tests in all below test directories all tests below current dir
Note: Test modules are imported under their fully qualied name as follows: nd basedir this is the rst upward (towards the root) directory not containing an __init__.py perform sys.path.insert(0, basedir) to make the fully qualied test module path importable. import path.to.test_module where the path is determined by converting path separators into . les. This means you must follow the convention of having directory and le names map to the import names.
1.5 Project examples

Here are some examples of projects using py.test (please send notes via Contact channels): PyPy, Python with a JIT compiler, running over 16000 tests the MoinMoin Wiki Engine tox, virtualenv/Hudson integration tool PIDA framework for integrated development PyPM ActiveStates package manager Fom a uid object mapper for FluidDB applib cross-platform utilities six Python 2 and 3 compatibility utilities pediapress MediaWiki articles mwlib mediawiki parser and utility library The Translate Toolkit for localization and conversion execnet rapid multi-Python deployment pylib cross-platform path, IO, dynamic code library Pacha conguration management in ve minutes
12
bbfreeze create standalone executables from Python scripts pdb++ a fancier version of PDB py-s3fuse Amazon S3 FUSE based lesystem waskr WSGI Stats Middleware guachi global persistent congs for Python modules Circuits lightweight Event Driven Framework pygtk-helpers easy interaction with PyGTK QuantumCore statusmessage and repoze openid plugin pydataportability libraries for managing the open web XIST extensible HTML/XML generator tiddlyweb optionally headless, extensible RESTful datastore fancycompleter for colorful tab-completion Paludis tools for Gentoo Paludis package manager Gerald schema comparison tool abjad Python API for Formalized Score control bu a microscopic build system katcp Telescope communication protocol over Twisted kss plugin timer
1.5.1 Some organisations using py.test

Square Kilometre Array, Cape Town Some Mozilla QA people use pytest to distribute their Selenium tests Tandberg Shootq Stups department of Heinrich Heine University Dsseldorf cellzome Open End, Gothenborg Laboraratory of Bioinformatics, Warsaw merlinux, Germany many more ... (please be so kind to send a note via Contact channels)
1.6 Some Issues and Questions

Note: If you dont nd an answer here, checkout the Contact channels to get help.
1.6. Some Issues and Questions
13
1.6.1 On naming, nosetests, licensing and magic

Why a py.test instead of a pytest command? Some of the reasons are historic, others are practical. py.test used to be part of the py package which provided several developer utilities, all starting with py.<TAB>, thus providing nice TAB-completion. If you install pip install pycmd you get these tools from a separate package. These days the command line tool could be called pytest since many people have gotten used to the old name and there is another tool named pytest we just decided to stick with py.test. How does py.test relate to nose and unittest? py.test and nose share basic philosophy when it comes to running and writing Python tests. In fact, you can run many tests written for nose with py.test. nose was originally created as a clone of py.test when py.test was in the 0.8 release cycle. Note that starting with pytest-2.0 support for running unittest test suites is majorly improved and you should be able to run many Django and Twisted test suites without modication. Whats this magic with py.test? Around 2007 (version 0.8) some people claimed that py.test was using too much magic. Partly this has been xed by removing unused, deprecated or complicated code. It is today probably one of the smallest, most universally runnable and most customizable testing frameworks for Python. However, py.test still uses many metaprogramming techniques and reading its source is thus likely not something for Python beginners. A second magic issue arguably the assert statement debugging feature. When loading test modules py.test rewrites the source code of assert statements. When a rewritten assert statement fails, its error message has more information than the original. py.test also has a second assert debugging technique. When an assert statement that was missed by the rewriter fails, py.test re-interprets the expression to show intermediate values if a test fails. This second technique suffers from caveat that the rewriting does not: If your expression has side effects (better to avoid them anyway!) the intermediate values may not be the same, confusing the reinterpreter and obfuscating the initial error (this is also explained at the command line if it happens). You can turn off all assertion debugging with py.test --assertmode=off.
1.6.2 function arguments, parametrized tests and setup

Is using funcarg- versus xUnit setup a style question? For simple applications and for people experienced with nose or unittest-style test setup using xUnit style setup probably feels natural. For larger test suites, parametrized testing or setup of complex test resources using funcargs may feel more natural. Moreover, funcargs are ideal for writing advanced test support code (like e.g. the monkeypatch, the tmpdir or capture funcargs) because the support code can register setup/teardown functions in a managed class/module/function scope. Why the pytest_funcarg__* name for funcarg factories? We like Convention over Conguration and didnt see much point in allowing a more exible or abstract mechanism. Moreover, is is nice to be able to search for pytest_funcarg__MYARG in a source code and safely nd all factory functions for the MYARG function argument.
14
Can I yield multiple values from a funcarg factory function? There are two conceptual reasons why yielding from a factory function is not possible: Calling factories for obtaining test function arguments is part of setting up and running a test. At that point it is not possible to add new test calls to the test collection anymore. If multiple factories yielded values there would be no natural place to determine the combination policy - in real-world examples some combinations often should not run. Use the pytest_generate_tests hook to solve both issues and implement the parametrization scheme of your choice.
1.6.3 py.test interaction with other packages

Issues with py.test, multiprocess and setuptools? On windows the multiprocess package will instantiate sub processes by pickling and thus implicitly re-import a lot of local modules. Unfortunately, setuptools-0.6.11 does not if __name__==__main__ protect its generated command line script. This leads to innite recursion when running a test that instantiates Processes. A good solution is to install Distribute as a drop-in replacement for setuptools and then re-install pytest. Otherwise you could x the script that is created by setuptools by inserting an if __name__ == __main__. Or you can create a pytest.py script with this content and invoke that with the python version:
import pytest if __name__ == __main__: pytest.main()
1.6. Some Issues and Questions
15
16
CHAPTER
TWO
USAGES AND EXAMPLES

Here is a (growing) list of examples. Contact us if you need more examples or have questions. Also take a look at the comprehensive documentation which contains many example snippets as well. Note: see Installation and Getting Started for basic introductionary examples
2.1 Demo of Python failure reports with py.test

Here is a nice run of several tens of failures and how py.test presents things (unfortunately not showing the nice colors here in the HTML that you get on the terminal - we are working on that):
assertion $ py.test failure_demo.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 39 items failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF ================================= FAILURES ================================= ____________________________ test_generative[0] ____________________________ param1 = 3, param2 = 6 def test_generative(param1, param2): assert param1 * 2 < param2 assert (3 * 2) < 6
> E
failure_demo.py:15: AssertionError _________________________ TestFailing.test_simple __________________________ self = <failure_demo.TestFailing object at 0x14b9890> def test_simple(self): def f(): return 42 def g(): return 43 > E E assert f() == g() assert 42 == 43 + where 42 = <function f at 0x14a5e60>()
17
and
43 = <function g at 0x14bc1b8>()
failure_demo.py:28: AssertionError ____________________ TestFailing.test_simple_multiline _____________________ self = <failure_demo.TestFailing object at 0x14b9b50> def test_simple_multiline(self): otherfunc_multi( 42, 6*9)
>
failure_demo.py:33: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ a = 42, b = 54 def otherfunc_multi(a,b): assert (a == b) assert 42 == 54
> E
failure_demo.py:12: AssertionError ___________________________ TestFailing.test_not ___________________________ self = <failure_demo.TestFailing object at 0x14b9790> def test_not(self): def f(): return 42 assert not f() assert not 42 + where 42 = <function f at 0x14bc398>()
> E E
failure_demo.py:38: AssertionError _________________ TestSpecialisedExplanations.test_eq_text _________________ self = <failure_demo.TestSpecialisedExplanations object at 0x14aa810> def test_eq_text(self): assert spam == eggs assert spam == eggs - spam + eggs
> E E E
failure_demo.py:42: AssertionError _____________ TestSpecialisedExplanations.test_eq_similar_text _____________ self = <failure_demo.TestSpecialisedExplanations object at 0x1576190> def test_eq_similar_text(self): assert foo 1 bar == foo 2 bar assert foo 1 bar == foo 2 bar - foo 1 bar ? ^ + foo 2 bar ? ^
> E E E E E
18
Chapter 2. Usages and Examples
failure_demo.py:45: AssertionError ____________ TestSpecialisedExplanations.test_eq_multiline_text ____________ self = <failure_demo.TestSpecialisedExplanations object at 0x14a7450> def test_eq_multiline_text(self): assert foo\nspam\nbar == foo\neggs\nbar assert foo\nspam\nbar == foo\neggs\nbar foo - spam + eggs bar
> E E E E E
failure_demo.py:48: AssertionError ______________ TestSpecialisedExplanations.test_eq_long_text _______________ self = <failure_demo.TestSpecialisedExplanations object at 0x14b9350> def test_eq_long_text(self): a = 1*100 + a + 2*100 b = 1*100 + b + 2*100 assert a == b assert 111111111111...2222222222222 == 1111111111111...2222222222222 Skipping 90 identical leading characters in diff Skipping 91 identical trailing characters in diff - 1111111111a222222222 ? ^ + 1111111111b222222222 ? ^
> E E E E E E E
failure_demo.py:53: AssertionError _________ TestSpecialisedExplanations.test_eq_long_text_multiline __________ self = <failure_demo.TestSpecialisedExplanations object at 0x15764d0> def test_eq_long_text_multiline(self): a = 1\n*100 + a + 2\n*100 b = 1\n*100 + b + 2\n*100 assert a == b assert 1\n1\n1\n1\n...n2\n2\n2\n2\n == 1\n1\n1\n1\n1...n2\n2\n2\n2\n Skipping 190 identical leading characters in diff Skipping 191 identical trailing characters in diff 1 1 1 1 1 - a2 + b2 2 2 2 2
> E E E E E E E E E E E E E E
failure_demo.py:58: AssertionError _________________ TestSpecialisedExplanations.test_eq_list _________________ self = <failure_demo.TestSpecialisedExplanations object at 0x1576350>
2.1. Demo of Python failure reports with py.test
19
> E E
def test_eq_list(self): assert [0, 1, 2] == [0, 1, 3] assert [0, 1, 2] == [0, 1, 3] At index 2 diff: 2 != 3
failure_demo.py:61: AssertionError ______________ TestSpecialisedExplanations.test_eq_list_long _______________ self = <failure_demo.TestSpecialisedExplanations object at 0x1576f10> def test_eq_list_long(self): a = [0]*100 + [1] + [3]*100 b = [0]*100 + [2] + [3]*100 assert a == b assert [0, 0, 0, 0, 0, 0, ...] == [0, 0, 0, 0, 0, 0, ...] At index 100 diff: 1 != 2
> E E
failure_demo.py:66: AssertionError _________________ TestSpecialisedExplanations.test_eq_dict _________________ self = <failure_demo.TestSpecialisedExplanations object at 0x1576390> def test_eq_dict(self): assert {a: 0, b: 1} == {a: 0, b: 2} assert {a: 0, b: 1} == {a: 0, b: 2} - {a: 0, b: 1} ? ^ + {a: 0, b: 2} ? ^
> E E E E E
failure_demo.py:69: AssertionError _________________ TestSpecialisedExplanations.test_eq_set __________________ self = <failure_demo.TestSpecialisedExplanations object at 0x14bd790> def test_eq_set(self): assert set([0, 10, 11, 12]) == set([0, 20, 21]) assert set([0, 10, 11, 12]) == set([0, 20, 21]) Extra items in the left set: 10 11 12 Extra items in the right set: 20 21
> E E E E E E E E
failure_demo.py:72: AssertionError _____________ TestSpecialisedExplanations.test_eq_longer_list ______________ self = <failure_demo.TestSpecialisedExplanations object at 0x157a7d0> def test_eq_longer_list(self): assert [1,2] == [1,2,3] assert [1, 2] == [1, 2, 3] Right contains more items, first extra item: 3
> E E
failure_demo.py:75: AssertionError _________________ TestSpecialisedExplanations.test_in_list _________________
20
self = <failure_demo.TestSpecialisedExplanations object at 0x157ab50> def test_in_list(self): assert 1 in [0, 2, 3, 4, 5] assert 1 in [0, 2, 3, 4, 5]
> E
failure_demo.py:78: AssertionError __________ TestSpecialisedExplanations.test_not_in_text_multiline __________ self = <failure_demo.TestSpecialisedExplanations object at 0x157a090> def test_not_in_text_multiline(self): text = some multiline\ntext\nwhich\nincludes foo\nand a\ntail assert foo not in text assert foo not in some multiline\ntext\nw...ncludes foo\nand a\ntail foo is contained here: some multiline text which includes foo ? +++ and a tail
> E E E E E E E E E
failure_demo.py:82: AssertionError ___________ TestSpecialisedExplanations.test_not_in_text_single ____________ self = <failure_demo.TestSpecialisedExplanations object at 0x14aaa50> def test_not_in_text_single(self): text = single foo line assert foo not in text assert foo not in single foo line foo is contained here: single foo line ? +++
> E E E E
failure_demo.py:86: AssertionError _________ TestSpecialisedExplanations.test_not_in_text_single_long _________ self = <failure_demo.TestSpecialisedExplanations object at 0x157ab90> def test_not_in_text_single_long(self): text = head * 50 + foo + tail * 20 assert foo not in text assert foo not in head head head head hea...ail tail tail tail tail foo is contained here: head head foo tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail ? +++
> E E E E
failure_demo.py:90: AssertionError ______ TestSpecialisedExplanations.test_not_in_text_single_long_term _______ self = <failure_demo.TestSpecialisedExplanations object at 0x1576ed0> def test_not_in_text_single_long_term(self): text = head * 50 + f*70 + tail * 20 assert f*70 not in text
>
21
E E E E
assert fffffffffff...ffffffffffff not in head head he...l tail tail ffffffffffffffffff...fffffffffffffffffff is contained here: head head fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffftail tail ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
failure_demo.py:94: AssertionError ______________________________ test_attribute ______________________________ def test_attribute(): class Foo(object): b = 1 i = Foo() assert i.b == 2 assert 1 == 2 + where 1 = <failure_demo.Foo object at 0x157a910>.b
> E E
failure_demo.py:101: AssertionError _________________________ test_attribute_instance __________________________ def test_attribute_instance(): class Foo(object): b = 1 assert Foo().b == 2 assert 1 == 2 + where 1 = <failure_demo.Foo object at 0x1584610>.b + where <failure_demo.Foo object at 0x1584610> = <class failure_demo.Foo>()
> E E E
failure_demo.py:107: AssertionError __________________________ test_attribute_failure __________________________ def test_attribute_failure(): class Foo(object): def _get_b(self): raise Exception(Failed to get attrib) b = property(_get_b) i = Foo() assert i.b == 2
>
failure_demo.py:116: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <failure_demo.Foo object at 0x157a3d0> def _get_b(self): raise Exception(Failed to get attrib) Exception: Failed to get attrib
> E
failure_demo.py:113: Exception _________________________ test_attribute_multiple __________________________ def test_attribute_multiple(): class Foo(object): b = 1 class Bar(object): b = 2 assert Foo().b == Bar().b assert 1 == 2 + where 1 = <failure_demo.Foo object at 0x157a1d0>.b
> E E
22
E E E
+ + +
where <failure_demo.Foo object at 0x157a1d0> = <class failure_demo.Foo>() and 2 = <failure_demo.Bar object at 0x157a9d0>.b where <failure_demo.Bar object at 0x157a9d0> = <class failure_demo.Bar>()
failure_demo.py:124: AssertionError __________________________ TestRaises.test_raises __________________________ self = <failure_demo.TestRaises instance at 0x157d7e8> def test_raises(self): s = qwe raises(TypeError, "int(s)")
>
failure_demo.py:133: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > E int(s) ValueError: invalid literal for int() with base 10: qwe
<0-codegen /home/hpk/p/pytest/_pytest/python.py:831>:1: ValueError ______________________ TestRaises.test_raises_doesnt _______________________ self = <failure_demo.TestRaises instance at 0x158ae60> def test_raises_doesnt(self): raises(IOError, "int(3)") Failed: DID NOT RAISE
> E
failure_demo.py:136: Failed __________________________ TestRaises.test_raise ___________________________ self = <failure_demo.TestRaises instance at 0x158bb90> def test_raise(self): raise ValueError("demo error") ValueError: demo error
> E
failure_demo.py:139: ValueError ________________________ TestRaises.test_tupleerror ________________________ self = <failure_demo.TestRaises instance at 0x157cd40> def test_tupleerror(self): a,b = [1] ValueError: need more than 1 value to unpack
> E
failure_demo.py:142: ValueError ______ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ______ self = <failure_demo.TestRaises instance at 0x157d488> def test_reinterpret_fails_with_print_for_the_fun_of_it(self): l = [1,2,3] print ("l is %r" % l) a,b = l.pop() TypeError: int object is not iterable
> E
failure_demo.py:147: TypeError
23
----------------------------- Captured stdout -----------------------------l is [1, 2, 3] ________________________ TestRaises.test_some_error ________________________ self = <failure_demo.TestRaises instance at 0x158a7e8> def test_some_error(self): if namenotexi: NameError: global name namenotexi is not defined
> E
failure_demo.py:150: NameError ____________________ test_dynamic_compile_shows_nicely _____________________ def test_dynamic_compile_shows_nicely(): src = def foo():\n assert 1 == 0\n name = abc-123 module = py.std.imp.new_module(name) code = py.code.compile(src, name, exec) py.builtin.exec_(code, module.__dict__) py.std.sys.modules[name] = module module.foo()
>
failure_demo.py:165: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def foo(): assert 1 == 0 assert 1 == 0
> E
<2-codegen abc-123 /home/hpk/p/pytest/doc/example/assertion/failure_demo.py:162>:2: AssertionError ____________________ TestMoreErrors.test_complex_error _____________________ self = <failure_demo.TestMoreErrors instance at 0x158f8c0> def test_complex_error(self): def f(): return 44 def g(): return 43 somefunc(f(), g())
>
failure_demo.py:175: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x = 44, y = 43 def somefunc(x,y): otherfunc(x,y)
>
failure_demo.py:8: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ a = 44, b = 43 def otherfunc(a,b): assert a==b assert 44 == 43
> E
24
failure_demo.py:5: AssertionError ___________________ TestMoreErrors.test_z1_unpack_error ____________________ self = <failure_demo.TestMoreErrors instance at 0x158c998> def test_z1_unpack_error(self): l = [] a,b = l ValueError: need more than 0 values to unpack
> E
failure_demo.py:179: ValueError ____________________ TestMoreErrors.test_z2_type_error _____________________ self = <failure_demo.TestMoreErrors instance at 0x15854d0> def test_z2_type_error(self): l = 3 a,b = l TypeError: int object is not iterable
> E
failure_demo.py:183: TypeError ______________________ TestMoreErrors.test_startswith ______________________ self = <failure_demo.TestMoreErrors instance at 0x14b65a8> def test_startswith(self): s = "123" g = "456" assert s.startswith(g) assert False + where False = <built-in method startswith of str object at 0x14902a0>(456) + where <built-in method startswith of str object at 0x14902a0> = 123.startswith
> E E E
failure_demo.py:188: AssertionError __________________ TestMoreErrors.test_startswith_nested ___________________ self = <failure_demo.TestMoreErrors instance at 0x158d518> def test_startswith_nested(self): def f(): return "123" def g(): return "456" assert f().startswith(g()) assert False + where False = <built-in method startswith of str object at 0x14902a0>(456) + where <built-in method startswith of str object at 0x14902a0> = 123.startswith + where 123 = <function f at 0x15806e0>() + and 456 = <function g at 0x1580aa0>()
> E E E E E
failure_demo.py:195: AssertionError _____________________ TestMoreErrors.test_global_func ______________________ self = <failure_demo.TestMoreErrors instance at 0x1593440> def test_global_func(self): assert isinstance(globf(42), float) assert False
> E
25
E E
+ +
where False = isinstance(43, float) where 43 = globf(42)
failure_demo.py:198: AssertionError _______________________ TestMoreErrors.test_instance _______________________ self = <failure_demo.TestMoreErrors instance at 0x15952d8> def test_instance(self): self.x = 6*7 assert self.x != 42 assert 42 != 42 + where 42 = 42 + where 42 = <failure_demo.TestMoreErrors instance at 0x15952d8>.x
> E E E
failure_demo.py:202: AssertionError _______________________ TestMoreErrors.test_compare ________________________ self = <failure_demo.TestMoreErrors instance at 0x1593758> def test_compare(self): assert globf(10) < 5 assert 11 < 5 + where 11 = globf(10)
> E E
failure_demo.py:205: AssertionError _____________________ TestMoreErrors.test_try_finally ______________________ self = <failure_demo.TestMoreErrors instance at 0x157cd88> def test_try_finally(self): x = 1 try: assert x == 0 assert 1 == 0
> E
failure_demo.py:210: AssertionError ======================== 39 failed in 0.23 seconds =========================
2.2 basic patterns and examples

2.2.1 pass different values to a test function, depending on command line options
Suppose we want to write a test that depends on a command line option. Here is a basic pattern how to achieve this:
# content of test_sample.py def test_answer(cmdopt): if cmdopt == "type1": print ("first") elif cmdopt == "type2": print ("second") assert 0 # to see what was printed
For this to work we need to add a command line option and provide the cmdopt through a function argument factory:
26
# content of conftest.py def pytest_addoption(parser): parser.addoption("--cmdopt", action="store", default="type1", help="my option: type1 or type2") def pytest_funcarg__cmdopt(request): return request.config.option.cmdopt
Lets run this without supplying our new command line option:
$ py.test -q test_sample.py collecting ... collected 1 items F ================================= FAILURES ================================= _______________________________ test_answer ________________________________ cmdopt = type1 def test_answer(cmdopt): if cmdopt == "type1": print ("first") elif cmdopt == "type2": print ("second") assert 0 # to see what was printed assert 0
> E
test_sample.py:6: AssertionError ----------------------------- Captured stdout -----------------------------first 1 failed in 0.03 seconds
And now with supplying a command line option:

$ py.test -q --cmdopt=type2 collecting ... collected 1 items F ================================= FAILURES ================================= _______________________________ test_answer ________________________________ cmdopt = type2 def test_answer(cmdopt): if cmdopt == "type1": print ("first") elif cmdopt == "type2": print ("second") assert 0 # to see what was printed assert 0
> E
test_sample.py:6: AssertionError ----------------------------- Captured stdout -----------------------------second 1 failed in 0.02 seconds
Ok, this completes the basic pattern. However, one often rather wants to process command line options outside of the test and rather pass in different or more complex objects. See the next example or refer to mysetup pattern: application specic test xtures for more information on real-life examples.
2.2. basic patterns and examples
27
2.2.2 dynamically adding command line options

Through addopts you can statically add command line options for your project. You can also dynamically modify the command line arguments before they get processed:
# content of conftest.py import sys def pytest_cmdline_preparse(args): if xdist in sys.modules: # pytest-xdist plugin import multiprocessing num = max(multiprocessing.cpu_count() / 2, 1) args[:] = ["-n", str(num)] + args
If you have the xdist plugin installed you will now always perform test runs using a number of subprocesses close to your CPU. Running in an empty directory with the above conftest.py:
$ py.test =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 gw0 I / gw1 I / gw2 I / gw3 I gw0 [0] / gw1 [0] / gw2 [0] / gw3 [0] scheduling tests via LoadScheduling ============================= in 0.52 seconds =============================
2.2.3 control skipping of tests according to command line option

Here is a conftest.py le adding a --runslow command line option to control skipping of slow marked tests:
# content of conftest.py import pytest def pytest_addoption(parser): parser.addoption("--runslow", action="store_true", help="run slow tests") def pytest_runtest_setup(item): if slow in item.keywords and not item.config.getvalue("runslow"): pytest.skip("need --runslow option to run")
We can now write a test module like this:

# content of test_module.py import pytest slow = pytest.mark.slow def test_func_fast(): pass @slow def test_func_slow(): pass
and when running it will see a skipped slow test:
28
$ py.test -rs # "-rs" means report details on the little s =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items test_module.py .s ========================= short test summary info ========================== SKIP [1] /tmp/doc-exec-42/conftest.py:9: need --runslow option to run =================== 1 passed, 1 skipped in 0.01 seconds ====================
Or run it including the slow marked test:

$ py.test --runslow =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items test_module.py .. ========================= 2 passed in 0.01 seconds =========================
2.2.4 writing well integrated assertion helpers

If you have a test helper function called from a test you can use the pytest.fail marker to fail a test with a certain message. The test support function will not show up in the traceback if you set the __tracebackhide__ option somewhere in the helper function. Example:
# content of test_checkconfig.py import pytest def checkconfig(x): __tracebackhide__ = True if not hasattr(x, "config"): pytest.fail("not configured: %s" %(x,)) def test_something(): checkconfig(42)
The __tracebackhide__ setting inuences py.test showing of tracebacks: the checkconfig function will not be shown unless the --fulltrace command line option is specied. Lets run our little function:
$ py.test -q test_checkconfig.py collecting ... collected 1 items F ================================= FAILURES ================================= ______________________________ test_something ______________________________ def test_something(): checkconfig(42) Failed: not configured: 42
> E
test_checkconfig.py:8: Failed 1 failed in 0.02 seconds
2.2. basic patterns and examples
29
2.2.5 Detect if running from within a py.test run

Usually it is a bad idea to make application code behave differently if called from a test. But if you absolutely must nd out if your application code is running from a test you can do something like this:
# content of conftest.py def pytest_configure(config): import sys sys._called_from_test = True def pytest_unconfigure(config): del sys._called_from_test
and then check for the sys._called_from_test ag:

if hasattr(sys, _called_from_test): # called from within a test run else: # called "normally"
accordingly in your application. Its also a good idea to rather use your own application module rather than sys for handling ag.
2.2.6 Adding info to test report header

Its easy to present extra information in a py.test run:
# content of conftest.py def pytest_report_header(config): return "project deps: mylib-1.1"
which will add the string to the test header accordingly:

$ py.test =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 project deps: mylib-1.1 collecting ... collected 0 items ============================= in 0.00 seconds =============================
You can also return a list of strings which will be considered as several lines of information. You can of course also make the amount of reporting information on e.g. the value of config.option.verbose so that you present more information appropriately:
# content of conftest.py def pytest_report_header(config): if config.option.verbose > 0: return ["info1: did you know that ...", "did you?"]
which will add info only when run with v:

$ py.test -v =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 -- /home/hpk/venv/0/bin/python
30
info1: did you know that ... did you? collecting ... collected 0 items ============================= in 0.00 seconds =============================
and nothing when run plainly:

$ py.test =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 0 items ============================= in 0.00 seconds =============================
2.3 mysetup pattern: application specic test xtures

Here is a basic useful step-by-step example for managing and interacting with application specic test setup. The goal is to have one place where we have the glue and test support code for bootstrapping and conguring application objects and allow test modules and test functions to stay ignorant of involved details.
2.3.1 step1: implementing the test/app-specic mysetup pattern

Lets write a simple test function using a mysetup funcarg:
# content of test_sample.py def test_answer(mysetup): app = mysetup.myapp() answer = app.question() assert answer == 42
To run this test py.test needs to nd and call a factory to obtain the required mysetup function argument. To make an according factory ndable we write down a specically named factory method in a local plugin
# content of conftest.py from myapp import MyApp def pytest_funcarg__mysetup(request): # "mysetup" factory function return MySetup() class MySetup: # instances of this are seen by test functions def myapp(self): return MyApp()
To run the example we stub out a a simple MyApp application object:

# content of myapp.py class MyApp: def question(self): return 6 * 9
You can now run the test:

$ py.test test_sample.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3
2.3. mysetup pattern: application specic test xtures
31
collecting ... collected 1 items test_sample.py F ================================= FAILURES ================================= _______________________________ test_answer ________________________________ mysetup = <conftest.MySetup instance at 0x2c1b128> def test_answer(mysetup): app = mysetup.myapp() answer = app.question() assert answer == 42 assert 54 == 42
> E
test_sample.py:4: AssertionError ========================= 1 failed in 0.02 seconds =========================
This means that our mysetup object was successfully instantiated and mysetup.app() returned an initialized MyApp instance. We can ask it about the question and if you are confused as to what the concrete question or answers actually mean, please see here.
2.3.2 step 2: checking a command line option and skipping tests

To add a command line option we update the conftest.py of the previous example to add a command line option and to offer a new mysetup method:
# content of ./conftest.py import pytest from myapp import MyApp def pytest_funcarg__mysetup(request): # "mysetup" factory function return MySetup(request) def pytest_addoption(parser): parser.addoption("--ssh", action="store", default=None, help="specify ssh host to run tests with")
class MySetup: def __init__(self, request): self.config = request.config def myapp(self): return MyApp() def getsshconnection(self): host = self.config.option.ssh if host is None: pytest.skip("specify ssh host with --ssh") return execnet.SshGateway(host)
Now any test function can use the mysetup.getsshconnection() method like this:
# content of test_ssh.py class TestClass: def test_function(self, mysetup):
32
conn = mysetup.getsshconnection() # work with conn
Running it yields:
$ py.test test_ssh.py -rs =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 1 items test_ssh.py s ========================= short test summary info ========================== SKIP [1] /tmp/doc-exec-37/conftest.py:22: specify ssh host with --ssh ======================== 1 skipped in 0.01 seconds =========================
If you specify a command line option like py.test --ssh=python.org the test will execute as expected. Note that neither the TestClass nor the test_function need to know anything about how to setup the test state. It is handled separately in your test setup glue code in the conftest.py le. It is easy to extend the mysetup object for further needs in the test code - and for use by any other test functions in the les and directories below the conftest.py le.
2.4 parametrizing tests

py.test allows to easily implement your own custom parametrization scheme for tests. Here we provide some examples for inspiration and re-use.
2.4.1 generating parameters combinations, depending on command line

Lets say we want to execute a test with different parameters and the parameter range shall be determined by a command line argument. Lets rst write a simple computation test:
# content of test_compute.py def test_compute(param1): assert param1 < 4
Now we add a test conguration like this:

# content of conftest.py def pytest_addoption(parser): parser.addoption("--all", action="store_true", help="run all combinations") def pytest_generate_tests(metafunc): if param1 in metafunc.funcargnames: if metafunc.config.option.all: end = 5 else: end = 2 for i in range(end): metafunc.addcall(funcargs={param1: i})
This means that we only run 2 tests if we do not pass --all:
2.4. parametrizing tests
33
$ py.test -q test_compute.py collecting ... collected 2 items .. 2 passed in 0.01 seconds
We run only two computations, so we see two dots. lets run the full monty:
$ py.test -q --all collecting ... collected 5 items ....F ================================= FAILURES ================================= _____________________________ test_compute[4] ______________________________ param1 = 4 def test_compute(param1): assert param1 < 4 assert 4 < 4
> E
test_compute.py:3: AssertionError 1 failed, 4 passed in 0.03 seconds
As expected when running the full range of param1 values well get an error on the last one.
2.4.2 Deferring the setup of parametrizing resources

The parametrization of test functions happens at collection time. It is often a good idea to setup possibly expensive resources only when the actual test is run. Here is a simple example how you can achieve that:
# content of test_backends.py import pytest def test_db_initialized(db): # a dummy test if db.__class__.__name__ == "DB2": pytest.fail("deliberately failing for demo purposes")
Now we add a test conguration that takes care to generate two invocations of the test_db_initialized function and furthermore a factory that creates a database object when each test is actually run:
# content of conftest.py def pytest_generate_tests(metafunc): if db in metafunc.funcargnames: metafunc.addcall(param="d1") metafunc.addcall(param="d2") class DB1: "one database object" class DB2: "alternative database object" def pytest_funcarg__db(request): if request.param == "d1": return DB1() elif request.param == "d2": return DB2()
34
else: raise ValueError("invalid internal test config")
Lets rst see how it looks like at collection time:

$ py.test test_backends.py --collectonly =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items <Module test_backends.py> <Function test_db_initialized[0]> <Function test_db_initialized[1]> ============================= in 0.01 seconds =============================
And then when we run the test:

$ py.test -q test_backends.py collecting ... collected 2 items .F ================================= FAILURES ================================= __________________________ test_db_initialized[1] __________________________ db = <conftest.DB2 instance at 0x2bf7bd8> def test_db_initialized(db): # a dummy test if db.__class__.__name__ == "DB2": pytest.fail("deliberately failing for demo purposes") Failed: deliberately failing for demo purposes
> E
test_backends.py:6: Failed 1 failed, 1 passed in 0.03 seconds
Now you see that one invocation of the test passes and another fails, as it to be expected.
2.4.3 Parametrizing test methods through per-class conguration

Here is an example pytest_generate_function function implementing a parametrization scheme similar to Michael Foords unittest parameterizer in a lot less code:
# content of ./test_parametrize.py import pytest def pytest_generate_tests(metafunc): # called once per each test function for funcargs in metafunc.cls.params[metafunc.function.__name__]: # schedule a new test function run with applied **funcargs metafunc.addcall(funcargs=funcargs) class TestClass: # a map specifying multiple argument sets for a test method params = { test_equals: [dict(a=1, b=2), dict(a=3, b=3), ], test_zerodivision: [dict(a=1, b=0), dict(a=3, b=2)], } def test_equals(self, a, b):
35
assert a == b def test_zerodivision(self, a, b): pytest.raises(ZeroDivisionError, "a/b")
Running it means we are two tests for each test functions, using the respective settings:
$ py.test -q collecting ... collected 6 items .FF..F ================================= FAILURES ================================= __________________________ test_db_initialized[1] __________________________ db = <conftest.DB2 instance at 0x19bcb90> def test_db_initialized(db): # a dummy test if db.__class__.__name__ == "DB2": pytest.fail("deliberately failing for demo purposes") Failed: deliberately failing for demo purposes
> E
test_backends.py:6: Failed _________________________ TestClass.test_equals[0] _________________________ self = <test_parametrize.TestClass instance at 0x19ca8c0>, a = 1, b = 2 def test_equals(self, a, b): assert a == b assert 1 == 2
> E
test_parametrize.py:17: AssertionError ______________________ TestClass.test_zerodivision[1] ______________________ self = <test_parametrize.TestClass instance at 0x19cd4d0>, a = 3, b = 2 def test_zerodivision(self, a, b): pytest.raises(ZeroDivisionError, "a/b") Failed: DID NOT RAISE
> E
test_parametrize.py:20: Failed 3 failed, 3 passed in 0.05 seconds
2.4.4 Parametrizing test methods through a decorator

Modifying the previous example we can also allow decorators for parametrizing test methods:
# content of test_parametrize2.py import pytest # test support code def params(funcarglist): def wrapper(function): function.funcarglist = funcarglist return function return wrapper
36
def pytest_generate_tests(metafunc): for funcargs in getattr(metafunc.function, funcarglist, ()): metafunc.addcall(funcargs=funcargs) # actual test code class TestClass: @params([dict(a=1, b=2), dict(a=3, b=3), ]) def test_equals(self, a, b): assert a == b @params([dict(a=1, b=0), dict(a=3, b=2)]) def test_zerodivision(self, a, b): pytest.raises(ZeroDivisionError, "a/b")
Running it gives similar results as before:

$ py.test -q test_parametrize2.py collecting ... collected 4 items F..F ================================= FAILURES ================================= _________________________ TestClass.test_equals[0] _________________________ self = <test_parametrize2.TestClass instance at 0x1cf1170>, a = 1, b = 2 @params([dict(a=1, b=2), dict(a=3, b=3), ]) def test_equals(self, a, b): assert a == b assert 1 == 2
> E
test_parametrize2.py:19: AssertionError ______________________ TestClass.test_zerodivision[1] ______________________ self = <test_parametrize2.TestClass instance at 0x1d02170>, a = 3, b = 2 @params([dict(a=1, b=0), dict(a=3, b=2)]) def test_zerodivision(self, a, b): pytest.raises(ZeroDivisionError, "a/b") Failed: DID NOT RAISE
> E
test_parametrize2.py:23: Failed 2 failed, 2 passed in 0.03 seconds
2.4.5 checking serialization between Python interpreters

Here is a stripped down real-life example of using parametrized testing for testing serialization between different interpreters. We dene a test_basic_objects function which is to be run with different sets of arguments for its three arguments:
* python1: first python interpreter * python2: second python interpreter * obj: object to be dumped from first interpreter and loaded into second interpreter """ module containing a parametrized tests testing cross-python serialization via the pickle module. """ import py
37
pythonlist = [python2.4, python2.5, python2.6, python2.7, python2.8] def pytest_generate_tests(metafunc): if python1 in metafunc.funcargnames: assert python2 in metafunc.funcargnames for obj in metafunc.function.multiarg.kwargs[obj]: for py1 in pythonlist: for py2 in pythonlist: metafunc.addcall(id="%s-%s-%s" % (py1, py2, obj), param=(py1, py2, obj)) @py.test.mark.multiarg(obj=[42, {}, {1:3},]) def test_basic_objects(python1, python2, obj): python1.dumps(obj) python2.load_and_is_true("obj == %s" % obj) def pytest_funcarg__python1(request): tmpdir = request.getfuncargvalue("tmpdir") picklefile = tmpdir.join("data.pickle") return Python(request.param[0], picklefile) def pytest_funcarg__python2(request): python1 = request.getfuncargvalue("python1") return Python(request.param[1], python1.picklefile) def pytest_funcarg__obj(request): return request.param[2] class Python: def __init__(self, version, picklefile): self.pythonpath = py.path.local.sysfind(version) if not self.pythonpath: py.test.skip("%r not found" %(version,)) self.picklefile = picklefile def dumps(self, obj): dumpfile = self.picklefile.dirpath("dump.py") dumpfile.write(py.code.Source(""" import pickle f = open(%r, wb) s = pickle.dump(%r, f) f.close() """ % (str(self.picklefile), obj))) py.process.cmdexec("%s %s" %(self.pythonpath, dumpfile)) def load_and_is_true(self, expression): loadfile = self.picklefile.dirpath("load.py") loadfile.write(py.code.Source(""" import pickle f = open(%r, rb) obj = pickle.load(f) f.close() res = eval(%r) if not res: raise SystemExit(1) """ % (str(self.picklefile), expression))) print (loadfile) py.process.cmdexec("%s %s" %(self.pythonpath, loadfile))
38
Running it (with Python-2.4 through to Python2.7 installed):

. $ py.test -q multipython.py collecting ... collected 75 items ....s....s....s....ssssss....s....s....s....ssssss....s....s....s....ssssss 48 passed, 27 skipped in 2.04 seconds
2.5 Changing standard (Python) test discovery

2.5.1 changing directory recursion
You can set the norecursedirs option in an ini-le, for example your setup.cfg in the project root directory:
# content of setup.cfg [pytest] norecursedirs = .svn _build tmp*
This would tell py.test to not recurse into typical subversion or sphinx-build directories or into any tmp prexed directory.
2.5.2 change naming conventions

You can congure different naming conventions by setting the python_files, python_classes and python_functions conguration options. Example:
# content of setup.cfg # can also be defined in in tox.ini or pytest.ini file [pytest] python_files=check_*.py python_classes=Check python_functions=check
This would make py.test look for check_ prexes in Python lenames, Check prexes in classes and check prexes in functions and classes. For example, if we have:
# content of check_myapp.py class CheckMyApp: def check_simple(self): pass def check_complex(self): pass
then the test collection looks like this:

$ py.test --collectonly =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items <Module check_myapp.py> <Class CheckMyApp> <Instance ()> <Function check_simple> <Function check_complex> ============================= in 0.01 seconds =============================
2.5. Changing standard (Python) test discovery
39
2.5.3 interpret cmdline arguments as Python packages

You can use the --pyargs option to make py.test try interpreting arguments as python package names, deriving their le system path and then running the test. For example if you have unittest2 installed you can type:
py.test --pyargs unittest2.test.test_skipping -q
which would run the respective test module. Like with other options, through an ini-le and the addopts option you can make this change more permanently:
# content of pytest.ini [pytest] addopts = --pyargs
Now a simple invocation of py.test NAME will check if NAME exists as an importable package/module and otherwise treat it as a lesystem path.
2.5.4 nding out what is collected

You can always peek at the collection tree without running tests like this:
. $ py.test --collectonly pythoncollection.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 3 items <Module pythoncollection.py> <Function test_function> <Class TestClass> <Instance ()> <Function test_method> <Function test_anothermethod> ============================= in 0.01 seconds =============================
2.6 Working with non-python tests

2.6.1 a basic example for specifying tests in Yaml les
Here is an example conftest.py (extracted from Ali Afshnars special purpose pytest-yamlwsgi plugin). This conftest.py will collect test*.yml les and will execute the yaml-formatted content as custom tests:
# content of conftest.py import pytest def pytest_collect_file(path, parent): if path.ext == ".yml" and path.basename.startswith("test"): return YamlFile(path, parent) class YamlFile(pytest.File): def collect(self): import yaml # we need a yaml parser, e.g. PyYAML raw = yaml.load(self.fspath.open()) for name, spec in raw.items(): yield YamlItem(name, self, spec)
40
class YamlItem(pytest.Item): def __init__(self, name, parent, spec): super(YamlItem, self).__init__(name, parent) self.spec = spec def runtest(self): for name, value in self.spec.items(): # some custom test execution (dumb example follows) if name != value: raise YamlException(self, name, value) def repr_failure(self, excinfo): """ called when self.runtest() raises an exception. """ if isinstance(excinfo.value, YamlException): return "\n".join([ "usecase execution failed", " spec failed: %r: %r" % excinfo.value.args[1:3], " no further details known at this point." ]) def reportinfo(self): return self.fspath, 0, "usecase: %s" % self.name class YamlException(Exception): """ custom exception for error reporting. """
You can create a simple example le:

# test_simple.yml ok: sub1: sub1 hello: world: world some: other
and if you installed PyYAML or a compatible YAML-parser you can now execute the test specication:
nonpython $ py.test test_simple.yml =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items test_simple.yml .F ================================= FAILURES ================================= ______________________________ usecase: hello ______________________________ usecase execution failed spec failed: some: other no further details known at this point. ==================== 1 failed, 1 passed in 0.24 seconds ====================
You get one dot for the passing sub1: sub1 check and one failure. Obviously in the above conftest.py youll want to implement a more interesting interpretation of the yaml-values. You can easily write your own domain specic testing language this way. Note: repr_failure(excinfo) is called for representing test failures. If you create custom collection nodes you can return an error representation string of your choice. It will be reported as a (red) string. 2.6. Working with non-python tests 41
reportinfo() is used for representing the test location and is also consulted for reporting in verbose mode:
nonpython $ py.test -v =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 -- /home/hpk/venv/0/bin/python collecting ... collected 2 items test_simple.yml:1: usecase: ok PASSED test_simple.yml:1: usecase: hello FAILED ================================= FAILURES ================================= ______________________________ usecase: hello ______________________________ usecase execution failed spec failed: some: other no further details known at this point. ==================== 1 failed, 1 passed in 0.07 seconds ====================
While developing your custom test collection and execution its also interesting to just look at the collection tree:
nonpython $ py.test --collectonly =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items <YamlFile test_simple.yml> <YamlItem ok> <YamlItem hello> ============================= in 0.07 seconds =============================
42
CHAPTER
THREE
PY.TEST REFERENCE DOCUMENTATION

3.1 pytest builtin helpers
3.1.1 builtin pytest.* functions and helping objects
You can always use an interactive Python prompt and type:
import pytest help(pytest)
to get an overview on the globally available helpers. unit and functional testing with Python. pytest.main(args=None, plugins=None) returned exit code integer, after an in-process testing run with the given command line arguments, preloading an optional list of passed in plugin objects. pytest.fail(msg=, pytrace=True) explicitely fail an currently-executing test with the given Message. if @pytrace is not True the msg represents the full failure information. pytest.skip(msg=) skip an executing test with the given message. Note: its usually better to use the py.test.mark.skipif marker to declare a test to be skipped under certain conditions like mismatching platforms or dependencies. See the pytest_skipping plugin for details. pytest.exit(msg) exit testing process as if KeyboardInterrupt was triggered. pytest.importorskip(modname, minversion=None) return imported module if it has a higher __version__ than the optionally specied minversion - otherwise call py.test.skip() with a message detailing the mismatch. pytest.raises(ExpectedException, *args, **kwargs) assert that a code block/function call raises @ExpectedException and raise a failure exception otherwise. If using Python 2.5 or above, you may use this function as a context manager:
>>> with raises(ZeroDivisionError): ... 1/0
Or you can specify a callable by passing a to-be-called lambda:
43
>>> raises(ZeroDivisionError, lambda: 1/0) <ExceptionInfo ...>
or you can specify an arbitrary callable with arguments:

>>> def f(x): return 1/x ... >>> raises(ZeroDivisionError, f, 0) <ExceptionInfo ...> >>> raises(ZeroDivisionError, f, x=0) <ExceptionInfo ...>
A third possibility is to use a string which which will be executed:

>>> raises(ZeroDivisionError, "f(0)") <ExceptionInfo ...>
pytest.xfail(reason=) xfail an executing test or setup functions with the given reason. pytest.deprecated_call(func, *args, **kwargs) assert that calling func(*args, **kwargs) triggers a DeprecationWarning.
3.1.2 builtin function arguments

You can ask for available builtin or project-custom function arguments by typing:
$ py.test --funcargs pytestconfig the pytest config object with access to command line opts. capsys enables capturing of writes to sys.stdout/sys.stderr and makes captured output available via capsys.readouterr() method calls which return a (out, err) tuple. capfd enables capturing of writes to file descriptors 1 and 2 and makes captured output available via capsys.readouterr() method calls which return a (out, err) tuple. tmpdir return a temporary directory path object which is unique to each test function invocation, created as a sub directory of the base temporary directory. The returned object is a py.path.local_ path object. monkeypatch The returned monkeypatch funcarg provides these helper methods to modify objects, dictionaries or os.environ:: monkeypatch.setattr(obj, name, value, raising=True) monkeypatch.delattr(obj, name, raising=True) monkeypatch.setitem(mapping, name, value) monkeypatch.delitem(obj, name, raising=True) monkeypatch.setenv(name, value, prepend=False) monkeypatch.delenv(name, value, raising=True) monkeypatch.syspath_prepend(path)
44
Chapter 3. py.test reference documentation
All modifications will be undone after the requesting test function has finished. The raising parameter determines if a KeyError or AttributeError will be raised if the set/deletion operation has no target. recwarn Return a WarningsRecorder instance that provides these methods: * pop(category=None): return last warning matching the category. * clear(): clear list of warnings See http://docs.python.org/library/warnings.html for information on warning categories.
3.2 basic test conguration

3.2.1 Command line options and conguration le settings
You can get help on command line options and values in INI-style congurations les by using the general help option:
py.test -h # prints options _and_ config file settings
This will display command line and conguration le settings which were registered by installed plugins.
3.2.2 How test conguration is read from conguration INI-les

py.test searches for the rst matching ini-style conguration le in the directories of command line argument and the directories above. It looks for le basenames in this order:
pytest.ini tox.ini setup.cfg
Searching stops when the rst [pytest] section is found. There is no merging of conguration values from multiple les. Example:
py.test path/to/testdir
will look in the following dirs for a cong le:

path/to/testdir/pytest.ini path/to/testdir/tox.ini path/to/testdir/setup.cfg path/to/pytest.ini path/to/tox.ini path/to/setup.cfg ... # up until root of filesystem
If argument is provided to a py.test run, the current working directory is used to start the search.
3.2.3 How to change command line options defaults

It can be tedious to type the same series of command line options every time you use py.test . For example, if you always want to see detailed info on skipped and xfailed tests, as well as have terser dot progress output, you can 3.2. basic test conguration 45
write it into a conguration le:

# content of pytest.ini # (or tox.ini or setup.cfg) [pytest] addopts = -rsxX -q
From now on, running py.test will add the specied options.
3.2.4 builtin conguration le options

minversion species a minimal pytest version required for running tests. minversion = 2.1 # will fail if we run with pytest-2.0 addopts add the specied OPTS to the set of command line arguments as if they had been specied by the user. Example: if you have this ini le content:
[pytest] addopts = --maxfail=2 -rf # exit after 2 failures, report fail info
issuing py.test test_hello.py actually means:

py.test --maxfail=2 -rf test_hello.py
Default is to add no options. norecursedirs Set the directory basename patterns to avoid when recursing for test discovery. The individual (fnmatch-style) patterns are applied to the basename of a directory to decide if to recurse into it. Pattern matching characters:
* ? [seq] [!seq] matches matches matches matches everything any single character any character in seq any char not in seq
Default patterns are .* _* CVS {args}. Setting a norecurse replaces the default. Here is an example of how to avoid certain directories:
# content of setup.cfg [pytest] norecursedirs = .svn _build tmp*
This would tell py.test to not look into typical subversion or sphinx-build directories or into any tmp prexed directory. python_files One or more Glob-style le patterns determining which python les are considered as test modules. python_classes One or more name prexes determining which test classes are considered as test modules. python_functions One or more name prexes determining which test functions and methods are considered as test modules. See change naming conventions for examples.
46
3.3 The writing and reporting of assertions in tests

3.3.1 assert with the assert statement
py.test allows you to use the standard python assert for verifying expectations and values in Python tests. For example, you can write the following:
# content of test_assert1.py def f(): return 3 def test_function(): assert f() == 4
to assert that your function returns a certain value. If this assertion fails you will see the return value of the function call:
$ py.test test_assert1.py ============================= test session starts ============================== platform linux2 -- Python 2.6.6 -- pytest-2.1.0.dev6 collecting ... collected 1 items test_assert1.py F =================================== FAILURES =================================== ________________________________ test_function _________________________________ def test_function(): assert f() == 4 assert 3 == 4 + where 3 = f()
> E E
test_assert1.py:5: AssertionError =========================== 1 failed in 0.01 seconds ===========================
py.test has support for showing the values of the most common subexpressions including calls, attributes, comparisons, and binary and unary operators. (See Demo of Python failure reports with py.test). This allows you to use the idiomatic python constructs without boilerplate code while not losing introspection information. However, if you specify a message with the assertion like this:
assert a % 2 == 0, "value was odd, should be even"
then no assertion introspection takes places at all and the message will be simply shown in the traceback. See Advanced assertion introspection for more information on assertion introspection.
3.3.2 assertions about expected exceptions

In order to write assertions about raised exceptions, you can use pytest.raises as a context manager like this:
import pytest with pytest.raises(ZeroDivisionError): 1 / 0
and if you need to have access to the actual exception info you may use:
3.3. The writing and reporting of assertions in tests
47
with pytest.raises(RuntimeError) as excinfo: def f(): f() f() # do checks related to excinfo.type, excinfo.value, excinfo.traceback
If you want to write test code that works on Python2.4 as well, you may also use two other ways to test for an expected exception:
pytest.raises(ExpectedException, func, *args, **kwargs) pytest.raises(ExpectedException, "func(*args, **kwargs)")
both of which execute the specied function with args and kwargs and asserts that the given ExpectedException is raised. The reporter will provide you with helpful output in case of failures such as no exception or wrong exception.
3.3.3 Making use of context-sensitive comparisons

New in version 2.0. py.test has rich support for providing context-sensitive information when it encounters comparisons. For example:
# content of test_assert2.py def test_set_comparison(): set1 = set("1308") set2 = set("8035") assert set1 == set2
if you run this module:

$ py.test test_assert2.py ============================= test session starts ============================== platform linux2 -- Python 2.6.6 -- pytest-2.1.0.dev6 collecting ... collected 1 items test_assert2.py F =================================== FAILURES =================================== _____________________________ test_set_comparison ______________________________ def test_set_comparison(): set1 = set("1308") set2 = set("8035") assert set1 == set2 assert set([0, 1, 3, 8]) == set([0, 3, 5, 8]) Extra items in the left set: 1 Extra items in the right set: 5
> E E E E E
test_assert2.py:5: AssertionError =========================== 1 failed in 0.01 seconds ===========================
Special comparisons are done for a number of cases: comparing long strings: a context diff is shown comparing long sequences: rst failing indices
48
comparing dicts: different entries See the reporting demo for many more examples.
3.3.4 Dening your own assertion comparison

It is possible to add your own detailed explanations by implementing the pytest_assertrepr_compare hook. _pytest.hookspec.pytest_assertrepr_compare(cong, op, left, right) return explanation for comparisons in failing assert expressions. Return None for no custom explanation, otherwise return a list of strings. The strings will be joined by newlines but any newlines in a string will be escaped. Note that all but the rst line will be indented sligthly, the intention is for the rst line to be a summary. As an example consider adding the following hook in a conftest.py which provides an alternative explanation for Foo objects:
# content of conftest.py from test_foocompare import Foo def pytest_assertrepr_compare(op, left, right): if isinstance(left, Foo) and isinstance(right, Foo) and op == "==": return [Comparing Foo instances:, vals: %s != %s % (left.val, right.val)]
now, given this test module:

# content of test_foocompare.py class Foo: def __init__(self, val): self.val = val def test_compare(): f1 = Foo(1) f2 = Foo(2) assert f1 == f2
you can run the test module and get the custom output dened in the conftest le:
$ py.test -q test_foocompare.py collecting ... collected 1 items F =================================== FAILURES =================================== _________________________________ test_compare _________________________________ def test_compare(): f1 = Foo(1) f2 = Foo(2) assert f1 == f2 assert Comparing Foo instances: vals: 1 != 2
> E E
test_foocompare.py:8: AssertionError 1 failed in 0.01 seconds
3.3. The writing and reporting of assertions in tests
49
3.3.5 Advanced assertion introspection

New in version 2.1. Reporting details about a failing assertion is achieved either by rewriting assert statements before they are run or re-evaluating the assert expression and recording the intermediate values. Which technique is used depends on the location of the assert, py.tests conguration, and Python version being used to run py.test. Note that for assert statements with a manually provided message, i.e. assert expr, message, no assertion introspection takes place and the manually provided message will be rendered in tracebacks. By default, if the Python version is greater than or equal to 2.6, py.test rewrites assert statements in test modules. Rewritten assert statements put introspection information into the assertion failure message. py.test only rewrites test modules directly discovered by its test collection process, so asserts in supporting modules which are not themselves test modules will not be rewritten. Note: py.test rewrites test modules on import. It does this by using an import hook to write a new pyc les. Most of the time this works transparently. However, if you are messing with import yourself, the import hook may interfere. If this is the case, simply use --assert=reinterp or --assert=plain. Additionally, rewriting will fail silently if it cannot write new pycs, i.e. in a read-only lesystem or a ziple. If an assert statement has not been rewritten or the Python version is less than 2.6, py.test falls back on assert reinterpretation. In assert reinterpretation, py.test walks the frame of the function containing the assert statement to discover sub-expression results of the failing assert statement. You can force py.test to always use assertion reinterpretation by passing the --assert=reinterp option. Assert reinterpretation has a caveat not present with assert rewriting: If evaluating the assert expression has side effects you may get a warning that the intermediate values could not be determined safely. A common example of this issue is an assertion which reads from a le:
assert f.read() != ...
If this assertion fails then the re-evaluation will probably succeed! This is because f.read() will return an empty string when it is called the second time during the re-evaluation. However, it is easy to rewrite the assertion and avoid any trouble:
content = f.read() assert content != ...
All assert introspection can be turned off by passing --assert=plain. New in version 2.1: Add assert rewriting as an alternate introspection technique.Changed in version 2.1: Introduce the --assert option. Deprecate --no-assert and --nomagic.
3.4 Injecting objects into test functions (funcargs)

3.4.1 Dependency injection through function arguments
py.test lets you inject objects into test functions and precisely control their life cycle in relation to the test execution. It is also possible to run a test function multiple times with different objects. The basic mechanism for injecting objects is also called the funcarg mechanism because objects are ultimatly injected by calling a test function with it as an argument. Unlike the classical xUnit approach funcargs relate more to Dependency Injection because they help to de-couple test code from objects required for them to execute. To create a value with which to call a test function a factory function is called which gets full access to the test function context and can register nalizers or invoke lifecycle-caching helpers. The factory can be implemented in same test class or test module, or in a per-directory conftest.py le or even in an external plugin. This allows full de-coupling of test code and objects needed for test execution. 50 Chapter 3. py.test reference documentation
A test function may be invoked multiple times in which case we speak of parametrized testing. This can be very useful if you want to test e.g. against different database backends or with multiple numerical arguments sets and want to reuse the same set of test functions. Basic injection example Lets look at a simple self-contained test module:
# content of ./test_simplefactory.py def pytest_funcarg__myfuncarg(request): return 42 def test_function(myfuncarg): assert myfuncarg == 17
This test function needs an injected object named myfuncarg. py.test will discover and call the factory named pytest_funcarg__myfuncarg within the same module in this case. Running the test looks like this:
$ py.test test_simplefactory.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 1 items test_simplefactory.py F ================================= FAILURES ================================= ______________________________ test_function _______________________________ myfuncarg = 42 def test_function(myfuncarg): assert myfuncarg == 17 assert 42 == 17
> E
test_simplefactory.py:5: AssertionError ========================= 1 failed in 0.02 seconds =========================
This means that indeed the test function was called with a myfuncarg argument value of 42 and the assert fails. Here is how py.test comes to call the test function this way: 1. py.test nds the test_function because of the test_ prex. The test function needs a function argument named myfuncarg. A matching factory function is discovered by looking for the name pytest_funcarg__myfuncarg. 2. pytest_funcarg__myfuncarg(request) is called and returns the value for myfuncarg. 3. the test function can now be called: test_function(42). This results in the above exception because of the assertion mismatch. Note that if you misspell a function argument or want to use one that isnt available, youll see an error with a list of available function arguments. You can always issue:
py.test --funcargs test_simplefactory.py
to see available function arguments (which you can also think of as resources).
3.4. Injecting objects into test functions (funcargs)
51
3.4.2 The funcarg request object

Each funcarg factory receives a request object which is tied to a specic test function call. A request object is passed to a funcarg factory and provides access to test conguration and context: class _pytest.python.FuncargRequest A request for function arguments from a test function. Note that there is an optional param attribute in case there was an invocation to metafunc.addcall(param=...). If no such call was done in a pytest_generate_tests hook, the attribute will not be present. function function object of the test invocation. cls class (can be None) where the test function was collected. module module where the test function was collected. keywords keywords of the test function item. New in version 2.0. config the pytest cong object associated with this request. FuncargRequest.addfinalizer(nalizer) add nalizer function to be called after test function nished execution. FuncargRequest.cached_setup(setup, teardown=None, scope=module, extrakey=None) return a testing resource managed by setup & teardown calls. scope and extrakey determine when the teardown function will be called so that subsequent calls to setup would recreate the resource. Parameters teardown function receiving a previously setup resource. setup a no-argument function creating a resource. scope a string value out of function, class, module or session indicating the caching lifecycle of the resource. extrakey added to internal caching key of (funcargname, scope). FuncargRequest.applymarker(marker) apply a marker to a single test function invocation. This method is useful if you dont want to have a keyword/marker on all function invocations. Parameters marker a _pytest.mark.MarkDecorator object created by a call to py.test.mark.NAME(...). FuncargRequest.getfuncargvalue(argname) Retrieve a function argument by name for this test function invocation. This allows one function argument factory to call another function argument factory. If there are two funcarg factories for the same test function argument the rst factory may use getfuncargvalue to call the second one and do something additional with the resource.
3.4.3 Parametrizing multiple calls to a test function

You can parametrize multiple runs of the same test function by adding new test function calls with different function argument values. Lets look at a simple self-contained example:
52
Basic generated test example Lets consider a test module which uses the pytest_generate_tests hook to generate several calls to the same test function:
# content of test_example.py def pytest_generate_tests(metafunc): if "numiter" in metafunc.funcargnames: for i in range(10): metafunc.addcall(funcargs=dict(numiter=i)) def test_func(numiter): assert numiter < 9
Running this:
$ py.test test_example.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 10 items test_example.py .........F ================================= FAILURES ================================= _______________________________ test_func[9] _______________________________ numiter = 9 def test_func(numiter): assert numiter < 9 assert 9 < 9
> E
test_example.py:7: AssertionError ==================== 1 failed, 9 passed in 0.03 seconds ====================
Note that the pytest_generate_tests(metafunc) hook is called during the test collection phase which is separate from the actual test running. Lets just look at what is collected:
$ py.test --collectonly test_example.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 10 items <Module test_example.py> <Function test_func[0]> <Function test_func[1]> <Function test_func[2]> <Function test_func[3]> <Function test_func[4]> <Function test_func[5]> <Function test_func[6]> <Function test_func[7]> <Function test_func[8]> <Function test_func[9]> ============================= in 0.00 seconds =============================
If you want to select only the run with the value 7 you could do:
3.4. Injecting objects into test functions (funcargs)
53
$ py.test -v -k 7 test_example.py # or -k test_func[7] =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 -- /home/hpk/venv/0/bin/python collecting ... collected 10 items test_example.py:6: test_func[7] PASSED ======================== 9 tests deselected by 7 ========================= ================== 1 passed, 9 deselected in 0.01 seconds ==================
You might want to look at more parametrization examples. The metafunc object metafunc objects are passed to the pytest_generate_tests hook. They help to inspect a testfunction and to generate tests according to test conguration or values specied in the class or module where a test function is dened: metafunc.funcargnames: set of required function arguments for given function metafunc.function: underlying python test function metafunc.cls: class object where the test function is dened in or None. metafunc.module: the module object where the test function is dened in. metafunc.config: access to command line opts and general cong Metafunc.addcall(funcargs=None, id=_notexists, param=_notexists) add a new call to the underlying test function during the collection phase of a test run. Note that request.addcall() is called during the test collection phase prior and independently to actual test execution. Therefore you should perform setup of resources in a funcarg factory which can be instrumented with the param. Parameters funcargs argument keyword dictionary used when invoking the test function. id used for reporting and identication purposes. If you dont supply an id the length of the currently list of calls to the test function will be used. param will be exposed to a later funcarg factory invocation through the request.param attribute. It allows to defer test xture setup activities to when an actual test is run.
3.5 extended xUnit style setup xtures

Python, Java and many other languages support xUnit style testing. This typically involves the call of a setup (xture) method before running a test function and teardown after it has nished. py.test supports a more ne-grained model of setup/teardown handling by optionally calling per-module and per-class hooks.
3.5.1 module level setup/teardown

If you have multiple test functions and test classes in a single module you can optionally implement the following xture methods which will usually be called once for all the functions:
54
def setup_module(module): """ setup up any state specific to the execution of the given module. """ def teardown_module(module): """ teardown any state that was previously setup with a setup_module method. """
3.5.2 class level setup/teardown

Similarly, the following methods are called at class level before and after all test methods of the class are called:
@classmethod def setup_class(cls): """ setup up any state specific to the execution of the given class (which usually contains tests). """ @classmethod def teardown_class(cls): """ teardown any state that was previously setup with a call to setup_class. """
3.5.3 method and function level setup/teardown

Similarly, the following methods are called around each method invocation:
def setup_method(self, method): """ setup up any state tied to the execution of the given method in a class. setup_method is invoked for every test method of a class. """ def teardown_method(self, method): """ teardown any state that was previously setup with a setup_method call. """
If you would rather dene test functions directly at module level you can also use the following functions to implement xtures:
def setup_function(function): """ setup up any state tied to the execution of the given function. Invoked for every test function in the module. """ def teardown_function(function): """ teardown any state that was previously setup with a setup_function call. """
Note that it possible that setup/teardown pairs are invoked multiple times per testing process.
3.5. extended xUnit style setup xtures
55
3.6 Capturing of the stdout/stderr output

3.6.1 Default stdout/stderr/stdin capturing behaviour
During test execution any output sent to stdout and stderr is captured. If a test or a setup method fails its according captured output will usually be shown along with the failure traceback. In addition, stdin is set to a null object which will fail on attempts to read from it because it is rarely desired to wait for interactive input when running automated tests. By default capturing is done by intercepting writes to low level le descriptors. This allows to capture output from simple print statements as well as output from a subprocess started by a test.
3.6.2 Setting capturing methods or disabling capturing

There are two ways in which py.test can perform capturing: le descriptor (FD) level capturing (default): All writes going to the operating system le descriptors 1 and 2 will be captured. sys level capturing: Only writes to Python les sys.stdout and sys.stderr will be captured. No capturing of writes to ledescriptors is performed. You can inuence output capturing mechanisms from the command line:
py.test -s # disable all capturing py.test --capture=sys # replace sys.stdout/stderr with in-mem files py.test --capture=fd # also point filedescriptors 1 and 2 to temp file
3.6.3 Using print statements for debugging

One primary benet of the default capturing of stdout/stderr output is that you can use print statements for debugging:
# content of test_module.py def setup_function(function): print ("setting up %s" % function) def test_func1(): assert True def test_func2(): assert False
and running this module will show you precisely the output of the failing function and hide the other one:
$ py.test =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 2 items test_module.py .F ================================= FAILURES ================================= ________________________________ test_func2 ________________________________
56
> E
def test_func2(): assert False assert False
test_module.py:9: AssertionError ----------------------------- Captured stdout -----------------------------setting up <function test_func2 at 0x238c410> ==================== 1 failed, 1 passed in 0.02 seconds ====================
3.6.4 Accessing captured output from a test function

The Dependency injection through function arguments allows test function a very easy way to access the captured output by simply using the names capsys or capfd in the test function signature. Here is an example test function that performs some output related checks:
def test_myoutput(capsys): # or use "capfd" for fd-level print ("hello") sys.stderr.write("world\n") out, err = capsys.readouterr() assert out == "hello\n" assert err == "world\n" print "next" out, err = capsys.readouterr() assert out == "next\n"
The readouterr() call snapshots the output so far - and capturing will be continued. After the test function nishes the original streams will be restored. Using capsys this way frees your test from having to care about setting/resetting output streams and also interacts well with py.tests own per-test capturing. If you want to capture on fd level you can use the capfd function argument which offers the exact same interface.
3.7 monkeypatching/mocking modules and environments

Sometimes tests need to invoke functionality which depends on global settings or which invokes code which cannot be easily tested such as network access. The monkeypatch function argument helps you to safely set/delete an attribute, dictionary item or environment variable or to modify sys.path for importing. See the monkeypatch blog post for some introduction material and a discussion of its motivation.
3.7.1 Simple example: patching os.path.expanduser

If, for instance, you want to pretend that os.expanduser returns a certain directory, you can use the monkeypatch.setattr() method to patch this function before calling into a function which uses it:
import os.path def getssh(): # pseudo application code return os.path.join(os.path.expanduser("~admin"), .ssh) def test_mytest(monkeypatch): def mockreturn(path): return /abc monkeypatch.setattr(os.path, expanduser, mockreturn) x = getssh() assert x == /abc/.ssh
3.7. monkeypatching/mocking modules and environments
57
After the test function nishes the os.path.expanduser modication will be undone.
3.7.2 Method reference of the monkeypatch function argument

class _pytest.monkeypatch.monkeypatch object keeping a record of setattr/item/env/syspath changes. setattr(obj, name, value, raising=True) set attribute name on obj to value, by default raise AttributeEror if the attribute did not exist. delattr(obj, name, raising=True) delete attribute name from obj, by default raise AttributeError it the attribute did not previously exist. setitem(dic, name, value) set dictionary entry name to value. delitem(dic, name, raising=True) delete name from dict, raise KeyError if it doesnt exist. setenv(name, value, prepend=None) set environment variable name to value. if prepend is a character, read the current environment variable value and prepend the value adjoined with the prepend character. delenv(name, raising=True) delete name from environment, raise KeyError it not exists. syspath_prepend(path) prepend path to sys.path list of import locations. undo() undo previous changes. This call consumes the undo stack. Calling it a second time has no effect unless you do more monkeypatching after the undo call. monkeypatch.setattr/delattr/delitem/delenv() all by default raise an Exception if the target does not exist. Pass raising=False if you want to skip this check.
3.8 xdist: pytest distributed testing plugin

The pytest-xdist plugin extends py.test with some unique test execution modes: Looponfail: run your tests repeatedly in a subprocess. After each run, py.test waits until a le in your project changes and then re-runs the previously failing tests. This is repeated until all tests pass. At this point a full run is again performed. multiprocess Load-balancing: if you have multiple CPUs or hosts you can use them for a combined test run. This allows to speed up development or to use special resources of remote machines. Multi-Platform coverage: you can specify different Python interpreters or different platforms and run tests in parallel on all of them. Before running tests remotely, py.test efciently rsyncs your program source code to the remote place. All test results are reported back and displayed to your local terminal. You may specify different Python versions and interpreters.
3.8.1 Installation of xdist plugin

Install the plugin with:
58
easy_install pytest-xdist # or pip install pytest-xdist
or use the package in develop/in-place mode with a checkout of the pytest-xdist repository
python setup.py develop
3.8.2 Usage examples

Speed up test runs by sending tests to multiple CPUs To send tests to multiple CPUs, type:
py.test -n NUM
Especially for longer running tests or tests requiring a lot of I/O this can lead to considerable speed ups. Running tests in a Python subprocess To instantiate a Python-2.4 subprocess and send tests to it, you may type:
py.test -d --tx popen//python=python2.4
This will start a subprocess which is run with the python2.4 Python interpreter, found in your system binary lookup path. If you prex the tx option value like this:
py.test -d --tx 3*popen//python=python2.4
then three subprocesses would be created and the tests will be distributed to three subprocesses and run simultanously. Running tests in looponfailing mode For refactoring a project with a medium or large test suite you can use the looponfailing mode. Simply add the --f option:
py.test -f
and py.test will run your tests. Assuming you have failures it will then wait for le changes and re-run the failing test set. File changes are detected by looking at looponfailingroots root directories and all of their contents (recursively). If the default for this value does not work for you you can change it in your project by setting a conguration option:
# content of a pytest.ini, setup.cfg or tox.ini file [pytest] looponfailroots = mypkg testdir
This would lead to only looking for le changes in the respective directories, specied relatively to the ini-les directory.
3.8. xdist: pytest distributed testing plugin
59
Sending tests to remote SSH accounts Suppose you have a package mypkg which contains some tests that you can successfully run locally. And you also have a ssh-reachable machine myhost. Then you can ad-hoc distribute your tests by typing:
py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
This will synchronize your mypkg package directory with a remote ssh account and then collect and run your tests at the remote side. You can specify multiple --rsyncdir directories to be sent to the remote side. Sending tests to remote Socket Servers Download the single-module socketserver.py Python program and run it like this:
python socketserver.py
It will tell you that it starts listening on the default port. You can now on your home machine specify this new socket host with something like this:
py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
Running tests on many platforms at once The basic command to run tests on multiple platforms is:
py.test --dist=each --tx=spec1 --tx=spec2
If you specify a windows host, an OSX host and a Linux environment this command will send each tests to all platforms - and report back failures from all platforms at once. The specications strings use the xspec syntax. Specifying test exec environments in an ini le pytest (since version 2.0) supports ini-style conguration. For example, you could make running with three subprocesses your default:
[pytest] addopts = -n3
You can also add default environments like this:

[pytest] addopts = --tx ssh=myhost//python=python2.5 --tx ssh=myhost//python=python2.6
and then just type:

py.test --dist=each
to run tests in each of the environments. Specifying rsync dirs in an ini-le In a tox.ini or setup.cfg le in your root project directory you may specify directories to include or to exclude in synchronisation:
60
[pytest] rsyncdirs = . mypkg helperpkg rsyncignore = .hg
These directory specications are relative to the directory where the conguration le was found.
3.9 temporary directories and les

3.9.1 the tmpdir test function argument
You can use the tmpdir function argument which will provide a temporary directory unique to the test invocation, created in the base temporary directory. tmpdir is a py.path.local object which offers os.path methods and more. Here is an example test usage:
# content of test_tmpdir.py import os def test_create_file(tmpdir): p = tmpdir.mkdir("sub").join("hello.txt") p.write("content") assert p.read() == "content" assert len(tmpdir.listdir()) == 1 assert 0
Running this would result in a passed test except for the last assert 0 line which we use to look at values:
$ py.test test_tmpdir.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 1 items test_tmpdir.py F ================================= FAILURES ================================= _____________________________ test_create_file _____________________________ tmpdir = local(/tmp/pytest-11/test_create_file0) def test_create_file(tmpdir): p = tmpdir.mkdir("sub").join("hello.txt") p.write("content") assert p.read() == "content" assert len(tmpdir.listdir()) == 1 assert 0 assert 0
> E
test_tmpdir.py:7: AssertionError ========================= 1 failed in 0.06 seconds =========================
3.9.2 the default base temporary directory

Temporary directories are by default created as sub-directories of the system temporary directory. The base name will be pytest-NUM where NUM will be incremented with each test run. Moreover, entries older than 3 temporary directories will be removed.
3.9. temporary directories and les
61
You can override the default temporary directory setting like this:
py.test --basetemp=mydir
When distributing tests on the local machine, py.test takes care to congure a basetemp directory for the sub processes such that all temporary data lands below a single per-test run basetemp directory.
3.10 skip and xfail: dealing with tests that can not succeed
If you have test functions that cannot be run on certain platforms or that you expect to fail you can mark them accordingly or you may call helper functions during execution of setup or test functions. A skip means that you expect your test to pass unless a certain conguration or condition (e.g. wrong Python interpreter, missing dependency) prevents it to run. And xfail means that your test can run but you expect it to fail because there is an implementation problem. py.test counts and lists skip and xfail tests separately. However, detailed information about skipped/xfailed tests is not shown by default to avoid cluttering the output. You can use the -r option to see details corresponding to the short letters shown in the test progress:
py.test -rxs # show extra info on skips and xfails
(See How to change command line options defaults)
3.10.1 Marking a test function to be skipped

Here is an example of marking a test function to be skipped when run on a Python3 interpreter:
import sys @pytest.mark.skipif("sys.version_info >= (3,0)") def test_function(): ...
During test function setup the skipif condition is evaluated by calling eval(sys.version_info >= (3,0), namespace). (New in version 2.0.2) The namespace contains all the module globals of the test function so that you can for example check for versions of a module you are using:
import mymodule @pytest.mark.skipif("mymodule.__version__ < 1.2") def test_function(): ...
The test function will not be run (skipped) if mymodule is below the specied version. The reason for specifying the condition as a string is mainly that py.test can report a summary of skip conditions. For information on the construction of the namespace see evaluation of skipif/xfail conditions. You can of course create a shortcut for your conditional skip decorator at module level like this:
win32only = pytest.mark.skipif("sys.platform != win32") @win32only def test_function(): ...
62
3.10.2 skip all test functions of a class

As with all function marking you can skip test functions at the whole class- or module level. Here is an example for skipping all methods of a test class based on the platform:
class TestPosixCalls: pytestmark = pytest.mark.skipif("sys.platform == win32") def test_function(self): "will not be setup or run under win32 platform"
The pytestmark special name tells py.test to apply it to each test function in the class. If your code targets python2.6 or above you can more naturally use the skipif decorator (and any other marker) on classes:
@pytest.mark.skipif("sys.platform == win32") class TestPosixCalls: def test_function(self): "will not be setup or run under win32 platform"
Using multiple skipif decorators on a single function is generally ne - it means that if any of the conditions apply the function execution will be skipped.
3.10.3 mark a test function as expected to fail

You can use the xfail marker to indicate that you expect the test to fail:
@pytest.mark.xfail def test_function(): ...
This test will be run but no traceback will be reported when it fails. Instead terminal reporting will list it in the expected to fail or unexpectedly passing sections. By specifying on the commandline:
pytest --runxfail
you can force the running and reporting of an xfail marked test as if it werent marked at all. As with skipif you can also mark your expectation of a failure on a particular platform:
@pytest.mark.xfail("sys.version_info >= (3,0)") def test_function(): ...
You can furthermore prevent the running of an xfail test or specify a reason such as a bug ID or similar. Here is a simple test le with the several usages:
import pytest xfail = pytest.mark.xfail @xfail def test_hello(): assert 0 @xfail(run=False) def test_hello2(): assert 0
3.10. skip and xfail: dealing with tests that can not succeed
63
@xfail("hasattr(os, sep)") def test_hello3(): assert 0 @xfail(reason="bug 110") def test_hello4(): assert 0 @xfail(pytest.__version__[0] != "17") def test_hello5(): assert 0 def test_hello6(): pytest.xfail("reason")
Running it with the report-on-xfail option gives this output:

example $ py.test -rx xfail_demo.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 6 items xfail_demo.py xxxxxx ========================= short test summary info ========================== XFAIL xfail_demo.py::test_hello XFAIL xfail_demo.py::test_hello2 reason: [NOTRUN] XFAIL xfail_demo.py::test_hello3 condition: hasattr(os, sep) XFAIL xfail_demo.py::test_hello4 bug 110 XFAIL xfail_demo.py::test_hello5 condition: pytest.__version__[0] != "17" XFAIL xfail_demo.py::test_hello6 reason: reason ======================== 6 xfailed in 0.05 seconds =========================
3.10.4 evaluation of skipif/xfail expressions

New in version 2.0.2. The evaluation of a condition string in pytest.mark.skipif(conditionstring) or pytest.mark.xfail(conditionstring) takes place in a namespace dictionary which is constructed as follows: the namespace is initialized by putting the sys and os modules and the pytest config object into it. updated with the module globals of the test function for which the expression is applied. The pytest config object allows you to skip based on a test conguration value which you might have added:
@pytest.mark.skipif("not config.getvalue(db)") def test_function(...): ...
64
3.10.5 imperative xfail from within a test or setup function

If you cannot declare xfail-conditions at import time you can also imperatively produce an XFail-outcome from within test or setup code. Example:
def test_function(): if not valid_config(): pytest.xfail("unsupported configuration")
3.10.6 skipping on a missing import dependency

You can use the following import helper at module level or within a test or test setup function:
docutils = pytest.importorskip("docutils")
If docutils cannot be imported here, this will lead to a skip outcome of the test. You can also skip based on the version number of a library:
docutils = pytest.importorskip("docutils", minversion="0.3")
The version will be read from the specied modules __version__ attribute.
3.10.7 imperative skip from within a test or setup function

If for some reason you cannot declare skip-conditions you can also imperatively produce a skip-outcome from within test or setup code. Example:
def test_function(): if not valid_config(): pytest.skip("unsupported configuration")
3.11 mark test functions with attributes

By using the pytest.mark helper you can instantiate decorators that will set named metadata on test functions.
3.11.1 Marking a single function

You can mark a test function with metadata like this:
import pytest @pytest.mark.webtest def test_send_http(): ...
This will set the function attribute webtest to a MarkInfo instance. You can also specify parametrized metadata like this:
# content of test_mark.py import pytest @pytest.mark.webtest(firefox=30) def test_receive(): pass
3.11. mark test functions with attributes
65
@pytest.mark.webtest("functional", firefox=30) def test_run_and_look(): pass
and access it from other places like this:

test_receive.webtest.kwargs[firefox] == 30 test_run_and_look.webtest.args[0] == "functional"
3.11.2 Marking whole classes or modules

If you are programming with Python2.6 you may use pytest.mark decorators with classes to apply markers to all of its test methods:
# content of test_mark_classlevel.py import pytest @pytest.mark.webtest class TestClass: def test_startup(self): pass def test_startup_and_more(self): pass
This is equivalent to directly applying the decorator to the two test functions. To remain compatible with Python2.5 you can also set a pytestmark attribute on a TestClass like this:
import pytest class TestClass: pytestmark = pytest.mark.webtest
or if you need to use multiple markers you can use a list:

import pytest class TestClass: pytestmark = [pytest.mark.webtest, pytest.mark.slowtest]
You can also set a module level marker:

import pytest pytestmark = pytest.mark.webtest
in which case it will be applied to all functions and methods dened in the module.
3.11.3 Using -k TEXT to select tests

You can use the -k command line option to select tests:
$ py.test -k webtest # running with the above defined examples yields =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 4 items test_mark.py .. test_mark_classlevel.py ..
66
========================= 4 passed in 0.01 seconds =========================
And you can also run all tests except the ones that match the keyword:
$ py.test -k-webtest =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 4 items ===================== 4 tests deselected by -webtest ===================== ======================= 4 deselected in 0.01 seconds =======================
Or to only select the class:

$ py.test -kTestClass =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 4 items test_mark_classlevel.py .. ==================== 2 tests deselected by TestClass ===================== ================== 2 passed, 2 deselected in 0.01 seconds ==================
3.11.4 API reference for mark related objects

class _pytest.mark.MarkGenerator Factory for MarkDecorator objects - exposed as a py.test.mark singleton instance. Example:
import py @py.test.mark.slowtest def test_function(): pass
will set a slowtest MarkInfo object on the test_function object. class _pytest.mark.MarkDecorator(name, args=None, kwargs=None) A decorator for test functions and test classes. When applied it will create MarkInfo objects which may be retrieved by hooks as item keywords. MarkDecorator instances are often created like this:
mark1 = py.test.mark.NAME # simple MarkDecorator mark2 = py.test.mark.NAME(name1=value) # parametrized MarkDecorator
and can then be applied as decorators to test functions:

@mark2 def test_function(): pass
class _pytest.mark.MarkInfo(name, args, kwargs) Marking object created by MarkDecorator instances. args positional argument list, empty if none specied kwargs keyword argument dictionary, empty if nothing specied
3.11. mark test functions with attributes
67
name name of attribute
3.12 asserting deprecation and other warnings.

3.12.1 recwarn function argument
You can use the recwarn funcarg to assert that code triggers warnings through the Python warnings system. Here is a simple self-contained test:
# content of test_recwarn.py def test_hello(recwarn): from warnings import warn warn("hello", DeprecationWarning) w = recwarn.pop(DeprecationWarning) assert issubclass(w.category, DeprecationWarning) assert hello in str(w.message) assert w.filename assert w.lineno
The recwarn function argument provides these methods: pop(category=None): return last warning matching the category. clear(): clear list of warnings
3.12.2 ensuring a function triggers a deprecation warning

You can also call a global helper for checking that a certain function call triggers a Deprecation warning:
import pytest def test_global(): pytest.deprecated_call(myfunction, 17)
3.13 unittest.TestCase support

py.test has limited support for running Python unittest.py style tests. It will automatically collect unittest.TestCase subclasses and their test methods in test les. It will invoke setUp/tearDown methods but also perform py.tests standard ways of treating tests such as IO capturing:
# content of test_unittest.py import unittest class MyTest(unittest.TestCase): def setUp(self): print ("hello") # output is captured def test_method(self): x = 1 self.assertEquals(x, 3)
Running it yields:
68
$ py.test test_unittest.py =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 1 items test_unittest.py F ================================= FAILURES ================================= ____________________________ MyTest.test_method ____________________________ self = <test_unittest.MyTest testMethod=test_method> def test_method(self): x = 1 self.assertEquals(x, 3) AssertionError: 1 != 3
> E
test_unittest.py:8: AssertionError ----------------------------- Captured stdout -----------------------------hello ========================= 1 failed in 0.02 seconds =========================
3.14 Running test written for nose

py.test has basic support for running tests written for nose.
3.14.1 Usage
type:
py.test # instead of nosetests
and you should be able to run your nose style tests and make use of py.tests capabilities.
3.14.2 Supported nose Idioms

setup and teardown at module/class/method level SkipTest exceptions and markers setup/teardown decorators yield-based tests and their setup general usage of nose utilities
3.14.3 Unsupported idioms / known issues

nose-style doctests are not collected and executed correctly, also doctest xtures dont work. no nose-conguration is recognized
3.14. Running test written for nose
69
3.15 doctest integration for modules and test les

By default all les matching the test*.txt pattern will be run through the python standard doctest module. You can change the pattern by issuing:
py.test --doctest-glob=*.rst
on the command line. You can also trigger running of doctests from docstrings in all python modules (including regular python test modules):
py.test --doctest-modules
You can make these changes permanent in your project by putting them into a pytest.ini le like this:
# content of pytest.ini [pytest] addopts = --doctest-modules
If you then have a text le like this:

# content of example.rst hello this is a doctest >>> x = 3 >>> x 3
and another like this:

# content of mymodule.py def something(): """ a doctest in a docstring >>> something() 42 """ return 42
then you can just invoke py.test without command line options:
$ py.test =========================== test session starts ============================ platform linux2 -- Python 2.6.6 -- pytest-2.0.3 collecting ... collected 1 items mymodule.py . ========================= 1 passed in 0.40 seconds =========================
70
CHAPTER
FOUR
WORKING WITH PLUGINS AND CONFTEST FILES

py.test implements all aspects of conguration, collection, running and reporting by calling well specied hooks. Virtually any Python module can be registered as a plugin. It can implement any number of hook functions (usually two or three) which all have a pytest_ prex, making hook functions easy to distinguish and nd. There are three basic locations types: builtin plugins: loaded from py.tests own pytest/plugin directory. external plugins: modules discovered through setuptools entry points conftest.py plugins: modules auto-discovered in test directories
4.1 conftest.py: local per-directory plugins

local conftest.py plugins contain directory-specic hook implementations. Session and test running activities will invoke all hooks dened in conftest.py les closer to the root of the lesystem. Example: Assume the following layout and content of les:
a/conftest.py: def pytest_runtest_setup(item): # called for running each test in a directory print ("setting up", item) a/test_in_subdir.py: def test_sub(): pass test_flat.py: def test_flat(): pass
Here is how you might run it:

py.test test_flat.py py.test a/test_sub.py # will not show "setting up" # will show "setting up"
Note: If you have conftest.py les which do not reside in a python package directory (i.e. one containing an __init__.py) then import conftest can be ambiguous because there might be other conftest.py les as well on your PYTHONPATH or sys.path. It is thus good practise for projects to either put conftest.py under a package scope or to never import anything from a conftest.py le. 71
4.2 Installing External Plugins / Searching

Installing a plugin happens through any usual Python installation tool, for example:
pip install pytest-NAME pip uninstall pytest-NAME
If a plugin is installed, py.test automatically nds and integrates it, there is no need to activate it. Here is a list of known plugins: pytest-capturelog: to capture and assert about messages from the logging module pytest-xdist: to distribute tests to CPUs and remote hosts, looponfailing mode, see also xdist: pytest distributed testing plugin pytest-cov: coverage reporting, compatible with distributed testing pytest-pep8: a --pep8 option to enable PEP8 compliance checking. oejskit: a plugin to run javascript unittests in life browsers (version 0.8.9 not compatible with pytest-2.0) You may discover more plugins through a pytest- pypi.python.org search.
4.3 Writing a plugin by looking at examples

If you want to write a plugin, there are many real-life examples you can copy from: a custom collection example plugin: a basic example for specifying tests in Yaml les around 20 builtin plugins which comprise py.tests own functionality around 10 external plugins providing additional features All of these plugins implement the documented well specied hooks to extend and add functionality.
4.4 Making your plugin installable by others

If you want to make your plugin externally available, you may dene a so-called entry point for your distribution so that py.test nds your plugin module. Entry points are a feature that is provided by setuptools or Distribute. py.test looks up the pytest11 entrypoint to discover its plugins and you can thus make your plugin available by denig it in your setuptools/distribute-based setup-invocation:
# sample ./setup.py file from setuptools import setup setup( name="myproject", packages = [myproject] # the following makes a plugin available to py.test entry_points = { pytest11: [ name_of_plugin = myproject.pluginmodule, ]
72
Chapter 4. Working with plugins and conftest les
}, )
If a package is installed this way, py.test will load myproject.pluginmodule as a plugin which can dene well specied hooks.
4.5 Plugin discovery order at tool startup

py.test loads plugin modules at tool startup in the following way: by loading all builtin plugins by loading all plugins registered through setuptools entry points. by pre-scanning the command line for the -p name option and loading the specied plugin before actual command line parsing. by loading all conftest.py les as inferred by the command line invocation (test les and all of its parent directories). Note that conftest.py les from sub directories are by default not loaded at tool startup. by recursively loading all plugins specied by the pytest_plugins variable in conftest.py les
4.6 Requiring/Loading plugins in a test module or conftest le

You can require plugins in a test module or a conftest le like this:
pytest_plugins = "name1", "name2",
When the test module or conftest plugin is loaded the specied plugins will be loaded as well. You can also use dotted path like this:
pytest_plugins = "myapp.testsupport.myplugin"
which will import the specied module as a py.test plugin.
4.7 Accessing another plugin by name

If a plugin wants to collaborate with code from another plugin it can obtain a reference through the plugin manager like this:
plugin = config.pluginmanager.getplugin("name_of_plugin")
If you want to look at the names of existing plugins, use the --traceconfig option.
4.8 Finding out which plugins are active

If you want to nd out which plugins are active in your environment you can type:
py.test --traceconfig
and will get an extended test header which shows activated plugins and their names. It will also print local plugins aka conftest.py les when they are loaded.
4.5. Plugin discovery order at tool startup
73
4.9 deactivate / unregister a plugin by name

You can prevent plugins from loading or unregister them:
py.test -p no:NAME
This means that any subsequent try to activate/load the named plugin will it already existing. See Finding out which plugins are active for how to obtain the name of a plugin.
74
Chapter 4. Working with plugins and conftest les
CHAPTER
FIVE
PY.TEST DEFAULT PLUGIN REFERENCE

You can nd the source code for the following plugins in the pytest repository. _pytest.assertion _pytest.capture support for presenting detailed information in failing assertions. per-test stdout/stderr capturing mechanisms, capsys and capfd function arguments. _pytest.config command line options, ini-le and conftest.py processing. _pytest.doctest discover and run doctests in modules and test les. _pytest.genscript generate a single-le self-contained version of py.test _pytest.helpconfig version info, help messages, tracing conguration. _pytest.junitxml report test results in JUnit-XML format, for use with Hudson and build integration servers. _pytest.mark generic mechanism for marking and selecting python functions. _pytest.monkeypatch monkeypatching and mocking functionality. _pytest.nose run test suites written for nose. _pytest.pastebin submit failure or test session information to a pastebin service. _pytest.pdb interactive debugging with PDB, the Python Debugger. _pytest.pytester (disabled by default) support for testing py.test and py.test plugins. _pytest.python Python test discovery, setup and run of test functions. _pytest.recwarn recording warnings during test function execution. _pytest.resultlog (disabled by default) create result information in a plain text le. _pytest.runner basic collect and runtest protocol implementations _pytest.main core implementation of testing process: init, session, runtest loop. _pytest.skipping support for skip/xfail functions and markers. _pytest.terminal terminal reporting of the full testing process. _pytest.tmpdir support for providing temporary directories to test functions. _pytest.unittest discovery and running of std-library unittest style tests.
75
76
Chapter 5. py.test default plugin reference
CHAPTER
SIX
PY.TEST HOOK REFERENCE

6.1 hook specication and validation
py.test calls hook functions to implement initialization, running, test execution and reporting. When py.test loads a plugin it validates that each hook function conforms to its respective hook specication. Each hook function name and its argument names need to match a hook specication. However, a hook function may accept fewer parameters by simply not specifying them. If you mistype argument names or the hook name itself you get an error showing the available arguments.
6.2 initialisation, command line and conguration hooks

_pytest.hookspec.pytest_cmdline_preparse(cong, args) modify command line arguments before option parsing. _pytest.hookspec.pytest_cmdline_parse(pluginmanager, args) return initialized cong object, parsing the specied args. _pytest.hookspec.pytest_namespace() return dict of name->object to be made globally available in the py.test/pytest namespace. This hook is called before command line options are parsed. _pytest.hookspec.pytest_addoption(parser) add optparse-style options and ini-style cong values via calls to parser.addoption and parser.addini(...). _pytest.hookspec.pytest_cmdline_main(cong) called for performing the main command line action. The default implementation will invoke the congure hooks and runtest_mainloop. _pytest.hookspec.pytest_configure(cong) called after command line options have been parsed. and all plugins and initial conftest les been loaded. _pytest.hookspec.pytest_unconfigure(cong) called before test process is exited.
6.3 generic runtest hooks

All all runtest related hooks receive a pytest.Item object.
77
_pytest.hookspec.pytest_runtest_protocol(item) implements the standard runtest_setup/call/teardown protocol including capturing exceptions and calling reporting hooks on the results accordingly. Return boolean True if no further hook implementations should be invoked. _pytest.hookspec.pytest_runtest_setup(item) called before pytest_runtest_call(item). _pytest.hookspec.pytest_runtest_call(item) called to execute the test item. _pytest.hookspec.pytest_runtest_teardown(item) called after pytest_runtest_call. _pytest.hookspec.pytest_runtest_makereport(item, call) return a _pytest.runner.TestReport object for _pytest.runner.CallInfo. the given pytest.Item and
For deeper understanding you may look at the default implementation of these hooks in _pytest.runner and maybe also in _pytest.pdb which interacts with _pytest.capture and its input/output capturing in order to immediately drop into interactive debugging when a test failure occurs. The _pytest.terminal reported specically uses the reporting hook to print information about a test run.
6.4 collection hooks

py.test calls the following hooks for collecting les and directories: _pytest.hookspec.pytest_ignore_collect(path, cong) return True to prevent considering this path for collection. This hook is consulted for all les and directories prior to calling more specic hooks. _pytest.hookspec.pytest_collect_directory(path, parent) called before traversing a directory for collection les. _pytest.hookspec.pytest_collect_file(path, parent) return collection Node or None for the given path. Any new node needs to have the specied parent as a parent. For inuencing the collection of objects in Python modules you can use the following hook: _pytest.hookspec.pytest_pycollect_makeitem(collector, name, obj) return custom item/collector for a python object in a module, or None.
6.5 reporting hooks

Session related reporting hooks: And here is the central hook for reporting about test execution:
78
Chapter 6. py.test hook reference
CHAPTER
SEVEN
REFERENCE OF IMPORTANT OBJECTS INVOLVED IN HOOKS

class _pytest.config.Config(pluginmanager=None) access to conguration values, pluginmanager and plugin hooks. classmethod fromdictargs(option_dict, args) constructor useable for subprocesses. getini(name) return conguration value from an ini le. If the specied name hasnt been registered through a prior parse.addini call (usually from a plugin), a ValueError is raised. getvalue(name, path=None) return name value looked set from command line options. (deprecated) if we cant nd the option also lookup the name in a matching conftest le. getvalueorskip(name, path=None) (deprecated) return getvalue(name) or call py.test.skip if no value exists. option command line option values, usually added via parser.addoption(...) or parser.getgroup(...).addoption(...) calls pluginmanager a pluginmanager instance class _pytest.config.Parser(usage=None, processopt=None) Parser for command line arguments. addini(name, help, type=None, default=None) add an ini-le option with the given name and description. addoption(*opts, **attrs) add an optparse-style option. getgroup(name, description=, after=None) get (or create) a named option Group. Name unique name of the option group. Description long description for help output. After name of other group, used for ordering help output. class _pytest.main.Node(name, parent) base class for all Nodes in the collection tree. Collector subclasses have children, Items are terminal nodes.
79
Class deprecated attribute Class, use pytest.Class File deprecated attribute File, use pytest.File Function deprecated attribute Function, use pytest.Function Instance deprecated attribute Instance, use pytest.Instance Item deprecated attribute Item, use pytest.Item Module deprecated attribute Module, use pytest.Module config the test cong object fspath lesystem path where this node was collected from listchain() return list of all parent collectors up to self, starting from root of collection tree. name a unique name with the scope of the parent parent the parent collector node. session the collection this node is part of class _pytest.runner.CallInfo(func, when) Result/Exception info a function invocation. excinfo None or ExceptionInfo object. when context of invocation: one of setup, call, teardown, memocollect class _pytest.runner.TestReport(nodeid, location, keywords, outcome, longrepr, when) Basic test report object (also used for setup and teardown calls if they fail). keywords a name -> value dictionary containing all keywords and markers associated with a test invocation. location a (lesystempath, lineno, domaininfo) tuple indicating the actual location of a test item - it might be different from the collected one e.g. if a method is inherited from a different module. longrepr None or a failure representation. nodeid normalized collection node id outcome test outcome, always one of passed, failed, skipped.
80
Chapter 7. Reference of important objects involved in hooks
when one of setup, call, teardown to indicate runtest phase.
81
82
Chapter 7. Reference of important objects involved in hooks
CHAPTER
EIGHT
TALKS AND TUTORIALS

8.1 tutorial examples and blog postings
basic usage and funcargs: pycon 2010 tutorial PDF and tutorial1 repository function arguments: mysetup pattern: application specic test xtures application setup in test functions with funcargs monkey patching done right (blog post, consult monkeypatch plugin for actual 1.0 API) test parametrization: generating parametrized tests with funcargs test generators and cached setup parametrizing tests, generalized (blog post) putting test-hooks into local or global plugins (blog post) distributed testing: simultaneously test your code on all platforms (blog entry) plugin specic examples: skipping slow tests by default in py.test (blog entry) many examples in the docs for plugins
8.2 conference talks and tutorials

ep2009-rapidtesting.pdf tutorial slides (July 2009): testing terminology basic py.test usage, le system layout test function arguments (funcargs) and test xtures existing plugins distributed testing
83
ep2009-pytest.pdf 60 minute py.test talk, highlighting unique features and a roadmap (July 2009) pycon2009-pytest-introduction.zip slides and les, extended version of py.test basic introduction, discusses more options, also introduces old-style xUnit setup, looponfailing and other features. pycon2009-pytest-advanced.pdf contain a slightly older version of funcargs and distributed testing, compared to the EuroPython 2009 slides.
84
Chapter 8. Talks and Tutorials
CHAPTER
NINE
FEEDBACK AND CONTRIBUTE TO PY.TEST

9.1 Contact channels
new issue tracker to report bugs or suggest features (for version 2.0 and above). You may also peek at the old issue tracker but please dont submit bugs there anymore. Testing In Python: a mailing list for Python testing tools and discussion. py-dev developers list pytest specic announcements and discussions. #pylib on irc.freenode.net IRC channel for random questions. private mail to Holger.Krekel at gmail com if you want to communicate sensitive issues commit mailing list merlinux.eu offers on-site teaching and consulting services.
9.2 Working from version control or a tarball

To follow development or start experiments, checkout the complete code and documentation source with mercurial:
hg clone https://bitbucket.org/hpk42/pytest/
You can also go to the python package index and download and unpack a TAR le:
http://pypi.python.org/pypi/pytest/
9.2.1 Activating a checkout with setuptools

With a working Distribute or setuptools installation you can type:
python setup.py develop
in order to work inline with the tools and the lib of your checkout. If this command complains that it could not nd the required version of py then you need to use the development pypi repository:
85
python setup.py develop -i http://pypi.testrun.org
86
Chapter 9. Feedback and contribute to py.test
CHAPTER
TEN
RELEASE ANNOUNCEMENTS
10.1 py.test 2.1.0: perfected assertions and bug xes
Welcome to the relase of pytest-2.1, a mature testing tool for Python, supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See the improved extensive docs (now also as PDF!) with tested examples here: http://pytest.org/ The single biggest news about this release are perfected assertions courtesy of Benjamin Peterson. You can now safely use assert statements in test modules without having to worry about side effects or python optimization (-OO) options. This is achieved by rewriting assert statements in test modules upon import, using a PEP302 hook. See http://pytest.org/assert.html#advanced-assertion-introspection for detailed information. The work has been partly sponsored by my company, merlinux GmbH. For further details on bug xes and smaller enhancements see below. If you want to install or upgrade pytest, just type one of:
pip install -U pytest # or easy_install -U pytest
best, holger krekel / http://merlinux.eu
10.1.1 Changes between 2.0.3 and 2.1.0

x issue53 call nosestyle setup functions with correct ordering x issue58 and issue59: new assertion code xes merge Benjamins assertionrewrite branch: now assertions for test modules on python 2.6 and above are done by rewriting the AST and saving the pyc le before the test module is imported. see doc/assert.txt for more info. x issue43: improve doctests with better traceback reporting on unexpected exceptions x issue47: timing output in junitxml for test cases is now correct x issue48: typo in MarkInfo repr leading to exception x issue49: avoid confusing error when initizaliation partially fails x issue44: env/username expansion for junitxml le path show releaselevel information in test runs for pypy reworked doc pages for better navigation and PDF generation report KeyboardInterrupt even if interrupted during session startup 87
x issue 35 - provide PDF doc version and download link from index page
10.2 py.test 2.0.3: bug xes and speed ups

Welcome to pytest-2.0.3, a maintenance and bug x release of pytest, a mature testing tool for Python, supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See the extensive docs with tested examples here: http://pytest.org/ If you want to install or upgrade pytest, just type one of:
There also is a bugx release 1.6 of pytest-xdist, the plugin that enables seemless distributed and looponfail testing for Python. best, holger krekel

x issue38: nicer tracebacks on calls to hooks, particularly early congure/sessionstart ones x missing skip reason/meta information in junitxml les, reported via http://lists.idyll.org/pipermail/testing-inpython/2011-March/003928.html x issue34: avoid collection failure with test prexed classes deriving from object. dont require zlib (and other libs) for genscript plugin without genscript actually being used. speed up skips (by not doing a full traceback represenation internally) x issue37: avoid invalid characters in junitxmls output
10.3 py.test 2.0.2: speedups
bug xes, improved xfail/skip expressions,
Welcome to pytest-2.0.2, a maintenance and bug x release of pytest, a mature testing tool for Python, supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See the extensive docs with tested examples here: http://pytest.org/ If you want to install or upgrade pytest, just type one of:
Many thanks to all issue reporters and people asking questions or complaining, particularly Jurko for his insistence, Laura, Victor and Brianna for helping with improving and Ronny for his general advise. best, holger krekel
88
Chapter 10. Release announcements

tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead x issue30 - extended xfail/skipif handling and improved reporting. If you have a syntax error in your skip/xfail expressions you now get nice error reports. Also you can now access module globals from xfail/skipif expressions so that this for example works now:
import pytest import mymodule @pytest.mark.skipif("mymodule.__version__[0] == "1") def test_function(): pass
This will not run the test function if the modules version string does not start with a 1. Note that specifying a string instead of a boolean expressions allows py.test to report meaningful information when summarizing a test run as to what conditions lead to skipping (or xfail-ing) tests. x issue28 - setup_method and pytest_generate_tests work together The setup_method xture method now gets called also for test function invocations generated from the pytest_generate_tests hook. x issue27 - collectonly and keyword-selection (-k) now work together Also, if you do py.test collectonly -q you now get a at list of test ids that you can use to paste to the py.test commandline in order to execute a particular test. x issue25 avoid reported problems with pdb and python3.2/encodings output x issue23 - tmpdir argument now works on Python3.2 and WindowsXP Starting with Python3.2 os.symlink may be supported. By requiring a newer py lib version the py.path.local() implementation acknowledges this. xed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular thanks to Laura Creighton who also revieved parts of the documentation. x slighly wrong output of verbose progress reporting for classes (thanks Amaury) more precise (avoiding of) deprecation warnings for node.Class|Function accesses avoid std unittest assertion helper code in tracebacks (thanks Ronny)
10.4 py.test 2.0.1: bug xes

Welcome to pytest-2.0.1, a maintenance and bug x release of pytest, a mature testing tool for Python, supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See extensive docs with tested examples here: http://pytest.org/ If you want to install or upgrade pytest, just type one of:
Many thanks to all issue reporters and people asking questions or complaining. Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt for their great coding contributions and many others for feedback and help. best, holger krekel
10.4. py.test 2.0.1: bug xes
89

rene and unify initial capturing so that it works nicely even if the logging module is used on an early-loaded conftest.py le or plugin. x issue12 - show plugin versions with version and tracecong and also document how to add extra information to reporting test header x issue17 (import-* reporting issue on python3) by requiring py>1.4.0 (1.4.1 is going to include it) x issue10 (numpy arrays truth checking) by rening assertion interpretation in py lib x issue15: make nose compatibility tests compatible with python3 (now that nose-1.0 supports python3) remove somewhat surprising same-conftest detection because it ignores conftest.py when they appear in several subdirs. improve assertions (not in), thanks Floris Bruynooghe improve behaviour/warnings when running on top of python -OO (assertions and docstrings are turned off, leading to potential false positives) introduce a pytest_cmdline_processargs(args) hook to allow dynamic computation of command line arguments. This xes a regression because py.test prior to 2.0 allowed to set command line options from conftest.py les which so far pytest-2.0 only allowed from ini-les now. x issue7: assert failures in doctest modules. unexpected failures in doctests will not generally show nicer, i.e. within the doctest failing context. x issue9: setup/teardown functions for an xfail-marked test will report as xfail if they fail but report as normally passing (not xpassing) if they succeed. This only is true for direct setup/teardown invocations because teardown_class/ teardown_module cannot closely relate to a single test. x issue14: no logging errors at process exit renements to collecting output on non-ttys rene internal plugin registration and tracecong output introduce a mechanism to prevent/unregister http://pytest.org/plugins.html#cmdunregister activate resultlog plugin by default x regression wrt yielded tests which due to the collection-before-running semantics were not setup as with pytest 1.3.4. Note, however, that the recommended and much cleaner way to do test parametrization remains the pytest_generate_tests mechanism, see the docs. plugins from the command line, see
10.5 py.test 2.0.0: docs++
asserts++, unittest++, reporting++, cong++,
Welcome to pytest-2.0.0, a major new release of py.test, the rapid easy Python testing tool. There are many new features and enhancements, see below for summary and detailed lists. A lot of long-deprecated code has been removed, resulting in a much smaller and cleaner implementation. See the new docs with examples here: http://pytest.org/2.0.0/index.html A note on packaging: pytest used to part of the py distribution up until version py-1.3.4 but this has changed now: pytest-2.0.0 only contains py.test related code and is expected to be backward-compatible to existing test code. If you want to install pytest, just type one of:
90
pip install -U pytest easy_install -U pytest
Many thanks to all issue reporters and people asking questions or complaining. Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt for their great coding contributions and many others for feedback and help. best, holger krekel
10.5.1 New Features

new invocations through Python interpreter and from Python:
python -m pytest # on all pythons >= 2.5
or from a python program:

import pytest ; pytest.main(arglist, pluginlist)
see http://pytest.org/2.0.0/usage.html for details. new and better reporting information in assert expressions if comparing lists, sequences or strings. see http://pytest.org/2.0.0/assert.html#newreport new conguration through ini-les (setup.cfg or tox.ini recognized), for example:
[pytest] norecursedirs = .hg data* addopts = -x --pyargs # dont ever recurse in such dirs # add these command line options by default
see http://pytest.org/2.0.0/customize.html improved standard unittest support. In general py.test should now better be able to run custom unittest.TestCases like twisted trial or Django based TestCases. Also you can now run the tests of an installed unittest package with py.test:
py.test --pyargs unittest
new -q option which decreases verbosity and prints a more nose/unittest-style dot output. many many more detailed improvements details
10.5.2 Fixes
x issue126 - introduce py.test.set_trace() to trace execution via PDB during the running of tests even if capturing is ongoing. x issue124 - make reporting more resilient against tests opening les on ledescriptor 1 (stdout). x issue109 - sibling conftest.py les will not be loaded. (and Directory collectors cannot be customized anymore from a Directorys conftest.py - this needs to happen at least one level up). x issue88 (nding custom test nodes from command line arg) x issue93 stdout/stderr is captured while importing conftest.py x bug: unittest collected functions now also can have pytestmark applied at class/module level
10.5. py.test 2.0.0: asserts++, unittest++, reporting++, cong++, docs++
91
10.5.3 Important Notes

The usual way in pre-2.0 times to use py.test in python code was to import py and then e.g. use py.test.raises for the helper. This remains valid and is not planned to be deprecated. However, in most examples and internal code youll nd import pytest and pytest.raises used as the recommended default way. pytest now rst performs collection of the complete test suite before running any test. This changes for example the semantics of when pytest_collectstart/pytest_collectreport are called. Some plugins may need upgrading. The pytest package consists of a 400 LOC core.py and about 20 builtin plugins, summing up to roughly 5000 LOCs, including docstrings. To be fair, it also uses generic code from the pylib, and the new py package to help with lesystem and introspection/code manipulation.
10.5.4 (Incompatible) Removals

py.test.cong is now only available if you are in a test run. the following (mostly already deprecated) functionality was removed: removed support for Module/Class/... collection node denitions in conftest.py les. They will cause nothing special. removed support for calling the pre-1.0 collection API of run() and join removed reading option values from conftest.py les or env variables. This can now be done much much better and easier through the ini-le mechanism and the addopts entry in particular. removed the disabled attribute in test classes. Use the skipping and pytestmark mechanism to skip or xfail a test class. py.test.collect.Directory does not exist anymore and it is not possible to provide an own Directory object. If you have used this and dont know what to do, get in contact. Well gure something out. Note that pytest_collect_directory() is still called but any return value will be ignored. This allows to keep old code working that performed for example py.test.skip() in collect() to prevent recursion into directory trees if a certain dependency or command line option is missing. see Changelog history for more detailed changes.
92
CHAPTER
ELEVEN
CHANGELOG HISTORY
11.1 Changes between 2.0.3 and 2.1.0.DEV
x issue53 call nosestyle setup functions with correct ordering x issue58 and issue59: new assertion code xes merge Benjamins assertionrewrite branch: now assertions for test modules on python 2.6 and above are done by rewriting the AST and saving the pyc le before the test module is imported. see doc/assert.txt for more info. x issue43: improve doctests with better traceback reporting on unexpected exceptions x issue47: timing output in junitxml for test cases is now correct x issue48: typo in MarkInfo repr leading to exception x issue49: avoid confusing error when initizaliation partially fails x issue44: env/username expansion for junitxml le path show releaselevel information in test runs for pypy reworked doc pages for better navigation and PDF generation report KeyboardInterrupt even if interrupted during session startup x issue 35 - provide PDF doc version and download link from index page
11.2 Changes between 2.0.2 and 2.0.3

x issue38: nicer tracebacks on calls to hooks, particularly early congure/sessionstart ones x missing skip reason/meta information in junitxml les, reported via http://lists.idyll.org/pipermail/testing-inpython/2011-March/003928.html x issue34: avoid collection failure with test prexed classes deriving from object. dont require zlib (and other libs) for genscript plugin without genscript actually being used. speed up skips (by not doing a full traceback represenation internally) x issue37: avoid invalid characters in junitxmls output
93

tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead x issue30 - extended xfail/skipif handling and improved reporting. If you have a syntax error in your skip/xfail expressions you now get nice error reports. Also you can now access module globals from xfail/skipif expressions so that this for example works now:
import pytest import mymodule @pytest.mark.skipif("mymodule.__version__[0] == "1") def test_function(): pass
This will not run the test function if the modules version string does not start with a 1. Note that specifying a string instead of a boolean expressions allows py.test to report meaningful information when summarizing a test run as to what conditions lead to skipping (or xfail-ing) tests. x issue28 - setup_method and pytest_generate_tests work together The setup_method xture method now gets called also for test function invocations generated from the pytest_generate_tests hook. x issue27 - collectonly and keyword-selection (-k) now work together Also, if you do py.test collectonly -q you now get a at list of test ids that you can use to paste to the py.test commandline in order to execute a particular test. x issue25 avoid reported problems with pdb and python3.2/encodings output x issue23 - tmpdir argument now works on Python3.2 and WindowsXP Starting with Python3.2 os.symlink may be supported. By requiring a newer py lib version the py.path.local() implementation acknowledges this. xed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular thanks to Laura Creighton who also revieved parts of the documentation. x slighly wrong output of verbose progress reporting for classes (thanks Amaury) more precise (avoiding of) deprecation warnings for node.Class|Function accesses avoid std unittest assertion helper code in tracebacks (thanks Ronny)

rene and unify initial capturing so that it works nicely even if the logging module is used on an early-loaded conftest.py le or plugin. allow to omit () in test ids to allow for uniform test ids as produced by Alfredos nice pytest.vim plugin. x issue12 - show plugin versions with version and tracecong and also document how to add extra information to reporting test header x issue17 (import-* reporting issue on python3) by requiring py>1.4.0 (1.4.1 is going to include it) x issue10 (numpy arrays truth checking) by rening assertion interpretation in py lib x issue15: make nose compatibility tests compatible with python3 (now that nose-1.0 supports python3) remove somewhat surprising same-conftest detection because it ignores conftest.py when they appear in several subdirs. improve assertions (not in), thanks Floris Bruynooghe
94
Chapter 11. Changelog history
improve behaviour/warnings when running on top of python -OO (assertions and docstrings are turned off, leading to potential false positives) introduce a pytest_cmdline_processargs(args) hook to allow dynamic computation of command line arguments. This xes a regression because py.test prior to 2.0 allowed to set command line options from conftest.py les which so far pytest-2.0 only allowed from ini-les now. x issue7: assert failures in doctest modules. unexpected failures in doctests will not generally show nicer, i.e. within the doctest failing context. x issue9: setup/teardown functions for an xfail-marked test will report as xfail if they fail but report as normally passing (not xpassing) if they succeed. This only is true for direct setup/teardown invocations because teardown_class/ teardown_module cannot closely relate to a single test. x issue14: no logging errors at process exit renements to collecting output on non-ttys rene internal plugin registration and tracecong output introduce a mechanism to prevent/unregister http://pytest.org/plugins.html#cmdunregister activate resultlog plugin by default x regression wrt yielded tests which due to the collection-before-running semantics were not setup as with pytest 1.3.4. Note, however, that the recommended and much cleaner way to do test parametraization remains the pytest_generate_tests mechanism, see the docs. plugins from the command line, see

pytest-2.0 is now its own package and depends on pylib-2.0 new ability: python -m pytest / python -m pytest.main ability new python invcation: pytest.main(args, plugins) to load some custom plugins early. try harder to run unittest test suites in a more compatible manner by deferring setup/teardown semantics to the unittest package. also work harder to run twisted/trial and Django tests which should now basically work by default. introduce a new way to set cong options via ini-style les, by default setup.cfg and tox.ini les are searched. The old ways (certain environment variables, dynamic conftest.py reading is removed). add a new -q option which decreases verbosity and prints a more nose/unittest-style dot output. x issue135 - marks now work with unittest test cases as well x issue126 - introduce py.test.set_trace() to trace execution via PDB during the running of tests even if capturing is ongoing. x issue123 - new python -m py.test invocation for py.test (requires Python 2.5 or above) x issue124 - make reporting more resilient against tests opening les on ledescriptor 1 (stdout). x issue109 - sibling conftest.py les will not be loaded. (and Directory collectors cannot be customized anymore from a Directorys conftest.py - this needs to happen at least one level up). introduce (customizable) assertion failure representations and enhance output on assertion failures for comparisons and other cases (Floris Bruynooghe) nose-plugin: pass through type-signature failures in setup/teardown functions instead of not calling them (Ed Singleton) 11.5. Changes between 1.3.4 and 2.0.0 95
remove py.test.collect.Directory (follows from a major refactoring and simplication of the collection process) majorly reduce py.test core code, shift function/python testing to own plugin x issue88 (nding custom test nodes from command line arg) rene tmpdir creation, will now create basenames better associated with test names (thanks Ronny) xpass (unexpected pass) tests dont cause exitcode!=0 x issue131 / issue60 - importing doctests in __init__ les used as namespace packages x issue93 stdout/stderr is captured while importing conftest.py x bug: unittest collected functions now also can have pytestmark applied at class/module level add ability to use class level for cached_setup helper x strangeness: mark.* objects are now immutable, create new instances

x issue111: improve install documentation for windows x issue119: x custom collectability of __init__.py as a module x issue116: doctestmodules work with __init__.py les as well x issue115: unify internal exception passthrough/catching/GeneratorExit x issue118: new tb=native for presenting cpython-standard exceptions

x issue113: assertion representation problem with triple-quoted strings (and possibly other cases) make conftest loading detect that a conftest le with the same content was already loaded, avoids surprises in nested directory structures which can be produced e.g. by Hudson. It probably removes the need to use confcutdir in most cases. x terminal coloring for win32 (thanks Michael Foord for reporting) x weirdness: make terminal width detection work on stdout instead of stdin (thanks Armin Ronacher for reporting) remove trailing whitespace in all py/text distribution les

11.8.1 New features
x issue103: introduce py.test.raises as context manager, examples:
with py.test.raises(ZeroDivisionError): x = 0 1 / x with py.test.raises(RuntimeError) as excinfo:
96
call_something() # you may do extra checks on excinfo.value|type|traceback here
(thanks Ronny Pfannschmidt) Funcarg factories can now dynamically apply a marker to a test invocation. This is for example useful if a factory provides parameters to a test which are expected-to-fail:
def pytest_funcarg__arg(request): request.applymarker(py.test.mark.xfail(reason="flaky config")) ... def test_function(arg): ...
improved error reporting on collection and import errors. This makes use of a more general mechanism, namely that for custom test item/collect nodes node.repr_failure(excinfo) is now uniformly called so that you can override it to return a string error representation of your choice which is going to be reported as a (red) string. introduce junitprex=STR option to prepend a prex to all reports in the junitxml le.
11.8.2 Bug xes / Maintenance

make tests and the pytest_recwarn plugin in particular fully compatible to Python2.7 (if you use the recwarn funcarg warnings will be enabled so that you can properly check for their existence in a cross-python manner). rene pdb: ignore xfailed tests, unify its TB-reporting and dont display failures again at the end. x assertion interpretation with the ** operator (thanks Benjamin Peterson) x issue105 assignment on the same line as a failing assertion (thanks Benjamin Peterson) x issue104 proper escaping for test names in junitxml plugin (thanks anonymous) x issue57 -f|looponfail to work with xpassing tests (thanks Ronny) x issue92 collectonly reporter and pastebin (thanks Benjamin Peterson) x py.code.compile(source) to generate unique lenames x assertion re-interp problems on PyPy, by defering code compilation to the (overridable) Frame.eval class. (thanks Amaury Forgeot) x py.path.local.pyimport() to work with directories streamline py.path.local.mkdtemp implementation and usage dont print empty lines when showing junitxml-lename add optional boolean ignore_errors parameter to py.path.local.remove x terminal writing on win32/python2.4 py.process.cmdexec() now tries harder to return properly encoded unicode objects on all python versions install plain py.test/py.which scripts also for Jython, this helps to get canonical script paths in virtualenv situations make path.bestrelpath(path) return ., note that when calling X.bestrelpath the assumption is that X is a directory.
11.8. Changes between 1.3.1 and 1.3.2
97
make initial conftest discovery ignore prexed arguments x resultlog plugin when used in an multicpu/multihost xdist situation (thanks Jakub Gustak) perform distributed testing related reporting in the xdist-plugin rather than having dist-related code in the generic py.test distribution x homedir detection on Windows ship distribute_setup.py version 0.6.13

11.9.1 New features
issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain congurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all congurations as xfail. issue102: introduce new maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the -x or exitrst option which is now equivalent to maxfail=1. Both -x and maxfail will now also print a line near the end indicating the Interruption. issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list. improve and rene letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed) You can use any combination of fsxX with the -r extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also xes issue99. make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and dont have global state.
11.9.2 Fixes / Maintenance

improved traceback presentation: - improved and unied reporting for tb=short option - Errors during test module imports are much shorter, (using tb=short style) - raises shows shorter more relevant tracebacks fulltrace now more systematically makes traces longer / inhibits cutting improve support for raises and other dynamically compiled code by manipulating pythons linecache.cache instead of the previous rather hacky way of creating custom code objects. This makes it seemlessly work on Jython and PyPy where it previously didnt. x issue96: make capturing more resilient against Control-C interruptions (involved somewhat substantial refactoring to the underlying capturing functionality to avoid race conditions). x chaining of conditional skipif/xfail decorators - so it works now as expected to use multiple @py.test.mark.skipif(condition) decorators, including specic reporting which of the conditions lead to skipping. x issue95: late-import zlib so that its not required for general py.test startup.
98
x issue94: make reporting more robust against bogus source code (and internally be more careful when presenting unexpected byte sequences)

deprecate report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of xfsX characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test. allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature. add a new pytest_ignore_collect(path, cong) hook to allow projects and plugins to dene exclusion behaviour for their directory structure - for example you may dene in a conftest.py this method:
def pytest_ignore_collect(path): return path.check(link=1)
to prevent even a collection try of any tests in symlinked dirs. new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module. extend and rene xfail mechanism: @py.test.mark.xfail(run=False) do not run the decorated test @py.test.mark.xfail(reason="...") prints the reason string in xfail summaries speciying --runxfail on command line virtually ignores xfail markers expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(...) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking (issue85) x junitxml plugin to handle tests with non-ascii output x/rene python3 compatibility (thanks Benjamin Peterson) xes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see http://bugs.jython.org/issue1491 . See pylib install documentation for how to work around. xes for handling of unicode exception values and unprintable objects (issue87) x unboundlocal error in assertionold code (issue86) improve documentation for looponfailing rene IO capturing: stdin-redirect pseudo-le now has a NOP close() method ship distribute_setup.py version 0.6.10 added links to the new capturelog and coverage plugins

rened usage and options for py.cleanup:
99
py.cleanup py.cleanup py.cleanup py.cleanup py.cleanup py.cleanup
# remove "*.pyc" and "*$py.class" (jython) files -e .swp -e .cache # also remove files with these extensions -s # remove "build" and "dist" directory next to setup.py files -d # also remove empty directories -a # synonym for "-s -d -e pip-log.txt" -n # dry run, only show what would be removed
add a new option py.test funcargs which shows available funcargs and their help strings (docstrings on their respective factory function) for a given test path display a short and concise traceback if a funcarg lookup fails early-load conftest.py les in non-dot rst-level sub directories. allows to conveniently keep and access testrelated options in a test subdir and still add command line options. x issue67: new super-short traceback-printing option: tb=line will print a single line for each failing (python) test indicating its lename, lineno and the failure value x issue78: always call python-level teardown functions even if the according setup failed. This includes renements for calling setup_module/class functions which will now only be called once instead of the previous behaviour where theyd be called multiple times if they raise an exception (including a Skipped exception). Any exception will be re-corded and associated with all tests in the according module/class scope. x issue63: assume <40 columns to be a bogus terminal width, default to 80 x pdb debugging to be in the correct frame on raises-related errors update apipkg.py to x an issue where recursive imports might unnecessarily break importing x plugin links
11.12 Changes between 1.2 and 1.1.1

moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin. new junitxml plugin: junitxml=path will generate a junit style xml le which is processable e.g. by the Hudson CI system. new option: genscript=path will generate a standalone py.test script which will not need any libraries installed. thanks to Ralf Schmitt. new option: ignore will prevent specied path from collection. Can be specied multiple times. new option: confcutdir=dir will make py.test only consider conftest les that are relative to the specied dir. new funcarg: pytestcong is the pytest cong object for access to command line args and can now be easily used in a test. install py.test and py.which with a -$VERSION sufx to disambiguate between Python3, python2.X, Jython and PyPy installed versions. new pytestcong funcarg allows access to test cong object new pytest_report_header hook can return additional lines to be displayed at the header of a test run. (experimental) allow py.test path::name1::name2::... for pointing to a test within a test collection directly. This might eventually evolve as a full substitute to -k specications. streamlined plugin loading: order is now as documented in customize.html: setuptools, ENV, commandline, conftest. also setuptools entry point names are turned to canonical namees (pytest_*) automatically skip tests that need capfd but have no os.dup
100
allow pytest_generate_tests to be dened in classes as well deprecate usage of disabled attribute in favour of pytestmark deprecate denition of Directory, Module, Class and Function nodes in conftest.py les. Use pytest collect hooks instead. collection/item node specic runtest/collect hooks are only called exactly on matching conftest.py les, i.e. ones which are exactly below the lesystem path of an item change: the rst pytest_collect_directory hook to return something will now prevent further hooks to be called. change: gleaf plugin now requires gleaf to run. Also change its long command line options to be a bit shorter (see py.test -h). change: pytest doctest plugin is now enabled by default and has a new option doctest-glob to set a pattern for le matches. change: remove internal py._* helper vars, only keep py._pydir robustify capturing to survive if custom pytest_runtest_setup code failed and prevented the capturing setup code from running. make py.test.* helpers provided by default plugins visible early - works transparently both for pydoc and for interactive sessions which will regularly see e.g. py.test.mark and py.test.importorskip. simplify internal plugin manager machinery simplify internal collection tree by introducing a RootCollector node x assert reinterpreation that sees a call containing keyword=... x issue66: invoke pytest_sessionstart and pytest_sessionnish hooks on slaves during dist-testing, report module/session teardown hooks correctly. x issue65: properly handle dist-testing if no execnet/py lib installed remotely. skip some install-tests if no execnet is available x docs, x internal bin/ script generation

introduce automatic plugin pkg_resources.iter_entry_points registration via pytest11 entrypoints via setuptools
x py.test dist-testing to work with execnet >= 1.0.0b4 re-introduce py.test.cmdline.main() for better backward compatibility svn paths: x a bug with path.check(versioned=True) for svn paths, allow % in svn paths, make svnwc.update() default to interactive mode like in 1.0.x and add svnwc.update(interactive=False) to inhibit interaction. rene distributed tarball to contain test and no pyc les try harder to have deprecation warnings for py.compat.* accesses report a correct location

adjust and improve docs
101
remove py.rest tool and internal namespace - it was never really advertised and can still be used with the old release if needed. If there is interest it could be revived into its own tool i guess. x issue48 and issue59: raise an Error if the module from an imported test le does not seem to come from the lepath - avoids same-name confusion that has been reported repeatedly merged Ronnys nose-compatibility hacks: now nose-style setup_module() and setup() functions are supported introduce generalized py.test.mark function marking reshufe / rene command line grouping deprecate parser.addgroup in favour of getgroup which creates option group add report command line option that allows to control showing of skipped/xfailed sections generalized skipping: a new way to mark python functions with skipif or xfail at function, class and modules level based on platform or sys-module attributes. extend py.test.mark decorator to allow for positional args introduce and test py.cleanup -d to remove empty directories x issue #59 - robustify unittest test collection make bpython/help interaction work by adding an __all__ attribute to ApiModule, cleanup initpkg use MIT license for pylib, add some contributors remove py.execnet code and substitute all usages with execnet proper x issue50 - cached_setup now caches more to expectations for test functions with multiple arguments. merge Jarkos xes, issue #45 and #46 add the ability to specify a path for py.lookup to search in x a funcarg cached_setup bug probably only occuring in distributed testing and module scope with teardown. many xes and changes for making the code base python3 compatible, many thanks to Benjamin Peterson for helping with this. consolidate builtins implementation to be compatible with >=2.3, add helpers to ease keeping 2 and 3k compatible code deprecate py.compat.doctest|subprocess|textwrap|optparse deprecate py.magic.autopath, remove py/magic directory move pytest assertion handling to py/code and a pytest_assertion plugin, add no-assert option, deprecate py.magic namespaces in favour of (less) py.code ones. consolidate and cleanup py/code classes and les cleanup py/misc, move tests to bin-for-dist introduce delattr/delitem/delenv methods to py.tests monkeypatch funcarg consolidate py.log implementation, remove old approach. introduce py.io.TextIO and py.io.BytesIO for distinguishing between text/unicode and byte-streams (uses underlying standard lib io.* if available) make py.unittest_convert helper script available which converts unittest.py style les into the simpler assert/direct-test-classes py.test/nosetests style. The script was written by Laura Creighton. simplied internal localpath implementation
102

xing packaging issues, triggered by fedora redhat packaging, also added doc, examples and contrib dirs to the tarball. added a documentation link to the new django plugin.

added a pytest_nose plugin which handles nose.SkipTest, setup/teardown and tries to report functions correctly. nose-style function/method/generator
capturing of unicode writes or encoded strings to sys.stdout/err work better, also terminalwriting was adapted and somewhat unied between windows and linux. improved documentation layout and content a lot added a help-cong option to show conftest.py / ENV-var names for all longopt cmdline options, and some special conftest.py variables. renamed conf_capture conftest setting to option_capture accordingly. x issue #27: better reporting on non-collectable items given on commandline (e.g. pyc les) x issue #33: added version ag (thanks Benjamin Peterson) x issue #32: adding support for incomplete paths to wcpath.status() Test prexed classes are not collected by default anymore if they have an __init__ method monkeypatch setenv() now accepts a prepend parameter improved reporting of collection error tracebacks simplied multicall mechanism and plugin architecture, renamed some internal methods and argnames
11.17 Changes between 1.0.0b9 and 1.0.0

more terse reporting try to show lesystem path relatively to current dir improve xfail output a bit
11.18 Changes between 1.0.0b8 and 1.0.0b9

cleanly handle and report nal teardown of test setup x svn-1.6 compat issue with py.path.svnwc().versioned() (thanks Wouter Vanden Hove) setup/teardown or collection problems now show as ERRORs or with big Es in the progress lines. they are reported and counted separately. dist-testing: properly handle test items that get locally collected but cannot be collected on the remote side often due to platform/dependency reasons simplied py.test.mark API - see keyword plugin documentation integrate better with logging: capturing now by default captures test functions and their immediate setup/teardown in a single stream
103
capsys and capfd funcargs now have a readouterr() and a close() method (underlyingly py.io.StdCapture/FD objects are used which grew a readouterr() method as well to return snapshots of captured out/err) make assert-reinterpretation work better with comparisons not returning bools (reported with numpy from thanks maciej jalkowski) reworked per-test output capturing into the pytest_iocapture.py plugin and thus removed capturing code from cong object item.repr_failure(excinfo) instead of item.repr_failure(excinfo, outerr)

pytest_unittest-plugin is now enabled by default introduced pytest_keyboardinterrupt hook and rened pytest_sessionnish hooked, added tests. workaround a buggy logging module interaction (closing already closed les). Thanks to Sridhar Ratnakumar for triggering. if plugins use py.test.importorskip for importing a dependency only a warning will be issued instead of exiting the testing process. many improvements to docs: - rened funcargs doc , use the term factory instead of provider - added a new talk/tutorial doc page - better download page - better plugin docstrings - added new plugins page and automatic doc generation script xed teardown problem related to partially failing funcarg setups (thanks MrTopf for reporting), pytest_runtest_teardown is now always invoked even if the pytest_runtest_setup failed. tweaked doctest output for docstrings in py modules, thanks Radomir.

renamed py.test.xfail back to py.test.mark.xfail to avoid two ways to decorate for xfail re-added py.test.mark decorator for setting keywords on functions (it was actually documented so removing it was not nice) remove scope-argument from request.addnalizer() because request.cached_setup has the scope arg. TOOWTDI. perform setup nalization before reporting failures apply modied patches from Andreas Kloeckner to allow test functions to have no func_code (#22) and to make -k and function keywords work (#20) apply patch from Daniel Peolzleithner (issue #23) resolve issue #18, multiprocessing.Manager() and redirection clash make __name__ == __channelexec__ for remote_exec code

plugin classes are removed: one now denes hooks directly in conftest.py or global pytest_*.py les.
104
added new pytest_namespace(cong) hook that allows to inject helpers directly to the py.test.* namespace. documented and rened many hooks added new style of generative tests via pytest_generate_tests hook that integrates well with function arguments.
11.22 Changes between 0.9.2 and 1.0.0b1

introduced new funcarg setup method, see doc/test/funcarg.txt introduced plugin architecuture and many new py.test plugins, see doc/test/plugins.txt teardown_method is now guaranteed to get called after a test method has run. new method: py.test.importorskip(mod,minversion) will either import or call py.test.skip() completely revised internal py.test architecture new py.process.ForkedFunc object allowing to fork execution of a function to a sub process and getting a result back. XXX lots of things missing here XXX

rened installation and metadata, created new setup.py, now based on setuptools/ez_setup (thanks to Ralf Schmitt for his support). improved the way of making py.* scripts available in windows environments, they are now added to the Scripts directory as .cmd les. py.path.svnwc.status() now is more complete and uses xml output from the svn command if available (Guido Wesdorp) x for py.path.svn* to work with svn 1.5 (Chris Lamb) x path.relto(otherpath) method on windows to use normcase for checking if a path is relative. py.tests traceback is better parseable from editors (follows the lenames:LINENO: MSG convention) (thanks to Osmo Salomaa) x to javascript-generation, py.test runbrowser should work more reliably now removed previously accidentally added py.test.broken and py.test.notimplemented helpers. there now is a py.__version__ attribute

This is a fairly complete list of changes between 0.9 and 0.9.1, which can serve as a reference for developers. allowing + signs in py.path.svn urls [39106] xed support for Failed exceptions without excinfo in py.test [39340] added support for killing processes for Windows (as well as platforms that support os.kill) in py.misc.killproc [39655] added setup/teardown for generative tests to py.test [40702] 11.22. Changes between 0.9.2 and 1.0.0b1 105
added detection of FAILED TO LOAD MODULE to py.test [40703, 40738, 40739] xed problem with calling .remove() on wcpaths of non-versioned les in py.path [44248] xed some import and inheritance issues in py.test [41480, 44648, 44655] fail to run greenlet tests when pypy is available, but without stackless [45294] small xes in rsession tests [45295] xed issue with 2.5 type representations in py.test [45483, 45484] made that internal reporting issues displaying is done atomically in py.test [45518] made that non-existing les are igored by the py.lookup script [45519] improved exception name creation in py.test [45535] made that less threads are used in execnet [merge in 45539] removed lock required for atomical reporting issue displaying in py.test [45545] removed globals from execnet [45541, 45547] refactored cleanup mechanics, made that setDaemon is set to 1 to make atexit get called in 2.5 (py.execnet) [45548] xed bug in joining threads in py.execnets servemain [45549] refactored py.test.rsession tests to not rely on exact output format anymore [45646] using repr() on test outcome [45647] added Reason classes for py.test.skip() [45648, 45649] killed some unnecessary sanity check in py.test.collect [45655] avoid using os.tmple() in py.io.fdcapture because on Windows its only usable by Administrators [45901] added support for locking and non-recursive commits to py.path.svnwc [45994] locking les in py.execnet to prevent CPython from segfaulting [46010] added export() method to py.path.svnurl xed -d -x in py.test [47277] xed argument concatenation problem in py.path.svnwc [49423] restore py.test behaviour that it exits with code 1 when there are failures [49974] dont fail on html les that dont have an accompanying .txt le [50606] xed utestconvert.py < input [50645] small x for code indentation in py.code.source [50755] x _docgen.py documentation building [51285] improved checks for source representation of code blocks in py.test [51292] added support for passing authentication to py.path.svn* objects [52000, 52001] removed sorted() call for py.apigen tests in favour of [].sort() to support Python 2.3 [52481]
106
CHAPTER
TWELVE
NEW PYTEST NAMES IN 2.0 (FLAT IS BETTER THAN NESTED)

If you used older version of the py distribution (which included the py.test command line tool and Python name space) you accessed helpers and possibly collection classes through the py.test Python namespaces. The new pytest Python module aty provides the same objects, following these renaming rules:
py.test.XYZ -> pytest.XYZ py.test.collect.XYZ -> pytest.XYZ py.test.cmdline.main -> pytest.main
The old py.test.* ways to access functionality remain valid but you are encouraged to do global renaming according to the above rules in your test code.
107
108
Chapter 12. New pytest names in 2.0 (at is better than nested)
CHAPTER
THIRTEEN
EXAMPLE: SPECIFYING AND SELECTING ACCEPTANCE TESTS

# ./conftest.py def pytest_option(parser): group = parser.getgroup("myproject") group.addoption("-A", dest="acceptance", action="store_true", help="run (slow) acceptance tests") def pytest_funcarg__accept(request): return AcceptFuncarg(request) class AcceptFuncarg: def __init__(self, request): if not request.config.option.acceptance: pytest.skip("specify -A to run acceptance tests") self.tmpdir = request.config.mktemp(request.function.__name__, numbered=True) def run(self, cmd): """ called by test code to execute an acceptance test. """ self.tmpdir.chdir() return py.process.cmdexec(cmd)
and the actual test function example:

def test_some_acceptance_aspect(accept): accept.tmpdir.mkdir("somesub") result = accept.run("ls -la") assert "somesub" in result
If you run this test without specifying a command line option the test will get skipped with an appropriate message. Otherwise you can start to add convenience and test support methods to your AcceptFuncarg and drive running of tools or applications and provide ways to do assertions about the output.
109
110
Chapter 13. example: specifying and selecting acceptance tests
CHAPTER
FOURTEEN
EXAMPLE: DECORATING A FUNCARG IN A TEST MODULE

For larger scale setups its sometimes useful to decorate a funcarg just for a particular test module. We can extend the accept example by putting this in our test module:
def pytest_funcarg__accept(request): # call the next factory (living in our conftest.py) arg = request.getfuncargvalue("accept") # create a special layout in our tempdir arg.tmpdir.mkdir("special") return arg class TestSpecialAcceptance: def test_sometest(self, accept): assert accept.tmpdir.join("special").check()
Our module level factory will be invoked rst and it can ask its request object to call the next factory and then decorate its result. This mechanism allows us to stay ignorant of how/where the function argument is provided - in our example from a conftest plugin. sidenote: the temporary directory used here are instances of the py.path.local class which provides many of the os.path methods in a convenient way.
111
112
Chapter 14. example: decorating a funcarg in a test module
PYTHON MODULE INDEX
p
pytest, 43
113
114
Python Module Index
INDEX
File (_pytest.main.Node attribute), 80 fromdictargs() (_pytest.cong.Cong class method), 79 addcall() (_pytest.python.Metafunc method), 54 addnalizer() (_pytest.python.FuncargRequest method), fspath (_pytest.main.Node attribute), 80 FuncargRequest (class in _pytest.python), 52 52 Function (_pytest.main.Node attribute), 80 addini() (_pytest.cong.Parser method), 79 function (_pytest.python.FuncargRequest attribute), 52 addoption() (_pytest.cong.Parser method), 79 addopts G conguration value, 46 (_pytest.python.FuncargRequest applymarker() (_pytest.python.FuncargRequest method), getfuncargvalue() method), 52 52 getgroup() (_pytest.cong.Parser method), 79 args (_pytest.mark.MarkInfo attribute), 67 getini() (_pytest.cong.Cong method), 79 C getvalue() (_pytest.cong.Cong method), 79 cached_setup() (_pytest.python.FuncargRequest method), getvalueorskip() (_pytest.cong.Cong method), 79 52 I CallInfo (class in _pytest.runner), 80 Class (_pytest.main.Node attribute), 79 importorskip() (in module pytest), 43 cls (_pytest.python.FuncargRequest attribute), 52 Instance (_pytest.main.Node attribute), 80 cong (_pytest.main.Node attribute), 80 Item (_pytest.main.Node attribute), 80 cong (_pytest.python.FuncargRequest attribute), 52 Cong (class in _pytest.cong), 79 K conguration value keywords (_pytest.python.FuncargRequest attribute), 52 addopts, 46 keywords (_pytest.runner.TestReport attribute), 80 minversion, 46 kwargs (_pytest.mark.MarkInfo attribute), 67 norecursedirs, 46 python_classes, 46 L python_les, 46 listchain() (_pytest.main.Node method), 80 python_functions, 46 location (_pytest.runner.TestReport attribute), 80 longrepr (_pytest.runner.TestReport attribute), 80 D delattr() (_pytest.monkeypatch.monkeypatch method), 58 M delenv() (_pytest.monkeypatch.monkeypatch method), 58 delitem() (_pytest.monkeypatch.monkeypatch method), main() (in module pytest), 43 MarkDecorator (class in _pytest.mark), 67 58 MarkGenerator (class in _pytest.mark), 67 deprecated_call() (in module pytest), 44 MarkInfo (class in _pytest.mark), 67 E minversion conguration value, 46 excinfo (_pytest.runner.CallInfo attribute), 80 Module (_pytest.main.Node attribute), 80 exit() (in module pytest), 43 module (_pytest.python.FuncargRequest attribute), 52 F monkeypatch (class in _pytest.monkeypatch), 58 fail() (in module pytest), 43 115
N
name (_pytest.main.Node attribute), 80 name (_pytest.mark.MarkInfo attribute), 67 Node (class in _pytest.main), 79 nodeid (_pytest.runner.TestReport attribute), 80 norecursedirs conguration value, 46
S
session (_pytest.main.Node attribute), 80 setattr() (_pytest.monkeypatch.monkeypatch method), 58 setenv() (_pytest.monkeypatch.monkeypatch method), 58 setitem() (_pytest.monkeypatch.monkeypatch method), 58 skip() (in module pytest), 43 syspath_prepend() (_pytest.monkeypatch.monkeypatch method), 58
O
option (_pytest.cong.Cong attribute), 79 outcome (_pytest.runner.TestReport attribute), 80
T
TestReport (class in _pytest.runner), 80
P
parent (_pytest.main.Node attribute), 80 Parser (class in _pytest.cong), 79 pluginmanager (_pytest.cong.Cong attribute), 79 pytest (module), 43 pytest_addoption() (in module _pytest.hookspec), 77 pytest_assertrepr_compare() (in module _pytest.hookspec), 49 pytest_cmdline_main() (in module _pytest.hookspec), 77 pytest_cmdline_parse() (in module _pytest.hookspec), 77 pytest_cmdline_preparse() (in module _pytest.hookspec), 77 pytest_collect_directory() (in module _pytest.hookspec), 78 pytest_collect_le() (in module _pytest.hookspec), 78 pytest_congure() (in module _pytest.hookspec), 77 pytest_ignore_collect() (in module _pytest.hookspec), 78 pytest_namespace() (in module _pytest.hookspec), 77 pytest_pycollect_makeitem() (in module _pytest.hookspec), 78 pytest_runtest_call() (in module _pytest.hookspec), 78 pytest_runtest_makereport() (in module _pytest.hookspec), 78 pytest_runtest_protocol() (in module _pytest.hookspec), 77 pytest_runtest_setup() (in module _pytest.hookspec), 78 pytest_runtest_teardown() (in module _pytest.hookspec), 78 pytest_uncongure() (in module _pytest.hookspec), 77 Python Enhancement Proposals PEP 8, 3 python_classes conguration value, 46 python_les conguration value, 46 python_functions conguration value, 46
U
undo() (_pytest.monkeypatch.monkeypatch method), 58
W
when (_pytest.runner.CallInfo attribute), 80 when (_pytest.runner.TestReport attribute), 80
X
xfail() (in module pytest), 44
R
raises() (in module pytest), 43
116
Index

Pytest

Uploaded by

Document Informationclick to expand document information

Copyright:

Available Formats

Pytest

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pytest

Uploaded by

Copyright:

Available Formats

pytest Documentation

holger krekel et alii

July 09, 2011

4.6 4.7 4.8 4.9 5 6

pytest Documentation, Release 2.1.0

Download latest version as PDF

pytest Documentation, Release 2.1.0

GETTING STARTED BASICS

pytest Documentation, Release 2.1.0

1.2 Installation and Getting Started

To check your installation has installed the correct version:

If you get an error checkout Known Installation issues.

1.2.2 Our rst test run

Thats it. You can execute the test function now:

Chapter 1. Getting started basics

pytest Documentation, Release 2.1.0

test_sample.py:5: AssertionError ========================= 1 failed in 0.02 seconds =========================

1.2.3 Asserting that a certain exception is raised

Running it with, this time in quiet reporting mode:

1.2.4 Grouping multiple tests in a class

1.2. Installation and Getting Started

pytest Documentation, Release 2.1.0

assert h in x def test_two(self): x = "hello" assert hasattr(x, check)

test_class.py:8: AssertionError 1 failed, 1 passed in 0.03 seconds

1.2.5 Going functional: requesting a unique temporary directory

Chapter 1. Getting started basics

pytest Documentation, Release 2.1.0

----------------------------- Captured stdout -----------------------------/tmp/pytest-10/test_needsfiles0 1 failed in 0.13 seconds

1.2.6 where to go next

1.2.7 Known Installation issues

1.3 Usage and Invocations

1.3. Usage and Invocations

pytest Documentation, Release 2.1.0

1.3.2 Getting help on version, option names, environment variables

1.3.3 Stopping after the rst (or N) failures

1.3.4 specifying tests / selecting tests

1.3.5 Modifying Python traceback printing

1.3.6 Dropping to PDB (Python Debugger) on failures

Chapter 1. Getting started basics

pytest Documentation, Release 2.1.0

1.3.7 Setting a breakpoint / aka set_trace()

# invoke PDB debugger and tracing

1.3.8 creating JUnitXML format les

to create an XML le at path.

1.3.9 creating resultlog format les

1.3.10 send test report to pocoo pastebin service

Currently only pasting to the http://paste.pocoo.org service is implemented.

1.3.11 calling pytest from Python code

1.3. Usage and Invocations

pytest Documentation, Release 2.1.0

You can specify additional plugins to pytest.main:

Running it will exit quickly:

1.4 Good Integration Practises

1.4.2 Use tox and Continuous Integration servers

1.4.3 Create a py.test standalone Script

1.4.4 Integrating with distutils / python setup.py test

Chapter 1. Getting started basics

pytest Documentation, Release 2.1.0