Python Testing and Continuous Integration: Glossary

Tests check whether the observed result, from running the code, is what was expected ahead of time.

Tests should ideally be written before the code they are testing is written, however some tests must be written after the code is written.

Assertions and exceptions are like alarm systems embedded in the software, guarding against exceptional bahavior.

Unit tests try to test the smallest pieces of code possible, usually functions and methods.

Integration tests make sure that code units work together properly.

Regression tests ensure that everything works the same today as it did yesterday.

Assertions are one line tests embedded in code.

The assert keyword is used to set an assertion.

Assertions halt execution if the argument is false.

Assertions do nothing if the argument is true.

The numpy.testing module provides tools numeric testing.

Assertions are the building blocks of tests.

Exceptions are effectively specialized runtime tests

Exceptions can be caught and handled with a try-except block

Many built-in Exception types are available

Design by Contract is a way of using Assertions for interface specification.

Pre-conditions are promises you agree to obey when calling a function.

Post-conditions are promises a function agrees to obey returning to you.

Functions are the atomistic unit of software.

Simpler units are easier to test than complex ones.

A single unit test is a function containing assertions.

Such a unit test is run just like any other function.

Running tests one at a time is pretty tedious, so we will use a framework instead.

The pytest command collects and runs tests starting with Test or test_.

. means the test passed

F means the test failed or erred

x is a known failure

s is a purposefully skipped test

Functions often fail at the edge of their range of validity

Edge case tests query the limits of a function’s behavior

Corner cases are where two edge cases meet

Integration tests interrogate the coopration of pieces of the software

Regression tests use past behavior as the expected result

Servers exist for automatically running your tests

Running the tests can be triggered by a GitHub pull request

CI allows cross-platform build testing

A .travis.yml file configures a build on the travis-ci servers

Many free CI servers are available

Test driven development is a common software development technique

By writing the tests first, the function requirements are very explicit

TDD is not for everyone

TDD requires vigilance for success

It may be necessary to set up “fixtures” composing the test environment.