Python Assertion and Unit Testing

In Debug > Settings Wheel > Configurations you can review your
config and edit the launch.json file. For example add arguments.

Python – VS Code debug

 Python – VS Code debug
{

• launch.json with custom args

"name": "Listen for XDebug",
"type": "php",
"request": "launch",
"port": 9000
{
},
// Use IntelliSense to learn about possible attributes.
{
// Hover to view descriptions of existing attributes.
"name": "Launch currently open script",
// For more information, visit:
"type": "php",
// https://go.microsoft.com/fwlink/?linkid=830387
"request": "launch",
"version": "0.2.0",
"program": "${file}",
"configurations": [
"cwd": "${fileDirname}",
{
"port": 9000
"name": "Python: Current File",
},
"type": "python",
{
"request": "launch",
"name": "PowerShell: Launch Current File",
"program": "${file}",
"type": "PowerShell",
"console": "integratedTerminal",
"request": "launch",
"cwd": "${fileDirname}",
"script": "${file}",
"args": [
"cwd": "${file}"
"--movie=flowers_google/flowers.mp4"
}
]
]
},
}
 Essential VS Code extensions
and keep your Python up to date
• VS Code: https://code.visualstudio.com/docs/python/python-tutorial
• To enable VS Code linting (a tool that analyzes source code to flag programming
errors, bugs, stylistic errors, suspicious constructs, etc.)

$ pip install pylint | To force a specific package version: $ pip install numpy==1.19.0

To list currently installed packages and versions $ pip list
• To upgrade all local packages use pip-review (from elevated console)

$ pip install pip-review

$ pip-review --local --interactive
 VS Code Settings (JSON)

To make various things work as it should in VS Code you may need to edit the View >
Command Palatte > Preferences: Open Settings (JSON) > settings.json file

For example to get Python to work correct with intellisense and pylint (in this case the
package cv2)

1. In VScode: CTRL + Shift + P

2. Choose "Preferences: Open Settings (JSON)"

3. Add the settings below to the settings JSON file
{
"python.linting.pylintEnabled": true,
"python.linting.enabled": true,
"python.linting.pylintArgs": ["--extension-pkg-whitelist=cv2"],
...
}


To make other essential adjustments search for: vscode python ”Open Settings
(JSON)” and your topic, program language etc.

More info at: https://code.visualstudio.com/docs/getstarted/tips-and-tricks
 Python Assert statements
• Assertions are statements that assert or
state a fact confidently in your program
• Assertions are simply boolean
expressions that checks if the
conditions return true or false
• If it is true, the program does nothing
and move to the next line of code. If it's
false, the program stops and throws an
error
• It is also a debugging tool as it brings
the program to halt as soon as any error
have occurred and shows the location
where in the program the error occurred
 Python assert statement example
• The condition is supposed to be always true. If the condition is false assert halts
the program and gives an AssertionError: assert <condition>, <error_message>
def calc_factorial(num): def test_negative_numbers_return_false():
if isinstance(num, str): # type(num) is str: checkcalc_factorial(-1, False, "test_negative_numbers_return_false")
print("Sorry, factorial does not exist for string input") def test_non_integer_return_false():
return False checkcalc_factorial(0.5, False, "test_non_integer_return_false")
elif num < 0: def test_when_input_is_zero_return_one():
print("Sorry, factorial does not exist for negative numbers") checkcalc_factorial(0, 1, "test_when_input_is_zero_return_one")
return False def test_when_input_is_three_teturn_six():
elif int(num) != num: checkcalc_factorial(3, 6, "test_when_input_is_three_return_six")
print("Sorry, factorial does not exist for real numbers") def test_string_input_return_false():
return False checkcalc_factorial('t', False, "test_string_input_return_false")
elif num == 0: if __name__ == '__main__':
print("The factorial of 0 is 1") try:
return 1 """
else: # change the value for a different result
factorial = 1 num = 7
for i in range(1, num + 1): # uncomment to take input from the user
factorial = factorial * i num = int(input("Enter a number: "))
print("The factorial of", num ,"is", factorial) calc_factorial(num):
return factorial """

def checkcalc_factorial(num, expected_value, assert_error):

# test code
test_negative_numbers_return_false()
assert_factorial.py
'''Test code below this line''' test_non_integer_return_false()
ret_val = calc_factorial(num) test_when_input_is_zero_return_one()
# assert <condition>, <error message> test_when_input_is_three_return_six()
# if condition is not satisfied (true) program will stop and throw an AssertionError test_string_input_return_false()
assert ret_val == expected_value, assert_error except AssertionError as ex:
print(f"{assert_error}: {ret_val == expected_value} ... OK") print(f"AssertionError: {ex}") # if assert condition is true
 Unit Testing with Python 1
• Python have several of testing frameworks available
– One is unittest, others are doctest, Nose and pytest
• unittest is inspired from JUnit and has a similar flavor as major unit testing
frameworks in other languages
– It is organized around test case classes which contain test case methods
– Naming follows Java camelCase in contrast to Pythons snake_case

PEP 8 style guide: https://www.python.org/dev/peps/pep-0008/
• unittest supports
– test automation,
– sharing of setup and shutdown code for tests,
– aggregation of tests into collections,
– and independence of the tests from the reporting framework
• Documentation
– https://docs.python.org/3/library/unittest.html
 Unit Testing with Python 2
• unittest supports some important concepts in an object-oriented way
• test fixture
– A test fixture represents the preparation needed to perform one or more tests, and any
associate cleanup actions. This may involve, for example, creating temporary or proxy
databases, directories, or starting a server process
• test case
– A test case is the individual unit of testing. It checks for a specific response to a particular
set of inputs. unittest provides a base class, unittest.TestCase, which may be used to
create new test cases
• test suite
– A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests
that should be executed together
• test runner (or test driver)
– A test runner is a component which orchestrates the execution of tests and provides the
outcome to the user. The runner may use a graphical interface, a textual interface, or
return a special value to indicate the results of executing the tests
App testing (black box) vs. unit testing (white box)
 Unit testing general
• Test methods recommended naming convention

(Test)_MethodToTest_ScenarioWeTest_ExpectedBehaviour

In the test method the pattern we use is ”tripple A”

Arrange, Act and Assert # Python
// In C# with MSTest, Nunit, xUnit class RoundtripCheck(unittest.TestCase):
[TestClass], [TestFixture], def test_to_roman_roundtrip_return_equal(self):
public class ReservationsTests
{ # The arrangement below is called tripple A
[TestMethod], [Test], [Fact] # Arrange - here we initialize our objects
public void CanBeCancelledBy_AdminCancelling_ReturnsTrue()
{ integer = known_values[1]
// The arrangement below is called tripple A # Act - here we act on the objects
// Arrange - here we initialize our objects numeral = to_roman(integer)
var reservation = new Reservation(); result = from_roman(numeral)
// Act - here we act on the objects # Assert - here we verify the result
var result = reservation.CanBeCancelledBy( self.assertEqual(integer, result)
new User { IsAdmin = true });
if __name__ == '__main__':
// Assert - here we verify the result unittest.main(argv=['first-arg-is-ignored'],
Assert.IsTrue(result);
} exit=False)
 unittest most common assert methods
https://docs.python.org/3/library/unittest.html#assert-methods
• Usually all assertions also take an optional message argument
• Template: self.assert***(first_arg, second_arg, msg=None) # msg is for error info etc.
 A very limited Python unittest example
factorial.py import unittest
import factorial
import sys
# derivation from base class is necessary -> unittest.TestCase
# make ZeroDivisionError pass # to run via console: python -m unittest --v
class MyZeroDivisionError(ValueError): class TestFactorial(unittest.TestCase):
pass test_fact_inp = (0, 1, 2, 4, 5, 6, 10,)
test_fact_out = (1, 1, 2, 24, 120, 720, 3628800,)
def fact(n):
def test_factorial_return_equal(self):
""" Factorial function, arg n: Number
""" Testing fact as res = fact(5)
returns: factorial of n """ self.assertEqual(res, 120) """
if n == 0: for inp, out in zip(self.test_fact_inp, self.test_fact_out):
return 1 result = factorial.fact(inp)
return n * fact(n - 1) message = f'factorial inp: {inp} and out: {out} gives an error message!'
self.assertEqual(out, result, message)
def div(n):
class TestDiv(unittest.TestCase):
""" Just divide """
@classmethod
if n == 0:
raise MyZeroDivisionError('division by zero!')
def setUpClass(cls): test_factorial.py
print('setupClass')
res = 10 / n
return res @classmethod
def tearDownClass(cls):
def main(n): print('\nteardownClass')
print(fact(n))
print(div(n)) def test_div_return_ZeroDivisionError(self):
""" Test exception raise due to run time error in the div function """
if __name__ == '__main__': self.assertRaises(factorial.MyZeroDivisionError, factorial.div, 0)
if len(sys.argv) > 1:
if __name__ == '__main__':
main(int(sys.argv[1])) unittest.main(argv=['first-arg-is-ignored'], exit=False)
BaseException
Python exceptions PermissionError
ProcessLookupError
Exception TimeoutError
ArithmeticError ReferenceError
FloatingPointError RuntimeError
OverflowError
ZeroDivisionError Read More! NotImplementedError
RecursionError
AssertionError StopIteration
AttributeError StopAsyncIteration
BufferError SyntaxError
EOFError https:// IndentationError
TabError
ImportError
ModuleNotFoundError julien.danjou.info/ SystemError
LookupError TypeError
IndexError python-exceptions- ValueError
UnicodeError
KeyError
MemoryError
guide/ UnicodeDecodeError
NameError UnicodeEncodeError
UnboundLocalError UnicodeTranslateError
OSError https://airbrake.io/ Warning
BytesWarning
BlockingIOError
ChildProcessError blog/python- DeprecationWarning
ConnectionError FutureWarning
BrokenPipeError exception-handling/ ImportWarning
PendingDeprecationWarning
ConnectionAbortedError
ConnectionRefusedError
class-hierarchy ResourceWarning
ConnectionResetError RuntimeWarning
FileExistsError SyntaxWarning
FileNotFoundError UnicodeWarning
InterruptedError UserWarning
IsADirectoryError GeneratorExit
NotADirectoryError KeyboardInterrupt
PermissionError SystemExit
 AssertRaises etc.
https://docs.python.org/3/library/unittest.html#assert-methods
• It is also possible to check the production of exceptions, warnings, and log
messages using the following assert methods

• Template: self.assert***(exception, callable/fun, *args, **keywords)

self.assertRaises(factorial.MyZeroDivisionError, factorial.div, 0)
 assertRaises and keywords with 1
• self.assertRaises(basic1.MyTypeError, basic1.my_split, 'hello world', 2)
• assertRises() test that a specific exception is raised when callable/fun is called with
any extra keyword arguments

The test

Passes if the specific exception is raised

Is an error if another exception is raised

Fails if no exception is raised
• AssertRises() can also return a context manager by using the with statement

Context managers are a way of allocating and releasing some sort of resource
exactly when/where you need it. Example using with and file access
with open('file_to_write.txt', 'w') as fh_cm:
fh_cm.write('file contents')
 assertRaises with keywords with 2
• assertRaises(exception, *, msg=None) using keyword with
• If a callable/fun argument is not provided assertRaises returns an optional context
manager which can be used to access exception details
• Used as a context manager, assertRaises fails if associated with body does not raise
# alternative call with an optional contex manager
with self.assertRaises(basic1.MyTypeError) as cm:
basic1.my_split('hello world', 2)
# (cm) that also can perform additional checks on the exception raised
self.assertEqual('Input separator must be a string', cm.exception.args[0])

• When used as a context manager, assertRaises() accepts the additional keyword

argument msg
• The context manager will store the caught exception object in its exception attribute
• This can be useful if the intention is to perform additional checks on the exception
raised
 AssertRaisesRegex etc.
• assertRaisesRegex can match a string representation of a raised exception

Template: self.assertRaisesRegex(exception, regex, callable/fun, *args, **keywords)

regex may be a regular expression object or a string - equivalent to a regex.search()
'''split should raise error with non-string input separator and if regex does not match'''
self.assertRaisesRegex(basic1.MyTypeError, 'Input', basic1.my_split, 'hello world', 2)
# alternative call with an optional contex manager
with self.assertRaisesRegex(basic1.TypeError, 'Input') as cm:
basic1.my_split('hello world', 2)
self.assertEqual('Input', cm.expected_regex.pattern)

• assertWarns(warning, callable, *args, **kwds), assertWarns(warning, *,

msg=None) and assertWarnsRegex(***) works in similar ways
• assertLogs(logger=None, level=None)

A context manager to test that at least one message is logged on the logger or one
of its children, with at least the given level
 unittest more specific assert methods
• More specific and type specific
asserts methods – all take at least
an optional message argument
 A Python unittest example 2
'''basic1.py reworked example From:
import basic1, unittest
https://docs.python.org/3/library/unittest.html
'''test_basic1.py reworked example from: https://docs.python.org/3/library/unittest.html
To run it: python basic1.py''' To run it: python -m unittest --v Here is a short script to test three string methods:'''
import sys class TestStringMethods(unittest.TestCase):
# my class definition of the TypeError exception def test_upper(self): # test case methods must start with test
# Calling reserved word pass does nothing self.assertEqual(basic1.my_upper('foo'), 'FOO')
class MyTypeError(ValueError): def test_isupper(self):
pass self.assertTrue(basic1.my_isupper('FOO'))
def my_upper(my_str):
return my_str.upper() basic1.py
self.assertFalse(basic1.my_isupper('Foo'))
def test_split(self):
test_basic1.py
def my_isupper(my_str):
ss = 'hello world'
self.assertEqual(basic1.my_split(ss, ' '), ['hello', 'world'])
return my_str.isupper()
def test_non_string_split(self):
def my_split(my_str, sep):
self.assertRaises(basic1.MyTypeError, basic1.my_split, 'hello world', 2)
# check if separator is a string # alternative call with an optional contex manager
if not isinstance(sep, str): with self.assertRaises(basic1.MyTypeError) as cm:
raise MyTypeError('''Input separator must be a string''') basic1.my_split('hello world', 2)
# (cm) that also can perform additional checks on the exception raised
return my_str.split(sep)
self.assertEqual('Input separator must be a string', cm.exception.args[0])
try: def test_non_string_split_regex(self):'
ss = input("Enter a string: ") self.assertRaisesRegex(basic1.MyTypeError, 'Input', basic1.my_split, 'hello world', 2)
print(f"my_upper: {my_upper(ss)}") # alternative call with an optional contex manager
print(f"my_isupper: {my_isupper(ss)}") with self.assertRaisesRegex(basic1.MyTypeError, 'Input') as cm:
print(f"my_split: {my_split(ss, ' ')}") basic1.my_split('hello world', 2)
# (cm) that also can perform additional checks on the exception raised
print(f"my_split: {my_split(ss, 2)}")
self.assertEqual('Input', cm.expected_regex.pattern)
except BaseException as ex:
print(f"TypeError: {ex}") if __name__ == '__main__':
sys.exit(0) unittest.main(argv=['first-arg-is-ignored'], exit=False)
 Unit Testing in practice 1
• Good unit tests
– Are fully automated, i.e. write code to test code
– Offer good coverage of the code under test, including boundary cases and error handling
paths
– Are easy to read and maintain – acts like some documentation for the source code
– Express the intent of the code under test – test cases do more than just check it
– Enables refactoring and updates of the code with confidence
• Not so good unit tests
– Monolitic tests: all test cases in a single function
• Ex. test_foo()
– Ad hoc tests: test cases are scattered across test functions
• Ex. test_1(), test_2(), ...
– Procedural tests: test cases bundled into a test method that directly correspond to a target
method (as in the basic1 code example with isupper())
• Ex. test_foo() → (is testing the function) foo()
 Unit Testing in practice 2
• A test is not a unit test if ...
– It talks to the database
– It communicates across the network
– It touches the file system
– It cannot run at the same time as any of your other unit tests
– You have to do special things to your environment (such as editing config files)
to run it
– https://www.artima.com/weblogs/viewpost.jsp?thread=126923
• Tests that do these things aren't bad. Often they are worth writing, and they
can be written in a unit test harness (as part of a specially prepared test
environment needed to execute the test)
• However, it is important to be able to separate them from true unit tests so
that we can keep a set of tests that we can run fast whenever we make our
changes
 Code coverage
• Coverage measurement is typically used to gauge the effectiveness of tests.
It can show which parts of your code are being executed (covered) by the
test suite, and which are not
• Function coverage – Has each function (or subroutine) in the program been called?
• Statement coverage – Has each statement in the program been executed?
• Edge coverage – has every edge (arrow) in the Control Flow Graph been executed?

Branch coverage – Has each branch (also called Decision-to-Decision path) of
each control structure (such as in if and case statements) been executed? For
example, given an if statement, have both the true and false branches been
executed? Notice that this is a subset of Edge coverage
• Condition coverage – Has each Boolean sub-expression
evaluated both to true and false?

https://en.wikipedia.org/wiki/Code_coverage
https://en.wikipedia.org/wiki/Control-flow_graph
 Coverage.py
• Coverage.py is a tool for measuring code
coverage of Python programs. It monitors your
program, noting which parts of the code have
been executed, then analyzes the source to
identify code that could have been executed but
was not
• Note that high coverage numbers does not mean
that your code is clean from bugs!
# install
$ pip install coverage
$ coverage help
usage: coverage <command> [options] [args]
# run your test code (.coverage is created) PS C:\python_unittesting> coverage report -m
$ coverage run --branch test_factorial.py Name Stmts Miss Branch BrPart Cover Missing

# report to console (from .coverage file) ---------------------------------------------------------------

factorial.py 19 8 8 2 56% 29-30, 33-36, 39-40, 27->29, 38->39
$ coverage report -m
test_factorial.py 14 0 4 1 94% 23->exit
# report to html (htmlcov folder is created)
---------------------------------------------------------------
$ coverage html TOTAL 33 8 12 3 71%
Recommended viewing and reading
• Python Tutorial: Unit Testing Your Code with the unittest Module

https://www.youtube.com/watch?v=6tNS--WetLI
• Dive into Python 3

https://github.com/diveintomark/diveintopython3
• Python 3.x unittest documentation

https://docs.python.org/3/library/unittest.html
• Code Coverage

https://coverage.readthedocs.io

https://en.wikipedia.org/wiki/Code_coverage
• Martin Fowler on TestCoverage

https://martinfowler.com/bliki/TestCoverage.html