ITECH7409: Software Testing

Lecture 4. Unit Testing. Part 1.

This lecture
In this lecture:
• Unit Testing
• Identifying Test Cases
• Boundary-Value Testing
• Equivalence Class Testing
Unit Testing: What it is?
Unit testing is the process of testing individual components in isolation of
other units.
There are several interpretations of what exactly constitutes a unit. Usually,
units are defined by teams/organizations that implement the code.
Let’s give examples:
• In a procedural language, a unit can be:
• A single procedure,
• A function,
• The smallest body of code that can be compiled and executed by itself etc.
• In an object-oriented language, it can be:
• A method within an object,
• A class,
• A group of logically connected classes etc.
Unit Testing: Identifying Test Cases
Two fundamental approaches are used to identify test cases:

1. Black-box testing also called functional or specification-based testing.

• The only information used is the specification of the software.
Therefore, the test cases have two distinct advantages:
• they are independent of how the software is implemented, so if the
implementation changes, the test cases are still useful;
• test case development can occur in parallel with the
implementation.
• On the negative side, specification-based test cases frequently suffer
from redundancies among test cases.
Unit Testing: Identifying Test Cases
Two fundamental approaches are used to identify test cases:

2. White-box testing also called structural or code-based testing.

• The implementation is known and used to identify test cases. This
allows the tester to identify test cases based on how the function is
implemented.
• Code-based testing is used for the definition and use of test coverage
metrics.
• Test coverage metrics provide a way to explicitly state the extent to
which a software item has been tested, which makes testing
management more meaningful.
More about Black-box Testing
In the rest of this lecture, we will focus on black-box testing , which:

• Complements white-box testing by uncovering different classes of

errors.
• Focuses on the functional requirements and the information domain
of the software.
• Is used during the later stages of testing after white box testing has
been performed.
• Aims to identify a set of input conditions that fully exercise all
functional requirements for a program:
More about Black-box Testing
Questions usually answered by Black-box testing:

• How is functional validity tested?

• How are system behavior and performance tested?
• What classes of input will make good test cases?
• Is the system particularly sensitive to certain input values?
• How are the boundary values of a data class isolated?
• What data rates and data volume can the system tolerate?
• What effect will specific combinations of data have on system
operation?
Black-box: Boundary Value Testing
Boundary Value Testing is the well-known black-box testing technique based
on the idea that errors tend to occur near the extreme values of an input
variable.
There are two independent considerations that apply to input domain
testing:
• The first asks whether or not we are concerned with invalid values of the
input variables.
• The second consideration is whether we make the “single fault”
assumption common to reliability theory.
Black-box: Boundary Value Testing
Taken together, the two considerations yield four variations of boundary
value testing:
• Normal boundary value testing – only valid extreme values are considered.
Single fault assumption is made.
• Robust boundary value testing – both valid and invalid extreme values are
considered. Single fault assumption is made.
• Worst-case boundary value testing – only valid extreme values are
considered. Multiple faults are considered.
• Robust worst-case boundary value testing – both valid and invalid extreme
values are considered. Multiple faults are considered.
Boundary Value Testing: Example 1
Let’s assume we want to test a function written in Python
def ITECH7409StudentGrade(lab_test, a1, a2, exam)
The function takes marks of four assessment tasks in ITECH7409 and calculates
the corresponding grade (for details see the unit description). The marks
satisfy the following constraints : .
Grade calculation is based on the value of :
• If , the grade is “F”,
• If , the grade is “MF”,
• If , the grade is “P”,
• If , the grade is “C”,
• If , the grade is “D”,
• If, the grade is “HD”.
Example 1: Normal Boundary Value Testing
The basic idea of normal boundary value testing is to use input variable
values at their:
• minimum,
• just above the minimum,
• a nominal value,
• just below their maximum, and
• at their maximum.

Please note, that single fault assumption is made.

Example 1: Normal Boundary Value Testing
Cases lab_test a1 a2 exam Expected
Output
1 0 15 15 15 MF
2 1 15 15 15 MF
3 9 15 15 15 P
4 10 15 15 15 P
5 5 0 15 15 F
6 5 1 15 15 F
7 5 29 15 15 C
8 5 30 15 15 C
Example 1: Normal Boundary Value Testing
Cases lab_test a1 a2 exam Expected
Output
9 5 15 0 15 F
10 5 15 1 15 F
11 5 15 29 15 C
12 5 15 30 15 C
13 5 15 15 0 F
14 5 15 15 1 F
15 5 15 15 29 C
16 5 15 15 30 C

Note. This form of testing is focused on the input domain; however, it would be a good
supplement to apply this technique to develop range-based test cases as well.
Example 1: Robust Normal Boundary Value
Testing
The basic idea of robust normal boundary value testing is to use input
variable values at:
• just below the minimum,
• minimum,
• just above the minimum,
• a nominal value,
• just below their maximum,
• at their maximum,
• just above their maximum.
Again, single fault assumption is made.
It is not difficult to see that the number of cases will increase to 24.
Example 1: Worst-Case and Robust Worst-Case Boundary Value
Testings
• Rejection of single fault assumption significantly increases the number of
test cases.
• In Worst-Case Boundary Value Testing all combinations of 5 values for
each of the variables (minimum, just above the minimum, a nominal
value, just below their maximum, maximum) should be considered. We
also rule out the case where all variables have nominal values. This
gives cases.
• For Robust Worst-Case Boundary Value Testing the number of cases
will be .
Exercise 1. List Robust Normal Boundary Value Test cases for the Example 1.
Exercise 2. Show that some of 624 cases for Worst-Case Boundary Value Testing for
the Example 1 coincide (redundancy).
Limitations of Boundary Value Analysis
• Boundary value analysis works well when the program to be tested is a
function of several independent variables.
• This means that the function’s domain is a Cartesian product of the
domains of each of its parameters. E.g., in the Example 1 function
ITECH7409StudentGrade(lab_test, a1, a2, exam)
is defined on the set

• However, as you will see in the next example, even independency of the
parameters doesn’t guarantee that all “crucial” cases will be analysed in
Boundary Value Analysis.
Exercise 3. In the textbook find definition of Cartesian product, read and
understand it. In case you have questions, ask your tutor for assistance.
Limitations of Boundary Value Analysis: Example
2
Let’s assume we want to apply a boundary value analysis to a function
def is_ITECH7409_D_Student (lab_test, a1, a2, exam)
that takes marks of four assessment tasks in ITECH7409, calculates the grade,
and if the grade is D, returns True, otherwise, returns False.
It is not difficult to see that boundary value analysis is of little value in testing
this function:
• Expected outputs for the cases in the Normal Boundary Value Testing table
are all below C. There are no D’s.
• Most of the “interesting” cases consist of “nominal” (non-boundary) values.
Exercise 4. Explain, why in testing function, is_ITECH7409_D_Student, it is
important to consider cases consisting of non-boundary values of the
parameters.
Limitations of Boundary Value Analysis
Overall, boundary value analysis has the following significant limitations:
• Parameters of the tested function should be of comparable (say, numerical)
types.
• There could be significant redundancies in boundary value analysis, where
different cases are essentially the same or very close in meaning.
• Also, usually boundary value analysis doesn’t cover all suspicious cases,
i.e., there could be huge gaps in test coverage.

Boundary value analysis test cases are rudimentary because they are derived
with no consideration of the nature of the function, nor of the semantic
meaning of the variables.
Mathematical Detour: Relations
Definition Given two sets A and B, a relation R is a subset of the Cartesian
product A × B.
In other words, a relation R on the set A × B is a subset of the set of all
ordered pairs from A × B.
Two notations are popular:
• when we wish to speak about the entire relation, we usually just write R⊆
A × B.
• for specific elements a ∈ A, b ∈ B, we write a R b if (a,b) ⊆ A × B.

Examples of relations: a = b, a < b, a ≠ b etc.

Some Types of Relations
Definition A relation R ⊆ A × A is
• Reflexive iff for all a ∈ A, (a,a)∈ R
• Symmetric iff (a,b)∈ R ⇒ (b,a)∈ R
• Antisymmetric iff (a,b), (b,a) ∈ R ⇒ a = b
• Transitive iff (a,b),(b,c) ∈ R ⇒ (a,c)∈ R

Examples:
• “=“ is reflexive, symmetric and transitive.
• “>” is antisymmetric and transitive.
• “≠” is symmetric.
Equivalence Relations and Partitions of Sets
Definition A relation R ⊆ A × A is an equivalence relation if R is reflexive,
symmetric, and transitive.
Definition Given a set A, and a set of subsets A1, A2, …, An of A, the subsets
are a partition of A iff A1 ∪ A2 ∪ … ∪ An = A, and i ≠ j ⇒ Ai ∩ Aj = ∅.

Example of a set partition

Equivalence Relations and Partitions of Sets
A very important connection exists between equivalence relations and
partitions of a set:
• Suppose we have some partition A1, A2, …, An of a set B, and we say that
two elements, b1 and b2 of B, are related (i.e., b1 R b2) if b1 and b2 are in the
same partition element.
• This relation is:
• Reflexive: any element is in its own partition,
• Symmetric: if b1 and b2 are in a partition element, then b2 and b1 are,
• Transitive: if b1 and b2 are in the same set, and if b2 and b3 are in the same set, then
b1 and b3 are in the same set.
• The relation defined from the partition is called the equivalence relation
induced by the partition.
Equivalence Relations and Partitions of Sets
• Conversely, if we start with an equivalence relation defined on a set, we
can define subsets according to elements that are related to each other.
• This turns out to be a partition and is called the partition induced by the
equivalence relation.
• The sets in this partition are known as equivalence classes.
• This result shows that partitions and equivalence relations are
interchangeable.
• The proof of the above result is almost trivial; however, we are not going to
discuss it in this course.
Exercise 5. Read the parts of the textbook where set partitions and relations
are discussed. In case you have questions, ask your tutor for assistance.
Equivalence Class Testing
• The use of equivalence classes as the basis for functional testing has two
motivations:
• to have a sense of complete testing (no gaps),
• to avoid redundancy.
• The idea is to introduce an equivalence relation on the set of all inputs to a
tested function that adequately reflects similarity of cases.
• Then we can proceed by testing one element from each equivalence class
and assuming that the remaining elements will behave similarly .
• It is not an easy task to find a suitable equivalence relation, and may
require lots of expertise, insight and imagination.
Equivalence Class Testing
As it was the case with the boundary value testing, equivalence class testing
has two deciding factors:
• robustness,
• single/multiple fault assumption.
Based on the two assumptions, there are four distinct forms of equivalence
class testing:
• Weak Normal – just a single fault assumption;
• Strong Normal – just a multiple fault assumption;
• Weak Robust – robustness + single fault assumption;
• Strong Robust – robustness + multiple fault assumption.
Equivalence Class Testing: Example 1
Revisited
Let’s see how equivalence class testing can be used in example 1:
• First, we need to introduce suitable partition (equivalence relation) on the
set of all inputs to the function ITECH7409StudentGrade(lab_test, a1, a2,
exam). Recall, that we defined auxiliary variable . We’ll be using it to define
partitions.
• The function’s definition suggests the following partition:
• ,
• ,
• ,
• If ,
• If ,
• If ,
• If,
• 10
Example 1: Equivalence Class Testing
Cases lab_test a1 a2 exam total_mark Expected
Output
1 -1 0 0 0 -1 N/A
2 5 10 10 10 35 F
3 6 12 12 12 42 MF
4 7 15 15 15 52 P
5 7 20 20 20 67 C
6 7 23 23 23 76 D
7 8 25 25 25 83 HD
8 8 35 30 30 103 N/A
Equivalence Class Testing: Example 1
Revisited
In equivalence class testing we are not required to choose only one test case
from each equivalence class. In addition, we may:

• Test boundaries of input

• Test boundaries of output
• Test each invalid input
• etc.

Exercise 6. What other test cases would you add to the table in the previous
slide? Explain your answer. Please be aware that there could be multiple
solutions.
Other Black-Box Testing Techniques: Special Value
Testing
• Special value testing, also called ad hoc testing, occurs when the test cases
are devised based on:
• domain knowledge,
• experience with similar programs,
• information about weak spots, etc.
• There are no guidelines used other than “best engineering judgment.”
• It is the most intuitive, least uniform and, probably, most widely practiced
form of functional testing.
• As a result, special value testing is very dependent on the abilities of the
tester.
Other Black-Box Testing Techniques: Random Testing
• In random testing the basic idea is that a random number generator is used
to pick test case values.
• The main question to be answered is: how many random test cases are
sufficient?
• To answer this question the tester requires expertise in probability and
statistics + the program domain knowledge.
 Black-Box Testing: Decision Tables
Decision tables are used to represent and analyse complex logical
relationships. They are ideal for describing situations in which several
combinations of actions are taken under varying sets of conditions.
Below the basic decision table terms are illustrated:
Stab Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6
c1 T T T F F F
Conditions c2 T T F T T F
c3 T F - T F -
a1 X X X
a2 X X
Actions
a3 X X
a4 X X
Possible Decision Table for Example 1
Stab Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8
T - - - - - - -
- T - - - - - -
- - T - - - - -
- - - T - - - -
- - - - T - - -
- - - - - T - -
- - - - - - T -
- - - - - - - T
Invalid X X
F X
MF X
P X
C X
D X
HD X