Software Testing and Quality Assurance

SEng5441

Chapter 4: Structural (White Box) Testing

(Control flow testing)

KIoT
Department of Software Engineering
1
 White Box testing
White-box testing(also called clear box testing, glass box testing, transparent box
testing, or structural testing)is a method of testing software that tests internal
structures or workings of an application [the tester has access to the internal data
structures and algorithms including the code that implement these.]

White box testing methods are especially useful for revealing design and code-
based control, logic and sequence defects, initialization defects, and data flow
defects.

2
Control flow refers to flow of control from one instruction to another

 The main idea in control flow testing is to appropriately select a few paths
in a program unit and observe whether or not the selected paths produce
3
the expected outcome.
Outline of Control Flow Testing

4
Figure 1: The process of generating test input data for control flow testing.
Basic Idea

• Two kinds of basic program statements:

– Assignment statements (Ex. x = 2*y; )

– Conditional statements (Ex. if(), for(), while(), …)

• Program path
– A program path is a sequence of statements from entry to exit.

– There can be a large number of paths in a program.

– There is an (input, expected output) pair for each path.

– Executing a path requires invoking the program unit with the right/correct test input.

– Paths are chosen by using the concepts of path selection criteria.

5
Outline of Control Flow Testing

• Inputs to the test generation process

– Source code
– Path selection criteria

• Generation of control flow graph (CFG)

– A CFG is a graphical representation of a program unit.
– Compilers are modified to produce CFGs if test generation is automated or You can draw one by hand.

• Selection of paths
– Enough entry/exit paths are selected to satisfy path selection criteria.

• Generation of test input data

– Two kinds of paths
• Feasible path/ Executable path: There exists input so that the path is executed.
• Infeasible path: There is no input to execute the path.
– Solve the path conditions to produce test input for each path.
6
Control Flow Graph
• Example code: ReturnAverage()
public static double ReturnAverage (int value[],
int AS, int MIN, int MAX)
{
int i, ti, tv, sum;
double av;
i = 0; ti = 0; tv = 0; sum = 0;
while (ti < AS && value[i] != -999)
{
ti++;
if (value[i] >= MIN && value[i] <= MAX)
{
tv++;
sum = sum + value[i];
}
i++;
}
if (tv > 0)
av = (double) sum/tv;
else
av = (double) -999;
return (av);
} Figure 2: A CFG representation of ReturnAverage(). 7
Numbers 1–13 are the nodes.
Paths in a Control Flow Graph

• A few paths in Figure 2.

– Path 1: 1-2-3(F)-10(T)-12-13

– Path 2: 1-2-3(F)-10(F)-11-13

– Path 3: 1-2-3(T)-4(T)-5-6(T)-

7(T)-8-9-3(F)-10(T)-12-13

– Path 4: 1-2-3(T)-4(T)-5-6(T)-7(T)-

8-9-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)

-10(T)-12-13

8
Path Selection Criteria

• Program paths are selectively executed.

• Question: What paths do I select for testing?
• The concept of path selection criteria is used to answer the above question.
• Advantages of selecting paths based on defined criteria:
– Ensure that all program constructs are executed at least once. (The programmer needs to
observe the outcome of executing each program construct)
– Repeated selection of the same path is avoided.(Executing the same path several times is a
waste of resources)
– One can easily identify what features have been tested and what not.
• Path selection criteria
1. Select all paths.
2. Select paths to achieve complete statement coverage.
3. Select paths to achieve complete branch coverage.
4. Select paths to achieve predicate coverage (Reading Assignments)
9
Path Selection Criteria…

1. All-path coverage criterion: Select all the paths in the program unit .

Advantage:
- it is desirable since it can detect error, except those due to missing path errors.

Disadvantages
– A program may contain a large number of paths, or even an infinite number of paths . It’s difficult

to select all possible paths in program in practice.

– Producing all the inputs that exercise all the program paths is difficult process.

– Example: check it in the previous slide (slide no. 8)

10
 Path Selection Criteria…
2. Statement coverage criterion
– Statement coverage means executing individual program statements (nodes) and observing the output.
– 100% statement coverage means all the statements have been executed at least once.
• Cover all assignment statements (i=0, ti=0,tv=0, sum=0) represented by node 2 in CFG
• Cover all conditional statements, ((ti < AS && value[i] != -999) represented by nodes 3,4, 6, 7 and 10
in CFG.
– Less than 100% statement coverage is unacceptable.

– Since a single entry–exit path includes many nodes, we need to select just a few paths to cover all the nodes of a CFG.

– while selecting paths:

• Select short paths.
• Select paths of increasingly longer length. Unfold a loop several times if there is a need.
• Select arbitrarily long, “complex” paths.

11
Path Selection Criteria…

–Example of Paths selected based on statement coverage

criterion and CFG

Table 1: Paths for statement coverage

of the CFG of Figure 3.

Finger 3 12
Path Selection Criteria…

Example 2: statement coverage

 Here are 8 statements in this code. In this code we
cannot cover all 8 statements in a single path as if 2
is valid then 4, 5, 6, 7 are not traversed, and if 4 is
valid then statement 2 and 3 will not be traversed.
Hence we will consider two paths so that we can
cover all the statements.
Set x= 1
Path – 1, 2, 3, 8
Output = 2
Set x= -1
Path = 1, 2, 4, 5, 6, 5, 6, 5, 7, 8
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑥𝑒𝑐𝑢𝑡𝑒𝑑 𝑠𝑎𝑡𝑒𝑚𝑒𝑛𝑡𝑠 Output = 2
𝑆𝑡𝑎𝑡𝑒𝑚𝑒𝑛𝑡 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒= *100
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑡𝑎𝑡𝑒𝑚𝑒𝑛𝑡𝑠

13
Path Selection Criteria…

3. Branch coverage criterion

– A branch coverage (also called Decision coverage)is an outgoing edge from a node in a CFG.

• A condition node has two outgoing branches – corresponding to the True and False values
of the condition.

• All the rectangle nodes have at most one outgoing branch (edge)

– Covering a branch means executing a path that contains the branch.

– Complete(100%) branch coverage means selecting enough number of paths such that
every condition evaluates to true at least once and to false at least once.

14
Path Selection Criteria
–Example of Paths selected based on Branch coverage
criterion and CFG

Table 2: Paths for branch coverage

of the CFG of Figure 3.

Figure 3: The dotted

arrows represent the
branches not covered by
the statement covering in
15
Table 1.
 Path Selection Criteria…
• For example, if the outcomes are binary, you need to test both True and False outcomes. The formula to calculate
Branch Coverage:
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑥𝑒𝑐𝑢𝑡𝑒𝑑 Braches
Branch 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒= *100
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 Branches

Demo(int a) { Scenario 1: Value of a is 2 Scenario 2: Value of a is 6

If (a> 5)
a=a*3
Print (a)
}

The code highlighted in yellow will be The code highlighted in yellow will
executed. Here the “No” outcome of the be executed. Here the “Yes”
Branch/decision If (a>5) is checked. outcome of the Branch/decision If
(a>5) is checked.
Branch Coverage = 50%
Branch Coverage = 50%

Test Case Value of A Output Branch Coverage

1 2 2 50%
2 6 18 50% 16
Generating Test Input

• Having identified path, a key question is how to make the path

execute, if possible.
– Generate input data that satisfy all the conditions on the path.
• Key concepts in generating test input data
1. Input vector
2. Predicate
3. Path condition
4. Predicate interpretation
5. Path predicate expression
6. Generating test input from path predicate expression

17
Generating Test Input

1-Input vector
– An input vector is a collection of all data entities read by the routine whose values must be
fixed prior to entering the routine.
– Members of an input vector can be as follows.
• Input arguments to the routine
• Global variables and constants
• Files
• Contents of registers (in Assembly language programming)
• Network connections
• Timers
– Example: The input vector of ReturnAverega() shown in Figure 2 is <value[], AS, MIN,
MAX>.

18
Generating Test Input
2- Predicate
– A predicate is a logical function evaluated at a decision point.
– Example of a predicate: the construct ti < AS is a predicate in node 3 of Figure 2.
– Example of a predicate: Value[i]<=Max is a predicate in node7 of Figure 2.
3- Path predicate
– A path predicate is the set of predicates associated with a path.
– The path in Figure 2 indicates that nodes 3, 4, 6, 7, and 10 are decision nodes
– An example path from Fig. 2:
1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13.
– The example of path predicate for the path shown in Figure 2.
ti < AS ≡ True
value[i] != -999 ≡ True
value[i] >= MIN ≡ True
value[i] <= MAX ≡ True
ti < AS ≡ False
tv > 0 ≡True
19
Generating Test Input

4-Predicate interpretation
– The path predicate shown in Figure 2 from the previous slide is composed of elements of
the input vector < value[], AS, MIN, MAX >, a vector of local variables < i, ti, tv >, and
the constant -999.
– Local variables play no role in selecting inputs that force a path to execute.
– Local variables can be eliminated by a process called symbolic execution.
– Predicate interpretation is defined as the process of
– symbolically substituting operations along a path in order to express the predicate only in terms of the
input vector and a constant vector.
– A predicate may have different interpretations depending on how control reaches the predicate.

– Examples of Predicate interpretation

 Example#1:
The predicate ti<AS can rewritten as 0<AS
 Example#2:
The predicate Value[i]>=MIN can rewritten as Value[0]>=MIN
20
Generating Test Input

5- Path predicate expression

– An interpreted path predicate is called a path predicate expression.
– A path predicate expression has the following attributes.
• It is void of local variables and is solely composed of elements of the input vector and possibly
a vector of constants
• It is a set of constraints in terms of the input vector, and, maybe, constants.
• Path forcing inputs can be generated by solving the constraints.
• If a path predicate expression has no solution, the path is infeasible.

21
 Generating Test Input…

Note: The bold entries in column 1 denote TABLE 3: Interpretation of Path Predicate of Path (1-2-3(T)-
4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13.) 22
interpreted predicates.
Cont.…
– example of Path predicate expression for the path shown in the previous slide.
0 < AS ≡ True …… (1)
value[0] != -999 ≡ True …… (2)
value[0] >= MIN ≡ True …… (3)
value[0] <= MAX ≡ True …… (4)
1 < AS ≡ False …… (5)
1>0 ≡ True …… (6)
– An example of infeasible path from Figure 2.
1-2-3(T)-4(F)-10(T)-12-13

TABLE 4: Interpretation of Path Predicate of

23
Path (1-2-3(T)-4(F)-10(T)-12-13)
Generating Test Input

– Path predicate expression for the path shown in Figure 2.

0 < AS ≡ True …… (1)
value[0] != -999 ≡ True …… (2)
0>0 ≡ True …… (3)
1-2-3(T)-4(F)-10(T)-12-13, is infeasible because the Path predicate expression is unsolved
because the constraint 0>0 ≡ True is unsatisfiable so we can not generating input data
from a path predicate expression.

24
Generating Test Input

6- Generating input data from a path predicate expression

– We must solve the corresponding path predicate expression in order to generate input data
which can force a program to execute a selected path.
– Consider the path predicate expression that is feasible of Figure 2
0 < AS ≡ True …… (1)
value[0] != -999 ≡ True …… (2)
value[0] >= MIN ≡ True …… (3)
value[0] <= MAX ≡ True …… (4)
1 < AS ≡ False …… (5)
1>0 ≡ True …… (6)

– Input data satisfying constraints for the above predicate expression.

AS =1
MIN = 25
MAX = 35
Value[0] = 30

– Note: The above set is not unique.

25
Summary

• Control flow is a fundamental concept in program execution.

• A program path is an instance of execution of a program unit.
• Select a set of paths by considering path selection criteria.
• Statement coverage
• Branch coverage
• Predicate coverage
• All paths
• From source code, derive a CFG (compilers are modified for this.)
• Select paths from a CFG based on path selection criteria.
• Extract path predicates from each path.
• Solve the path predicate expression to generate test input data.
• There are two kinds of paths.
• feasible
• infeasible 26