Control Flow Testing

Course Code: CSC4133 Course Title: Software Quality and Testing

Dept. of Computer Science

Faculty of Science and Technology

Lecturer No: 15 Week No: 9 Semester: Fall 2021-22

Lecturer: S.M. Abdur Rouf Bhuiyan [[email protected]]
Lecture Outline

• Static Analysis and Control Flow Testing

• Basic Idea of Control Flow Testing
• Outline of Control Flow Testing
• Control Flow Graph (CFG)
• Paths in a Control Flow Graph
• Path Selection Criteria
• Feasible & Infeasible Paths
Objectives and Outcomes
• Objectives: To understand the control flow testing as a
means of static analysis, to understand path selection
criteria in control flow testing, to understand feasible
and infeasible paths.
• Outcomes: Students are expected to be able to explain
control flow testing, be able to explain four path
selection criteria, be able to explain feasible and
infeasible paths.
 Static Analysis

• Static Analysis is the task of analyzing a test object (e.g.

source code, script, requirements) without executing
the test object.
• Possible aspects to be checked with static analysis are:
 programming rules and standards (syntax, convention)
 programming design (control flow analysis)
 use of data (data flow analysis)
 complexity of the programming structure (metrics e.g.
Cyclomatic number)
 Control Flow Testing
• The Control flow diagram shows the program flow and
allows for the detection of “dead branches” and
“unreachable code”.
• Aim is to find defects caused by wrong construction of
program code (dead branches, dead code etc.)
• Control flow graph, Directed graph
• Nodes represent statement
• Edge represent control flow as decision
Control Flow Testing
• Abnormality can easily be detected:
– loop
– dead branch
– multiple return
 Basic Idea of Control Flow Testing

• Two kinds of basic program statements:

 Assignment statements (Example: x = 2*y; )
 Conditional statements (Example: if(), for(), while(), …)
• Control flow
 In the absence of conditional statements, program
instructions are executed in the sequence they appear.
 Successive execution of program statements is viewed as
flow of control.
 Conditional statements alter the default flow, sequential
control flow in a program unit
Basic Idea of Control Flow Testing

• Program path
 A program path is a sequence of statements from entry to
exit.
 There can be a large number of paths in a program.
 There is a (input, expected output) pair for each path.
 Executing a path requires invoking the program unit with the
right test input.
 Paths are chosen by using the concepts of path selection
criteria.
 Basic Idea of Control Flow Testing

• Function calls are mechanism to provide abstraction in program design.

• A call to a program function leads to a control entering the called function.
• When the called function executes its return statement, we say that control
exits from the function.
• A program unit can be viewed as having a well-defined entry point and a
well-defined exit point.
• The execution of a sequence of instructions from the entry point to the exit
point of a program unit is called a program path.
• There can be a large, even infinite # of paths in a program unit. Each program
path can be characterized by an input and an expected output.
Basic Idea of Control Flow Testing

• Control Flow Testing (CFT) is a kind of structural testing, which is

performed by programmers to test code written by them. The
concept is applied to small units of code, such as a function or
method.
• Test cases for CFT are derived from the source code, such as a
program unit (e.g. a method), rather than from the entire
program.
• Structurally, a path is a sequence of statements in a program unit.
For a given set of input data, the program unit executes a certain
path. For another set of input data, the unit may execute a
different path.
 Basic Idea of Control Flow Testing

• The main idea in CFT is to appropriately select a few

paths in a program unit & observe whether or not the
selected paths produce the expected outcome.
By executing a few paths in a program unit, the
programmer tries to assess the behavior of the entire
program unit.
Basic Idea of Control Flow Testing

• CFT is a structural testing strategy that uses the program’s

control flow as a model.
• CFT techniques are based on judiciously selecting a set of test
paths through the program.
• The set of paths chosen is used to achieve a certain measure of
testing thoroughness.
• e.g., pick enough paths to assure that every source statement
is executed at least once
• CFT is most applicable to new software for unit testing.
 Control Flow Graphs

• The control flow graph (CFG) is a graphical

representation of a program’s control structure.
 Paths

• A path through a program is a sequence of statements

that starts at an entry, junction, or decision and ends
at another junction, decision, or exit.
• A path may go through several junctions, decisions;
one or more times.
• Paths consist of segments. The smallest segment is a
link. A link is a single process that lies between 2
nodes.
 Paths
• The length of a path is the number of links in a path.
• An entry/exit path or a complete path is a path that
starts at a routine’s entry and ends at the same
routine’s exit.
• Generally, there are many branching conditions in a
program unit, and thus there are numerous paths.
• If there is no branching condition in a program unit,
then there is just one path in that program unit.
Outline of Control Flow Testing

Figure 1: The process of generating test input data for CFT

Outline of Control Flow Testing

• Inputs to the test generation process

 Source code
 Path selection criteria ==>statement, branch, …
• Generation of control flow graph (CFG)
 A CFG is a graphical representation of a program unit
• Selection of paths
 Enough entry/exit paths are selected to satisfy path selection
criteria
Outline of Control Flow Testing

• Generation of test input data

Two kinds of paths
 Executable path: There exists input so that the path is
executed
 Infeasible path: There is no input to execute the path
• Feasibility Test of a Path
 Checking the feasibility of selected path ==> meet path
selection criteria
 If some chosen paths infeasible ==> select new path to meet
the criteria
Control Flow Graph Symbols

Figure 2: Symbols in a control flow graph

 Control Flow Graph
•Example code: openfiles()
FILE *fptr1, *fptr2, *fptr3; /* These are global variables. */
int openfiles(){
/*
This function tries to open files "file1", "file2", and "file3"
for read access, and returns the number of files
successfully opened. The file pointers of the opened files
are put in the global variables.
*/
int i = 0;
if(
((( fptr1 = fopen("file1", "r")) != NULL) && (i++) && (0)) ||
((( fptr2 = fopen("file2", "r")) != NULL) && (i++) && (0)) ||
((( fptr3 = fopen("file3", "r")) != NULL) && (i++))
);
return(i);
}
Figure 3: A function to open three files
 Control Flow Graph

Figure 4: A high-level CFG representation of openfiles()

Control Flow Graph

Figure 5: A detailed CFG representation of openfiles().

 Control Flow Graph
Example code: ReturnAverage()
public static double ReturnAverage(int value[], int AS, int MIN, int MAX){
/* Function: ReturnAverage Computes the average of all those numbers in the input array in the
positive range [MIN, MAX]. The maximum size of the array is AS. But, the array size could be
smaller than AS in which case the end of input is represented by -999. */
int i, ti, tv, sum;
double av;
i = 0; ti = 0; tv = 0; sum = 0;
while (ti < AS && value[i] != -999) {
ti++;
if (value[i] >= MIN && value[i] <= MAX) {
tv++;
sum = sum + value[i];
}
i++;
}
if (tv > 0)
av = (double)sum/tv;
else
av = (double) -999;
return (av);
} Figure6: A function to compute the average of selected integers in an array.
Control Flow Graph

Figure 7: A CFG representation of ReturnAverage().

 Paths in a Control Flow Graph

• A few paths in the last Figure :

 Path 1: 1-2-3(F)-10(T)-12-13
 Path 2: 1-2-3(F)-10(F)-11-13
 Path 3: 1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13
 Path 4: 1-2-3(T)-4(T)-5-6-7(T)-8-9-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-
10(T)-12- 13
Path Selection Criteria

• There are many paths between the entry and exit points
of a typical routine.
• Even a small routine can have a large number of paths.
 Path Selection Criteria

• Question: What paths do I select for testing?

 The concept of path selection criteria is used to answer the
question
• Advantages of selecting paths based on defined criteria:
 Ensure that all program constructs are executed at least once.
 Repeated selection of the same path is avoided.
 One can easily identify what features have been tested and
what not.
Path Selection Criteria

• Well-known Path selection criteria:

1) Select all paths
2) Select paths to achieve complete statement coverage
3) Select paths to achieve complete branch coverage
4) Select paths to achieve predicate coverage
[1] All-path coverage criterion

• All-path coverage criterion: Select all the paths in the

program unit under consideration.
 100% path coverage.
 Execute all possible control flow paths through the program.
• If all the paths in a CFG are selected, then one can
detect all faults, except those due to missing path
errors.
 [1] All-path coverage criterion

• A program may contain a large number of paths, or even

infinite # of paths. The openfiles() unit has 25+ paths.
• If one selects all possible paths in a program, then we say
that the all-path selection criteria has been satisfied.
• Selecting all the inputs will exercise all the program
paths.
• All-path selection criterion is desirable, but difficult to
achieve in practice.
[1] All-path coverage criterion

Input Path
<No, No, No> 1-2-3(F)-8-9(F)-14-15(F)-19-21
<Yes, No, No> 1-2-3(T)-4(F)-6-8-9(F)-14-15(F)-19-21
<Yes, Yes, Yes> 1-2-3(T)-4(F)-6-8-9(T)-10(T)-11-13(F)-14- 15(T)
-16(T)-18-20-21

Table : Inputs and paths in openfiles()

[2] Statement coverage criterion

• Statement coverage criterion:

• 100% statement coverage.
• Execute all statements in a program at least once under some
test.
• Statement coverage means executing individual program
statements and observing the output.
• 100% statement coverage means all the statements have been
executed at least once.
 Cover all assignment statements
 Cover all conditional statement
[2] Statement coverage criterion

• Less than 100% statement coverage is unacceptable.

SCPath1 1-2-3(F)-10(F)-11-13
SCPath2 1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13

Table : Paths for statement coverage of the CFG of Figure 7

 [3] Branch coverage criterion

• Branch coverage criterion:

• A branch is an outgoing edge from a node in a CFG.
• A condition node has two outgoing branches – corresponding to the
True and False values of the condition.
• All rectangular nodes have at most one outgoing branch (edge)
• The exit node of a CFG does not have an outgoing branch
• Covering a branch means selecting a path that contains the
branch.
• 100% branch coverage means selecting a set of paths such that
every branch is included in at least one path.
[3] Branch coverage criterion

BCPath 1 1-2-3(F)-10(F)-11-13
BCPath 2 1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13
BCPath 3 1-2-3(T)-4(F)-10(F)-11-13
BCPath 4 1-2-3(T)-4(T)-5-6(F)-9-3(F)-10(F)-11-13
BCPath 5 1-2-3(T)-4(T)-5-6(T)-7(F)-9-3(F)-10(F)-11-13

Table : Paths for branch coverage of the flow graph of Figure 7

[4] Predicate coverage criterion

• Predicate coverage criterion

 If all possible combinations of truth values of the
conditions affecting a path have been explored
under some tests, then we say that predicate
coverage has been achieved.
 Which Paths to Pick?
• You must pick enough paths to achieve statement and
branch coverage.

• Question: What is the fewest number of paths to

achieve statement and branch coverage?
 It is better to take many simple paths than a few
complicated ones.
- There is no harm in taking paths that will exercise
the same code more than once.
 Containing Infeasible Paths
• A program unit may contain a large number of paths.
 Path selection becomes a problem. Some selected paths may be
infeasible.
 Apply a path selection strategy:
• Select as many short paths as feasible
• Choose longer paths
 A large # of infeasible paths in CFG complicate the process of test
selection.
 To simplify path-based unit testing, it is useful to reduce the # of infeasible
paths in a program unit.
 There are efforts to write code with fewer/no infeasible paths.
Effectiveness of Control-flow Testing

• About 65% of all bugs can be caught in unit testing.

• Unit testing is dominated by control-flow testing methods.
• Statement and branch testing dominates control-flow testing.
• Studies show that control-flow testing catches 50% of all bugs caught
during unit testing.
 About 33% of all bugs.
• Control-flow testing is more effective for unstructured code than for
code that follows structured programming.
• Experienced programmers can bypass drawing CFGs by doing path
selection on the source.
 Limitations of Control-flow Testing

• Control-flow testing as a sole testing technique is

limited:
 Interface mismatches and mistakes are not caught.
 Not all initialization mistakes are caught by control-
flow testing.
 Specification mistakes are not caught.
Books

• Software Testing and Quality Assurance: Theory and Practice, by

Kshirasagar Naik, Priyadarshi Tripathy
References

1. Software Quality Engineering: Testing, Quality Assurance and

Quantifiable Improvement, by Jeff Tian
2. Software Quality Assurance: From Theory to Implementation, by
Daniel Galin
3. Software Testing and Continuous Quality Improvement, by William E.
Lewis
4. The Art of Software Testing, by Glenford J. Myers, Corey Sandler and
Tom Badgett