Software Testing and Quality

Assurance
Part 5
Graph Coverage

1
 Outline
Covering Graphs
Definition of a Graph
Paths in a Graph
Tests Paths in a Graph

2
 Covering Graphs
Graphs are the most commonly used
structure for testing
Graphs can come from many sources
Control flow graphs
Design structure
FSMs and statecharts
Use cases
Tests usually are intended to ‘cover’ the graph in
some way.

3
 Definition of a Graph
A set N of nodes, N is not empty

A set N of initial nodes, N is not empty

0 0

A set N of final nodes, N is not empty

f f

A set E of edges, each edge from one node

to another
( n , n ), i is predecessor, j is successor
i j

4
 Three Example Graphs

0 0 1 2 0

Not a
1 2 3 4 5 6 1 valid 2
graph

3 7 8 9 3

N0 = { 0 } N0 = { 0, 1, 2 } N0 = { }
Nf = { 3 } Nf = { 7, 8, 9 } Nf = { 3 }
5
 Paths in Graphs

Path : A sequence of nodes – [n 1, n2, …, nM]

Each pair of nodes is an edge
A path DOES NOT have to start from an initial node
Length : The number of edges
A single node is a path of length 0
Subpath : A subsequence of nodes in p is a
subpath of p
Reach (n) : Subgraph that can be reached from n

6
 Paths in Graphs
Paths
[ 0, 3, 7 ]
[ 1, 4, 8, 5, 1 ]
0 1 2
[ 2, 6, 9 ]

3 4 5 6 Reach (0) = { 0, 3, 4, 7, 8, 5, 1, 9 }
Reach (0,1,2) = N

7 8 9 Reach([6,9]) = { 9 }

Square brackets denote edge

7
 Test Paths and SESEs

Test Path : A path that starts at an initial node and ends at

a final node
Test paths represent execution of test cases
Some test paths can be executed by many tests
Some test paths cannot be executed by any tests

8
 Test Paths and SESEs
SESE graphs : All test paths start at a
single node and end at another node
Single-entry, single-exit
N0 and Nf have exactly one node

Double-diamond graph
1 4
Four test paths
[ 0, 1, 3, 4, 6 ]
0 3 6 [ 0, 1, 3, 5, 6 ]
[ 0, 2, 3, 4, 6 ]
2 5 [ 0, 2, 3, 5, 6 ]
9
 Visiting and Touring

Visit : A test path p visits node n if n is in p

A test path p visits edge e if e is in p
Tour : A test path p tours subpath q if q is a subpath of p

Path [ 0, 1, 3, 4, 6 ]
Visits nodes 0, 1, 3, 4, 6
Visits edges (0, 1), (1, 3), (3, 4), (4, 6)
Tours subpaths (0, 1, 3), (1, 3, 4), (3, 4, 6), (0, 1, 3, 4), (1, 3, 4, 6)

10
 Tests and Test Paths

 path (t) : The test path executed by test t

 path (T) : The set of test paths executed by the set
of tests T
 Each test executes one and only one test path (but
not necessarily the same path)
 A location in a graph (node or edge) can be reached
from another location if there is a sequence of
edges from the first location to the second
Syntactic reach : A subpath exists in the graph
Semantic reach : A test exists that can execute
that subpath
11
 Tests and Test Paths

test 1 many-to-one
Test
test 2 Path
test 3

Deterministic software – a test always

executes the same test path
This is obviously also valid for one-to-one (test
to test-path) relationships

12
 Tests and Test Paths
many-to-many Test Path 1
test 1

test 2 Test Path 2

test 3 Test Path 3

Non-deterministic software – a test can execute

different test paths

13
 Coverage Criteria of Graphs

We use graphs in testing as follows :

Developing a model of the software as a graph
Requiring tests to visit or tour specific sets of
nodes, edges or subpaths
Remember…
Test Requirements (TR): Specific elements about
that graphs that need to be covered by at least one
test case.
Coverage Criterion :A rule or collection of rules that
are impose the test requirements on a test set to
cover a graph.
14
 Testing and Covering Graphs

• Satisfaction : Given a set TR of test

requirements for a criterion C, a set of tests T
satisfies C on a graph if and only if for every
test requirement in TR, there is a test path in
path(T) that meets the test requirement tr.
• Structural Coverage Criteria : Defined on a
graph just in terms of nodes and edges
• Data Flow Coverage Criteria : Requires a
graph to be annotated with references to
variables
15
 Node Coverage
The first (and simplest) criteria require that
each node in a graph be executed

Node Coverage (NC) : TR contains each

reachable node in G.

16
 Edge Coverage

Edge coverage is slightly stronger than node coverage

Edge Coverage (EC) : TR contains each

reachable path of length up to 1, inclusive, in
G.
• The “length up to 1” allows for graphs with one
node and no edges
• EC subsumes NC and in fact all coverage criteria
are built to subsume a simpler criteria.

17
 Node vs. Edge Coverage
• For a simple if-else structure. Notice how
we need more test paths to satisfy the TR
of EC.

Node Coverage : TR = { 0, 1, 2 }
0 Test Path = [ 0, 1, 2 ]

1 Edge Coverage : TR = { (0,1), (0, 2), (1, 2) }

2 Test Paths = [ 0, 1, 2 ]
[ 0, 2 ]

18
 Covering Multiple Edges

 Edge-pair coverage requires pairs of edges, or

subpaths of length 2
Edge-Pair Coverage (EPC) : TR contains each
reachable path of length up to 2, inclusive, in
• G.
The “length up to 2” is used to include graphs
that have less than 2 edges

19
 Path Coverage

Complete Path Coverage (CPC) : TR contains

all paths in G.
• Unfortunately, this is impossible if the graph has a
loop, so a weak compromise is to specify the
paths we want.
• These paths could be provided by the customer in
the form of usage scenarios.
Specified Path Coverage (SPC) : TR contains a
set S of test paths, where S is supplied as a
parameter.

20
 Structural Coverage Example
Node Coverage
TR = { 0, 1, 2, 3, 4, 5, 6 }
0
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ]

1 Edge Coverage
TR = { (0,1), (0,2), (1,2), (2,3), (2,4), (3,6), (4,5), (4,6),
2 (5,4) }
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 2, 4, 5, 4, 6 ]
Edge-Pair Coverage
3 4 TR = { [0,1,2], [0,2,3], [0,2,4], [1,2,3], [1,2,4], [2,3,6],
[2,4,5], [2,4,6], [4,5,4], [5,4,5], [5,4,6] }
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 2, 3, 6 ]
5
6 [ 0, 2, 4, 5, 4, 5, 4, 6 ]

Complete Path Coverage

Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ]
21 [ 0, 1, 2, 4, 5, 4, 5, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 5, 4, 5, 4, 6 ] …
 Loops in Graphs
If a graph contains a loop, it has an infinite
number of paths

Thus, CPC is not feasible.

SPC may not be satisfactory because the

results are subjective and vary from one
system to the other.

22
 Simple Paths and Prime
Paths

Simple Path : A path from node ni to nj is

simple if no node appears more than once,
except if the first and last nodes are the
same.
No internal loops.
But a round-trip loop is in itself a simple path
since the first and last nodes are the same.

23
 Simple Paths and Prime
Paths
Prime Path : A simple path that does not appear
as a proper subpath of any other simple path
Simple Paths : [ 0, 1, 3, 0 ], [ 0, 2, 3, 0], [ 1, 3, 0, 1 ],
[ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ],
[ 2, 3, 0, 1 ], [ 0, 1, 3 ], [ 0, 2, 3 ], [ 1, 3, 0 ], [ 2, 3, 0 ],
[ 3, 0, 1 ], [3, 0, 2 ], [ 0, 1], [ 0, 2 ], [ 1, 3 ], [ 2, 3 ], [ 3, 0 ],
[0], [1], [2], [3]

Prime Paths : [ 0, 1, 3, 0 ], [ 0, 2, 3, 0], [ 1, 3, 0, 1 ],

[ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ],
[ 2, 3, 0, 1 ]
0
1 2
3
24
 Prime Path Coverage

A simple, elegant and finite criterion that requires

loops to be executed but not infinitely.

Prime Path Coverage (PPC) : TR contains each

prime path in G.

25
 Round Trips

Round-Trip Path : A prime path that starts and ends

at the same node

Simple Round Trip Coverage (SRTC) : TR

contains at least one round-trip path for each
reachable node in G that begins and ends a
round-trip path.
Complete Round Trip Coverage (CRTC) : TR
contains all round-trip paths for each
reachable node in G.

26
Finding Prime Paths
‘!’ means path ‘*’ means
terminates path cycles
Simple Len 0 Len 1 Len 2 Len 3
paths [0] [0, 1] [0, 1, 2] [0, 1, 2, 3]
[1] [0, 2] [0, 2, 3] [0, 1, 2, 4]
0 [2] [1, 2] [0, 2, 4] [0, 2, 3, 6] !
[3] [2, 3] [1, 2, 3] [0, 2, 4, 6] !
1 [4] [2, 4] [1, 2, 4] [0, 2, 4, 5] !
[5] [3, 6] ! [2, 3, 6] ! [1, 2, 3, 6] !
[6] ! [4, 6] ! [2, 4, 6] ! [1, 2, 4, 5] !
2
[4, 5] [2, 4, 5] ! [1, 2, 4, 6] !
[5, 4] [4, 5, 4] *
3 4 [5, 4, 6] !
[5, 4, 5] *
5
6 Len 4
[0, 1, 2, 3, 6] !
[0, 1, 2, 4, 6] ! Prime Paths
27
[0, 1, 2, 4, 5] !
 Infeasible Test Requirements
An infeasible test requirement cannot be
satisfied, for example:
 Unreachable statement (dead code).
 A subpath that can only be executed if a
contradiction occurs (x<0 && x>0).
 Most test criteria have some infeasible test
requirements.
It is usually undeterminable whether all
test requirements are feasible.

28
 Data Flow Criteria

Goal: Try to ensure that values are computed and

used correctly
Definition (def) : A location where a value
for a variable is stored into memory
Use : A location where a variable’s value is
accessed

29
 Data Flow Criteria

Z = X*2 Defs: def (0) = {X}

1 4 def (4) = {Z}
X = 42
0 3 6 def (5) = {Z}
2 5 Uses: use (4) = {X}
Z = X-8 use (5) = {X}

30
 DU Pairs and DU Paths
du pair : (Def-Use) A pair of locations (li, lj)
such that a variable v is defined at li and
used at lj
def-clear : A path from li to lj is def-clear
with respect to variable v if v is not given
another value at any of the nodes or edges
in the path
Reach : If there is a def-clear path from li to
lj with respect to v, the def of v at li reaches
the use at lj
31
 DU Pairs and DU Paths
du-path : A simple path that is def-clear
with respect to v from a def of v to a use of
v
du (ni, v) – the set of du-paths that start at
ni
du (ni, nj, v) – the set of du-paths from ni to
nj

32
 Data Flow Test Criteria

• First, we make sure every def reaches a use

All-defs coverage (ADC) : For each set of du-

paths S = du (n, v), TR contains at least one
path d in S.
• Then we make sure that every def reaches all
possible uses
All-uses coverage (AUC) : For each set of du-
paths to uses S = du (ni, nj, v), TR contains at
least one path d in S.
33
 Data Flow Test Criteria

• Finally, we cover all the paths between defs and

uses
All-du-paths coverage (ADUPC) : For each set S
= du (ni, nj, v), TR contains every path d in S.

34
 Data Flow Testing Example

Z = X*2
1 4
X = 42
0 3 6
2 5
Z = X-8
All-du-paths
All-defs All-uses for X
for X for X [ 0, 1, 3, 4 ]
[ 0, 1, 3, 4 [ 0, 1, 3, 4 [ 0, 2, 3, 4 ]
] ]
35 [ 0, 1, 3, 5 ]
[ 0, 1, 3, 5
 Graph Coverage Criteria
Subsumption Complete
Path Coverage
CPC

Prime Path
Coverage
PPC
All-DU-
Paths
Coverage Edge-
ADUP Pair
Coverag
e
All-uses
Coverage EPC
AUC Edge
Coverage
EC
All-defs
Coverage
ADC Node
Coverage
NC
36
 Control Flow Graphs (CFG) –
Source Code
A CFG models all executions of a method
by describing control structures
Nodes : Statements or sequences of
statements (basic blocks)
Edges : Transfers of control

37
 Control Flow Graphs (CFG) –
Source Code
Basic Block : A sequence of statements
such that if the first statement is
executed, all statements will be (no
branches)
CFGs are sometimes annotated with
extra information
branch predicates
defs
uses
Rules for translating statements into
graphs …

38
 CFG : The if Statement
if (x < y)
{
y = 0; 1
x = x + 1; x<y x >= y
} y=0
x=x+1 2 3 x=y
else
{
x = y; 4
}
if (x < y) 1
{ x<y
y = 0; y=0 x >= y
x=x+1 2
x = x + 1;
}
3
39
 CFG : The if-Return Statement
if (x < y)
{ 1
return; x<y
} x >= y
return 2
print (x);
return;
print (x)
3 return

No edge from node 2 to 3.

The return nodes must be distinct.

40
 Loops
Loops require “extra” nodes to be added

Nodes that do not represent statements or

basic blocks

41
 CFG : while and for Loops
x = 0;
x=0 1
dummy node
while (x < y)
{
2
y = f (x, y); x<y x >= y implicitly
x = x + 1; x=0 1
initializes loop
} 3 4
y =f(x,y)
x=x+1 2
x<y x >= y
for (x = 0; x < y; x++)
{ y = f (x, y) 3 5
y = f (x, y);
}
4 x=x+1

implicitly
42 increments loop
 CFG : The case (switch)
Structure
read ( c) ;
switch ( c ) read ( c );
{ 1
c == ‘N’
case ‘N’: c == ‘Y’ default
y = 25;
break; 2 3 4
case ‘Y’: y = 25; y = 0;
break; y = 50; break;
y = 50; break;
break;
default: 5
print (y);
y = 0;
break;
}
print (y);
43
 Example Control Flow – Stats
public static void computeStats (int [ ] numbers)
{
int length = numbers.length;
double med, var, sd, mean, sum, varsum;

sum = 0;
for (int i = 0; i < length; i++)
{
sum += numbers [ i ];
}
med = numbers [ length / 2 ];
mean = sum / (double) length;

varsum = 0;
for (int i = 0; i < length; i++)
{
varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean));
}
var = varsum / ( length - 1.0 );
sd = Math.sqrt ( var );

System.out.println ("length: " + length);

System.out.println ("mean: " + mean);
System.out.println ("median: " + med);
System.out.println ("variance: " + var);
System.out.println ("standard deviation: " + sd);
44
}
 Control Flow Graph for Stats
public static void computeStats (int [ ] numbers)
{
int length = numbers.length; 1
double med, var, sd, mean, sum, varsum;

sum = 0;
for (int i = 0; i < length; i++) 2 i=0
{
sum += numbers [ i ];
}
med = numbers [ length / 2 ]; i >=
mean = sum / (double) length; 3 length
varsum = 0; i < length
for (int i = 0; i < length; i++)
{ i++ 4
varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean)); 5
} i=0
var = varsum / ( length - 1.0 );
sd = Math.sqrt ( var );

System.out.println ("length: " + length);

6
System.out.println ("mean: " + mean); i < length
System.out.println ("median: " + med); i >= length
System.out.println ("variance: " + var);
System.out.println ("standard deviation: " + sd); 7 8
}
45 i++
Control Flow TRs and Test Paths –
EC
1

Edge Coverage
2 TR Test Path
A. [ 1, 2 ] [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ]
3
B. [ 2, 3 ]
C. [ 3, 4 ]
4 D. [ 3, 5 ]
5 E. [ 4, 3 ]
F. [ 5, 6 ]
G. [ 6,
6
7]
H. [ 6,
7 8]
8
46 I. [ 7, 6 ]
Control Flow TRs and Test Paths
– EPC
1 Edge-Pair Coverage
TR Test Paths
2 A. [ 1, 2, 3 ] i. [ 1, 2, 3, 4, 3, 5, 6, 7, 6,
B. [ 2, 3, 4 ] 8 ]
C. [ 2, 3, 5 ] ii. [ 1, 2, 3, 5, 6, 8 ]
3 D. [ 3, 4, 3 ] iii. [ 1, 2, 3, 4, 3, 4, 3, 5, 6,
E. [ 3, 5, 6 ] 7,
4
5 F. [ 4, 3, 5 ] 6, 7, 6, 8 ]
G. [ 5, 6, 7 ]
H. [ 5, 6, 8 ]
6 I. [ 6, 7, 6 ]
J. [ 7, 6, 8 ]
K. [ 4, 3, 4 ]
7 8 L. [ 7, 6, 7 ]
47
 Control Flow TRs and Test Paths
– PPC
1
Prime Path Coverage
TR Test Paths
A. [ 3, 4, 3 ] i. [ 1, 2, 3, 4, 3, 5, 6, 7, 6,
2 B. [ 4, 3, 4 ] 8]
C. [ 7, 6, 7 ] ii. [ 1, 2, 3, 4, 3, 4, 3,
D. [ 7, 6, 8 ] 5, 6, 7, 6, 7, 6, 8 ]
3
E. [ 6, 7, 6 ] iii. [ 1, 2, 3, 4, 3, 5, 6, 8 ]
4 F. [ 1, 2, 3, 4 ] iv. [ 1, 2, 3, 5, 6, 7, 6, 8 ]
5 G. [ 4, 3, 5, 6, 7 ] v. [ 1, 2, 3, 5, 6, 8 ]
H. [ 4, 3, 5, 6, 8 ]
I. [ 1, 2, 3, 5, 6, 7 ]
6 J. [ 1, 2, 3, 5, 6, 8 ]

7 8
48
 CFG for Stats – With Defs &
Uses 1 def (1) = { numbers, sum, length }

2 def (2) = { i }

3 use (3, 5) = { i, length }

use (3, 4) = { i, length }
def (5) = { med, mean, varsum, i }
4 5 use (5) = { numbers, length, sum }
def (4) = { sum, i }
use (4) = { sum, numbers, i }
6 use (6, 8) = { i, length }
use (6, 7) = { i, length }
def (8) = { var, sd }
7 8 use (8) = { varsum, length, mean,
def (7) = { varsum, i } med, var, sd }
49
use (7) = { varsum, numbers, i, mean }
Defs and Uses Tables for
Stats
Node Def Use
Edge Use
1 { numbers, sum,
length } (1, 2)
2 {i} (2, 3)
3 (3, 4) { i, length }
4 { sum, i } { numbers, i, sum } (4, 3)
5 { med, mean, { numbers, length, sum } (3, 5) { i, length }
varsum, i }
(5, 6)
6
(6, 7) { i, length }
7 { varsum, i } { varsum, numbers, i,
mean } (7, 6)

8 { var, sd } { varsum, length, var, (6, 8) { i, length }

mean, med, var, sd } 50
 DU Paths for Stats
variable DU Pairs DU Paths variable DU Pairs DU Paths
numbers (1, 4) [ 1, 2, 3, 4 ] mean (5, 7) [ 5, 6, 7 ]
(1, 5) [ 1, 2, 3, 5 ] (5, 8) [ 5, 6, 8 ]
(1, 7) [ 1, 2, 3, 5, 6, 7 ] varsum (5, 7) [ 5, 6, 7 ]
length (1, 5) [ 1, 2, 3, 5 ] (5, 8) [ 5, 6, 8 ]
(1, 8) [ 1, 2, 3, 5, 6, 8 ] (7, 7) [ 7, 6, 7 ]
(1, (3,4)) [ 1, 2, 3, 4 ] (7, 8) [ 7, 6, 8 ]
(1, (3,5)) [ 1, 2, 3, 5 ] i (2, 4) [ 2, 3, 4 ]
(1, (6,7)) [ 1, 2, 3, 5, 6, 7 ] (2, (3,4)) [ 2, 3, 4 ]
(1, (6,8)) [ 1, 2, 3, 5, 6, 8 ] (2, (3,5)) [ 2, 3, 5 ]
med (5, 8) [ 5, 6, 8 ] (4, 4) [ 4, 3, 4 ]
(4, (3,4)) [ 4, 3, 4 ]
var (8, 8) No path needed
(4, (3,5)) [ 4, 3, 5 ]
sd (8, 8) No path needed
(5, 7) [ 5, 6, 7 ]
sum (1, 4) [ 1, 2, 3, 4 ] (5, (6,7)) [ 5, 6, 7 ]
(1, 5) [ 1, 2, 3, 5 ] (5, (6,8)) [ 5, 6, 8 ]
(4, 4) [ 4, 3, 4 ] (7, 7) [ 7, 6, 7 ]
(4, 5) [ 4, 3, 5 ] (7, (6,7)) [ 7, 6, 7 ]
51 (7, (6,8)) [ 7, 6, 8 ]
 Test Cases and Test Paths1
2
There are 12 unique
[ 1, 2, 3, 4DU
] paths
[ 4, 3, 4 ] 3
[ 1, 2, 3, 5 ] [ 4, 3, 5 ]
[ 1, 2, 3, 5, 6, 7 ] [ 5, 6, 7 ]
[ 1, 2, 3, 5, 6, 8 ] [ 5, 6, 8 ] 4 5
[ 2, 3, 4 ] [ 7, 6, 7 ]
[ 2, 3, 5 ] [ 7, 6, 8 ]
6

7 8

52
 Test Cases and Test Paths1
2
Remove the subsumed
[ 1, 2, 3, sub-paths
4] [ 4, 3, 4 ] 3
[ 1, 2, 3, 5 ] [ 4, 3, 5 ]
[ 1, 2, 3, 5, 6, 7 ] [ 5, 6, 7 ]
[ 1, 2, 3, 5, 6, 8 ] [ 5, 6, 8 ] 4 5
[ 2, 3, 4 ] [ 7, 6, 7 ]
[ 2, 3, 5 ] [ 7, 6, 8 ]
6

7 8

53
 Test Cases and Test Paths1
2

[ 1, 2, 3, 4 ] [ 4, 3, 4 ] 3
[ 4, 3, 5 ]
[ 1, 2, 3, 5, 6, 7 ]
[ 1, 2, 3, 5, 6, 8 ] 4 5
[ 7, 6, 7 ]
[ 7, 6, 8 ]
6

1 expect a loop not to be

“entered” 7 8
3 require at least one iteration
of a loop
54 2 require at least two
iterations of a loop
 Test Cases and Test Paths

Test Case : numbers = (44) ; length = 1

Test Path : [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ]
Note: At least one iteration of a loop

Test Case : numbers = (2, 10, 15) ; length = 3

Test Path : [ 1, 2, 3, 4, 3, 4, 3, 4, 3, 5, 6, 7, 6, 7, 6,
7, 6, 8 ]
Note: At least two iterations of a loop
Other DU paths require arrays with length 0 to
skip loops
But the method fails with divide by zero on the
statement … A fault was
55 mean = sum / (double) length; found
 Fault Discovered
public static void computeStats (int [ ] numbers)
{
int length = numbers.length;
double med, var, sd, mean, sum, varsum;

sum = 0;
for (int i = 0; i < length; i++)
{
sum += numbers [ i ];
}
med = numbers [ length / 2 ];
mean = sum / (double) length;
varsum = 0;
for (int i = 0; i < length; i++)
{
varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean));
}
var = varsum / ( length - 1.0 );
sd = Math.sqrt ( var );

System.out.println ("length: " + length);

System.out.println ("mean: " + mean);
System.out.println ("median: " + med);
System.out.println ("variance: " + var);
System.out.println ("standard deviation: " + sd);
56 }
 OO Software and Designs
Emphasis on modularity and reuse puts
complexity in the design connections

Testing design relationships is more

important than before

Graphs are based on the connections

among the software components
 Connections are dependency relations, also called
couplings

57
 Call Graph

 The most common graph for structural design

testing
 Nodes : Units (in Java – methods)
 Edges : Calls to units
A
Node coverage : call every unit
at least once (method
B C D
coverage)
E F Edge coverage : execute every
Example call call at least once (call
58 graph coverage)
 Inheritance & Polymorphism
A
Classes are not executable,
so this graph is not directly
testable
B
We need objects
C D A
a
Example inheritance
hierarchy graph B objects
b

C D What is
c d coverage on
this graph ?
59
 Coverage on Inheritance
Graph
 Create an object for each class ?

This seems weak because there is no execution
 Create an object for each class and apply call
coverage (i.e. edge coverage)?
OO Call Coverage : TR contains each reachable
node in the call graph of an object instantiated for
each class in the class hierarchy.
OO Object Call Coverage : TR contains each
reachable node in the call graph of every object
instantiated for each class in the class hierarchy.
60
 Call Graphs on Classes

 Node and edge coverage of class call graphs

often do not work very well
 Individual methods might not call each other
at all! Class stack
???
public void push (Object o)
public Object pop ( )
public boolean isEmpty
(Object o)
push pop isEmpty

Data flow is probably more

61
appropriate …
 Data Flow at the Design
Level

 Data flow couplings among units and classes are

more complicated than control flow couplings
 When values are passed, they “change names”
 Many different ways to share data
 Finding defs and uses can be difficult – finding which
uses a def can reach is very difficult
 When software gets complicated … testers should
get interested
 That’s where the faults are!
62
 Data Flow at the Design
Level

• Caller : A unit that invokes another unit

• Callee : The unit that is called
• Callsite : Statement or node where the call
appears
• Actual parameter : Variable in the caller
• Formal parameter : Variable in the callee

63
 Example Call Site

A Caller
Callsite 
B (x) Actual
 Parameter
interface end A
Callee
B (Y)
 Formal
Parameter
end B

64
 Example Call Site

 Applying data flow criteria to def-use pairs

between units is too expensive
 Too many possibilities
 But this is integration testing, and we really only
care about the interface …

65
 Inter-procedural DU Pairs
 If we focus on the interface, then we just need to
consider the last definitions of variables before
calls and returns and first uses inside units and
after calls

 Last-def : The set of nodes that define a variable x

and has a def-clear path from the node through a
callsite to a use in the other unit
 Can be from caller to callee (parameter or shared
variable) or from callee to caller as a return value


66 First-use : The set of nodes that have uses of a
 Example Inter-procedural DU
Pairs
Caller
1 x=5
F x = 14 last-def
 10 B (int
y=G callsite 2 x=4 y)
DU pair (x)
 11 Z = y 12 T = y
first-use 3 x=3
print (y)

Callee 13 print
4 B
(y)
G print (a) (x)
(a) first-use
DU  Last Defs
pair b = 42 2, 3
last-def

return First Uses
(b) 11, 12

67
 Example – Quadratic
25 ok = Root (X, Y, Z);
1 // Program to compute the quadratic root for 26 if (ok)
two numbers 27 System.out.println
2 import java.lang.Math; 28 (“Quadratic: ” + Root1 + Root2);
3 29 else
4 class Quadratic 30 System.out.println (“No Solution.”);
5{ 31 }
6 private static float Root1, Root2; 32
7 33 // Three positive integers, finds quadratic root
8 public static void main (String[] argv) 34 private static boolean Root (int A, int B, int C)
9 { 35 {
10 int X, Y, Z; 36 float D;
11 boolean ok; 37 boolean Result;
12 int controlFlag = Integer.parseInt (argv[0]); 38 D = (float) Math.pow ((double)B,
13 if (controlFlag == 1) (double2-4.0)*A*C );
14 { 39 if (D < 0.0)
15 X = Integer.parseInt (argv[1]); 40 {
16 Y = Integer.parseInt (argv[2]); 41 Result = false;
17 Z = Integer.parseInt (argv[3]);
18 } 42 return (Result);
19 else 43 }
20 { 44 Root1 = (float) ((-B +
Math.sqrt(D))/(2.0*A));
21 X = 10; 45 Root2 = (float) ((-B –
22 Y = 9; Math.sqrt(D))/(2.0*A));
23 Z = 12; 46 Result = true;
24 } 47 return (Result);
68 48 } / /End method Root
49
 Example – Quadratic
1 // Program to compute the quadratic root for two numbers
2 import java.lang.Math;
3
4 class Quadratic
5{
6 private static float Root1, Root2; shared variables
7
8 public static void main (String[] argv)
9 {
10 int X, Y, Z;
11 boolean ok;
12 int controlFlag = Integer.parseInt (argv[0]);
13 if (controlFlag == 1)
14 {
15 X = Integer.parseInt (argv[1]);
last-defs 16 Y = Integer.parseInt (argv[2]);
17 Z = Integer.parseInt (argv[3]);
18 }
19 else
20 {
21 X = 10;
22 Y = 9;
23 Z = 12;
69 24 }
 25 ok = Root (X, Y, Z);
first-use 26 if (ok)
27 System.out.println
28 (“Quadratic: ” + Root1 + Root2);
29 else
30 System.out.println (“No Solution.”);
31 }
32
33 // Three positive integers, finds the quadratic root
34 private static boolean Root (int A, int B, int C)
first-use 35 {
36 float D;
37 boolean Result;
38 D = (float) Math.pow ((double)B, (double2-4.0)*A*C);
39 if (D < 0.0)
40 {
last-def 41 Result = false;
42 return (Result);
43 }
44 Root1 = (float) ((-B + Math.sqrt(D)) / (2.0*A));
last-defs 45 Root2 = (float) ((-B – Math.sqrt(D)) / (2.0*A));
46 Result = true;
47 return (Result);
48 } / /End method Root
49
70 50 } // End class Quadratic
 Quadratic – Coupling DU-
pairs
Pairs of locations: method name, variable name,
statement
(main (), X, 15) – (Root (), A, 38)
(main (), Y, 16) – (Root (), B, 38)
(main (), Z, 17) – (Root (), C, 38)
(main (), X, 21) – (Root (), A, 38)
(main (), Y, 22) – (Root (), B, 38)
(main (), Z, 23) – (Root (), C, 38)
(Root (), Root1, 44) – (main (), Root1, 28)
(Root (), Root2, 45) – (main (), Root2, 28)
(Root (), Result, 41) – ( main (), ok, 26 )
71
(Root (), Result, 46) – ( main (), ok, 26 )
 Coupling Data Flow Notes
 Only variables that are defined then used are
useful.
• Implicit initializations of class and global variables

• Transitive DU-pairs are too expensive to handle

– A calls B, B calls C, and there is a variable
defined in A and used in C
• Arrays : a reference to one element is considered
to be a reference to all elements

72
 Design Specifications
 A design specification describes aspects of
what behavior software should exhibit

 A design specification may or may not reflect

the implementation
More accurately – the implementation may
not exactly reflect the spec
Design specifications are often called models
of the software
 Two types of descriptions are used in this
chapter
1.Sequencing constraints on class methods
2.State behavior descriptions of software

73
 Sequencing Constraints

 Sequencing constraints are rules that impose

constraints on the order in which methods
may be called
 They can be encoded as a preconditions or
other specifications
Class stack
???
public void push (Object o)
public Object pop ( )
public boolean isEmpty ( )

74
 Sequencing Constraints
Tests can be created for these classes as
sequences of method calls
Sequencing constraints give an easy and
effective way to choose which sequences to
use

75
 Sequencing Constraints
Overview
 Sequencing constraints might be
Expressed explicitly
Expressed implicitly
Not expressed at all
 Testers should derive them if they do not exist
Look at existing design documents
Look at requirements documents
Ask the developers
Last choice : Look at the implementation
 If they don’t exist, expect to find more faults !
 Remember that sequencing constraints do not
capture all behavior

76
 Queue Example
public int DeQueue()
{
// Pre: At least one element must be on the queue.
……
public EnQueue (int e)
{
// Post: e is on the end of the queue.

 Sequencing constraints are implicitly embedded in the pre

and postconditions
 EnQueue () must be called before DeQueue ()

 Does not include the requirement that we must have at least

as many Enqueue () calls as DeQueue () calls

77 
Can be handled by state behavior techniques
 File ADT Example
class FileADT has three methods:
• open (String fName) // Opens file with
name fName
• close () // Closes the file and makes it
unavailable
• write
Valid (String textLine)
sequencing // Writes
constraints on a line of
text to the file
FileADT: s1 open (f)

1. An open (f) must be executed

before every write (t) write(t) s2 s3
2. An open (f) must be executed
before every close () s4 s5
3. A write (f) may not be executed write (t)
after a close () unless there is an
78 open (f) in between s6 close ()
 Static Checking
Is there a path that violates any of the sequencing constraints ?

 Is there a path to a write() that does not go

s1 open (f) through an open() ?
 Is there a path to a close() that does not go
through an open() ?
write(t) s2 s3
 Is there a path from a close() to a write()?
 Is there a path from an open() to a close() that
s4 s5
does not go through a write() ? (“write-clear”
write (t) path)
s6 close ()

79
[ 1, 3, 4, 6 ] – ADT use
anomaly!
 Static Checking
Consider the following graph :

s1 open (f)

s2 s3
write
s4 s5 (t)
write
(t) close ()
s6 s7
close () s8

80 [ 7, 3, 4 ] – close () before
write () !
 Generating Test
Requirements
[ 1, 3, 4, 6 ] – ADT use anomaly!
s1 open (f)
• But it is possible that the logic
write (t) of the program does not allow
s2 s3
the pair of edges [1, 3, 4]
• That is – the loop body must be
s4 s5
write (t)
taken at least once
• Determining this is
s6 close () undecidable – so static
methods are not enough
 Use the sequencing constraints to generate
test requirements
 The goal is to violate every sequencing
81
constraint
 Test Requirements for
FileADT
 Cover every path from the start node to every
node that contains a write() such that the path
does not go through a node containing an open()
 Cover every path from the start node to every
node that contains a close() such that the path
does not go through a node containing an open()
 Cover every path from every node that contains a
close() to every node that contains a write()
 Cover every path from every node that contains
an open() to every node that contains a close()
such that the path does not go through a node
containing a write()

If program is correct, all test requirements

82
will be infeasible
 Testing State Behavior

 A finite state machine (FSM) is a graph that

describes how software variables are modified
during execution
 Nodes : States, representing sets of values for key
variables
 Edges : Transitions, possible changes in the state
switch up

Off On

83
switch
 Finite State Machine – Two
Variables

Tropical Depression Tropical Storm

circulation = yes circulation = yes
windspeed < 39mph windspeed = 39..73 mph

Something Else
circulation = no
windspeed = 0..38 mph

Major Hurricane Hurricane

circulation = yes circulation = yes
windspeed >= 110 mph windspeed 74..109 mph

84 Other variables may exist but are not

 Finite State Machines are
Common
 FSMs can accurately model many kinds of
software
Embedded and control software (think
electronic gadgets)
Abstract data types
Compilers and operating systems
Web applications
 Creating FSMs can help find software problems
 Numerous languages for expressing FSMs
UML statecharts
Automata
State tables (SCR)
Petri nets
 Limitations : FSMs are not always practical for
85 programs that have lots of states (for example,
GUIs)
 Annotations on FSMs
 FSMs can be annotated with different types of
actions
Actions on transitions
Entry actions to nodes
Exit actions on nodes
 Actions can express changes to variables or
conditions on variables
Preconditions (guards) : conditions that
must be true for transitions to be taken
Triggering events : changes to variables
that cause transitions to be taken

86
 Example Annotations

open elevator
door

Closed Open

pre: elevSpeed = 0

trigger: openButton = pressed

87
 Covering FSMs
 Node coverage : execute every state (state
coverage)
 Edge coverage : execute every transition (transition
coverage)
 Edge-pair coverage : execute pairs of transitions
(transition-pair)
 Data flow:
Nodes often do not include defs or uses of
variables
Defs of variables in triggers are used immediately
(the next state)
Defs and uses are usually computed for guards
FSMs typically only model a subset of the variables
 Generating FSMs is often harder than covering them
…

88
 Deriving FSMs
 With some projects, a FSM (such as a
statechart) was created during design
Tester should check to see if the FSM is still
current with respect to the implementation

 If not, it is very helpful for the tester to derive

the FSM

89
Modeling State Variables to Build
FSMs
State variables are usually defined early.
First identify all state variables, then
choose which are relevant.
In theory, every combination of values
for the state variables defines a different
state.
In practice, we must identify ranges, or
sets of values, that are all in one state.
Some states may not be feasible.
FSMs easily becomes messy so we must
be careful.
90
 State Variable Model for
Watch
up

mode = TIME down mode = TIME down mode = TIME

watch = 12:00 up watch = 12:01.. 12:59 up watch = 01:00..12:59
down next
next next
next next next
next next next
up
mode = STOP down mode = STOP down mode = STOP
stopw = 12:00 up stopw = 12:01.. 12:59 up stopw = 01:00..12:59
down next next
next next next next
next next next
up
down down
mode = ALARM mode = ALARM mode = ALARM
alarm = 12:00 up alarm = 12:01.. 12:59 up alarm = 01:00..12:59
next nextnext next next next next
91 next next
down
 Using Implicit or Explicit
Specifications
 Relies on explicit requirements or formal
specifications that describe software behavior

 These could be derived by the tester

92
 Tradeoffs in Applying Graph
Coverage Criteria to FSMs
 There are two advantages
1.Tests can be designed before implementation
2.Analyzing FSMs is much easier than analyzing
source code
 There are three disadvantages
1.Some implementation decisions are not
modeled in the FSM
2.There is some variation in the results because
of the subjective nature of deriving FSMs
3.Tests have to be “mapped” to actual inputs of
the program – the names that appear in the
FSM may not be the same as the names in the
program

93
 Graph Coverage from
Requirements
Graph Coverage for Use Cases
Use Cases as Activity Diagrams
Covering Activity Diagrams
Node Coverage
Edge Coverage
Data Flow Coverage

94
 UML Use Cases
UML use cases are often used to express
software requirements

They help express computer application

workflow

95
 Simple Use Case Example

Withdraw
Funds
ATM
User Get Balance

Transfer
Funds

 Actors : Humans or software components that use

the software being modeled
 Use cases : Shown as circles or ovals
 Node Coverage : Try each use case once …
96
Use Case diagrams, by themselves, are not very
useful for testing
 Elaboration
Use cases are commonly elaborated (or
documented)
Elaborations are written textually
Details of operation
Alternatives model choices and conditions
during execution
Exceptional flows and optional behavior
Use case descriptions should first be
transformed into an activity diagram
The activity diagrams will become the
graphs which we analyze and try to cover
97
 Use Cases to Activity
Diagrams
Activity diagrams indicate flow among activities
Activities should model user level steps
Flow among steps are edges
Alternatives are sequential branch nodes
(decision diamonds)
Exceptional flows are modeled the same way as
Alternative flows
Activity diagrams usually have some helpful
characteristics:
Few loops
Simple predicates (variables)
Helps expose hidden DU pairs

98
 ATM Withdraw Activity Graph
[ card
Insert ATM Card recognized ]
Prompt for PIN Enter PIN
[ card not [ card
[ < 3 tries ]
recognized ] expired ]
[ >= 3 tries ] Confiscate
Eject Card [ card not
Card
[card lost ] expired ]
[ invalid PIN ]
[card not
Prompt for Transaction Check PIN lost ]
[ valid PIN ]

[ valid [ not [ sufficient

[ select account ] exceeded ] funds ]
withdraw ]
Select Acct #
[ select
[ invalid [ daily amount [ insufficient [ ATM out
balance or
account ] exceeded ] funds ] of funds ]
transfer ]
[ not out
Eject Card of funds ]

Print Welcome
Eject Card Print Receipt Dispense Cash
Message

99
 Covering Activity Graphs
Node Coverage
Cover every action
Edge Coverage
Cover every transition
Data flow coverage (after detailed design is
completed)
All-defs
All uses
All DU paths

100
 Covering Activity Graphs
Scenario Testing
Scenario : A complete path through a use
case activity graph
Should make semantic sense to the users
Number of paths often finite
If not, scenarios defined based on domain
knowledge
Use “specified path coverage”, where the set
S of paths is the set of scenarios
Note that specified path coverage does not
necessarily subsume edge coverage, but
scenarios should be defined so that it does

101