Lecture 10: Structural Testing -

Paths and Data Flow

Gregory Gay
DIT635 - February 18, 2022
Test Adequacy Criteria
Compromise between
the impossible and the inadequate

• Can we measure “good testing”?

• Test adequacy criteria “score” tests by measuring
completion of test obligations.
• Checklists of properties that must be met by test cases.

2
Structural Coverage Criteria
• Criteria based on exercising:
• Statements (nodes of CFG)
• Branches (edges of CFG)
• Decisions and Conditions
• Paths
• … and many more
• Measurements used as adequacy criteria

3
Elements Vs. Paths
• Statement, Branch, boolean A = …
Condition Coverage all focus boolean B = …
boolean expr = A || B;
on one element at a time.
if (expr && C) {
• A test executes a path, not a System.out.println(“Here I am!”);
single element. }

• Each element on that path is

dependent on the others.

4
Elements Vs. Paths
• There are different control boolean A = … Fault in definition
paths through a program… boolean B = …
boolean expr = A || B; Corrupts definition
• … And different ways that if (expr && C) {
of expr if B = False

data passed along paths can System.out.println(“Here I am!”);

influence execution. }
expr can corrupt outcome if C = True
• Important to examine not
just elements, but paths.

5
Today’s Goals
• Introduce Path Coverage
• Data Flow Coverage Criteria
• Focus on how information spreads through a program.
• Based on Definition-Use Pairs
• (Where is X defined? Where is each definition of X used?)

6
Path Coverage

7
Path Coverage
• Path coverage requires that all paths through the
CFG are covered.
Paths:
False C A, B, C, E, G
A, B, D, E, G
A B False E A, B, D, F, G
D G
True F
True

• Coverage = Number of Paths Covered

Number of Total Paths
8
Path Coverage
public int flipSome(int[] A, int N, int X) i=0
{
int i=0;
while (i<N and A[i] <X) i<N and A[i] <X
{ True
if (A[i]<0) False A[i]<0
A[i] = - A[i]; True
i++; False A[i] = - A[i];
}
return A; return(1)
} i++

Path coverage is a powerful coverage metric, but is often impractical.

● How many paths does this have?
● Each loop cycle is a separate path!
9
 Path Coverage
How many cases
for Statement
Branch
Path

loop <= 20

10
Path coverage with (loop <= 20) requires:
3,656,158,440,062,976 test cases

If you run 1000 tests per second, this will

take 116,000 years.

However, there are ways to get some of the benefits of

path coverage without the cost...

11
Path Coverage
• Theoretically, a very strong coverage metric.
• Many faults emerge through sequences of interactions.
• But… Generally impossible to achieve.
• Loops result in an infinite number of path variations.
• Even bounding number of loop executions leaves an
infeasible number of tests.

12
Boundary Interior Coverage
• Groups paths that differ only in the subpath they
follow when repeating the body of a loop.
• Executing loop 20 times is different than executing it
twice, but same subpaths repeat over and over.
• Unroll loop in CFG into distinct subpaths, and cover those
instead of worrying about loop cycles.

13
Boundary Interior Coverage
A A -> B -> M A

B B

A -> B -> C -> E -> L -> B

M C M C

D E A -> B -> C -> D -> DF -> L -> BE

F L
F G G
A -> B -> C -> D -> G -> H -> LB-> B
H I L H I

L B L L
A -> B -> C -> D -> G -> I -> L -> B
B B
14
 Boundary Interior Coverage Tests
● [ ], 0, 10
public int flipSome(int[] A, int N, int X) ● [-1], 1, 10
{ ● [1], 1, 10
int i=0; i=0 A
while (i<N and A[i] <X) i=0
{
if (A[i]<0) B
i<N and A[i] <X
A[i] = - A[i]; i<N and A[i] <X True
i++; True
}
False A[i]<0
False A[i]<0 D True
return A; True
} False A[i] = - A[i];
False
A[i] = - A[i]; E
return(1) C
Paths: return(1) i++ F i++
● A, B, C i++ F
● A, B, D, F, B
● A, B, D, E, F, B i<N and A[i] <X B i<N and A[i] <X B
15
Boundary Interior Example
1. public int doSomething(int x, int y)
1
2. {
3. while(y > 0) {
4. if(x > 0) { F 3 T
5. y = y - x;
6. if (y > 0) 14
7. System.out.println(“Y: “ + y); T 4 F
8. }else {
9. x = x + 1;
5 9
10. if (x <= 0)
11. System.out.println(X: “ + x); 6 10
12. } T F T
F
13. }
7 11
14. return x + y;
15. }

16
 Paths:
Boundary Interior Example ● 1, 3-F, 14
● 1, 3-T, 4-T, 5, 6-T, 7, 3
1. public int doSomething(int x, int y) ● 1, 3-T, 4-T, 5, 6-F, 3
1
2. { ● 1, 3-T, 4-F, 9, 10-T, 11, 3
3. while(y > 0) { ● 1, 3-T,4-F, 9, 10-F, 3
4. if(x > 0) { F 3 T
5. y = y - x;
6. if (y > 0) 14 Tests:
7. System.out.println(“Y: “ + y); T 4 F ● 10, -1
8. }else { ● 3, 4
9. x = x + 1;
5 9 ● -1, 1
10. if (x <= 0)
11. System.out.println(X: “ + x); 6 10
12. } T F T
F
13. }
7 11
14. return x + y;
15. }
3 3 3 3
17
Number of Paths
• Boundary Interior Coverage
if (a) S1;
removes bounds number of
loop paths. if (b) S2;
• However, number of paths can if (c) S3;
still be exponential. …
• N non-loop branches results in if (x) SN;
2N paths.
• Additional limitations may
need to be imposed.

18
Data Flow

19
Control Flow
1<x
• Capture how execution
T F
navigates between blocks
of statements. /* continue */
x--;
• We care about a
statement’s effect only
when it affects the path.
• Deemphasizes information
being transmitted.

20
Data Flow
• Program statements compute and transform data…
• Reason about data dependence
• A variable is used here.
• Where does its value come from?
• Is this assigned value ever used?
• Is this variable properly initialized?
• If the expression assigned to a variable is changed what
else would be affected?

21
Data Flow
• Basis of the optimization performed by compilers.
• Used to derive test cases.
• Have we covered the dependencies?
• Used to detect faults and other anomalies.
• When can we cache result of a calculation instead of
recalculating it?
• Can we eliminate a variable definition?

22
Definition-Use Pairs
• Data is defined.
• … and data is used.
• Pairs of definitions and uses capture flow of
information through the program.
• Definitions occur when variables are declared, initialized,
assigned values, or received as parameters.
• Uses occur in expressions, conditional statements,
parameter passing, return statements.

23
 Definitions and Uses
1. min = 1; 1. def - min
2. max = N; 2. def - max, use - N
3. mid = ((min + (max - min))/2); 3. def - mid, use - min,
4. while (A[mid] != x or min <= max){ max
5. mid = ((min + (max - min))/2); 4. use - A[mid], mid, x,
6. if (x > A[mid]){ min, max
7. min = mid + 1 5. def - mid, use - min,
max
8. } else { 6. use - x, A[mid], mid
9. max = mid - 1; 7. def - min, use - mid
10. } 8. -
11. } 9. def - max, use - mid
24
Definitions and Uses
min = 1; max =
mid = ((min + (max - min))/2);
N;
1. def - min
2. def - max, use - N
A[mid] != 3. def - mid, use - min,
x or min max
<= max 4. use - A[mid], mid, x,
min, max
mid = ((min + (max 5. def - mid, use - min,
- min))/2); max
min = mid
+ 1; 6. use - x, A[mid], mid
x > 7. def - min, use - mid
A[mid] 8. -
max = 9. def - max, use - mid
mid -1;

25
Definition-Use (DU) Pairs
• We can say there is a DU pair when:
• There is a def (definition) of variable x at location A.
• Variable x is used at location B.
• A control-flow path exists from A to B.
• and the path is definition-clear for x from A to B.
• If variable is redefined, original def is killed and pair
is now between new definition and its use in B.

26
 Example - Definition-Use Pairs
1. min = 1;
1. def - min
2. max = N; DU Pairs
2. def - max, use - N
3. mid = ((min + (max - min))/2); min:def
3. (1, -3), (1, use
mid, 4), (1, 5),
- min,
4. while (A[mid] != x or min <= max){ (7, 4),
max(7, 5)
5. mid = ((min + (max - min))/2); max:use
4. (2, -3), (2, 4),mid,
A[mid], (1, 5),
x,
6. if (x > A[mid]){ (9, 4), (9,
min, max5)
N: (0,
5. def2)- mid, use - min,
7. min = mid + 1 mid:max
(3, 4), (5, 6), (5, 7),
8. } else { (5, 9),
6. use(5,- 4)
x, A[mid], mid
9. max = mid - 1; x: (0, 4), (0,
7. def - min, 6) use - mid
A: (0,
8. - 4), (0, 6)
10. }
11. } 9. def - max, use - mid
27
Example - GCD 1. def: x, y
1. public int gcd(int x, int y){ 2. def: tmp
2. int tmp; 3. use: y
3. while(y!=0){ 4. use: x, y
4. tmp = x % y; def: tmp
5. x = y;
5. use: y
6. y = tmp;
7. } def: x
8. return x; 6. use: tmp
9. } def: y
7. -
8. use: x
28
Example - GCD public int gcd(int x, int y) {
int tmp;
A
1. public int gcd(int x, int y){
2. int tmp; while (y != 0) { B
3. while(y!=0){
4. tmp = x % y;
tmp = x % y C
5. x = y;
6. y = tmp;
x=y
7. } D
8. return x;
9. } 1. def: x, y y = tmp; E
Def-Use Pairs 2. def: tmp
3. use:
x: (1, 4),y(5, 4), (5, 4.
8),use:
(1, 8)x, y def: tmp return x; F
5. use:
y: (1, 3),y(1,
def:
4), x(1, 6.
5),use: tmp
(6, 3), (6, def: y 5)
4), (6,
8.
tmp:use:
(4, x
6)
29
Example - collapseNewlines
7. public static String collapseNewlines(String argStr)
8. {
9. char last = argStr.charAt(0);
10. StringBuffer argBuf = new StringBuffer();
11.
12. for(int cldx = 0; cldx < argStr.length(); cldx++)
13. {
14. char ch = argStr.charAt(cldx);
Variable
Variable Definitions
D-U Pairs Uses
15. if(ch != ‘\n’ || last != ‘\n’)
16. { argStr
argStr 7 (7, 9), (7,12),9,(7,12,
14)14
17. argBuf.append(ch); last
last 9, 18 (9, 15), (18, 15)
15
18. last = ch;
argBuf
argBuf 10, 17 (17, 22) 22
19. }
20. } cldx
cldx 12 (12, 12), (12,12,
14)14
21.
ch
ch 14 (14, 15), (14,15,
17),17,
(14,
1818)
22. return argBuf.toString();
23. }
30
Let’s Take a Break

31
Dealing With Arrays and Pointers
• Arrays and pointers (including object references
and arguments) introduce issues.
• It is not possible to determine whether two access refer to
the same storage location.
• a[x] = 13;
k = a[y];
• Are these a def-use pair?
• a[2] = 42;
i = b[2];
• Are these a def-use pair?
• Aliasing = two names refer to the same memory location.

32
Aliasing
• Aliasing is when two names refer to the same
memory location.
• int[] a = new int[3]; int[] b = a;
a[2] = 42;
i = b[2];
• a and b are aliases.
• Worse in C:
p = &b;
*(p + i) = k;
33
Uncertainty
• Aliasing introduces uncertainty.
• Instead of definition or use of one variable, may have a
potential def or use of a set of variables.
• Proper treatment depends on purpose of analysis:
• Safest method - treat any use of a potential alias of V as
a use of V.
• Creates more def-use pairs (some may not be real pairs),
but avoids missed pairs.

34
Dealing With Uncertainty
• Treat all potential aliases as definitions and uses:
a[1] = 13; Def of a[1], use of a[2].
k = a[2];

a[x] = 13; Def and use of array a.

k = a[y];

• Can be very imprecise.

• They are only the same if x and y are the same.

35
Dealing With Uncertainty
• Option 2: Treat uncertainty about aliases like
uncertainty about control flow.
a[x] = 13; a[x] = 13;
k = a[y]; if(x == y) k = a[x];
else k = a[y];

• Rewrite code to make references explicit.

• In transformed code, all array references are distinct.

36
Situational Def-Use Pairs
• ++counter, counter++, counter+=1
counter = counter + 1
• Use of counter then a new definition.
• char *ptr = *otherPtr
• Definition of string *ptr
• Use of memory index ptr, string *otherPtr, and memory
index otherPtr.
• ptr++
• Use of memory index ptr, definition of both memory index and
string *ptr (change to index moves pointer to a new location).

37
Dealing With Nonlocal Information
• fromCust and toCust may be
references to same object. public void transfer(Customer fromCust,
• from/toHome and from/toWork. Customer toCust){
PhoneNum fromHome =
• Option 1 - treat all nonlocal fromCust.getHomePhone();
PhoneNum fromWork =
variables of same type as fromCust.getWorkPhone();
potential aliases. PhoneNum toHome =
toCust.getHomePhone();
• Option 2 - Introduce additional PhoneNum toWork =
control flow toCust.getWorkPhone();
}
• if (fromHome.equals(fromWork))

38
Data Flow Coverage Criteria

39
Overcoming Limitations of Path Coverage
• We can potentially expose many faults by targeting
particular paths of execution.
• What are the important paths to cover?
• Some methods impose heuristic limitations.
• Use data flow to select paths based on how one element
can affect the computation of another.

40
Choosing the Paths
• Computing the wrong value leads to a failure only
when that value is used.
• Pair definitions with usages.
• Ensure that definitions are actually used by covering
paths from definitions to uses.
• All DU Pair Coverage, All DU Paths Coverage, All
Definitions Coverage
• Varying power and cost.

41
All DU Pair Coverage
• Requires each DU pair be exercised in at least one
program execution.
• Cover any path between a definition and its use.
• Coverage = number exercised DU pairs
number of DU pairs
• Can easily achieve structural coverage without
covering all DU pairs.

42
All DU Pairs Coverage Example
1. public int doSomething(int x, int y)
1
2. {
3. while(y > 0) {
4. if(x > 0) { F 3 T
5. y = y - x;
6. if (y > 0) X: 14
7. System.out.println(“Y: “ + y); (1, 4), (1,T5), (1,
4 9), (1,
F 14)
8. }else { (9, 10), (9, 11), (9, 5), (9, 9), (9, 14)
9. x = x + 1;
5 9
10. if (x <= 0) Y:
11. System.out.println(X: “ + x);
(1, 3), (1,65), (1, 14)
10
(5, 6), (5, 7), (5, 3), (5, 5), (5, 14)
12. } T F T
F
13. }
7 11
14. return x + y;
15. }

43
 Tests:
X: (1, 4), (1, 5), (1, 9), (1, 14), (9, 10), (9, 11), (9, 5), (9, 9), (9, 14)
1. -1, 1
Y: (1, 3), (1, 5), (1, 14), (5, 6), (5, 7), (5, 3), (5, 5), (5, 14)
2. 3, 7
3. -2, 1
1. public int doSomething(int x, int y)
1
2. {
3. while(y > 0) {
4. if(x > 0) { F 3 T
5. y = y - x;
6. if (y > 0) 14
7. System.out.println(“Y: “ + y); T 4 F
8. }else {
9. x = x + 1;
5 9
10. if (x <= 0)
11. System.out.println(X: “ + x); 6 10
12. } T F T
F
13. }
7 11
14. return x + y;
15. }

44
All DU Paths Coverage
• A use may be reachable along several paths from
the definition.
• Cover all simple (non-looping) paths at least once.
• Can reveal faults where a path is exercised that should
use a certain definition but doesn’t.

Coverage = number of exercised DU paths

number of DU paths
45
All DU Paths Example
DU Pair (2, 8) for X can be
1. ... 2
reached along multiple paths.
● 2, 3T, 4T, 5, 8
2. int x = 1; ● 2, 3T, 4F, 8
3. if (y > 7) { 3 ● 2, 3F, 8
4. if (z > 5) { T
5. z = x + 5; 4
F
6. } T Tests:
7. } F ● y = 10, z = 6
● y = 10, z = 3
8. y = x + 7; 5
● y = 2, z = (anything)
9. ... 8

46
Path Explosion Problem
• Even without looping public void countBits(char ch){
int count = 0;
paths, number of DU if (ch & 1) ++count;
paths can be exponential. if (ch & 2)
if (ch & 4)
++count;
++count;
• Code between definition if (ch & 8) ++count;
if (ch & 16) ++count;
and use can be irrelevant if (ch & 32) ++count;
to that variable, but if (ch & 64) ++count;
if (ch & 128) ++count;
contains many paths. System.out.println(ch + “ has ” +
count + “bits set to 1”);
}

47
All Definitions Coverage
• All DU Pairs/All DU Paths may be too expensive in
some situations.
• Pair each definition with at least one use.
• Skips many DU pairs, but ensures each definition tried.
Coverage = number of covered definitions
number of definitions

48
X: (1, 4), (1, 5), (1, 9), (1, 14), (9, 10), (9, 11), (9, 5), (9, 9), (9, 14) X: Definitions on lines 1, 9
Y: (1, 3), (1, 5), (1, 14), (5, 6), (5, 7), (5, 3), (5, 5), (5, 14) Y: Definitions on lines 1, 5

1. public int doSomething(int x, int y)

1
2. {
3. while(y > 0) {
4. if(x > 0) { F 3 T
5. y = y - x;
6. if (y > 0) 14
7. System.out.println(“Y: “ + y); T 4 F
8. }else {
9. x = x + 1;
5 9
10. if (x <= 0)
11. System.out.println(X: “ + x); 6 10
12. } T F T
F
13. }
● Any test covers (1, -) pairs. 7 11
14. return x + y;
● Reaching lines 5, 9 gets
15. }
(5,14) and (9,14) pairs.

49
Infeasibility Problem
• Metrics may ask for impossible test cases.
• Path-based metrics may require infeasible
combinations of feasible elements.
• Alias analysis may add additional infeasible paths.
• All Definitions, All DU-Pairs Coverage reasonable.
• All DU-Paths is much harder!

50
Activity - DU Pair Coverage
1. int doSomething(int x, int y)
• Identify all DU pair 2. {
• Write your own test 3. while(y > 0) {
4. if(x > 0) {
cases to achieve All
5. y = y - x;
DU Pair Coverage. 6. }else {
• Hint - remember that 7. x = x + 1;
there is a loop. 8. }
9. }
https://bit.ly/3rCsWlN 10. return x + y;
11. }
51
Activity - DU Pairs
1. int doSomething(int x, int y)
2. { Variable Defs Uses

3. while(y > 0) { x 1, 7 4, 5, 7, 10

4. if(x > 0) { y 1, 5 3, 5, 10

5. y = y - x;
Variable D-U Pairs
6. }else {
x (1, 4), (1, 5), (1, 7), (1, 10), (7, 4),
7. x = x + 1; (7, 5), (7, 7), (7, 10)
8. } y (1, 3), (1, 5), (1, 10), (5, 3), (5, 5),
9. } (5, 10)

10. return x + y;
11. }

52
 Variable Defs Uses

Activity - DU Pairs x 1, 7 4, 5, 7, 10

y 1, 5 3, 5, 10
1. int doSomething(int x, int y)
Variable D-U Pairs
2. {
x (1, 4), (1, 5), (1, 7), (1, 10), (7, 4),
3. while(y > 0) { (7, 5), (7, 7), (7, 10)
4. if(x > 0) { y (1, 3), (1, 5), (1, 10), (5, 3), (5, 5),
5. y = y - x; (5, 10)

6. }else { Test 1: (x = 1, y = 2)
7. x = x + 1; Covers lines 1, 3, 4, 5, 3, 4, 5, 3, 10
8. } Test 2: (x = -1, y = 1)
Covers lines 1, 3, 4, 6, 7, 3, 4, 6, 7, 3, 4, 5, 3, 10
9. } Test 3: (x = 1, y = 0)
10. return x + y; Covers lines 1, 3, 8
11. }

53
We Have Learned
• Control-flow and data-flow capture important paths
in program execution.
• Analysis of how variables are defined and then
used and the dependencies between definitions
and usages can help us reveal important faults.
• Many forms of analysis can be performed using
data flow information.

54
We Have Learned
• If there is a fault in a computation, we can observe
it by looking at where the computation is used.
• By identifying DU pairs and paths, we can create
tests that trigger faults along those paths.
• All DU Pairs coverage
• All DU Paths coverage
• All Definitions coverage

55
Next Time
• Exercise Session - Structural Testing
• Using Meeting Planner code.
• Next Wednesday - Fault-Based Testing
• Pezze & Young - Ch 16
• Assignment 2
• Due February 27! We have covered everything on it.

56