Data flow testing

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 1

 Learning objectives
• Understand why data flow criteria have been
designed and used
• Recognize
R i andd di
distinguish
ti i hbbasic
i DF criteria
it i
– All DU pairs, all DU paths, all definitions
• Understand how the infeasibility problem
impacts data flow testing
• Appreciate limits and potential practical uses
of data flow testing
g

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 2

 Motivation
• Middle ground in structural testing
– Node and edge coverage don’t test interactions
– Path-based
Path based criteria require impractical number of
test cases
• And only a few paths uncover additional faults
faults, anyway
– Need to distinguish “important” paths
• Intuition: Statements interact through data
flow
– Value computed in one statement,
statement used in another
– Bad value computation revealed only when it is used

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 3

 Data flow concept

1
x = ....

2 • Value of x at 6 could be
if ....
computed at 1 or at 4
3
.... 4 x = .... • Bad computation at 1 or
4 could be revealed only
5
... if they are used at 6
6 • (1,6) and (4,6) are
y = x + ... def-use (DU) pairs
– defs at 1,4
– use at 6

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 4

 Terms
• DU pair: a pair of definition and use for some
variable, such that at least one DU path exists
from the definition to the use
x = ... is a definition of x
= ... x ... is a use of x
• DU path: a definition-clear path on the CFG
starting
t ti ffrom a definition
d fi iti tot a use off a same
variable
– Definition clear: Value is not replaced on path
– Note – loops could create infinite DU paths between
a def
d f andd a use
(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 5
 Definition-clear path
• 1
1,2,3,5,6
2 3 5 6 is a definition
definition-
1
x = ....
clear path from 1 to 6
– x is not re-assigned
g
2 between 1 and 6
if ....
• 1,2,4,5,6 is not a
3
.... 4 x = .... d fi iti
definition-clear
l path
th
from 1 to 6
5
... – the value of x is “killed”
killed
(reassigned) at node 4
6
y = x + ... • (1,6)
( , ) is a DU p
pair
because 1,2,3,5,6 is a
definition-clear path

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 6

 Adequacy criteria
• All DU pairs: Each DU pair is exercised by at
least one test case
• All DU paths:
th E Each
h simple
i l (non
( llooping)
i ) DU pathth
is exercised by at least one test case
• All definitions: For each definition, there is at
least one test case which exercises a DU pair
containing it
– (Every computed value is used somewhere)
Corresponding coverage fractions can also be
defined

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 7

 Difficult cases
• x[i] = ... ; ... ; y = x[j]
– DU pair (only) if i==j
• p = &x
& ; ... ; **p = 99 ; ... ; q = x
– *p is an alias of x
• m.putFoo(...); ... ; y=n.getFoo(...);
– Are m and n the same object?
– Do m and n share a “foo” field?

• Problem of aliases: Which references are

(always or sometimes) the same?
(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 8
Data flow coverage with complex structures
• Arrays and pointers are critical for data flow analysis
– Under-estimation of aliases may fail to include some DU pairs
– Over-estimation,, on the other hand,, mayy introduce unfeasible
test obligations
• For testing, it may be preferrable to accept under-
estimation
ti ti off alias
li sett rather
th ththan over-estimation
ti ti or
expensive analysis
– Controversial: In other applications (e.g.,
(e g compilers)
compilers), a
conservative over-estimation of aliases is usually required
– Alias analysis may rely on external guidance or other global
analysis to calculate good estimates
– Undisciplined use of dynamic storage, pointer arithmetic, etc.
may make the whole analysis infeasible

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 9

 Infeasibility
• Suppose cond has not
1 if (cond) changed between 1 and 5
• Or the conditions could be
2 3 x = .... different, but the first
....
implies the second
4
...
• Then (3,5)
(3 5) is not a
(feasible) DU pair
5 • But it is difficult or
if (cond)
impossible to determine
which pairs are infeasible
6 y = x + ... 7
....
• Infeasible test
obligations are a problem
• No test case can cover
th
them

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 10

 Infeasibility
• The path
path-oriented
oriented nature of data flow analysis
makes the infeasibility problem especially
relevant
– Combinations of elements matter!
– Impossible
p to ((infallibly)
y) distinguish
g feasible from
infeasible paths. More paths = more work to check
manually.
• IIn practice,
ti reasonable
bl coverage iis ((often,
ft nott
always) achievable
– Number of paths is exponential in worst case,
case but
often linear
– All DU paths is more often impractical

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 11

 Summary
• Data flow testing attempts to distinguish
“important” paths: Interactions between
statements
• Intermediate between simple statement and branch
coverage and more expensive path-based structural testing
• C
Cover D
Def-Use
f U (DU) pairs:
i F From computation
t ti off
value to its use
• Intuition: Bad computed value is revealed only when it is
used
• Levels: All DU pairs, all DU paths, all defs (some use)
• Limits: Aliases, infeasible paths
• Worst case is bad (undecidable properties, exponential
blowup of paths),
paths) so pragmatic compromises are required

(c) 2007 Mauro Pezzè & Michal Young Ch 13, slide 12