CONTENTS

1. Overview of structural testing

2. Statement testing
3. Branch testing & Condition testing
4. Path testing: DD paths
5. Test Coverage matrics
6. Basic path testing
7. Data flow testing
8. Slice based
9. Text execution
10. Scaffolding
11. Test oracles
12. Capture and reply
13. conclusion
 Module 3 is Divided into 3 Sessions

SESSION-01 SESSION-02 SESSION-03

Introduction to Basic path testing and Slice based testing
Structural Testing data flow testing and test
execution
 Overview Of Structural Testing

 Structural testing, also known as glass box testing or white box

testing is an approach where the tests are derived from the
knowledge of the software's structure or internal implementation.
 The other names of structural testing includes clear box testing,
open box testing, logic driven testing or path driven testing.
Structural Testing Techniques:
⚫ Statement Coverage - This technique is aimed at exercising all
programming statements with minimal tests.
⚫ Branch Coverage - This technique is running a series of tests to
ensure that all branches are tested at least once.
⚫ Path Coverage - This technique corresponds to testing all
possible paths which means that each statement and branch are
covered.
 Contd,…

Advantages of Structural Testing:

⚫ Forces test developer to reason carefully about implementation
⚫ Reveals errors in "hidden" code
⚫ Spots the Dead Code or other issues with respect to
best programming practices.
Disadvantages of Structural Box Testing:
⚫ Expensive as one has to spend both time and money to perform
white box testing.
⚫ Every possibility that few lines of code is missed accidentally.
⚫ In-depth knowledge about the programming language
is necessary to perform white box testing.
 Statement testing

statement testing A test strategy in which each statement of a

program is executed at least once. It is equivalent to finding
a path (or set of paths) through the control-flow graph that
contains all the nodes of the graph. It is a weaker testing strategy
than path testing or branch testing because it (usually) requires
the least number of test cases.
Example“: print “hello world”

NOTE: statement is a command that the programmer gives to the

computer.
 Branch testing & Condition
testing
 Branch Coverage technique involves checking whether every
possible path or branch is covered. Branching is actually a jump
from one decision point to another.
Example: if condition
 Condition testing aims to cover the various conditions and its
consecutive flow. A condition or predicate when evaluates to true
must execute the next relevant line of code that follows.
Example: if else condition
 Path testing: DD paths

What is Path Testing?

⚫ Path Testing is a structural testing method based on the source
code or algorithm and NOT based on the specifications. It can be
applied at different levels of granularity.
Path Testing Assumptions:
⚫ The Specifications are Accurate
⚫ The Data is defined and accessed properly
⚫ There are no defects that exist in the system other than those that
affect control flow
 Contd,…

Path Testing Techniques:

⚫ Control Flow Graph (CFG) - The Program is converted into
Flow graphs by representing the code into nodes, regions and
edges.
⚫ Decision to Decision path (D-D) - The CFG can be broken into
various Decision to Decision paths and then collapsed into
individual nodes.
⚫ Independent (basis) paths - Independent path is a path through a
DD-path graph which cannot be reproduced from other paths by
other methods.
 Program Graphs

Definition
⚫ Given a program written in an imperative programming language,
its program graph is a directed graph in which nodes are
statement fragments, and edges represent flow of control.
⚫ If i and j are nodes in the program graph, an edge exists from
node i to node j if and only if the statement fragment
corresponding to node j can be executed immediately after the
statement fragment corresponding to node i.
 Style Choices for Program
Graphs
⚫ Deriving a program graph from a given program is an easy
process. It is illustrated here with four of the basic structured
programming constructs (Figure 3.1).
⚫ Line numbers refer to statements and statement fragments. An
element of judgment can be used here: sometimes it is convenient
to keep a fragment as a separate node; other times it seems better
to include this with another portion of a statement.
Figure 3.1: program graphs of four structured
programming constructs.
 DD-Paths

⚫ The best-known form of code-based testing is based on a

construct known as a decision-to-decision path (DD-path)
(Miller, 1977).
⚫ The name refers to a sequence of statements that, in Miller’s
words, begins with the “outway” of a decision statement and ends
with the “inway” of the next decision statement.
⚫ We will define DD-paths in terms of paths of nodes in a program
graph. In graph theory, these paths are called chains, where a
chain is a path in which the initial and terminal nodes are distinct,
and every interior node has indegree = 1 and outdegree = 1.
 Contd,…

Definition
A DD-path is a sequence of nodes in a program graph such that
⚫ Case 1: It consists of a single node with indeg = 0.
⚫ Case 2: It consists of a single node with outdeg = 0.
⚫ Case 3: It consists of a single node with indeg ≥ 2 or outdeg ≥ 2.
⚫ Case 4: It consists of a single node with indeg = 1 and outdeg =
1.
⚫ Case 5: It is a maximal chain of length ≥ 1.
 Contd,…

⚫ Cases 1 and 2 establish the unique source and sink nodes of the
program graph of a structured program as initial and final DD-
paths.
⚫ Case 3 deals with complex nodes; it assures that no node is
contained in more than one DD-path.
⚫ Case 4 is needed for “short branches”; it also preserves the one-
fragment, one DD-path principle.
⚫ Case 5 is the “normal case,” in which a DD-path is a single entry,
single-exit sequence of nodes (a chain).
⚫ The “maximal” part of the case 5 definition is used to determine
the final node of a normal (nontrivial) chain.
 Contd,…

Figure 3.2: Chain of nodes in a directed graph.

Figure 3.3: Program graph of triangle
Figure3.4:
DD-path
graph for
triangle
program.
 3.3 Test Coverage Metrics

What is Test Coverage?

Test coverage is defined as a metric in Software Testing that
measures the amount of testing performed by a set of test. It will
include gathering information about which parts of a program are
executed when running the test suite to determine which branches
of conditional statements have been taken.
In simple terms, it is a technique to ensure that your tests are
testing your code or how much of your code you exercised by
running the test.
3.3.1 Program Graph–Based Coverage
Metrics
⚫ Given a program graph, we can define the following set of test
coverage metrics. We will use them to relate to other published
sets of coverage metrics.
⚫ Common text coverage models are:
 Node coverage
 Edge coverage
 Chain coverage
 Path coverage
 Node coverage

Definition
⚫ Given a set of test cases for a program, they constitute node
coverage if, when executed on the program , every node in the
program graph is traversed. Denote this level of coverage as
Gnode, where the G stands for program graph. Since nodes
correspond to statement fragments, this guarantees that every
statement fragment is executed by some test case. If we are
careful about defining statement fragment nodes, this also
guarantees that statement fragments that are outcomes of a
decision-making statement are executed.
 Edge Coverage

Definition
⚫ Given a set of test cases for a program, they constitute edge
coverage if, when executed on the program, every edge in the
program graph is traversed. Denote this level of coverage as
Gedge.
⚫ The difference between Gnode and Gedge is that, in the latter, we
are assured that all outcomes of a decision-making statement are
executed. In our triangle problem , nodes 9, 10, 11, and 12 are a
complete if–then–else statement.
⚫ If we required nodes to correspond to full statements, we could
execute just one of the decision alternatives and satisfy the
statement coverage criterion.
 Contd,…

⚫ Because we allow statement fragments, it is natural to divide such

a statement into separate nodes (the condition test, the true
outcome, and the false outcome).
⚫ Doing so results in predicate outcome coverage. Whether or not
our convention is followed, these coverage metrics require that
we find a set of test cases such that, when executed, every
node of the program graph is traversed at least once.
 Chain Coverage

Definition
⚫ Given a set of test cases for a program, they constitute chain
coverage if, when executed on the program, every chain of length
greater than or equal to 2 in the program graph is traversed.
⚫ Denote this level of coverage as Gchain. The Gchain coverage is
the same as node coverage in the DD-path graph that
corresponds to the given program graph. Since DD-paths are
important in E.F. Miller’s original formulation of test covers we
now have a clear connection between purely program graph
constructs and Miller’s test covers.
 Path Coverage

Definition
⚫ Given a set of test cases for a program, they constitute path
coverage if, when executed on the program, every path from the
source node to the sink node in the program graph is traversed.
⚫ Denote this level of coverage as Gpath. This coverage is open to
severe limitations when there are loops in a program E.F. Miller
partially anticipated this when he postulated the C2 metric for
loop coverage.
⚫ Referring back to Chapter 4, observe that every loop in a
program graph represents a set of strongly (3-connected) nodes.
⚫ To deal with the size implications of loops, we simply exercise
every loop, and then form the condensation graph of the original
program graph, which must be a directed acyclic graph.
 Test coverage metrics

⚫ Measures the amount of testing executed.

⚫ Info about measured and unmeasured areas in testing.
⚫ 100% coverage not equals to 100% tested.
⚫% coverage=NO. of test areas covered/Total No. of
test areas*100
Why we need?
 To find areas that are mentioned in requirement doc. But
not covered in test cases.
 Helps to take decision about quality of product.
 Bridges the gap between requirement and test case.
 Contd,…

Approaches
 Program graph based coverage metrics:
 G-program graph.
 Gnode- node coverage node-statement
fragment
 Gedge- edge coverage
YES 9
NO

10 11

12
 Contd,…

 Gchain- chain coverage similar to Gnode

 linearly arranged one after the other.
 chain length > 2 is traversed.
 Gpath- path coverage
 every path from source to sink is traversed, not even single
loop is left out.
 Limitation-loops because loops are strongly connected nodes.
 Every loop has to be traversed like acyclic graph.

s s
 3.3.2 Metric-Based Testing/E.F miller’s

⚫ Organized view of the extent to which the program is

tested makes it possible to sensibly manage the testing process.
⚫ When DD-path coverage is attained by set of test cases, roughly
80% of all false are revealed.
⚫ E.F millers coverage metrics, here nodes are full statements but
formulation allows fragments to be nodes.
⚫ Concentrated about “WHAT TO TEST not HOW TO TEST”
 Contd,…

⚫ “Test coverage metrics are a device to measure the extent

to which a set of test cases covers a program”
⚫ Several widely accepted test coverage metrics are used; most of
those in Table 3.1 are due to the early work of Miller (1977).
⚫ Most quality organizations now expect the C1 metric (DD-path
coverage) as the minimum acceptable level of test coverage.
⚫ E.F Miller observes that when DD-path coverage is attained by a
set of test cases, roughly 85% of all faults are revealed.
⚫ The test coverage metrics in table 3.1 tells us what to test but not
how to test it.
⚫ Millers test coverage metrics are based on program graphs.
⚫ We take a closure look at techniques that exercise source code in
terms of metrics in table 3.1
Table 3.1 Structural Test Coverage Metrics
 Contd,…

 C0-statement testing:
 Related that to Gnode of program graph.
 Every node is traversed and tested.
 So we can say that there a gap in test coverage, if
some statements are not executed.
 C1-DD path testing:
 Gchain
 Checks each predicate outcome has been executed.
 Every edge in DD path graph is traversed.
 Contd,…

 C1p- every predicate:

 Predicate outcome, stating that every outcome
of decision(predicate) must be traversed.
 If there is a branching statement ex. If then else then both T & F
condition are tested.
 C1p=Gedge of previous approach.
 C2-C1+loop coverage:
 Gedge
 Loop testing includes 1)to traverse the loop 2) To Exit
 Contd,…

 Cd-C1+dependent pair:
 Dependent pair means variables define din one DD path
is referencing to the other.
a
 b and g are dependent pair.
 c and g are infeasible because b c
they represent two deferent conditions.
d

f g
 Contd,…

 Cik-k repetition, complex loop coverage:

 Extends loop coverage metrics.
 Full path containing loops.
 Cmcc- multiple condition coverage:
 Multiple conditions are traversed before to the
coming conclusion.
 Cstat- statistically significant:
 Comfort levels of customer/user.
 Cinfi- All possible path:
 Make sense of program is having trillions of loops.
 Statement and Predicate Testing

 Statement coverage is one of the widely used software testing. It

comes under white box testing.
 Statement coverage technique is used to design white box
test
cases.
 This technique involves execution of all statements of the source
code at least once.
 It is used to calculate the total number of executed statements in
the source code out of total statements present in the source code.
 Because our formulation allows statement fragments to be
individual nodes, the statements and predicate levels (C0 and C1)
collapse into one consideration.
 In our triangle problem nodes 9, 10, 11 and 12 are a complete if-
then-else statements.
 Contd,…

⚫ If we required nodes to correspond to full statements, we could

execute just one of the decision alternatives and satisfy the
statement coverage criterion.
⚫ Because we allow statement fragments, it is “natural” to divide
such a statement into three nodes.
⚫ Doing so results in predicate outcome coverage.
⚫ Whether or not our convention is followed, these coverage
metrics require that we find a set of test cases such that, when
executed, every node of the program graph is traversed at least
once.
 3.3.2.2 DD-Path Testing

⚫ A decision-to-decision path, or DD-path, is a path of execution

(usually through a flow graph representing a program, such as
a flow chart) between two decisions.
⚫ A DD-path is a set of nodes in a program graph such that one of
the following holds
⚫ t consists of a single node with in-degree = 0 (initial node)
⚫ It consists of a single node with out-degree = 0 (terminal node)
⚫ It consists of a single node with in-degree ≥ 2 or out-degree ≥ 2
(decision/merge points)
⚫ It consists of a single node with in-degree = 1 and out-degree = 1
⚫ It is a maximal chain of length ≥ 1.
 3.3.2.4 Dependent Pairs of DD-
Paths
⚫ Identification of dependencies must be made at the code level.
This cannot be done just by considering program graphs.
⚫ The most common dependency among pairs of DD-paths is the
define/reference relationship, in which a variable is defined
(receives a value) in one DD-path and is referenced in another
DD-path.
⚫ The importance of these dependencies is that they are closely
related to the problem of infeasible paths.
 3.3.2.7 Multiple Condition
Coverage
• Miller’s CMCC metric addresses the question of testing decisions
made by compound conditions.
⚫ Instead of simply traversing such predicates to their true and false
outcomes, we should investigate the different ways that each
outcome can occur.
⚫ One possibility is to make a decision table; a compound condition
of three simple conditions will have eight rules (see Table 3.2),
yielding eight test cases.
 3.3.2.8 Loop
Coverage
⚫ The condensation graphs provide us with an elegant resolution to
the problems of testing loops.
⚫ Loop testing has been studied extensively, and with good reason
loops are a highly fault prone portion of source code.
Figure 3.5: Concatenated, nested, and knotted loops.
 3.3.2.9 Test Coverage
Analyzers
⚫ Coverage analyzers are a class of test tools that offer automated
support for this approach to testing management.
⚫ With a coverage analyzer, the tester runs a set of test cases on a
program that has been “instrumented” by the coverage analyzer.
⚫ The analyzer then uses information produced by the
instrumentation code to generate a coverage report.
⚫ In the common case of DD-Path coverage, for example, the
instrumentation identifies and labels all DD-Paths in an original
program.
⚫ When the instrumented program is executed with test cases, the
analyzer tabulates the DD-Paths traversed by each test case.
 3.4 Basis Path Testing

⚫ In the 1970’s, Thomas McCabe came up with the idea of using a

vector space to carry out path testing.
⚫ A vector space is a set of elements along with certain operations
that can be performed upon these elements.
⚫ What makes vector spaces an attractive proposition to testers is
that they contain a basis.
⚫ The basis of a vector space contains a set of vectors that are
independent of one another, and have a spanning property; this
means that everything within the vector space can be expressed in
terms of the elements within the basis.
 Contd,…

⚫ The method devised by McCabe to carry out basis path testing

has four steps. These are:
1. Compute the program graph.
2. Calculate the cyclomatic complexity.
3. Select a basis set of paths.
4. Generate test cases for each of these paths.
 3.4.1 McCabe Basis Path Testing

• In order to show the workings of McCabe’s basis path method,

we will progress through each step before finally demonstrating
how it could be utilized as a structural testing method.
• To begin, we need a program graph from which to construct a
basis. Figure 3.6 shows an example taken from [McCabe, 1982].
• This is a commonly used example, as it demonstrates how the
basis of a graph containing a loop is computed.
• It should be noted that the graph is strongly connected;
that is, there exists an edge from the sink node to the source
node.
• This is necessary, for reasons that will be made clear
shortly.
Contd,…

Figure 3.6
McCabe’s
strongly
connected
program graph
 Contd,…

⚫ In graph theory, there exists a proof that states that the cyclomatic
complexity of a strongly connected graph is the number of
linearly independent circuits in a graph.
⚫ McCabe realised that path testing commonly makes use of
program graphs, and so felt that this theorem could be used to
find a basis that could be tested.
⚫ The cyclomatic complexity of a strongly connected graph is
provided by the formula V(G) = e – n + p.
⚫ The number of edges is represented by e, the number of nodes
by n and the number of connected areas by p.
 Contd,…

• If we apply this formula to the graph given in Figure 1.7,

the number of linearly independent circuits is:
V(G) = e – n + p = 11 – 7 + 1 = 5
• If we now delete the edge from G to A, we can see that we have
to identify 5 different independent paths to form our basis.
• The five linearly independent paths of our graph are as follows:
Path 1: A, B, C, G.
• Path 2: A, B, C, B, C, G.
• Path 3: A, B, E, F, G.
• Path 4: A, D, E, F, G.
• Path 5: A, D, F, G.
Observations on McCabe’s Basis Path Method

 In our study of FT, we observed that gaps and redundancies can

both exit and, at the same time cannot be recognized.
 Problem was that FT removes us too far from the code.
 Path testing also provides us with a set of metrics acts as cross-
checks on FT.
⚫ Rightly so, because two major soft spots occur in the McCabe
view:
 one is that testing the set of basis paths is sufficient (it is not), and
the other has to do with the yoga-like contortions we went
through to make program paths look like a vector space.
 Essential Complexity

⚫ Remove all structured components from the graph G to produce a

reduce graph R.
⚫ When carrying out his work on the basis path testing method,
McCabe developed the notion of what is now known as essential
complexity.
⚫ This is the term given for using the cyclomatic complexity to
produce a condensation graph; the result is a graph that can be
used to assist in both programming and the testing procedure.
⚫ The concept behind essential complexity is that the program
graph of a piece of software is traversed until a structured
programming construct is discovered; examples of these
constructs are shown in Figure 3.7.
 Contd,…

Once located,
the structured programming
construct is collapsed into a single
node and the graph traversal
continues.
The desired outcome of this
procedure is to end up with a graph
of V(G) = 1, that is, a program
made up of one node.

Figure 3.7 McCabe’s

Structured Constructs
 Contd,…

If a graph cannot

be reduced to one in
which there is a
cyclomatic complexity
of 1, then it means
that the program must
contain an
unstructured
programming construct.
Some examples of
these unstructured
constructs are
shown in Figure 3.8.
 Contd,…

⚫ The if–then–else construct involving nodes B, C, D, and E is

condensed into node a, and then the three if–then constructs are
condensed onto nodes b, c, and d.
⚫ The remaining if–then–else (which corresponds to the IF
IsATriangle statement) is condensed into
⚫ node e, resulting in a condensed graph with cyclomatic
complexity V(G) = 1.
⚫ In general, when a program is well structured (i.e., is
composed solely of the structured programming constructs), it
can always be reduced to a graph with one path.
Figure 3.9 Condensing with respect to structured
programming constructs.
 Guidelines and Observations

⚫ In our study of functional testing, we observed that gaps and

redundancies can both exist, and at the same time, cannot be
recognized.
⚫ McCabe was partly right when he observed: “It is important to
understand that these are purely criteria that measure the quality
of testing, and not a procedure to identify test cases”.
⚫ He was referring to the DD-Path coverage metric and the
cyclomatic complexity metric that requires at least the cyclomatic
number of distinct program paths must be traversed.
 Data Flow Testing

What is Data Flow Testing?

⚫ Data flow testing is a family of test strategies based on selecting
paths through the program's control flow in order to explore
sequences of events related to the status of variables or data
objects.
⚫ Dataflow Testing focuses on the points at which variables receive
values and the points at which these values are used.
⚫ Early “data flow” analyses often centered on a set of faults that
are now known as define/reference anomalies:
• a variable that is defined but never used (referenced)
• a variable that is used but never defined
• a variable that is defined twice before it is used
 Contd,…

⚫ Within the context of define/use testing, with respect to variables

there are two types of nodes: defining nodes and usage nodes.
The nodes are defined as follows:
 Defining nodes, referred to as DEF(v, n): Node n in the
program graph of P is a defining node of a variable v in the set V
if and only if at n, v is defined. For example, with respect to a
variable x, nodes containing statements such as “input x” and “x
= 2” would both be defining nodes.
 Usage nodes, referred to as USE(v, n): Node n in the program
graph of P is a usage node of a variable v in the set V if and only
if at n, v is used. For example, with respect to a variable x, nodes
containing statements such as “print x” and “a = 2 + x” would
both be usage nodes.
 Contd,…

⚫ Usage nodes can be split into a number of types, depending on

how the variable is used.
⚫ The two major types of usage node are:
 P-use: predicate use – the variable is used when making
a decision (e.g. if b > 6).
 • C-use: computation use – the variable is used in a computation
(for example, b = 3 + d – with respect to the variable d).
 There are also three other types of usage node, which are
all, in effect, subclasses of the C-use type:
 Contd,…

 O-use: output use – the value of the variable is output to the

external environment (for instance, the screen or a printer).
 • L-use: location use – the value of the variable is used, for
instance, to determine which position of an array is used (e.g.
a[b]).
 • I-use: iteration use – the value of the variable is used to
control the number of iterations made by a loop (for example: for
(int i = 0; i <= 10; i++)).
Commission
problem and
its program
graph.
 Du-paths for
Stocks
⚫ First, let us look at a simple path: the du-path for the variable
stocks. We have DEF(stocks, 15) and USE(stocks, 17), so the
path <15, 17> is a du-path with respect to stocks.
⚫ No other defining nodes are used for stocks; therefore, this path is
also definition clear.
Dc-path- only one node present.
 Du-paths for
Locks
⚫ Two defining and two usage nodes make the locks variable more
interesting: we have DEF(locks, 13), DEF(locks, 19), USE(locks,
14), and USE(locks, 16). These yield four du-paths;
⚫ p1 = <13, 14>
⚫ p2 = <13, 14, 15, 16>
⚫ p3 = <19, 20, 14>
⚫ p4 = <19, 20, 14, 15, 16>
 Contd,…

⚫ NOTE: du-paths p1 and p2 refer to the priming value of

locks, which is read at node 13.
⚫ The locks variable has a predicate use in the while statement
(node 14), and if the condition is true (as in path p2), a
computation use at statement 16.
⚫ The other two du-paths start near the end of the while loop
and occur when the loop repeats.
⚫ These paths provide the loop coverage discussed in
Chapter 8—bypass the loop, begin the loop, repeat the loop,
and exit the loop.
⚫ All these du-paths are definition clear.
Du-paths for locks.
 Du-paths for
Sales
⚫ There is one defining node for sales; therefore, all the du-paths
with respect to sales must be definition clear.
⚫ They are interesting because they illustrate predicate
and computation uses. The first three du-paths are easy:
⚫ p10 = <27, 28>
⚫ p11 = <27, 28, 29>
⚫ p12 = <27, 28, 29, 30, 31, 32, 33>
⚫ Notice that p12 is a definition-clear path with three usage nodes;
it also contains paths p10 and p11.
⚫ If wewere testing with p12, we know we would also
have
covered the other two paths.
 Contd,…

⚫ The IF, ELSE IF logic in statements 29 through 40 highlights an

ambiguity in the original research.
⚫ Two choices for du-paths begin with path p11: one choice is the
path <27, 28, 29, 30, 31, 32, 33>, and the other is the path <27,
28, 29, 34>.
⚫ The remaining du-paths for sales are
⚫ p13 = <27, 28, 29, 34>
⚫ p14 = <27, 28, 29, 34, 35, 36, 37>
⚫ p15 = <27, 28, 29, 34, 38>
 Du-paths for
Commission
⚫ it is time for a change of pace.
⚫ In statements 29 through 41, the calculation of commission
is controlled by ranges of the variable sales.
⚫ Statements 31 to 33 build up the value of commission by using
the memory location to hold intermediate values.
⚫ consider du-paths that begin with the three “real”
⚫ defining nodes: DEF(commission, 33), DEF(commission,
and
37), DEF(commission, 39). one usage node is
Only USE(commission, 41). used:
⚫ Below code explains the above .
Contd,…

Commission
problem code
with Unique
numbers.
 Define/Use Test Coverage Metrics

⚫ The whole point of analyzing a program with definition/use paths

is to define a set of test coverage metrics known as the Rapps–
Weyuker data flow metrics.
⚫ The first three of these are equivalent to three of E.F. Miller’s
metrics All-Paths, All-Edges, and All-Nodes.
⚫ The others presume that define and usage nodes have been
identified for all program variables, and that du-paths have been
identified with respect to each variable.
⚫ T is a set of paths in the program graph G(P) of a program P,
with the set V of variables.
 Contd,…

⚫ It is not enough to take the cross product of the set of DEF nodes
with the set of USE nodes for a variable to define du-paths.
⚫ This mechanical approach can result in infeasible paths.
⚫ In the next definitions, we assume that the define/use paths are all
feasible.
 Contd,…

Definition
⚫ The set T satisfies the All-Defs criterion for the program P if and
only if for every variable v ∈ V, T contains definition-clear paths
from every defining node of v to a use of v.
Definition
⚫ The set T satisfies the All-Uses criterion for the program P if and
only if for every variable v ∈ V, T contains definition-clear paths
from every defining node of v to every use of v, and to the
successor node of each USE(v, n).
 Contd,…

Definition
⚫ The set T satisfies the All-P-Uses/Some C-Uses criterion for the
program P if and only if for every variable v ∈ V, T contains
definition-clear paths from every defining node of v to every
predicate use of v; and if a definition of v has no P-uses, a
definition-clear path leads to at least one computation use.
⚫ Definition
⚫ The set T satisfies the All-C-Uses/Some P-Uses criterion for the
program P if and only if for every variable v ∈ V, T contains
definition clear paths from every defining node of v to every
computation use of v; and if a definition of v has no C-uses, a
definition-clear path leads to at least one predicate use.
 Contd,…

Definition
⚫ The set T satisfies the All-DU-paths criterion for the program P if
and only if for every variable v ∈ V, T contains definition-clear
paths from every defining node of v to every use of v and to the
successor node of each USE(v, n), and that these paths are either
single loop traversals or they are cycle free.
⚫ These test coverage metrics have several set-theory-based
relationships, which are referred to as “subsumption” in Rapps
and Weyuker (1985).
⚫ These relationships are shown in Figure below.
R apps–Weyuker hierarchy of data flow coverage metrics.
Define/Use Testing for Object-Oriented Code

⚫ All of the define/use definitions thus far make no mention of

where the variable is defined and where it is used.
⚫ In a procedural code, this is usually assumed to be within a unit,
but it can involve procedure calls to improperly coupled units.
⚫ We might make this distinction by referring to these definitions as
“context free”; that is, the places where variables are defined and
used are independent.
⚫ The object-oriented paradigm changes this—we must now
consider the define and use locations with respect to class
aggregation, inheritance, dynamic binding, and polymorphism.
⚫ The bottom line is that data flow testing for object-oriented code
moves from the unit level to the integration level.
 Slice-Based Testing

⚫ Originally proposed by [weiser 88] and [Gallager 91] in software

maintenance.
⚫ Useful for
 Software debugging
 Software maintenance
 Program understanding
 Qualification of functional cohesion
 Program Slicing

⚫ A complementary method of program decomposition that applied

after the program is written/designed.
⚫ Purpose
 Debugging (subset of behavior is fixed)
 Maintenance(sunset of program is improved)
⚫ Work on actual program
 Specify program behaviors using sets of variables at some sets of
statements.
 Slicing Criterion

⚫ Slicing criterion for a program specifies a windows for observing

its behavior.
 Where a window
 Specified as a statement and a set of variables
 Allows the observations of values of the specified
variables before execution of particular statement
 Example: S(x,24) (i.e value of x at line 24)
 Informal definition of SBT

⚫ Slice based testing technique

 Divide a program into components (slices)
⚫ A program slice?
 Refresh to a set of program statements that may affect
the computation of variable v at statement s.
Example
 Formal definition of SBT

⚫ Given a program P, program graph G(P) in which statements are

numbered and a set V of variables in P and slice on the variable
set V at statement n, S(V,n)
 The set of node(line) numbers of all the statements fragments in P
prior to n that contribute to the values of the variables in V at
statement fragment n.
 The notation of contribution:
USE
⚫ The notation of contribute can be by the USE
described relationship
⚫ P-use used in a predicate (decision)
⚫ C-use used in computation
⚫ O-use used for output
⚫ L-use used for location (pointers, subscripts)
⚫ I-use iteration (internal counters, loop indices)
 The notation of contribution:
Definitions
⚫ Most of the literature on program slices just uses P-uses and C-
uses.
⚫While we are at it, we identify two forms of definition nodes:
I-def defined by input(e.g: read)
A-def defined by assignment(e.g: a:=X+Y)
 Properties of slice

⚫ There are two properties in slice

 The slice must have been obtained from the original program by
statement deletion
 The behavior of the slice must correspond to the behavior
of program as observed through the window of the slicing
criterion.
 Example

⚫ The commission problem is used in this book because it contains

interesting data flow properties, and these are not present in the
triangle problem.
⚫ In the following, except where specifically noted, we are
speaking of static backward slices and we only include nodes
corresponding to executable statement fragments.
⚫ There are 42 “interesting” static backward slices in
⚫ our example. They are named in Table below We will take a
selective look at some interesting slices.
⚫ The first six slices are the simplest—they are the nodes where
variables are initialized.
Example
 Contd,…
⚫ S1: S(lockPrice, 7) = {7} ⚫ S11: S(locks, 16) = {13, 14, 19,
⚫ S2: S(stockPrice, 8) = {8} 20}
⚫ S3: S(barrelPrice, 9) = {9} ⚫ S12: S(totalLocks, 16) = {10, 13,
⚫ S4: S(totalLocks, 10) = {10}
14, 16, 19, 20}
⚫ S13: S(stocks, 17) = {13, 14, 15,
⚫ S5: S(totalStocks, 11) = {11}
19, 20}
⚫ S6: S(totalBarrels, 12) = {12}
⚫ S14: S(totalStocks, 17) = {11, 13,
⚫ S7: S(locks, 13) = {13}
14, 15, 17, 19, 20}
⚫ S8: S(locks, 14) = {13, 14, 19,
⚫ S15: S(barrels, 18) = {12, 13, 14,
20} 15, 19, 20}
⚫ S9: S(stocks, 15) = {13, 14, 15,
⚫ S16: S(totalBarrels, 18) =
19, 20} {12, 13, 14, 15, 18, 19, 20}
⚫ S10: S(barrels, 15) = {13, 14, 15,
⚫ S17: S(locks, 19) = {13, 14, 19,
19, 20} 20}
 Contd,…

⚫ Slices 7 through 17 focus on the sentinel controlled while loop in

which the totals for locks, stocks, and barrels are accumulated.
⚫ The locks variable has two uses in this loop: a P-use at fragment
14 and C-use at statement 16. It also has two defining nodes, at
statements 13 and 19.
⚫ The stocks and barrels variables have a defining node at 15, and
computation uses at nodes 17 and 18, respectively.
⚫ Notice the presence of all relevant statement fragments in slice 8.
⚫ Slices 18, 19, and 20 are output statements, and none of the
variables is defined; hence, the corresponding statements are not
included in these slices.
 Contd,…
⚫ S18: S(totalLocks, 21) = {10, ⚫ S25: S(totalStocks, 25) = {11,
13, 14, 16, 19, 20} 13, 14, 15, 17, 19, 20}
⚫ S19: S(totalStocks, 22) = {11, ⚫ S26: S(stockSales, 25) =
13, 14, 15, 17, 19, 20} {8, 11, 13, 14, 15, 17, 19, 20,
⚫ S20: S(totalBarrels, 23) = {12, 25}
13, 14, 15, 18, 19, 20} ⚫ S27: S(barrelPrice, 26) = {9}
⚫ S21: S(lockPrice, 24) = {7} ⚫ S28: S(totalBarrels, 26) = {12,
⚫ S22: S(totalLocks, 24) = {10, 13, 14, 15, 18, 19, 20}
13, 14, 16, 19, 20} ⚫ S29: S(barrelSales, 26) =
⚫ S23: S(lockSales, 24) = {7, 10, {9, 12, 13, 14, 15, 18, 19, 20,
13, 14, 16, 19, 20, 24} 26}
⚫ S24: S(stockPrice, 25) = {8} ⚫ S30: S(sales, 27) = {7, 8, 9, 10,
11, 12, 13, 14, 15, 16, 17, 18,
19, 20, 24, 25, 26, 27}
Contd,…

Figure : 9.8
Partial lattice
of slices on
commission.
 Contd,…

Figure 9.9 Full lattice on commission.

 Contd,…

⚫ Looking at slices as sets of fragment numbers (Figure 9.8) is

correct in terms of our definition, but it is also helpful to see how
slices are composed of sets of previous slices. We do this next,
and show the final lattice in Figure 9.9.
 Contd,…

⚫ S1: S(lockPrice, 7) = {7}

⚫ S2: S(stockPrice, 8) = {8}
⚫ S3: S(barrelPrice, 9) = {9}
⚫ S4: S(totalLocks, 10) = {10}
⚫ S5: S(totalStocks, 11) = {11}
⚫ S6: S(totalBarrels, 12) = {12}
⚫ S7: S(locks, 13) = {13}
⚫ S8: S(locks, 14) = S7 𝖴 {14, 19, 20}
⚫ S9: S(stocks, 15) = S8 𝖴 {15}
⚫ S10: S(barrels, 15) = S8
⚫ S11: S(locks, 16) = S8
⚫ S12: S(totalLocks, 16) = S4 𝖴 S11 𝖴 {16}
⚫ S13: S(stocks, 17) = S9 = {13, 14, 19, 20}
 Contd,…

⚫ S14: S(totalStocks, 17) = S5 𝖴 S13 𝖴 {17}

⚫ S15: S(barrels, 18) = S6 𝖴 S10
⚫ S16: S(totalBarrels, 18) = S6 𝖴 S15 𝖴 {18}
⚫ S18: S(totalLocks, 21) = S12
⚫ S19: S(totalStocks, 22) = S14
⚫ S20: S(totalBarrels, 23) = S16
⚫ S21: S(lockPrice, 24) = S1
⚫ S22: S(totalLocks, 24) = S12
⚫ S23: S(lockSales, 24) = S21 𝖴 S22 𝖴 {24}
⚫ S24: S(stockPrice, 25) = S2
⚫ S25: S(totalStocks, 25) = S14
 Contd,…

⚫ S26: S(stockSales, 25) = S24 𝖴 S25 𝖴 {25}

⚫ S27: S(barrelPrice, 26) = S3
⚫ S28: S(totalBarrels, 26) = S20
⚫ S29: S(barrelSales, 26) = S27 𝖴 S28 𝖴 {26}
⚫ S30: S(sales, 27) = S23 𝖴 S26 𝖴 S29 𝖴 {27}
⚫ S31: S(sales, 28) = S30
⚫ S32: S(sales, 29) = S30
⚫ S33: S(sales, 33) = S30
⚫ S34: S(sales, 34) = S30
⚫ S35: S(sales, 37) = S30
 Contd,…

⚫ S36: S(sales, 39) = S30

⚫ S37: S(commission, 31) = S30 𝖴 {29, 30, 31}
⚫ S38: S(commission, 32) = S37 𝖴 {32}
⚫ S39: S(commission, 33) = S38 𝖴 {33}
⚫ S40: S(commission, 36) = S30 𝖴 {29, 34, 35, 36}
⚫ S41: S(commission, 37) = S40 𝖴 {37}
⚫ S42: S(commission, 39) = S30 𝖴 {29, 34, 38, 39}
⚫ S43: S(commission, 41) = S39 𝖴 S41 𝖴 S42
 Style and Technique

⚫ 1. Never make a slice S(V, n) for which variables v of V do not

appear in statement fragment n. This possibility is permitted by
the definition of a slice, but it is bad practice. As an example,
suppose we defined a slice on the locks variable at node 27.
Defining such slices necessitates tracking the values of all
variables at all points in the program.
⚫ 2. Make slices on one variable. The set V in slice S(V, n) can
contain several variables, and sometimes such slices are useful.
The slice S(V, 27) where
⚫ V = {lockSales, stockSales, barrelSales}
⚫ contains all the elements of the slice S30: S(sales, 27) except
statement 27.
 Contd,…

⚫ 3. Make slices for all A-def nodes. When a variable is computed

by an assignment statement, a slice on the variable at that
statement will include (portions of ) all du-paths of the variables
used in the computation. Slice S30: S(sales, 27) is a good
example of an A-def slice. Similarly for variables defined by
input statements (I-def nodes), such as S10: S(barrels, 15).
⚫ 4. There is not much reason to make slices on variables that occur
in output statements. Slices on O-use variables can always be
expressed as unions of slices on all the A-defs (and I-defs) of the
O-use variable.
 Contd,…

⚫ 5. Make slices for P-use nodes. When a variable is used in a

predicate, the slice on that variable at the decision statement
shows how the predicate variable got its value. This is very useful
in decision-intensive programs such as the triangle program and
NextDate.
⚫ 6. Consider making slices compliable.
 Guidelines and Observations

⚫ 1. Slices don’t map nicely into test cases (because the other, non-
related code is still in an executable path). On the other hand,
they are a handy way to eliminate interaction among variables.
Use the slice composition approach to re-develop difficult
sections of code, and these slices before you splice (compose)
them with other slices.
⚫ 2. Relative complements of slices yield a “diagnostic” capability.
The relative complement of a set B with respect to another set A
is the set of all elements of A that are not elements of B. It is
denoted as A -B. Consider the relative complement set
S(commission, 48) - S(sales, 35):
 Contd,…

⚫ S(commission, 48) = {3, 4, 5,36,18,19, 20, 23, 24, 25, 26, 27, 34,
38, 39, 40, 44,45,47}
⚫ S(sales, 35) = {3, 4, 5, 36, 18, 19, 20, 23, 24, 25, 26, 27}
⚫ S(commission, 48) - S(sales, 35) = {34, 38, 39, 40, 44,45,47}
⚫ If there is a problem with commission at line 48, we can divide
the program into two parts, the computation of sales at line 34,
and the computation of commission between lines 35 and 48. If
sales is OK at line 34, the problem must lie in the relative
complement; if not, the problem may be in either portion.
 Contd,…

⚫ 3. There is a many-to-many relationship between slices and DD-

Paths: statements in one slice may be in several DD-Paths, and
statements in one DD-Path may be in several slices.
⚫ 4. If you develop a lattice of slices, it’s convenient to postulate a
slice on the very first statement. This way, the lattice of slices
always terminates in one root node. Show equal slices with a
two-way arrow.
⚫ 5. Slices exhibit define/reference information. Consider the
following slices on num_locks:
 Contd,…

⚫ S(num_locks, 17) = φ
⚫ S(num_locks, 24) = {17, 20, 27?}
⚫ S(num_locks, 31) = {17, 20, 24, 27}
⚫ S(num_locks, 34) = {17, 20, 24, 27}
⚫ S(num_locks, 17) is the first definition of num_locks.
⚫ S(num_locks, 24) - S(num_locks, 17) is a definition-clear, define
reference path.
⚫ When slices are equal, the corresponding paths are
definition- clear.
 Test Execution

⚫ Whereas test design, even when supported by tools, requires

insight and ingenuity in similar measure to other facets of
software design, test execution must be sufficiently automated for
frequentreexecution without little human involvement.
⚫ This chapter describes approaches for creating the run-time
support for generating and managing test data, creating
scaffolding for test execution, and automatically distinguishing
between correct and incorrect test case executions.
 From Test Case Specifications to Test Cases

⚫ If the test case specifications produced in test design already

include concrete input values and expected results, as for
example in the category-partition method, then producing a
complete test case may be as simple as filling a template with
those values.
⚫ A more general test case specification may designate many
possible concrete test cases, and it may be desirable to generate
just one instance or many.
⚫ There is no clear, sharp line between test case design and test case
generation.
⚫ A rule of thumb is that, while test case design involves judgment
and creativity, test case generation should be a mechanical step.
 Contd,…

⚫ Automatic generation of concrete test cases from more abstract

test case specifications reduces the impact of small interface
changes in the course of development.
⚫ Corresponding changes to the test suite are still required with
each program change, but changes to test case specifications are
likely to be smaller and more localized than changes to the
concrete test cases.
 Scaffolding

⚫ During much of development, only a portion of the full system is

available for testing.
⚫ In modern development methodologies, the partially developed
system is likely to consist of one or more runnable programs and
may even be considered a version or prototype of the final system
from very early in construction, so it is possible at least to
execute each new portion of the software as it is constructed, but
the external interfaces of the evolving system may not be ideal
for testing; often additional code must be added.
 Contd,…

⚫ For example, even if the actual subsystem for placing an order

with a supplier is available and fully operational, it is probably
not desirable to place a thousand supply orders each night as part
of an automatic test run.
 Generic versus Specific Scaffolding

⚫ The simplest form of scaffolding is a driver program that runs a

single, specific test case.
⚫ If, for example, a test case specification calls for executing
method calls in a particular sequence, this is easy to accomplish
by writing the code to make the method calls in that sequence.
⚫ Writing hundreds or thousands of such test-specific drivers, on
the other hand, may be cumbersome and a disincentive to
thorough testing.
⚫ At the very least one will want to factor out some of the common
driver code into reusable modules.
 Contd,…

⚫ Sometimes it is worthwhile to write more generic test drivers that

essentially interpret test case specifications.
⚫ At least some level of generic scaffolding support can be
used across a fairly wide class of applications.
 Test Oracles

⚫ It is little use to execute a test suite automatically if execution

results must be manually inspected to apply a pass/fail criterion.
⚫ Relying on human intervention to judge test outcomes is not
merely expensive, but also unreliable. Even the most
conscientious and hard-working person cannot maintain the
level of attention required to identify one failure in a
hundred program executions, little more one or ten
thousand.
 Contd,…

Figure 3.8: A test harness with a comparison-based test oracle

processes test cases consisting of (program input, predicted output)
pairs.
 Self-Checks as Oracles

⚫ A program or module specification describes all correct program

behaviors, so an oracle based on a specification need not be
paired with a particular test case.
⚫ Self-check assertions may be left in the production version of a
system, where they provide much better diagnostic information
than the uncontrolled application crash the customer may
otherwise report.
 Contd,…

Figure 3.8: When self-checks are embedded in the program, test

cases need not include predicted outputs.
 Contd,…

⚫ If this is not acceptable - for instance, if the cost of a runtime

assertion check is too high – most tools for assertion processing
also provide controls for activating and deactivating assertions.
⚫ It is generally considered good design practice to make assertions
and self-checks be free of side- effects on program state. Side-
effect free assertions are essential when assertions may be
deactivated, because otherwise suppressing assertion checking
can introduce program failures that appear only when one is not
testing.
Contd,…
 Capture and Replay

⚫ Sometimes it is difficult to either devise a precise description of

expected behavior or adequately characterize correct behavior for
effective self-checks.
⚫ For example, while many properties of a program with a
graphical interface may be specified in a manner suitable for
comparison-based or self-check oracles, some properties are
likely to require a person to interact with the program and judge
its behavior.
 Contd,…

⚫ If one cannot completely avoid human involvement in test case

execution, one can at least avoid unnecessary repetition of this
cost and opportunity for error. The principle is simple.
⚫ The first time such a test case is executed, the oracle function is
carried out by a human, and the interaction sequence is captured.
⚫ Provided the execution was judged (by the human tester) to be
correct, the captured log now forms an (input, predicted output)
pair for subsequent automated retesting.
 Contd,…

⚫ The savings from automated retesting with a captured log

depends on how many build- and test cycles we can continue to
use it in, before it is invalidated by some change to the program.
⚫ Distinguishing between significant and insignificant variations
from predicted behavior, in order to prolong the effective lifetime
of a captured log, is a major challenge for capture/replay testing.
⚫ Capturing events at a more abstract level suppresses insignificant
changes.
⚫ For example, if we log only the actual pixels of windows and
menus, then changing even a typeface or background color can
invalidate an entire suite of execution logs.
THANK
YOU