TEST CASE DESIGN

UNIT II
Introduction to Testing Design Strategies
• The Smart Tester
• Test Case Design Strategies
• Using Black-Box approach to Test Case Design
– Random Testing
– Equivalence Class Partitioning
– Boundary Value Analysis
– Decision tables
– Requirements based
– Positive & Negative Testing
– Cause effect Graphing
– Partitioning State Based Testing
– Error Guessing
– Compatibility Testing
– User Documentation Testing
– Domain Testing
The Smart Tester
Test Case Design Strategies
Using Black-Box approach to Test Case Design
✓ Random Testing

—if the valid input domain for a module is all positive

integers between 1 and 100, the tester using this approach
would randomly, or unsystematically, select values from
within that domain; for example, the values 55, 24, 3 might
be chosen.
 Some of the issues that remain open are the following:

• Are the three values adequate to show that the module meets
its specification when the tests are run? Should additional or
fewer values be used to make the most effective use of
resources?
• Are there any input values, other than those selected, more
likely to reveal defects? For example, should positive integers
at the beginning or end of the domain be specifically selected
as inputs?
• Should any values outside the valid domain be used as test
inputs? For example, should test data include floating point
values, negative values, or integer values greater than 100?
Equivalence Class Partitioning

• If a tester is viewing the software-under-test as a black box with

well defined inputs and outputs.
• A good approach to selecting test inputs is to use a method
called equivalence class partitioning.
• Using equivalence class partitioning a test value in a particular
class is equivalent to a test value of any other member of that
class.
• Therefore, if one test case in a particular equivalence class
reveals a defect, all the other test cases based on that class
would be expected to reveal the same defect.
This technique has the following advantages:

1. It eliminates the need for exhaustive testing, which is not

feasible.
2. It guides a tester in selecting a subset of test inputs with a
high probability of detecting a defect.
3. It allows a tester to cover a larger domain of inputs/outputs
with a smaller subset selected from an equivalence class.
The input conditions usually come from a description in the
specification of the software to be tested.
The tester uses the conditions to partition the input domain into
equivalence classes and then develops a set of tests cases to cover
(include) all the classes.

The tester and the analyst interact during the analysis phase to
develop -
(i) a set of testable requirements, and
(ii) a correct and complete input/output specification.
From these the tester develops -
(i) a high-level test plan, and
(ii) a preliminary set of black box test cases for the system.
Both the plan and the test cases undergo further development in
subsequent life cycle phases.
important points related to equivalence class partitioning
• The tester must consider both valid and invalid equivalence
classes. Invalid classes represent erroneous or unexpected
inputs.
• Equivalence classes may also be selected for output
conditions.
• The derivation of input or outputs equivalence classes is a
heuristic process.
• Some conditions only give the tester guidelines for
identifying the partitions. (Discuss later)
• In some cases it is difficult for the tester to identify
equivalence classes.
Equivalence Class Partitioning
E.g.
✓ Suppose the specification for a module says that an input, the length of a
widget in millimeters, lies in the range 1–499;
Then select one valid equivalence class that includes all values from 1 to
499. Select a second equivalence class that consists of all values less than
1, and a third equivalence class that consists of all values greater than 499.
✓ If the specification for a paint module states that the colors RED, BLUE,
GREEN and YELLOW are allowed as inputs, then select one valid
equivalence class that includes the set RED, BLUE, GREEN and YELLOW,
and one invalid equivalence class for all other inputs.
✓ If the specification for a module states that the first character of a part
identifier must be a letter, then select one valid equivalence class where
the first character is a letter, and one invalid class where the first character
is not a letter.
Boundary Value Analysis
• With experience, testers soon realize that many defects occur
directly on, and above and below, the edges of equivalence
classes.
• equivalence class partitioning directs the tester to select test
cases from any element of an equivalence class,
• boundary value analysis requires that the tester select
elements close to the edges, so that both the upper and lower
edges of an equivalence class are covered by test cases.
The rules-of-thumb boundary value analysis -
1. If an input condition for the software-under-test is specified
as a range of values, develop valid test cases for the ends of
the range, and invalid test cases for possibilities just above
and below the ends of the range.
2. If an input condition for the software-under-test is specified
as a number of values, develop valid test cases for the
minimum and maximum numbers as well as invalid test cases
that include one lesser and one greater than the maximum
and minimum.
3. If the input or output of the software-under-test is an
ordered set, such as a table or a linear list, develop tests that
focus on the first and last elements of the set.
Boundary Value Analysis
Example
Other Black Box Test Design Approaches
There are alternative methods to equivalence class
partitioning/boundary value analysis that a tester can use to
design test cases based on the functional specification for the
software to be tested.
• Cause-and-effect graphing
• State transition testing
• Error guessing.
Cause-and-effect graphing
• A major weakness with equivalence class partitioning is that it
does not allow testers to combine conditions.
• Cause-and-effect graphing is a technique that can be used to
combine conditions and derive an effective set of test cases that
may disclose inconsistencies in a specification.
• The specification must be transformed into a graph that
resembles a digital logic circuit.
• The tester should have knowledge of Boolean logic.
• The graph must be converted to a decision table that the tester
uses to develop test cases.
The steps in developing test cases with a cause-and-effect graph are
1. The tester must decompose the specification of a complex software
component into lower-level units.
2. For each specification unit, the tester needs to identify causes and their
effects.
• A cause is a distinct input condition or an equivalence class of input
conditions.
• An effect is an output condition or a system transformation.
3. From the cause-and-effect information, a Boolean cause-and-effect graph is
created.
• Nodes in the graph are causes and effects.
• Causes are placed on the left side of the graph and effects on the right.
• Logical relationships are expressed using standard logical operators such
as AND, OR, and NOT, and are associated with arcs.
4. The graph may be annotated with constraints that describe combinations of
causes and/or effects that are not possible due to environmental or
syntactic constraints.
5. The graph is then converted to a decision table.
6. The columns in the decision table are transformed into test cases.
Samples of cause-and-effect graph notations.
EXAMPLE
Suppose we have a specification for a module that allows a
user to perform a search for a character in an existing string.
The specification states that the user must input the length of
the string and the character to search for. If the string length is
out-of-range an error message will appear. If the character
appears in the string, its position will be reported. If the
character is not in the string, the message “not found” will be
output.

The input conditions, or causes are as follows:

C1: Positive integer from 1 to 80
C2: Character to search for is in string
The output conditions, or effects are:
E1: Integer out of range
E2: Position of character in string
E3: Character not found
The rules or relationships can be described as follows:
If C1 and C2, then E2.
If C1 and not C2, then E3.
If not C1, then E1.
• The next step is to develop a decision table. The decision table
reflects the rules and the graph and shows the effects for all
possible combinations of causes. Columns list each combination
of causes, and each column represents a test case.
• A decision table will have a row for each cause and each effect.
• The entries are reflection of the rules and the entities in the
cause-and-effect graph.
• Entries in the table can be represented by a “1” for a cause or
effect that is present, a “0” represents the absence of a cause or
effect, and a “—” indicates a “don’t care” value.
• A decision table for our simple example is shown in Table 4.3
where C1, C2, C3 represent the causes, E1, E2, E3 the effects,
and columns T1, T2, T3 the test cases.
In this example, three test cases are called for. If the existing
string is “abcde,” then possible tests are the following:

Inputs Length Character to search for Outputs

T1 5 c 3
T2 5 w Not found
T3 90 Integer out of range
State Transition Graph Based Testing

• State transition testing is useful for both procedural and object-

oriented development.
• It is based on the concepts of states and finite-state machines
and allows the tester to view the developing software in term of
its states, transitions between states, and the inputs and events
that trigger state changes.
• A state is an internal configuration of a system or component. It
is defined in terms of the values assumed at a particular time for
the variables that characterize the system or component.
• A finite-state machine is an abstract machine that can be
represented by a state graph having a finite number of states and
a finite number of transitions between states.
• During the specification phase a state transition graph (STG)
may be generated for the system as a whole and/or specific
modules.
• In object-oriented development the graph may be called a
state chart.
• STG/state charts are commonly depicted by a set of nodes
(circles, ovals, rounded rectangles) which represent states.
These usually will have a name or number to identify the state.
• A set of arrows between nodes indicate what inputs or events
will cause a transition or change between the two linked
states.
• Outputs/actions occurring with a state transition are also
depicted on a link or arrow.
• S1 and S2 are the two states of interest.
• The black dot represents a pointer to the initial state from outside
the machine.
• Many STGs also have “error” states and “done” states, the latter to
indicate a final state for the system. The arrows display
inputs/actions that cause the state transformations in the arrow
directions.
• For example, the transition from S1 to S2 occurs with input, or
event B.
• Action 3 occurs as part of this state transition. This is represented
by the symbol “B/act3.”
Error Guessing
• Designing test cases using the error guessing approach is based
on the tester’s/developer’s past experience with code similar to
the code-under test, and their intuition as to where defects may
lurk in the code.
• Code similarities may extend to the structure of the code, its
domain, the design approach used, its complexity, and other
factors.
• The tester/developer is sometimes able to make an educated
“guess” as to which types of defects may be present and design
test cases to reveal them.
• Error guessing is an ad hoc approach to test design in most
cases.
Using the White Box Approach to Test Case Design
✓ Test Adequacy Criteria
✓ Coverage & Control Flow Graphs
✓ Code Logic
✓ Paths –Their Role in White-Box based Test Design
✓ Code Complexity Testing
✓ Evaluating Test Adequacy Criteria
• black box test design approach that considers the software to
be tested with a well-defined set of inputs and outputs that
are described in a specification.
• white box test design approach, It will be examined where the
tester has knowledge of the internal logic structure of the
software under test.
• The tester’s goal is to determine if all the logical and data
elements in the software unit are functioning properly.
• This is called the white box, or glass box approach to test case
design.
Test Adequacy Criteria
• The goal for white box testing is to ensure that the internal
components of a program are working properly.
• A common focus is on structural elements such as statements
and branches.
• The tester develops test cases that exercise these structural
elements to determine if defects exist in the program
structure.
• By exercising all of the selected structural elements the tester
hopes to improve the chances for detecting defects.
• Testers need a framework for deciding which structural
elements to select as the focus of testing, for choosing the
appropriate test data, and for deciding when the testing
efforts are adequate enough to terminate the process with
confidence that the software is working properly.

• Such a framework exists in the form of test adequacy criteria.

• The criteria can be viewed as representing minimal standards

for testing a program.
The application scope of adequacy criteria also includes:

1. Helping testers to select properties of a program to focus on

during test.
2. Helping testers to select a test data set for a program based on
the selected properties.
3. Supporting testers with the development of quantitative
objectives for testing.
4. Indicating to testers whether or not testing can be stopped for
that program.
• If a test data adequacy criterion focuses on the structural
properties of a program it is said to be a program-based
adequacy criterion.
• They use either logic and control structures, data flow,
program text, or faults as the focal point of an adequacy
evaluation.
Test Adequacy Criteria

• Test data set is statement, or branch, adequate if a test set T

for program P causes all the statements, or branches, to be
executed respectively.
• If there are, for example, four branches in the software unit,
and only two are executed by the planned set of test cases,
then the degree of branch coverage is 50%. All four of the
branches must be executed by a test set in order to achieve
the planned testing goal. When a coverage goal is not met, as
in this example, the tester develops additional test cases and
re-executes the code. This cycle continues until the desired
level of coverage is achieved.
Coverage and Control Flow Graphs
• The application of coverage analysis is typically associated
with the use of control and data flow models to represent
program structural elements and data.
• The logic elements most commonly considered for coverage
are based on the flow of control in a unit of code.
For example,
1. program statements;
2. decisions/branches (these influence the program flow of
control);
3. conditions (expressions that evaluate to true/false, and do
not contain any other true/false-valued expressions);
4. combinations of decisions and conditions;
5. paths (node sequences in flow graphs).
All structured programs can be built from three basic primes-
1. sequential (e.g., assignment statements).
2. decision (e.g., if/then/else statements).
3. and iterative (e.g., while,for loops).

* A program prime is an atomic programming unit.

The flow graph can be used by the tester to evaluate the code with
respect to its testability, as well as to develop white box test cases.

Figure 5.2
Figure 5.3
• There are commercial tools that will generate control flow
graphs from code and in some cases from pseudo code.
• The tester can use tool support for developing control flow
graphs especially for complex pieces of code.
• A control flow representation for the software under test
facilitates the design of white box–based test cases as it clearly
shows the logic elements needed to design the test cases
using the coverage criterion of choice.
Covering Code Logic
• testers can use the concepts and tools to decide on the target
logic elements and the degree of coverage that makes sense
in terms of the type of software, its mission or safety
criticalness, and time and resources available.
• For example, if the tester selects the logic element “program
statements,” this indicates that she will want to design tests
that focus on the execution of program statements.
• If the goal is to satisfy the statement adequacy/ coverage
criterion, then the tester should develop a set of test cases so
that when the module is executed, all (100%) of the
statements in the module are executed at least once.
• In terms of a flow graph model of the code, satisfying this
criterion requires that all the nodes in the graph are exercised
at least once by the test cases.
• If the tests achieve this goal, the test data would satisfy the
statement adequacy criterion.
Paths: Their Role in White Box–Based Test Design
• In previous Section the role of a control flow graph as an aid
to white box test design was described.
• It was also mentioned that tools were available to generate
control flow graphs.
• These tools typically calculate a value for a software attribute
called McCabe’s Cyclomatic Complexity V(G) from a flow
graph.
• The cyclomatic complexity attribute is very useful to a tester.
• The complexity value is usually calculated from the control
flow graph (G) by the formula
V(G) = E - N + 2

Where E - Number of edges

N - Number of nodes
* This formula can be applied to flow graphs where there are no
disconnected components.
Paths: Their Role in White Box–Based Test Design
the cyclomatic complexity of the
flow graph in Figure 5.3 is
calculated as follows:

E = 7, N = 6
V(G) = 7 - 6 + 2 = 3

The cyclomatic complexity value

of a module is useful to the
tester in several ways.
One of its uses is to provide an
approximation of the number of
test cases needed for branch
coverage in a module of Figure 5.3
structured code.
Paths: Their Role in White Box–Based Test Design
• If the testability of a piece of software is defined in terms of
the number of test cases required to adequately test it, then
McCabes’ cyclomatic complexity provides an approximation of
the testability of a module.
• The tester can use the value of V(G) along with past project
data to approximate the testing time and resources required
to test a software module.
• In addition, the cyclomatic complexity value and the control
flow graph give the tester another tool for developing white
box test cases using the concept of a path. A definition for this
term is given below.
Paths: Their Role in White Box–Based Test Design
• A path is a sequence of control flow nodes usually beginning
from the entry node of a graph through to the exit node.
• A path may go through a given segment of the control flow
graph one or more times.
• We usually designate a path by the sequence of nodes it
encompasses.
For example, one path from the graph in Figure 5.2 is

1-2-3-4-8
• Cyclomatic complexity is a measure of the number of so-
called “independent” paths in the graph.
• An independent path is a special kind of path in the flow
graph.
• Deriving a set of independent paths using a flow graph can
support a tester in identifying the control flow features in the
code and in setting coverage goals.
Paths: Their Role in White Box–Based Test Design
• A tester identifies a set of independent paths for the software
unit by starting out with one simple path in the flow graph
and iteratively adding new paths to the set by adding new
edges at each iteration until there are no more new edges to
add.
• The independent paths are defined as any new path through
the graph that introduces a new edge that has not be
traversed before the path is defined.
• A set of independent paths for a graph is sometimes called a
basis set.
• For most software modules it may be possible to derive a
number of basis sets.
Paths: Their Role in White Box–Based Test Design
• If we examine the flow graph in Figure 5.3, we can derive the
following set of independent paths starting with the first path
identified above.
(i) 1-2-3-4-8
(ii) 1-2-3-4-5-6-7-4-8
(iii) 1-2-3-4-5-7-4-8

• The number of independent paths in a basis set is equal to

the cyclomatic complexity of the graph.
• For this example they both have a value of 3. Recall that the
cyclomatic complexity for a flow graph also gives us an
approximation (usually an upper limit) of the number of tests
needed to achieve branch (decision) coverage.
 Paths: Their Role in White Box–Based Test Design

• If we prepare white box test cases so that the inputs cause the
execution of all of these paths, we can be reasonably sure
that we have achieved complete statement and decision
coverage for the module.
• Testers should be aware that although identifying the
independent paths and calculating cyclomatic complexity in a
module of structured code provides useful support for
achieving decision coverage goals, in some cases the number
of independent paths in the basis set can lead to an over
approximation of the number of test cases needed for
decision (branch) coverage.
Paths: Their Role in White Box–Based Test Design
• One additional logic-based testing criterion based on the path
concept should be mentioned.
• It is the strongest program-based testing criterion, and it calls
for complete path coverage; that is, every path (as
distinguished from independent paths) in a module must be
exercised by the test set at least once.
• This may not be a practical goal for a tester.
• For example, even in a small and simple unit of code there
may be many paths between the entry and exit nodes. Adding
even a few simple decision statements increases the number
of paths.
• Every loop multiplies the number of paths based on the
number of possible iterations of the loop since each iteration
constitutes a different path through the code. Thus, complete
path coverage for even a simple module may not be practical,
and for large and complex modules it is not feasible.
 Additional White Box Test Design Approaches
Data Flow and White Box Test Design
• Some basic concepts that define the role of variables in a
software component need to be introduced.
• We say a variable is defined in a statement when its value is
assigned or changed.
For example, in the statements
Y = 26 * X
Read (Y)
Here the variable Y is defined, that is, it is assigned a new value.
In data flow notation this is indicated as a def for the variable Y.

We say a variable is used in a statement when its value is

utilized in a statement. The value of the variable is not
changed.
• A more detailed description of variable usage is given by Rapps
and Weyuker.
• They describe a predicate use (p-use) for a variable that
indicates its role in a predicate.
• A computational use (c-use) indicates the variable’s role as a
part of a computation.
• In both cases the variable value is unchanged.

For example, in the statement

Y = 26 * X
the variable X is used. Specifically, it has a c-use.
In the statement
if (X > 98)
Y = max
- X has a predicate or p-use.
There are other data flow roles for variables such as undefined or
dead, but these are not relevant to the subsequent discussion.
An analysis of data flow patterns for specific variables is often very
useful for defect detection.
For example,
- use of a variable without a definition occurring first indicates a
defect in the code.
- The variable has not been initialized.
- Smart compilers will identify these types of defects.
- Testers and developers can utilize data flow tools that will
identify and display variable role information.
- These should also be used prior to code reviews to facilitate the
work of the reviewers.
Using their data flow descriptions, Rapps and Weyuker identified
several data-flow based test adequacy criteria that map to
corresponding coverage goals. These are based on test sets that
exercise specific path segments.
for example:
All def
All p-uses
All c-uses/some p-uses
All p-uses/some c-uses
All uses
All def-use paths

The strongest of these criteria is all def-use paths. This includes all p-
and c-uses.
We say a path from a variable definition to a use is called a def-use
path.
To satisfy the all def-use criterion the tester must identify and classify
occurrences of all the variables in the software under test.
A tabular summary is useful.
Then for each variable, test data is generated so that all definitions
and all uses for all of the variables are exercised during test.
• The variables of interest are sum, i, n, and number.
• Since the goal is to satisfy the all def-use criteria we will need to
tabulate the def-use occurrences for each of these variables.
• The data flow role for each variable in each statement of the
example is shown beside the statement in italics.
• On the table each def-use pair is assigned an identifier.
• Line numbers are used to show occurrence of the def or use.
• Note that in some statements a given variable is both defined and
used.
The tester then generates test data to exercise all of these def-use
pairs In many cases a small set of test inputs will cover several or
all def-use paths.
For this example, two sets of test data would cover all the def-use
pairs for the variables:
Test data set 1: n = 0
Test data set 2: n = 5, number = 1,2,3,4,5

Set 1 covers pair 1 for n, pair 2 for sum, and pair 1 for i. Set 2
covers pair 1 for n, pair 1 for number, pairs 1,3,4 for sum, and
pairs 1,2,3,4 for i. Note even for this small piece of code there are
four tables and four def-use pairs for two of the variables.
• In white box testing methods, the data flow approach is most
effective at the unit level of testing. When code becomes more
complex and there are more variables to consider it becomes
more time consuming for the tester to analyze data flow roles,
identify paths, and design the tests.
• Other problems with data flow oriented testing occur in the
handling of dynamically bound variables such as pointers.
• Finally, there are no commercially available tools that provide
strong support for data flow testing, such as those that support
control-flow based testing.
Loop Testing
• Loops are among the most frequently used control structures.
Experienced software engineers realize that many defects are
associated with loop constructs. These are often due to poor
programming practices and lack of reviews. Therefore, special
attention should be paid to loops during testing.
• Beizer has classified loops into four categories:
1) simple,
2) nested,
3) concatenated,
4) unstructured.
• He advises that if instances of unstructured loops are found in
legacy code they should be redesigned to reflect structured
programming techniques.
• Testers can then focus on the remaining categories of loops.
• Loop testing strategies focus on detecting common defects
associated with these structures.
Loop Testing
example
Suppose in a simple loop that can have a range of zero to n
iterations, test cases should be developed so that there are:
a) zero iterations of the loop, i.e., the loop is skipped in its
entirely;
b) one iteration of the loop;
c) two iterations of the loop;
d) k iterations of the loop where k < n;
e) n - 1 iterations of the loop;
f) n + 1 iterations of the loop (if possible)

• Zhu has described a historical loop count adequacy criterion

that states that in the case of a loop having a maximum of n
iterations, tests that execute the loop zero times, once, twice,
and so on up to n times are required.
Loop Testing
• Beizer has some suggestions for testing nested loops where
the outer loop control variables are set to minimum values
and the inner most loop is exercised as above.
• The tester then moves up one loop level and finally tests all
the loops simultaneously.
• This will limit the number of tests to perform; however, the
number of test under these circumstances is still large and the
tester may have to make trade-offs.
• Beizer also has suggestions for testing concatenated loops
Mutation Testing
• Mutation testing is another approach to test data generation
that requires knowledge of code structure, but it is classified
as a fault-based testing approach.
• It considers the possible faults that could occur in a software
component as the basis for test data generation and
evaluation of testing effectiveness.
Mutation testing makes two major assumptions:
1. The competent programmer hypothesis.
This states that a competent programmer writes programs that
are nearly correct. Therefore, we can assume that there are no
major construction errors in the program; the code is correct
except for a simple error(s).
2. The coupling effect.
This effect relates to questions a tester might have about how
well mutation testing can detect complex errors since the
changes made to the code are very simple.
DeMillo states that test data that can distinguish all programs
differing from a correct one only by simple errors are sensitive
enough to distinguish it from programs with more complex
errors.
• In this testing the original code component is modified in a
simple way to provide a set of similar components that are
called mutants.
• Each mutant contains a fault as a result of the modification.
• The original test data is then run with the mutants.
• If the test data reveals the fault in the mutant (the result of the
modification) by producing a different output
• as a result of execution, then the mutant is said to be killed.
• If the mutants do not produce outputs that differ from the
original with the test data, then the test data are not capable of
revealing such defects.
• The tests cannot distinguish the original from the mutant. The
tester then must develop additional test data to reveal the fault
and kill the mutants.
A test set T is said to be mutation adequate for program P
provided that for every inequivalent mutant Pi of P there is an
element t in T such that Pi(t) is not equal to P(t).
• Mutations are simple changes in the original code component.
• for example: constant replacement, arithmetic operator
replacement, data statement alteration, statement deletion, and
logical operator replacement.
• There are existing tools that will easily generate mutants. Tool
users need only to select a change operator.
In Figure 5.2, first mutation could be to change line 7 from

i = i + 1 to i = i + 2.
If we rerun the tests used for branch coverage, this mutant will
be killed, that is, the output will be different than for the
original code.
 Another change we could make is in line 5, from
if a[i] > 0 to if a[i] < 0.
This mutant would also be killed by the original test data.
Therefore, we can assume that our original tests would have
caught this type of defect.

However, if we made a change in line 5 to read

if a[i] >= 0,
this mutant would not be killed by our original test data.

To measure the mutation adequacy of a test set T for a

program P we can use what is called a mutation score (MS),
which is calculated as follows

Equivalent mutants are discarded from the mutant set because

they do not contribute to the adequacy of the test set.
• Mutation testing is useful in that it can show that certain faults
as represented in the mutants are not likely to be present since
they would have been revealed by test data.
• It also helps the tester to generate hypotheses about the
different types of possible faults in the code and to develop test
cases to reveal them.
• Tthere are tools to support developers and testers with
producing mutants.
• In fact, many hundreds of mutants can be produced easily.
However, running the tests, analyzing results, and developing
additional tests, if needed, to kill the mutants are all time
consuming.
• For these reasons mutation testing is usually applied at the unit
level.
Evaluating Test Adequecy Criteria
• Most of the white box testing approaches we have discussed
so far are associated with application of an adequacy
criterion.
• Testers are often faced with the decision of which criterion to
apply to a given item under test given the nature of the item
and the constraints of the test environment (time, costs,
resources).
• One source of information the tester can use to select an
appropriate criterion is the test adequacy criterion hierarchy.
• This figure describes a
subsumes relationship among
the criteria.
• Satisfying an adequacy
criterion at the higher levels
of the hierarchy implies a
greater thoroughness in
testing.
• The criteria at the top of the
hierarchy are said to subsume
those at the lower levels.
• Achieving all definition-use
(def-use) path adequacy
means the tester has also
achieved both branch and
statement adequacy.
• As a conscientious tester you might at first reason that your
testing goal should be to develop tests that can satisfy the most
stringent criterion.
• Application of the so-called “stronger” criteria usually requires
more tester time and resources.
• Testing conditions, and the nature of the software should guide
your choice of a criterion.
• Weyuker presents a set of axioms that allow testers to formalize
properties which should be satisfied by any good program-based
test data adequacy criterion.
Testers can use the axioms to :
• recognize both strong and weak adequacy criteria; a tester may
decide to use a weak criterion, but should be aware of its
weakness with respect to the properties described by the axioms;
• focus attention on the properties that an effective test data
adequacy criterion should exhibit;
• select an appropriate criterion for the item under test;
• stimulate thought for the development of new criteria;
• the axioms are the framework with which to evaluate these new
criteria.
The axioms are based on the following set of assumptions:
1) programs are written in a structured programming language;
2) programs are SESE (single entry/single exit);
3) all input statements appear at the beginning of the program;
4) all output statements appear at the end of the program.
The axioms/properties described by Weyuker are the following
1. Applicability Property
2. Non exhaustive Applicability Property
3. Monotonicity Property
4. Inadequate Empty Set
5. Anti extensionality Property
6. General Multiple Change Property
7. Anti decomposition Property
8. Anti composition Property
9. Renaming Property
10. Complexity Property
11. Statement Coverage Property
1 . Applicability Property
“For every program there exists an adequate test set.” What this axiom
means is that for all programs we should be able to design an adequate test
set that properly tests it.

2 . Non-exhaustive Applicability Property

“For a program P and a test set T, P is adequately tested by the test set T,
and T is not an exhaustive test set.”
3 . Monotonicity Property
If a test set T is adequate for program P, and if T is equal to, or a subset of T',
then T' is adequate for program P.
4 . Inadequate Empty Set
“An empty test set is not an adequate test for any program.” If a program is
not tested at all, a tester cannot claim it has been adequately tested!
5 . Anti extensionality Property
“There are programs P and Q such that P is equivalent to Q, and T is
adequate for P, but T is not adequate for Q.” We can interpret this axiom as
saying that just because two programs are semantically equivalent (they
may perform the same function) does not mean we should test them the
same way.
6 . General Multiple Change Property
“There are programs P and Q that have the same shape, and there is a test
set T such that T is adequate for P, but is not adequate for Q.”
7 . Anti decomposition Property
“There is a program P and a component Q such that T is adequate for P, T' is
the set of vectors of values that variables can assume on entrance to Q for
some t in T, and T is not adequate for Q.” This axiom states that although an
encompassing program has been adequately tested, it does not follow that
each of its components parts has been properly tested.
8 . Anti composition Property
“There are programs P and Q, and test set T, such that T is adequate for P,
and the set of vectors of values that variables can assume on entrance to Q
for inputs in T is adequate for Q, but T is not adequate for P; Q (the
composition of P and Q).”
9 . Renaming Property
“If P is a renaming of Q, then T is adequate for P only if T is adequate for Q. A
program P is a renaming of Q if P is identical to Q expect for the fact that all
instances of an identifier, let us say a in Q have been replaced in P by an
identifier, let us say b, where “b” does not occur in Q, or if there is a set of
such renamed identifiers.”
10. Complexity Property
“For every n, there is a program P such that P is adequately tested by a size
n test set, but not by any size n - 1 test set.”
This means that for every program, there are other programs that require
more testing.

11. Statement Coverage Property

“If the test set T is adequate for P, then T causes every executable statement
of P to be executed.” Ensuring that their test set executed all statements in a
program is a minimum coverage goal for a tester.