Foundations of Software Testing

Slides based on: Draft V1.0 August 17, 2005

Test Generation: Finite State Models Aditya P. Mathur
Purdue University
Fall 2005
Last update: September 14, 2005
W and Wp methods
These slides are copyrighted. They are for use
with the Foundations of Software Testing
book by Aditya Mathur. Please use the slides
but do not remove the copyright notice.
Aditya P. Mathur 2005
2
Learning Objectives
UIO method is not covered in these slides. It is left for the students to read
on their own (Section 5.6).
The Wp method for test generation
What are Finite State Models?
The W method for test generation
Automata theoretic versus control-flow based test generation
Aditya P. Mathur 2005
3
Where are these methods used?
Conformance testing of communications protocols--this is where it all
started.
Testing of any system/subsystem modeled as a finite state machine,
e.g. elevator designs, automobile components (locks, transmission,
stepper motors, etc), nuclear plant protection systems, steam boiler
control, etc.)
Finite state machines are widely used in modeling of all kinds of
systems. Generation of tests from FSM specifications assists in testing
the conformance of implementations to the corresponding FSM model.
Warning: It will be a mistake to assume that the test generation
methods described here are applicable only to protocol testing!
Aditya P. Mathur 2005 4
Strings and languages
Aditya P. Mathur 2005
5
Strings
Strings play an important role in testing. A string serves as
a test input. Examples: 1011; AaBc; Hello world.
A collection of strings also forms a language. For example, a set
of all strings consisting of zeros and ones is the language of
binary numbers. In this section we provide a brief introduction to
strings and languages.
Aditya P. Mathur 2005
6
Alphabet
A collection of symbols is known as an alphabet. We use
an upper case letter such as X and Y to denote alphabets.
Though alphabets can be infinite, we are concerned only with
finite alphabets. For example, X={0, 1} is an alphabet consisting
of two symbols 0 and 1. Another alphabet is Y={dog, cat, horse,
lion}that consists of four symbols ``dog", ``cat", ``horse", and
``lion".
Aditya P. Mathur 2005
7
Strings over an Alphabet
A string over an alphabet X is any sequence of zero or
more symbols that belong to X. For example, 0110 is a
string over the alphabet {0, 1}. Also, dog cat dog dog lion
is a string over the alphabet {dog, cat, horse, lion}.
We will use lower case letters such as p, q, r to denote strings. The
length of a string is the number of symbols in that string. Given a string
s, we denote its length by |s|. Thus |1011|=4 and |dog cat dog|=3. A
string of length 0, also known as an empty string, is denoted by c.
Note that c denotes an empty string and also stands for element
of when used with sets.
Aditya P. Mathur 2005
8
String concatenation
Let s1 and s2 be two strings over alphabet X. We write s1.s2 to
denote the concatenation of strings s1 and s2.
For example, given the alphabet X={0, 1}, and two strings 011
and 101 over X, we obtain 011.101=011101. It is easy to see that
|s1.s2|=|s1|+|s2|. Also, for any string s, we have s. c =s and c.s=s.
Aditya P. Mathur 2005
9
Languages
A set L of strings over an alphabet X is known as a language. A
language can be finite or infinite.
The following sets are finite languages over the binary alphabet
{0, 1}:
C: The empty set
{c}: A language consisting only of one string of length zero
{00, 11, 0101}: A language containing three strings
Aditya P. Mathur 2005
10
Regular expressions
Given a finite alphabet X, the following are regular expressions
over X:
If a belongs to X, then a is a regular expression that denotes the set
{a}.
Let r1 and r2 be two regular expressions over the alphabet X that
denote, respectively, sets L1 and L2. Then r1.r2 is a regular
expression that denotes the set L1.L2.
Aditya P. Mathur 2005
11
Regular expressions (contd.)
If r is a regular expression that denotes the set L then r
+
is a
regular expression that denotes the set obtained by concatenating
L with itself one or more times also written as L
+
Also, r
*
known
as the Kleene closure of r, is a regular expression. If r denotes the
set L then r
*
denotes the set {c} L
+
.
If r1 and r2 are regular expressions that denote, respectively, sets L1
and L2, then r1r2 is also a regular expression that denotes the set L1
L2.
Aditya P. Mathur 2005 12
Review of FSMs
Aditya P. Mathur 2005
13
What is an Finite State Machine?
Quick review
A finite state machine, abbreviated as FSM, is an abstract
representation of behavior exhibited by some systems.
An FSM is derived from application requirements. For example, a
network protocol could be modeled using an FSM.
Not all aspects of an applications requirements are specified by an
FSM. Real time requirements, performance requirements, and
several types of computational requirements cannot be specified by
an FSM.
Aditya P. Mathur 2005
14
Requirements specification or design
specification?
An FSM could serve any of two roles: as a specification of the
required behavior and/or as a design artifact according to which an
application is to be implemented.
The role assigned to an FSM depends on whether it is a part of the
requirements specification or of the design specification.
Note that FSMs are a part of UML 2.0 design notation.
Aditya P. Mathur 2005
15
Where are FSMs used?
Modeling GUIs, network protocols, pacemakers, Teller machines,
WEB applications, safety software modeling in nuclear plants, and
many more.
While the FSMs considered in examples are abstract machines,
they are abstractions of many real-life machines.
Aditya P. Mathur 2005
16
FSM and statcharts
Note that FSMs are different from statecharts. While FSMs can be
modeled using statecharts, the reverse is not true.
Techniques for generating tests from FSMs are different from
those for generating tests from statecharts.
The term state diagram is often used to denote a graphical
representation of an FSM or a statechart.
Aditya P. Mathur 2005
17
FSM (Mealy machine): Formal definition
An FSM (Mealy) is a 6-tuple: (X, Y, Q, q
0
, o, O), where:
,
X is a finite set of input symbols also known as the input
alphabet.
Y is a finite set of output symbols also known as the output
alphabet,
Q is a finite set states,
q
0
in Q is the initial state,
o: Q x XQ is a next-state or state transition function, and
O: Q x XY is an output function
Aditya P. Mathur 2005
18
FSM (Moore machine): Formal definition
An FSM (Moore) is a 7-tuple: (X, Y, Q, q
0,
o, O, F), where:
,
X , Y, Q, q
0
,
and
o are the same as in FSM (Mealy)
O: Q Y is an output function
FeQ is the set of final or accepting or finish states.
Aditya P. Mathur 2005
19
FSM: Formal definition (contd.)
Mealy machines are due to G. H. Mealy (1955 publication)
Moore machines are due to E. F. Moore (1956 publication)
Aditya P. Mathur 2005
20
Test generation from FSMs
Our focus
Requirements FSM Test generation algorithm
FSM based
Test inputs
Test driver Application
Test inputs
Observed behavior Oracle Pass/fail
Application Test inputs
Test generation for application
Blue: Generated
data
Aditya P. Mathur 2005 21
Embedded systems and Finite State
Machines (FSMs)
Aditya P. Mathur 2005
22
Embedded systems
Many real-life devices have computers embedded in them.
For example, an automobile has several embedded
computers to perform various tasks, engine control being
one example. Another example is a computer inside a toy
for processing inputs and generating audible and visual
responses.
Such devices are also known as embedded systems. An
embedded system can be as simple as a child's musical keyboard
or as complex as the flight controller in an aircraft. In any case,
an embedded system contains one or more computers for
processing inputs.
Aditya P. Mathur 2005
23
Specifying embedded systems
An embedded computer often receives inputs from its
environment and responds with appropriate actions. While
doing so, it moves from one state to another.
The response of an embedded system to its inputs depends on its
current state. It is this behavior of an embedded system in
response to inputs that is often modeled by a finite state
machine (FSM).
Aditya P. Mathur 2005
24
FSM: lamp example
(a) Lamp switch can be turned clockwise.
(b) Lamp switch can be turned clockwise and counterclockwise..
Simple three state lamp behavior:
Aditya P. Mathur 2005
25
FSM: Actions with state transitions
(a) Notice ADD, INIT, ADD,OUT actions.
(b) INIT: Initialize num. ADD: Add to num. OUT: Output num.
Machine to convert a sequence of decimal digits to an integer:
Aditya P. Mathur 2005
26
FSM: Formal definition
An FSM is a quintuple: (X, Y, Q, q
0,
o, O), where:
,
X is a finite set of input symbols also known as the input
alphabet.
Y is a finite set of output symbols also known as the output
alphabet,
Q is a finite set states,
Aditya P. Mathur 2005
27
FSM: Formal definition (contd.)
q
0
in Q is the initial state,
o: Q x XQ is a next-state or state transition function, and
O: Q x XY is an output function.
In some variants of FSM more than one state could be
specified as an initial state. Also, sometimes it is
convenient to add F_ Q as a set of final or accepting
states while specifying an FSM.
Aditya P. Mathur 2005
28
State diagram representation of FSM
A state diagram is a directed graph that contains nodes
representing states and edges representing state
transitions and output functions.
Each node is labeled with the state it represents. Each directed
edge in a state diagram connects two states. Each edge is labeled
i/o where i denotes an input symbol that belongs to the input
alphabet X and o denotes an output symbol that belongs to the
output alphabet O. i is also known as the input portion of the
edge and o its output portion.
Aditya P. Mathur 2005
29
Tabular representation of FSM
A table is often used as an alternative to the state
diagram to represent the state transition function o and
the output function O.
The table consists of two sub-tables that consist of one or more
columns each. The leftmost sub table is the output or the action
sub-table. The rows are labeled by the states of the FSM. The
rightmost sub-table is the next state sub-table.
Aditya P. Mathur 2005
30
Tabular representation of FSM:
Example
The table given below shows how to represent
functions o and O for the DIGDEC machine.
Aditya P. Mathur 2005
31
Properties of FSM
Completely specified: An FSM M is said to be completely
specified if from each state in M there exists a transition for
each input symbol.
Strongly connected: An FSM M is considered strongly
connected if for each pair of states (q
i
q
j
) there exists an input
sequence that takes M from state q
i
to q
j
.
Aditya P. Mathur 2005
32
Properties of FSM: Equivalence
V-equivalence: Let M
1
=(X, Y, Q
1
, m
1
0
, T
1
, O
1
) and M
2
=(X, Y,
Q
2
, m
2
0
, T
2
, O
2
) be two FSMs. Let V denote a set of non-
empty strings over the input alphabet X i.e. V_ X
+
.
Let q
i
and q
j
, be two states of machines M
1
and M
2
,
respectively. q
i
and q
i
are considered V-equivalent if O
1
(q
i
,
s)=O
2
(v, q
j
) for all s in V.
Aditya P. Mathur 2005
33
Properties of FSM: Distinguishability
Stated differently, states q
i
and q
j
are considered V-equivalent
if M
1
and M
2
, when excited in states q
i
and q
j
, respectively,
yield identical output sequences.
States q
i
and q
j
are said to be equivalent if O
1
(q
i
, r)=O
2
(q
j
, r)
for any set V. If q
i
and q
j
are not equivalent then they are said to
be distinguishable. This definition of equivalence also applies
to states within a machine. Thus machines M
1
and M
2
could be
the same machine.
Aditya P. Mathur 2005
34
Properties of FSM: k-equivalence
k-equivalence: Let M
1
=(X, Y, Q
1
, m
1
0
, T
1
, O
1
) and M
2
=(X, Y,
Q
2
, m
2
0
, T
2
, O
2
) be two FSMs.
States q
i
c Q
1
and q
j
c Q
2
are considered k-equivalent if, when
excited by any input of length k, yield identical output
sequences.
Aditya P. Mathur 2005
35
Properties of FSM: k-equivalence
(contd.)
States that are not k-equivalent are considered k-distinguishable.
Once again, M
1
and M
2
may be the same machines implying
that k-distinguishability applies to any pair of states of an
FSM.
It is also easy to see that if two states are k-distinguishable for
any k>0 then they are also distinguishable for any n> k. If M
1
and M
2
are not k-distinguishable then they are said to be k-
equivalent.
Aditya P. Mathur 2005
36
Properties of FSM: Machine Equivalence
Machine equivalence: Machines M
1
and M
2
are said to be
equivalent if (a) for each state o in M1 there exists a state o ' in
M
2
such that o and o ' are equivalent and (b) for each state o in
M
2
there exists a state o ' in M
1
such that o and o ' are
equivalent.
Machines that are not equivalent are considered distinguishable.
Minimal machine: An FSM M is considered minimal if the
number of states in M is less than or equal to any other FSM
equivalent to M.
Aditya P. Mathur 2005 37
Faults Targeted
Aditya P. Mathur 2005
38
Faults in implementation
An FSM serves to specify the correct requirement or design of an
application. Hence tests generated from an FSM target faults
related to the FSM itself.
What faults are targeted by the tests generated using an FSM?
Aditya P. Mathur 2005
39
Fault model
q0
q1
a/1
b/0
b/1
a/1
Correct design
q0
q1
a/0
b/0
b/1
a/1
Operation error Transfer error
q0
q1
a/1
b/0
b/1
a/1
Aditya P. Mathur 2005
40
Fault model (contd.)
q0
a/0
b/0
Missing state error Extra state error
q0
q1
a/1
b/0
b/1
a/1
q2
a/1
Aditya P. Mathur 2005
41
Fault model (contd.)
q0
a/0
b/0
Missing state error Extra state error
q0
q1
a/1
b/0
b/1
a/1
q2
a/1
Aditya P. Mathur 2005 42
Test generation using Chows method
Aditya P. Mathur 2005
43
Assumptions for test generation
Minimality: An FSM M is considered minimal if the number of
states in M is less than or equal to any other FSM equivalent to
M.
Completely specified: An FSM M is said to be completely specified
if from each state in M there exists a transition for each input
symbol.
Aditya P. Mathur 2005
44
Overall algorithm used in Chows
method
Step 1: Estimate the maximum number of states (m) in the correct
implementation of the given FSM M.
Step 2: Construct the characterization set W for M.
Step 3: (a) Construct the testing tree for M and (b) generate the
transition cover set P from the testing tree.
Step 4: Construct set Z from W and m.
Step 5: Desired test set=P.Z
Aditya P. Mathur 2005
45
Step 1: Estimation of m
This is based on a knowledge of the implementation. In the absence
of any such knowledge, let m=|Q|.
Aditya P. Mathur 2005
46
Step 2: Construction of W. What is W?
Let M=(X, Y, Q, q1, o, O) be a minimal and complete FSM.
Given states qi and qj in Q, W contains a string s such that:
O(qi, s)=O(qj, s)
W is a finite set of input sequences that distinguish the behavior of
any pair of states in M. Each input sequence in W is of finite length.
Aditya P. Mathur 2005
47
Example of W
W={baaa,aa,aaa}
O(baaa,q1)=1101
O(baaa,q2)=1100
Thus baaa distinguishes state q1 from q2 as O(baaa,q1) = O(baaa,q2)
Aditya P. Mathur 2005
48
Steps in the construction of W
Step 1: Construct a sequence of k-equivalence partitions of Q denoted
as P1, P2, Pm, m>0.
Step 2: Traverse the k-equivalence partitions in reverse order to obtain
distinguishing sequence for each pair of states.
Aditya P. Mathur 2005
49
What is a k-equivalence partition of Q?
A k-equivalence partition of Q, denoted as P
k
, is a collection of n finite
sets E
k1
, E
k2
E
kn
such that

n
i=1
E
ki
=Q
States in E
ki
are k-equivalent.
If state v is in E
ki
and v in E
kj
for i=j, then u and v are k-distinguishable.
Aditya P. Mathur 2005
50
How to construct a k-equivalence partition?
Given an FSM M, construct a 1-equivalence partition, start with a
tabular representation of M.
Current
state
Output Next state
a b a b
q1 0 1 q1 q4
q2 0 1 q1 q5
q3 0 1 q5 q1
q4 1 1 q3 q4
q5 1 1 q2 q5
Aditya P. Mathur 2005
51
Construct 1-equivalence partition
Group states identical in their Output entries. This gives us 1-partition
P
1
consisting of E
1
={q1, q2, q3} and E
2
={q4, q5}.
E
Current
state
Output Next state
a b a b
1 q1 0 1 q1 q4
q2 0 1 q1 q5
q3 0 1 q5 q1
2 q4 1 1 q3 q4
q5 1 1 q2 q5
Aditya P. Mathur 2005
52
Construct 2-equivalence partition: Rewrite P
1
table
Rewrite P
1
table. Remove the output columns. Replace a state entry q
i
by q
ij
where j is the group number in which lies state q
i
.
E
Current
state
Next state
a b
1 q1 q11 q42
q2 q11 q52
q3 q52 q11
2 q4 q31 q42
q5 q21 q52
Group number
P
1
Table
Aditya P. Mathur 2005
53
Construct 2-equivalence partition: Construct P
2
table
Group all entries with identical second subscripts under the next state
column. This gives us the P
2
table. Note the change in second
subscripts.
E
Current
state
Next state
a b
1 q1 q11 q43
q2 q11 q53
2 q3 q53 q11
3 q4 q32 q43
q5 q21 q53
P
2
Table
Aditya P. Mathur 2005
54
Construct 3-equivalence partition: Construct P
3
table
Group all entries with identical second subscripts under the next state
column. This gives us the P
3
table. Note the change in second
subscripts.
E
Current
state
Next state
a b
1 q1 q11 q43
q2 q11 q54
2 q3 q54 q11
3 q4 q32 q43
4 q5 q21 q54
P
3
Table
Aditya P. Mathur 2005
55
Construct 4-equivalence partition: Construct P
4
table
Continuing with regrouping and relabeling, we finally arrive at P
4
table.
E
Current
state
Next state
a b
1 q1 q11 q44
2 q2 q11 q55
3 q3 q55 q11
4 q4 q33 q44
5 q5 q22 q55
P
4
Table
Aditya P. Mathur 2005
56
k-equivalence partition: Convergence of the process
The process is guaranteed to converge.
When the process converges, and the machine is minimal, each state
will be in a separate group.
The next step is to obtain the distinguishing strings for each state.
Aditya P. Mathur 2005
57
Finding the distinguishing sequences: Example
Let us find a distinguishing sequence for states q1 and q2.
Find tables P
i
and P
i+1
such that (q1, q2) are in the same group in P
i
and different groups in P
i+1
. We get P
3
and P
4
.
Initialize z=c. Find the input symbol that distinguishes q1 and q2 in
table P3. This symbol is b. We update z to z.b. Hence z now becomes
b.
Aditya P. Mathur 2005
58
Finding the distinguishing sequences: Example (contd.)
The next states for q1 and q2 on b are, respectively, q4 and q5.
We move to the P
2
table and find the input symbol that distinguishes
q4 and q5.
Let us select a as the distinguishing symbol. Update z which now
becomes ba.
The next states for states q4 and q5 on symbol a are, respectively,
q3 and q2. These two states are distinguished in P
1
by a and b. Let
us select a. We update z to baa.
Aditya P. Mathur 2005
59
Finding the distinguishing sequences: Example (contd.)
The next states for q3 and q2 on a are, respectively, q1 and q5.
Moving to the original state transition table we obtain a as the
distinguishing symbol for q1 and q5
We update z to baaa. This is the farthest we can go backwards
through the various tables. baaa is the desired distinguishing
sequence for states q1 and q2. Check that o(q1,baaa)=o(q2,baaa).
Aditya P. Mathur 2005
60
Finding the distinguishing sequences: Example (contd.)
Using the procedure analogous to the one used for q1 and q2, we can
find the distinguishing sequence for each pair of states. This leads us
to the following characterization set for our FSM.
W={a, aa, aaa, baaa}
Aditya P. Mathur 2005
61
Chows method: where are we?
Step 4: Construct set Z from W and m.
Step 5: Desired test set=P.Z
Step 1: Estimate the maximum number of states (m) in the correct
implementation of the given FSM M.
Step 2: Construct the characterization set W for M.
Done
Step 3: (a) Construct the testing tree for M and (b) generate the
transition cover set P from the testing tree.
Next (a)
Aditya P. Mathur 2005
62
Step 3: (a) Construct the testing tree for M
A testing tree of an FSM is a tree rooted at the initial state. It contains at least
one path from the initial state to the remaining states in the FSM. Here is how
we construct the testing tree.
State q0, the initial state, is the root of the testing tree. Suppose that the testing
tree has been constructed until level k . The (k+1)th level is built as follows.
Select a node n at level k. If n appears at any level from 1 through k , then n is a
leaf node and is not expanded any further. If n is not a leaf node then we
expand it by adding a branch from node n to a new node m if o(n, x)=m for
x X . This branch is labeled as x. This step is repeated for all nodes at level k.
Aditya P. Mathur 2005
63
Example: Construct the testing tree for M
Start here, initial
state is the root.
q1 becomes leaf,
q4 can be
expanded.
No further
expansion
possible
.
.
.
M
Aditya P. Mathur 2005
64
Chows method: where are we?
Step 4: Construct set Z from W and m.
Step 5: Desired test set=P.Z
Step 1: Estimate the maximum number of states (m) in the correct
implementation of the given FSM M.
Step 2: Construct the characterization set W for M.
Done
Step 3: (a) Construct the testing tree for M and (b) generate the
transition cover set P from the testing tree.
Next, (b)
Aditya P. Mathur 2005
65
Step 3: (b) Find the transition cover set from the testing tree
A transition cover set P is a set of all strings representing sub-paths, starting
at the root, in the testing tree. Concatenation of the labels along the edges of a
sub-path is a string that belongs to P. The empty string (c) also belongs to P.
P={c, a, b, bb, ba, bab, baa, baab, baaa, baaab,
baaaa}
Aditya P. Mathur 2005
66
Chows method: where are we?
Step 5: Desired test set=P.Z
Step 1: Estimate the maximum number of states (m) in the correct
implementation of the given FSM M.
Step 2: Construct the characterization set W for M.
Done
Step 3: (a) Construct the testing tree for M and (b) generate the
transition cover set P from the testing tree.
Done
Step 4: Construct set Z from W and m.
Next
Aditya P. Mathur 2005
67
Step 4: Construct set Z from W and m
For m=n, we get
Z = X
0
.W=W
Given that X is the input alphabet and W the characterization set, we
have:
Z = X
0
.W X
1
.W .. X
m-1-n
.W X
m-n
.W
For X={a, b}, W={a, aa, aaa, baaa}, m=6
Z = W X
1
.W ={a, aa, aaa, baaa} {a, b}.{a, aa, aaa, baaa}
={a, aa, aaa, baaa, aa, aaa, aaaa, baaaa, ba, baa, baaa,
bbaaa}
Aditya P. Mathur 2005
68
Chows method: where are we?
Step 1: Estimate the maximum number of states (m) in the correct
implementation of the given FSM M.
Step 2: Construct the characterization set W for M.
Done
Step 3: (a) Construct the testing tree for M and (b) generate the
transition cover set P from the testing tree.
Done
Step 4: Construct set Z from W and m.
Done
Step 5: Desired test set=P.Z Next
Aditya P. Mathur 2005
69
Step 5: Desired test set=P.Z
The test inputs based on the given FSM M can now be derived as:
T=P.Z
Do the following to test the implementation:
1. Find the expected response to each element of T.
2. Generate test cases for the application. Note that even though
the application is modeled by M, there might be variables to be
set before it can be exercised with elements of T.
3. Execute the application and check if the response matches. Reset
the application to the initial state after each test.
Aditya P. Mathur 2005
70
Example 1: Testing an erroneous application
Correct design
M1 M2
M
t1=baaaaaa
M1(t1)=1101001
M(t1)=1101001
t2=baaba
M2(t2)=11001
M(t2)=11011
Error revealing
test cases
Aditya P. Mathur 2005
71
Example 2: Extra state. N=5, m=6.
M1 M2
t1=baaba M(t1)=11011 M1(t1)=11001
t2=baaa M(t2)=1101 M2(t2)=1100
Aditya P. Mathur 2005
72
Error detection process: in-class discussion
Given m=n, each test case t is of the form r.s where r is in P
and s in W. r moves the application from initial state q0 to state
qj. Then, s=as takes it from qi to state qj or qj.
Aditya P. Mathur 2005 73
Automata theoretic versus control theoretic
methods for test generation
Aditya P. Mathur 2005
74
Automata-theoretic vs. Control theoretic techniques
The W and the Wp methods are considered automata-theoretic
methods for test generation.
In contrast, many books on software testing mention control
theoretic techniques for test generation. Let us understand the
difference between the two types of techniques and their fault
detection abilities.
Aditya P. Mathur 2005
75
Control theoretic techniques
State cover: A test set T is considered adequate with respect to
the state cover criterion for an FSM M if the execution of M
against each element of T causes each state in M to be
visited at least once.
Transition cover: A test set T is considered adequate with respect
to the branch/transition cover criterion for an FSM M if the
execution of M against each element of T causes each
transition in M to be taken at least once
Aditya P. Mathur 2005
76
Control theoretic techniques (contd.)
Switch cover: A test set T is considered adequate with respect to
the 1-switch cover criterion for an FSM M if the execution of
M against each element of T causes each pair of transitions
(tr1, tr2) in M to be taken at least once, where for some input
substring ab tr1: qi=o(qj, a) and tr_2: qk= o(qi, b) and qi,
qj, qk are states in M.
Aditya P. Mathur 2005
77
Control theoretic techniques (contd.)
Boundary interior cover: A test set T is considered adequate with
respect to the boundary-interior cover criterion for an FSM M if
the execution of M against each element of T causes each loop (a
self-transition) across states to be traversed zero times and at least
once. Exiting the loop upon arrival covers the ``boundary"
condition and entering it and traversing the loop at least once
covers the ``interior" condition.
Aditya P. Mathur 2005
78
Control theoretic technique: Example 1
Consider the following machines, a correct one (M1) and one with a
transfer error (M1).
t=abba covers all states but does
not not reveal the error. Both
machines generate the same
output which is 0111.
Will the tests generated by the
W method reveal this error?
Check it out!
Aditya P. Mathur 2005
79
Control theoretic technique: Example 2
Consider the following machines, a correct one (M2) and one with a
transfer error (M2).
There are 12 branch pairs, such
as (tr1, tr2), (tr1, tr3), tr6, tr5).
Consider the test set: {bb, baab,
aabb, aaba, abbaab}. Does it
cover all branches? Does it
reveal the error?
Are the states in M2 1-
distinguishable?
Aditya P. Mathur 2005
80
Control theoretic technique: Example 3
Consider the following machines, a correct one (M3) and one with a
transfer error (M3).
Consider T={t1: aab, t2: abaab}.
T1 causes each state to be
entered but loop not traversed.
T2 causes each loop to be
traversed once.
Is the error revealed by T?
Aditya P. Mathur 2005 81
The Partial W (Wp) method
Aditya P. Mathur 2005
82
The partial W (Wp) method
Tests are generated from minimal, complete, and connected FSM.
Size of tests generated is generally smaller than that generated using the
W-method.
Test generation process is divided into two phases: Phase 1: Generate a
test set using the state cover set (S) and the characterization set (W).
Phase 2: Generate additional tests using a subset of the transition cover
set and state identification sets.
What is a state cover set? A state identification set?
Aditya P. Mathur 2005
83
State cover set
Given FSM M with input alphabet X, a state cover set S is a finite non-
empty set of strings over X* such that for each state qi in Q, there is a
string in S that takes M from its initial state to qi.
S={c, b, ba, baa, baaa}
S is always a subset of the
transition cover set P. Also, S
is not necessarily unique.
Aditya P. Mathur 2005
84
State identification set
Given an FSM M with Q as the set of states, an identification set
for state qieQ is denoted by W
i
and has the following
properties:
(a) W
i
_ W , 1s isn [Identification set is a subset of W.]
(b) O(qi, s)= O(qj, s) , for 1sjs n , j= i , se W
i
[For each state
other than qi, there is a string in Wi that distinguishes qi from
qj.]
(c) No subset of W
i
satisfies property (b). [W
i
is minimal.]
Aditya P. Mathur 2005
85
State identification set: Example
Si Sj X o(Si,x) o(Sj,x)
1 2 baaa 1 0
3 aa 0 1
4 a 0 1
5 a 0 1
2 3 aa 0 1
4 a 0 1
5 a 0 1
3 4 a 0 1
5 a 0 1
4 5 aaa 1 0
Last element of the output string
W
1
=W
2
={baaa, aa, a}
W
3
={a aa} W
4
=W
5
={a, aaa}
Aditya P. Mathur 2005
86
Wp method: Example: Step 1: Compute S, P, W,
Wi,W
W
1
=W
2
={baaa, aa, a}
W
3
={a aa} W
4
=W
5
={a, aaa}
S={c, b, ba, baa, baaa}
P={c, a, b, bb, ba, bab, baa, baab, baaa, baaab, baaaa}
W={a, aa, aaa, baaa}
W={W1, W2, W3, W4, W5}
Aditya P. Mathur 2005
87
Wp method: Example: Step 2: Compute T1 [m=n]
T1=S. W={c, b, ba, baa, baaa}.{a, aa, aaa, baaa}
Elements of T1 ensure that the each state of the FSM is covered
and distinguished from the remaining states.
Aditya P. Mathur 2005
88
Wp method: Example: Step 3: Compute R and o
[m=n]
R=P-S={c, a, b, bb, ba, bab, baa, baab, baaa, baaab, baaaa}-{c, b,
ba, baa, baaa}
={a, bb, bab, baab, baaab, baaaa}
Let each element of R be denoted as r
i1
, r
i2
,r
ik
.
o(r
ik
, m)=q
ij
, where meX (the alphabet)
Aditya P. Mathur 2005
89
Wp method: Example: Step 4: Compute T2 [m=n]
T2=RW=
k
(j=1)
(r
ij
}. W
ij
, where W
ij
is the identification set for state
q
ij
.
o(q1, a)=q1 o(q1, bb)=q4 o(q1, bab)=q5
o(q1, baab)=q5 o(q1, baaab)=q5 o(q1, baaaa)=q1
T2=({a}. W
1
) ({bb}.W
4
) ({bab}.W
5
) ({baab}.W
5
)
{baaab}.W
5
) ({baaaa}. W
1
)
={abaaa, aaa, aa} {bba, bbaaa} {baba, babaaa}
{baaba, baabaaa} {baaaba, baaabaaa} {baaaabaaa,
baaaaaa, baaaaa}
Aditya P. Mathur 2005
90
Wp method: Example: Savings
Test set size using the W method= 44
Test set size using the Wp method= 34 (20 from T1+14 from T2)
Aditya P. Mathur 2005
91
Testing using the Wp method
Testing proceeds in two phases.
While tests from phase 1 ensure state coverage, they do not ensure
all transition coverage. Also, even when tests from phase cover all
transitions, they do not apply the state identification sets and hence
not all transfer errors are guaranteed to be revealed by these tests.
Tests from T1 are applied in phase 1. Tests from T2 are applied in phase 2.
Aditya P. Mathur 2005
92
Wp method:
T1=S. X[m-n], where X[m-n] is the set union of X
i
, 1sis (m-n)
T2= T2=R. X[m-n] W
Both sets T1 and T2 are computed a bit differently, as follows:
Aditya P. Mathur 2005
93
Summary
Behavior of a large variety of applications can be modeled using
finite state machines (FSM). GUIs can also be modeled using FSMs
The W and the Wp methods are automata theoretic methods to
generate tests from a given FSM model.
Tests so generated are guaranteed to detect all operation errors,
transfer errors, and missing/extra state errors in the implementation
given that the FSM representing the implementation is complete,
connected, and minimal. What happens if it is not?
Aditya P. Mathur 2005
94
Summary (contd.)
Automata theoretic techniques generate tests superior in their fault
detection ability than their control-theoretic counterparts.
Control-theoretic techniques, that are often described in books on
software testing, include branch cover, state cover, boundary-interior,
and n-switch cover.
The size of tests sets generated by the W method is larger than
generated by the Wp method while their fault detection effectiveness
are the same.