Model Checking

Edmund M. Clarke, Jr.

Computer Science Department
Carnegie Mellon University
Pittsburgh, PA 15213

1
 Temporal Logic Model Checking

Specification Language: A propositional temporal logic.

Verification Procedure: Exhaustive search of the state space of
the concurrent system to determine truth of specification.

 E. M. Clarke and E. A. Emerson. Synthesis of

synchronization skeletons for branching time temporal logic.
In Logic of programs: workshop, Yorktown Heights, NY, May
1981, volume 131 of Lecture Notes in Computer Science.
Springer-Verlag, 1981.
 J.P. Quielle and J. Sifakis. Specification and verification of
concurrent systems in CESAR. In Proceedings of the Fifth
International Symposium in Programming, volume 137 of
Lecture Notes in Computer Science. Springer-Verlag, 1981.

2
 Why Model Checking?

Advantages:
 No proofs!!!
 Fast
 Counterexamples
 No problem with partial specifications
 Logics can easily express many concurrency properties

Main Disadvantage: State Explosion Problem

 Too many processes
 In digital hardware terms: too many latches
Much progress recently!!

3
 Model of Computation

a b
State Transition Graph or
Kripke Model

b c c

a b

b c c

a b c c

Infinite Computation Tree

(Unwind State Graph to obtain Infinite Tree)

4
 Model of Computation (Cont.)

Formally, a Kripke structure is a triple M = hS; R; Li, where

 S is the set of states,
 R  S  S is the transition relation, and
 L : S ! P (AP ) gives the set of atomic propositions true in
each state.

We assume that R is total (i.e., for all states s 2 S there exists a

state s0 2 S such that (s; s0) 2 R).

A path in M is an infinite sequence of states,  = s0; s1; : : : such

that for i  0, (si; si+1) 2 R.

We write  i to denote the suffix of  starting at si.

Unless otherwise stated, all of our results apply only to finite

Kripke structures.

5
 Computation Tree Logics

Temporal logics may differ according to how they handle

branching in the underlying computation tree.

In a linear temporal logic, operators are provided for describing

events along a single computation path.

In a branching-time logic the temporal operators quantify over

the paths that are possible from a given state.

6
 The Logic CTL

The computation tree logic CTL combines both branching-time

and linear-time operators.

In this logic a path quantifier can prefix an assertion composed of

arbitrary combinations of the usual linear-time operators.

1. Path quantifier:
 A—“for every path”
 E—“there exists a path”
2. Linear-time operators:
 Xp—p holds next time.
 Fp—p holds sometime in the future
 Gp—p holds globally in the future
 pUq—p holds until q holds

7
 Path Formulas and State Formulas

The syntax of state formulas is given by the following rules:

 If p 2 AP , then p is a state formula.
 If f and g are state formulas, then :f and f _ g are state
formulas.
 If f is a path formula, then E(f ) is a state formula.

Two additional rules are needed to specify the syntax of path

formulas:
 If f is a state formula, then f is also a path formula.
 If f and g are path formulas, then :f , f _ g, Xf , and U fg
are path formulas.

8
 State Formulas (Cont.)

If f is a state formula, the notation M; s j= f means that f holds

at state s in the Kripke structure M .

Assume f1 and f2 are state formulas and g is a path formula. The

relation M; s j= f is defined inductively as follows:

1. s j= p , p 2 L(s).
2. s j= :f1 , s 6j= f1.
3. s j= f1 _ f2, s j= f1 or s j= f2.
4. s j= E(g) , there exists a path  starting with
s such that  j= g.

9
 Path Formulas (Cont.)

If f is a path formula, M;  j= f means that f holds along path 

in Kripke structure M .

Assume g1 and g2 are path formulas and f is a state formula. The

relation M;  j= f is defined inductively as follows:
1.  j= f , s is the first state of  and s j= f .
2.  j= :g1 ,  6j= g1.
3.  j= g1 _ g2 ,  j= g1 or  j= g2.
4.  j= Xg1 , 1 j= g1.
5.  j=U g1g2 , there exists a k  0 such that
k j= g2 and for 0  j < k, j j= g1.

10
 Standard Abbreviations

The customary abbreviations will be used for the connectives of

propositional logic.

In addition, we will use the following abbreviations in writing

temporal operators:

 A(f )  : E(:f )
 Ff U truef
 Gf  :F:f

11
 CTL and LTL

CTL is a restricted subset of CTL that permits only

G
branching-time operators—each of the linear-time operators ,
FX, , and U must be immediately preceded by a path quantifier.
Example: AG(EF p)

LTL consists of formulas that have the form A f where f is a

path formula in which the only state subformulas permitted are
atomic propositions.
Example: A(FG p)

12
 Expressive Power

It can be shown that the three logics discussed in this section

have different expressive powers.

For example, there is no CTL formula that is equivalent to the

LTL formula A(FG p).

Likewise, there is no LTL formula that is equivalent to the CTL

formula AG(EF p).

The disjunction A(FG p) _ AG(EF p) is a CTL formula that is

not expressible in either CTL or LTL.

13
 Basic CTL Operators

There are eight basic CTL operators:

 AX and EX,
 AG and EG,
 AF and EF,
 AU and EU

Each of these can be expressed in terms of EX, EG, and EU:

 AX f = : EX(:f )
 AG f = : EF(:f )
 AF f = : EG(:f )
 EF f = E[true U f ]
 A[f U g]  : E[:g U :f ^ :g] ^ : EG :g

14
 Basic CTL Operators

The four most widely used CTL operators are illustrated below.
Each computation tree has the state s0 as its root.

g g g

g g g g g g

. . . . . . . .
. . . . . . . .
. . . . . . . .

M; s0 j= AG g M; s0 j= AF g
g

g g

. . . . . . . .
. . . . . . . .
. . . . . . . .

M; s0 j= EF g M; s0 j= EG g

15
 Typical CTL Formulas

 EF(Started ^ :Ready): it is possible to get to a state where

Started holds but Ready does not hold.
 AG(Req ) AFAck): if a Request occurs, then it will be
eventually Acknowledged.
 AG(AF DeviceEnabled): DeviceEnabled holds infinitely
often on every computation path.
 AG(EF Restart): from any state it is possible to get to the
Restart state.

16
 Model Checking Problem

Let M be the state–transition graph obtained from the concurrent

system.

Let f be the specification expressed in temporal logic.

Find all states s of M such that

M; s j= f

and check if initial states are among these.

Efficient model checking algorithms exist for CTL.

 E. M. Clarke, E. A. Emerson, and A. P. Sistla. Automatic

verification of finite-state concurrent systems using temporal
logic specifications. ACM Trans. Programming Languages
and Systems, 8(2):pages 244–263, 1986.

17
 Explicit Traversal

Preprocessor Model Checker

(EMC)

CTL formulas

State Transition Graph True or Counterexample

4
10 to 10 5states

18
 Symbolic Model Checking

Method used by most “industrial strength” model checkers:

 uses boolean encoding for state machine and sets of states.

 can handle much larger designs – hundreds of state variables.
 BDDs traditionally used to represent boolean functions.

19
 Symbolic Model Checking with BDDs

Ken McMillan implemented a version of the CTL model

checking algorithm using Binary Decision Diagrams in 1987.

Carl Pixley independently developed a similar algorithm, as did

the French researchers, Coudert and Madre.

BDDs enabled handling much larger concurrent systems.

(usually, an order of magnitude increase in hardware latches!)

 J. R. Burch, E. M. Clarke, K. L. McMillan, D. L. Dill, and

J. Hwang. Symbolic model checking: 1020 states and beyond.
Information and Computation, 98(2):pages 142–170, 1992.
 K. L. McMillan. Symbolic Model Checking. Kluwer
Academic Publishers, 1993.

20
Fixpoint Algorithms

EF p = p _ EX EF p
p

21
 Fixpoint Algorithms (cont.)

Key properties of EF p:
1. EF p = p _ EX EF p
2. U = p _ EX U implies EF p  U

We write EF p = Lfp U:p _ EX U .

How to compute EF p:
U0 = False
U1 _
= p EX U0
U2 _
= p EX U1
U3 _
= p EX U2
..

22
 = EF
M; s0 j ?
p

s
0

U0 = ;

23
 = EF
M; s0 j ?
p

s
0

U1 = p _ EX U0

24
 = EF
M; s0 j ?
p

s
0

U2 = p _ EX U1

25
 = EF
M; s0 j ?
p

s
0

U3 = p _ EX U2

26
 Ordered Binary Decision Trees and Diagrams

Ordered Binary Decision Tree for the two-bit comparator, given

by the formula
f (a1; a2; b1; b2) = (a1 $ b1) ^ (a2 $ b2);
is shown in the figure below:

a
1
0 1

b b
1 1

0 0 1
1

a a a a
aa 22 aa 22 aa 2 aa 2
2 2 22 22

0 1 0 1 0 1 0 1

b b b b b b b b
2 b 2 2 b 2 2 b 2 2 b 2
1 1 1 1

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1

27
 From Binary Decision Trees to Diagrams

An Ordered Binary Decision Diagram (OBDD) is an ordered

decision tree where
 All isomorphic subtrees are combined, and
 All nodes with isomorphic children are eliminated.

Given a parameter ordering, OBDD is unique up to isomorphism.

 R. E. Bryant. Graph-based algorithms for boolean function

manipulation. IEEE Transactions on Computers,
C-35(8):677–691, 1986.

28
 OBDD for Comparator Example

If we use the ordering a1 < b1 < a2 < b2 for the comparator

function, we obtain the OBDD below:

a
1
0 1

b 1 b
1 1

a 1 0
2
1
0

b b
2 1 2

0
0 1

1 0

29
 Variable Ordering Problem

The size of an OBDD depends critically on the variable ordering.

If we use the ordering a1 < a2 < b1 < b2 for the comparator

function, we get the OBDD below:

aa1
1
0 1

a aa
2 22

0 1 0 1

b 0 b b 1 b
1 1 1 1

0 1 0
1

1 0
1
b b
2 2
0
0
1
1

1 0

30
 Variable Ordering Problem (Cont.)

For an n-bit comparator:

 if we use the ordering a1 < b1 < : : : < an < bn, the number of
vertices will be 3n + 2.
 if we use the ordering a1 < : : : < an < b1 : : : < bn, the
number of vertices is 3
2n 1.

Moreover, there are boolean functions that have exponential size

OBDDs for any variable ordering.

An example is the middle output (nth output) of a combinational

circuit to multiply two n bit integers.

31
 Logical operations on OBDD’s

 Logical negation: :f (a; b; c; d)

Replace each leaf by its negation
 Logical conjunction: f (a; b; c; d) ^ g(a; b; c; d)
– Use Shannon’s expansion as follows,
f
g = a
(f ja
gja) + a
(f ja
gja)
to break problem into two subproblems. Solve subproblems
recursively.
– Always combine isomorphic subtrees and eliminate
redundant nodes.
– Hash table stores previously computed subproblems
– Number of subproblems bounded by jf j
jg j.

32
 Logical operations (cont.)

 Boolean quantification: 9a : f (a; b; c; d)

– By definition,
9a : f = f ja _ f ja
– f (a; b; c; d)ja : replace all a nodes by left sub-tree.
– f (a; b; c; d)ja: replace all a nodes by right sub-tree.
Using the above operations, we can build up OBDD’s for
complex boolean functions from simpler ones.

33
 Symbolic Model Checking Algorithm

How to represent state-transition graphs with Ordered Binary

Decision Diagrams:

Assume that system behavior is determined by n boolean state

variables v1; v2; : : : ; vn.

The Transition relation T will be given as a boolean formula in

terms of the state variables:

T (v1; : : : ; vn; v10 ; : : : ; vn0 )

where v1; : : : vn represents the current state and v10 ; : : : ; vn0
represents the next state.

Now convert T to a OBDD!!

34
 Symbolic Model Checking (cont.)

Representing transition relations symbolically:

a a, b

Boolean formula for transition relation:

(a ^ :b ^ a0 ^ b0)
_ (a ^ b ^ a0 ^ b0)
_ (a ^ b ^ a0 ^ :b0)
Now, represent as an OBDD!

35
 Symbolic Model Checking (cont.)

Consider f = EX p.

Now, introduce state variables and transition relation:

f (v) = 9v0[T (v ; v0) ^ p(v0)]

Compute OBDD for relational product on right side of formula.

36
 Symbolic Model Checking (cont.)

How to evaluate fixpoint formulas using OBDDs:

EF p = Lfp U: p _ EX U
Introduce state variables:

v ) _ 9v0 [T (v ; v0) ^ U (v0)]

EF p = Lfp U: p(
Now, compute the sequence

U0(v ); U1(v); U2(v); : : :

until convergence.

Convergence can be detected since the sets of states Ui(

v) are
represented as OBDDs.

37
 Notable Examples

The following examples illustrate the power of model checking

to handle industrial size problems.
They come from many sources, not just my research group.

 Edmund M. Clarke, Jeannette M. Wing, et al. Formal

methods: State of the art and future directions. ACM
Computing Surveys, 28(4):626–643, December 1996.

38
 Notable Examples–IEEE Futurebus+

 In 1992 Clarke and his students at CMU used SMV to verify

the cache coherence protocol in the IEEE Futurebus+
Standard.
 They constructed a precise model of the protocol and
attempted to show that it satisfied a formal specification of
cache coherence.
 They found a number of previously undetected errors in the
design of the protocol.
 This was the first time that formal methods have been used to
find errors in an IEEE standard.
 Although development started in 1988, all previous attempts
to validate Futurebus+ were based on informal techniques.

39
 Notable Examples–HDLC

 A High-level Data Link Controller (HDLC) was being

designed at AT&T in Madrid.
 In 1996 researchers at Bell Labs offered to check some
properties of the design. The design was almost finished, so
no errors were expected.
 Within five hours, six properties were specified and five were
verified, using the FormalCheck verifier.
 The sixth property failed, uncovering a bug that would have
reduced throughput or caused lost transmissions.
 The error was corrected in a few minutes and formally
verified.

40
 Notable Examples–PowerPC 620
Microprocessor

 Richard Raimi and Jim Lear at Somerset used Motorola’s

Verdict model checker to debug a hardware laboratory failure.
 Initial silicon of PowerPC 620 microprocessor crashed during
boot of an operating system.
 With run time in seconds, Verdict produced example of BIU
deadlock causing the failure.
 Paper on this published at 1997 IEEE International Test
Conference.

41
 Problems with BDDs

 BDDs are a canonical representation. Often become too large.

 Selecting right variable ordering very important for obtaining
small BDDs.

– Often time consuming or needs manual intervention.

– Sometimes, no space efficient variable ordering exists.

Next, we describe an alternative approach to symbolic model

checking that uses SAT procedures.

42
 Advantages of SAT Procedures

 SAT procedures also operate on boolean expressions but do

not use canonical forms.

 Do not suffer from the potential space explosion of BDDs.

 Can handle functions with 100s to 1000s of variables.
 Very efficient implementations available.

43
 Bounded Model Checking

 Bounded model checking uses a SAT procedure instead of

BDDs.

 We construct boolean formula that is satisfiable iff there is a

specific finite path of length k in underlying machine.

 We look for longer and longer paths by incrementing the

bound k .

 After some number of iterations, we may conclude no such

path exists and specification holds.

 For example, to verify safety properties, number of iterations

is bounded by diameter of finite state machine.

 Armin Biere, Alessandro Cimatti, Edmund Clarke, and

Yunshan Zhu. Symbolic model checking without BDDs. In
W. Rance Cleaveland, editor, Tools and Algorithms for the
Construction and Analysis of Systems (TACAS), Amsterdam,
March 1999.
44
 Main Advantages of SAT Approach

 Bounded model checking works quickly. This is due to depth

first nature of SAT search procedures.

 It finds finite paths of minimal length. This helps user

understand the example more easily.

 It uses much less space than BDD based approaches.

 Does not need manually selected variable order or costly
reordering. Default splitting heuristics usually sufficient.

45
 Example To Illustrate New Technique

Two-bit counter with an erroneous transition:

00 11

01 10

 Each state s is represented by two state variables s[1] and s[0].

 In initial state, value of the counter is 0. Thus,
I (s) = :s[1] ^ :s[0]:
 Let
inc(s; s0) = (s0[0] $ :s[0]) ^ (s0[1] $ (s[0]  s[1])):
 Define
T (s; s0) = inc(s; s0) _ (s[1] ^ :s[0] ^ s0[1] ^ :s0[0])
 Have deliberately added erroneous transition!!
46
 Example (Cont.)

 Suppose we want to know if counter will eventually reach

state (11).

 Can specify the property by AFq, where q(s) = s[1] ^ s[0].

On all execution paths, there is a state where q (s) holds.

 Equivalently, we can check if there is a path on which counter

never reaches state (11).

 This is expressed by EGp, where p(s) = :s[1] _ :s[0].

There exists a path such that p(s) holds globally along it.

47
 Example (Cont.)

 In bounded model checking, we consider paths of length k.

 We start with k = 0 and increment k until a witness is found.
 Assume k equals 2. Call the states s0, s1, s2.
 We formulate constraints on s0, s1, and s2 in propositional
logic.

 Constraints guarantee that (s0; s1; s2) is a witness for EGp

and, hence, a counterexample for AFq.

48
 Example (Cont.)

 First, we constrain (s0; s1; s2) to be a valid path starting from

the initial state.

 Obtain a propositional formula

[[ M ]] = I (s0) ^ T (s0; s1) ^ T (s1 ; s2):

49
 Example (Cont.)

 Second, we constrain the shape of the path.

 The sequence of states s0; s1; s2 can be a loop.
 If so, there is a transition from s2 to the initial state s0, s1 or
itself.

 We write lL = T (s2; sl) to denote the transition from s2 to a

state sl where l 2 [0; 2].
W
 We define L as 2l=0 lL. Thus :L denotes the case where no
loop exists.

50
 Example (Cont.)

 The temporal property Gp must hold on (s0; s1; s2).

 If no loop exists, Gp does not hold and [[ Gp ]] is false.
 To be a witness for Gp, the path must contain a loop
(condition L, given previously).

 Finally, p must hold at every state on the path

[[ Gp ]] = p(s0 ) ^ p(s1 ) ^ p(s2 ):

 We combine all the constraints to obtain the propositional

formula
_2
[[ M ]] ^ ((:L ^ false) _ (l L ^ [[ Gp ]])):
l=0

51
 Example (Cont.)

 In this example, the formula is satisfiable.

 Truth assignment corresponds to counterexample path (00),
(01), (10) followed by self-loop at (10).

 If self-loop at (10) is removed, then formula is unsatisfiable.

52
 General Translation

 Given M , f and k, we construct a propositional formula

[[ M; f ]]k .
 The variables s0; : : : ; sk in [[ M; f ]]k are states on path .
 Each si is a vector of state variables.
 If [[ M; f ]]k is satisfiable, then f is valid along .

53
 Complexity

 Size of [[ M; f ]]k is polynomial in size of f if common

subformulas are shared.

 It is linear in k and in the size of formulas for T , I and p 2 A.

 Thus, existential bounded model checking can be reduced in
polynomial time to SAT.

54
 General Strategy

To construct [[ M; f ]]k :

 We define a propositional formula [[ M ]]k that constrains

s0; : : : ; sk to be on a valid path  in M .
 We give the translation of LTL formula f to a propositional
formula that constrains  to satisfy f .

55
 Sequential Multiplier Example

SMV1 SMV2 SATO PROVER

bit sec MB sec MB sec MB sec MB
0 919 13 25 79 0 0 0 1
1 1978 13 25 79 0 0 0 1
2 2916 13 26 80 0 0 0 1
3 4744 13 27 82 0 0 1 2
4 6580 15 33 92 2 0 1 2
5 10803 25 67 102 12 0 1 2
6 43983 73 258 172 55 0 2 2
7 >17h 1741 492 209 0 7 3
8 >1GB 473 0 29 3
9 856 1 58 3
10 1837 1 91 3
11 2367 1 125 3
12 3830 1 156 4
13 5128 1 186 4
14 4752 1 226 4
15 4449 1 183 5
sum 71923 2202 23970 1066

Model Checking: 16x16 bit sequential shift and add multiplier

with overflow flag and 16 output bits.

56
 DME Example

SMV1 SMV2 SATO PROVER SATO PROVER

k=5 k=5 k = 10 k = 10
cells sec MB sec MB sec MB sec MB sec MB sec MB
4 846 11 159 217 0 3 1 3 3 6 54 5
5 2166 15 530 703 0 4 2 3 9 8 95 5
6 4857 18 1762 703 0 4 3 3 7 9 149 6
7 9985 24 6563 833 0 5 4 4 15 10 224 8
8 19595 31 >1GB 1 6 6 5 16 12 323 8
9 >10h 1 6 9 5 24 13 444 9
10 1 7 10 5 36 15 614 10
11 1 8 13 6 38 16 820 11
12 1 9 16 6 40 18 1044 11
13 1 9 19 8 107 19 1317 12
14 1 10 22 8 70 21 1634 14
15 1 11 27 8 168 22 1992 15

Model Checking: Liveness for one user in the DME.

57
 “Buggy” DME Example

SMV1 SMV2 SATO PROVER

cells sec MB sec MB sec MB sec MB
4 799 11 14 44 0 1 0 2
5 1661 14 24 57 0 1 0 2
6 3155 21 40 76 0 1 0 2
7 5622 38 74 137 0 1 0 2
8 9449 73 118 217 0 1 0 2
9 segmentation 172 220 0 1 1 2
10 fault 244 702 0 1 0 3
11 413 702 0 1 0 3
12 719 702 0 2 1 3
13 843 702 0 2 1 3
14 1060 702 0 2 1 3
15 1429 702 0 2 1 3

Model Checking: Counterexample for liveness in a buggy DME

implementation.

58
 Queue Example

SMV1 SMV2 SMV3 SMV4 SATO PROVER

L sec MB sec MB sec MB sec MB sec MB sec MB
12 18 10 4 55 11 17 7 51 60 7 9 2
13 44 13 6 60 29 20 11 56 68 8 11 2
14 109 19 11 70 37 27 20 65 287 12 15 2
15 291 31 18 86 82 40 36 81 102 10 19 2
16 711 55 43 196 207 66 80 197 411 6 6 2
17 2126 102 159 393 573 119 191 393 1701 16 45 3
18 6103 195 459 753 1857 223 422 754 302 14 58 3
19 23405 383 1491 920 5765 430 1101 817 1551 20 70 3
20 >17h >1GB 30809 845 9136 977 1377 20 86 3
21 >1GB >1GB >40h 99 3
22 120 3
23 149 4
24 167 4

Invariant Checking: Comparison of two queue implementations.

 L is the length of the queues.

 SMV3 = SMV1 with invariant checking.
 SMV4 = SMV2 with invariant checking.

59
 Combinational Circuit Example

Circuit #ins #outs #gates sec

Industry1 203 8 738 233
Industry2 317 232 15242 8790
Industry3 96 32 1032 210

Equivalence checking using SAT procedures: Three industrial

examples from Fujitsu.

None could be done within a day using BDD based equvalence

checkers!!

60
 Model Checking with the
Partial Order Reduction

 P. Godefroid. Using partial orders to improve automatic

verification methods. In Proc. 2nd Workshop on Computer
Aided Verification, volume 531 of Lecture Notes in Computer
Science, pages 176–185, Rutgers, June 1990. Springer-Verlag.
 D. Peled. Combining partial order reductions with on-the-fly
model-checking. In D. L. Dill, editor, Proc. 1994 Workshop
on Comput.-Aided Verification, volume 818 of LNCS, pages
377–390. Springer-Verlag, June 1994.
 A. Valmari. A stubborn attack on state explosion. In Proc.
2nd Workshop on Computer Aided Verification, volume 531 of
Lecture Notes in Computer Science, pages 156–165, Rutgers,
June 1990. Springer-Verlag.

 G. J. Holzmann and D. Peled. An improvement in formal

verification. In Formal Description Techniques 1994, pages
197–211, Bern, Switzerland, 1994. Chapman & Hall.

61
 Asynchronous Computation

The interleaving model for asynchronous systems allows

concurrent events to be ordered arbitrarily.

To avoid discriminating against any particular ordering, the

events are interleaved in all possible ways.

The ordering between independent transitions is largely

meaningless!!

62
 The State Explosion Problem

Allowing all possible orderings is a potential cause of the state

explosion problem.

To see this, consider n transitions that can be executed

concurrently.

In this case, there are n! different orderings and 2n different states

(one for each subset of the transitions).

If the specification does not distinguish between these sequences,

it is beneficial to consider only one with n + 1 states.

1 3
2
? R
3
2 1
? 3R 1 R ?2

3 2
R ? 1

63
 Partial Order Reduction

The partial order reduction is aimed at reducing the size of the

state space that needs to be searched.

It exploits the commutativity of concurrently executed

transitions, which result in the same state.

Thus, this reduction technique is best suited for asynchronous

systems.

(In synchronous systems, concurrent transitions are executed

simultaneously rather than being interleaved.)

s
 
R
s1 s2

 
R
r

64
 Partial Order Reduction (Cont.)

The method consists of constructing a reduced state graph.

The full state graph, which may be too big to fit in memory, is
never constructed.

The behaviors of the reduced graph are a subset of the behaviors

of the full state graph.

The justification of the reduction method shows that the

behaviors that are not present do not add any information.

65
 Partial Order Reduction (Cont.)

The name partial order reduction comes from early versions of

the algorithms that were based on the partial order model of
program execution.

However, the method can be described better as model checking

using representatives, since the verification is performed using
representatives from the equivalence classes of behaviors.

 D. Peled. All from one, one for all: on model checking using
representatives. In Proc. 5th Workshop on Comput.-Aided
Verification, pages 409–423, 1993.

66
 Modified Kripke Structures

The transitions of a system play a significant role in the partial

order reduction.

The partial order reduction is based on the dependency relation

that exists between the transitions of a system.

Thus, we modify the definition of a Kripke structure slightly.

A state transition system is a quadruple (S; T; S0; L) where

 the set of states S , the set of initial states S0, and the labeling
function L are defined as for Kripke structures, and

 T is a set of transitions such that for each  2 T ,   S  S .

A Kripke structure M = (S; R; S0; L) may be obtained by
defining R so that

R(s; s0) , 9 2 T [ (s; s0) ]:

67
 Basic Definitions

A transition  2 T is enabled in a state s if there is a state s0 such

that (s; s0) holds.

Otherwise,  is disabled in s. The set of transitions enabled in s

is enabled(s).

A transition  is deterministic if for every state s there is at most

one state s0 such that (s; s0).

When  is deterministic we often write s0 = (s) instead of

(s; s0).
Note:We will only consider deterministic transitions!

68
 Basic Definitions (Cont.)

A path  from a state s0 is a finite or infinite sequence

0 1
 = s0 ! s1 ! : : :
such that for every i, i(si; si+1) holds.

Here, we do not require paths to be infinite. Moreover, any prefix

of a path is also a path.

If  is finite, then the length of  is the number of transitions in 

and will be denoted by j j

69
 Reduced State Graph

Goal is to reduce the number of states considered in model

checking, while preserving the correctness of the property.

Will assume that a reduced state graph is first generated

explicitly using depth-first search.

The model checking algorithm is then applied to the resulting

state graph, which has fewer states and edges.

This speeds up the construction of the graph and uses less

memory, thus resulting in a more efficient model checking
algorithm.

 Actually, the reduction can be applied on-the-fly while doing

the model checking.

 The DFS can also be replaced by breadth first search and

combined with symbolic model checking.

70
 Depth-First-Search Algorithm

1 hash(s0);
2 set on stack (s0 );
3 expand state(s0 );

4 procedure expand state(s)

5 work set(s) := ample(s);
6 while work set(s) is not empty do
7 let  2 work set(s);
8 work set(s) := work set(s) n fg;
9 s0 := (s);
10 if new(s0 ) then
11 hash(s0);
12 set on stack (s0 );
13 expand state(s0);
14 end if;
15 create edge(s; ; s0);
16 end while;
17 set completed(s);
18 end procedure

71
 Depth-First-Search Algorithm (Cont.)

The reduction is performed by modifying the standard DFS

algorithm to construct the reduced state graph.

The search starts with an initial state s0 (line 1) and proceeds

recursively.

For each state s it selects only a subset ample(s) of the enabled

transitions enabled(s) (in line 5).

The DFS explores only successors generated by these transitions

(lines 6-16).

 The DFS algorithm constructs the reduced state graph

directly.

 Constructing the full state graph and later reducing it would

defy the purpose of the reduction.

72
 Depth-First-Search Algorithm (Cont.)

When model checking is applied to the reduced state graph

 it terminates with a positive answer when the property holds

for the original state graph.

 it produces a counterexample, otherwise

Note: The counterexample may differ from one obtained using

the full state graph.

73
 Ample Sets

In order to implement the algorithm we must find a systematic

way of calculating ample(s) for any given state s.

The calculation of ample(s) needs to satisfy three goals:

1. When ample(s) is used instead of enabled(s), enough

behaviors must be retained so DFS gives correct results.

2. Using ample(s) instead of enabled(s) should result in a

significantly smaller state graph.

3. The overhead in calculating ample(s) must be reasonably

small.

74
 Dependence and Independence

An independence relation I  T  T is a symmetric,

antireflexive relation such that for s 2 S and (;  ) 2 I :
 2 enabled(s) then  2 enabled( (s)).
Enabledness If ;
Commutativity ;  2 enabled(s) then ( (s)) =  ((s)).

The dependency relation D is the complement of I , namely

D = (T  T ) n I:

Note

 The enabledness condition states that a pair of independent

transitions do not disable one another.

 However, that it is possible for one to enable another.

75
 Potential Problems

Suppose that  and  commute:

s
 
R
s1 s2

 
R
r

It does not matter whether  is executed before  or vice versa in

order to reach the state r from s.

It is tempting to select only one of the transitions originating

from s.

This is not appropriate for the following reasons:

Problem 1: The checked property might be sensitive to the

choice between the states s1 and s2, not only the states s and r.

Problem 2: The states s1 and s2 may have other successors in

addition to r, which may not be explored if either is eliminated.

76
 Visible and Invisible Transitions

Let L : S ! 2AP be the function that labels each state with a set
of atomic propositions.

A transition  2 T is invisible with respect to AP 0  AP if for

each pair s; s0 2 S such that s0 = (s),
L(s) \ AP 0 = L(s0) \ AP 0:
Thus, a transition is invisible when its execution from any state
does not change the value of the propositional variables in AP 0 .

A transition is visible if it is not invisible.

77
 Stuttering Equivalence

Stuttering refers to a sequence of identically labeled states along

a path in a Kripke structure.

Let  and  be two infinite paths:

 = s0 !0
s1 ! 1
: : : and  = r0 !
0
r1 !
1
:::
Then  and  are stuttering equivalent, denoted  st , if there
are two infinite sequences of integers

0= i0 < i1 < i2 < : : : and 0 = j0 < j1 < j2 < : : :

such that for every k  0,

L(sik ) = L(sik +1) = : : : = L(sik+1 1) =

L(rjk ) = L(rjk +1) = : : : = L(rjk+1 1):

Stuttering equivalence can be defined similarly for finite paths.

78
 Stuttering Equivalence (Cont.)

A finite sequence of identically labeled states is called a block.

Intuitively, Two paths are stuttering equivalent if they can be

partitioned into blocks, so states in the k th block of one are
labeled the same as states in the k th block of the other.

Note: Corresponding blocks may have different lengths!

p; q p; q p; :q :p; :q :p; :q p; :q

p; q p; :q p; :q p; :q :p; :q p; :q

79
 Stuttering Equivalence Example

Consider the diagram used to illustrate commutativity again.

s
 
R
s1 s2

 
R
r

Suppose that at least one transition, say , is invisible, then

L(s) = L(s1) and L(s2) = L(r).
Consequently,
s s1 r st s s2 r
Note: The paths s s1 r and s s2 r are stuttering equivalent!!

80
 LTL and Stuttering Equivalence

A
An LTL formula f is invariant under stuttering if and only if
for each pair of paths  and  0 such that  st  0,
 j= f if and only if 0 j= f:
We denote the subset of the logic LTL without the next time
operator by LTL X .

Theorem. Any LTL X property is invariant under stuttering.

81
 Stuttering Equivalent Structures

Without loss of generality, assume that M has initial state s0 and

that M 0 has initial state s00.

Then the two structures M and M 0 are stuttering equivalent if

and only if

 For each path  of M that starts in s0 there is a path 0 of M 0

starting in s00 such that  st  0 .

 For each path 0 of M 0 that starts in s00 there is a path  of M

starting in s0 such that  0 st  .

Corollary. Let M and M 0 be two stuttering equivalent

structures. Then, for every LTL X property f A
M; s0 j= f if and only if M 0; s00 j= Af:

82
 DFS Algorithm and Ample Sets

Commutativity and invisibility allow us to avoid generating some

of the states when the specification is invariant under stuttering,

Based on this observation, it is possible to devise a systematic

way of selecting an ample set for any given state.

The ample sets will be used by the DFS algorithm to construct a

reduced state graph so that for every path not considered there is
a stuttering equivalent path that is considered.

This guarantees that the reduced state graph is stuttering

equivalent to the full state graph.

We say that state s is fully expanded when

ample(s) = enabled(s):

In this case, all of the successors of that state will be explored by

the DFS algorithm.

83
 Correctness of Reduction

Will state four conditions for selecting ample(s)  enabled(s)

so satisfaction of the LTL X specifications is preserved.

The reduction will depend on the set of propositions AP 0 that

appear in the LTL X formula.

Condition C0 is very simple:

C0 ample(s) = ; if and only if enabled(s) = ;.

Intuitively, if the state has at least one successor, then the reduced
state graph also contains a successor for this state.

84
 Correctness of Reduction (Cont.)

Condition C1 is the most complicated constraint.

C1 Along every path in the full state graph that starts at s, the
following condition holds:
A transition that is dependent on a transition in ample(s) can
not be executed without one in ample(s) occurring first.

Note that Condition C1 refers to paths in the full state graph.

Obviously, we need a way of checking that C1 holds without

actually constructing the full state graph.

Later, we will show how to restrict C1 so that ample(s) can be

calculated based on the current state s.

85
 Correctness of Reduction (Cont.)

Lemma. The transitions in enabled(s) n ample(s) are all

independent of those in ample(s).

Proof:
Let 2 enabled(s) n ample(s).
Suppose that ( ;  ) 2 D, where  2 ample(s).
Since is enabled in s, there is a path starting with in the full
graph.
But then a transition dependent on some transition in ample(s) is
executed before a transition in ample(s).
This contradicts condition C1.

86
 Correctness of Reduction (Cont.)

If we always choose the next transition from ample(s), we will

not omit any paths that are essential for correctness.

Condition C1 implies that such a path will have one of two

forms:

 The path has a prefix 01 : : : m, where  2 ample(s) and

each i is independent of all transitions in ample(s) including
.
 The path is an infinite sequence of transitions 01 : : : where
each i is independent of all transitions in ample(s).

87
 Correctness of Reduction (Cont.)

If along a sequence of transitions 01 : : : m executed from s,

If none of the transitions in ample(s) have occurred, then all the

transitions in ample(s) remain enabled.

This is because each i is independent of the transitions in

ample(s) and, therefore, cannot disable them.

88
 Correctness of Reduction (Cont.)

In the first case, assume that the sequence of transitions

01 : : : m reaches a state r.
This sequence will not be considered by the DFS algorithm.

By applying the enabledness and commutativity conditions m

times, we can construct a sequence 01 : : : m, that also
reaches r.

Thus, if the reduced state graph does not contain the sequence
01 : : : m that reaches r, we can construct from s another
sequence that reaches r.

s = s0

0 
s1
1  r0
s2 0
2  r1
1
sm m
 2 r2

m
r = rm

89
 Another Correctness Condition

Consider the two sequences of states:

  = s0s1 : : : smr generated by 01 : : : m, and

  = sr0r1 : : : rm generated by 01 : : : m.
In order to discard  , we want  and  to be stuttering equivalent.

This is guaranteed if  is invisible, since then L(si ) = L(ri ) for

0  i  m.

Thus, the checked property will not be able to distinguish

between the two sequences above.

C2 If s is not fully expanded, then every  2 ample(s) is

invisible.

 This condition is called invisibility.

90
 Correctness of Reduction (Cont.)

Now consider the case in which an infinite path 012 : : : that

starts at s does not include any transition from ample(s).

By Condition C2 all transitions in ample(s) are invisible. Let 

be such a transition.

Then the path generated by the sequence 012 : : : is

stuttering equivalent to the one generated by 012 : : : .

Again, even though the path 012 : : : is not in the reduced state

graph, a stuttering equivalent path is included.

91
 Problem with Correctness Conditions

C1 and C2 are not yet sufficient to guarantee that the reduced

state graph is stuttering equivalent to the full state graph.

In fact, there is a possibility that some transition will actually be

delayed forever because of a cycle in the constructed state graph.
? ?

3 1

? ^
 2

Assume that  is independent of the transitions 1, 2 and 3 and

that 1, 2 and 3 are interdependent.

The process on the left can execute the  exactly once.

Assume there is one proposition p, which is changed from true to

false by  , so that  is visible.

The process on the right performs the invisible transitions 1, 2

and 3 repeatedly in a loop.

92
 Problem with Correctness Condition (Cont.)

The full state graph of the system is shown below:

? ?
s1
 
3 1 1
?
  ^  2 ^
s3 s2
2
 3 1 
? ^ ?

2

Starting with the initial state s1, we can select ample(s1 ) = f1g

and generate s2 = 1(s1).

Next, we can select ample(s2 ) = f2g and generate s3 = 2(s2 ).

When we reach s3, we can select ample(s3) = f3g and close

the cycle (s1; s2; s3).

(Easy to see that C0, C1 and C2 are satisfied at each step.)

But, the reduced state graph does not contain any sequences
where p is changed from true to false!!

93
 Cycle Closing Condition

Note that at each state on the cycle (s1 ; s2; s3; s1)  is deferred to
a possible future state.

When the cycle is closed, the construction terminates, and

transition  is ignored!!

To prevent this situation from occurring we need one more

condition:

C3 A cycle is not allowed if it contains a state in which some

transition  is enabled, but is never included in ample(s) for
any state s on the cycle.

Condition C3 is called the Cycle closing condition.

94
 Problem 1 Again

Consider the diagram used to illustrate commutativity of actions.

s
 
R
s1 s2

 
R
r

Assume that the DFS reduction algorithm chooses  as ample(s)

and does not include state s1 in the reduced graph.

By Condition C2,  must be invisible; thus s; s2; r and s; s1 ; r are

stuttering equivalent.

Since we are only interested in stuttering invariant properties, we

can’t distinguish between the two sequences.

95
 Problem 2 Again

Assume that there is a transition enabled from s1.

s
 
R
s1 s2

 
R R
r

Note that cannot be dependent on  . Otherwise, the sequence

; violates C1.
Thus, is independent of  . Since it is enabled in s1, it must also
be enabled in state r.

Assume that , when executed from r, results in state r0 and

when executed from s1 results in state s01.

Since  is invisible, the two state sequences s; s1; s01 and

s; s2; r; r0 are stuttering equivalent.
Therefore, properties that are invariant under stuttering will not
distinguish between the two.

96
 Heuristics for Ample Sets

We assume that the concurrent program is composed of

processes and that each process has a program counter.

 pci(s) will denote the program counter of a process Pi in a

state s.

 pre() is a set of transitions that includes the transitions

whose execution may enable .

 dep() is the set of transitions that are dependent on .

 Ti is the set of transitions of process Pi.
 Ti(s) = Ti \ enabled(s) denotes the set of transitions of Pi
that are enabled in the state s.

 currenti(s) is the set of transitions of Pi that are enabled in

some state s0 such that pci (s0) = pci(s).

97
 Heuristics for Ample Sets (Cont.)

We now describe the dependency relation for the different

models of computation.

 Pairs of transitions that share a variable, which is changed by

at least one of them, are dependent.

 Pairs of transitions belonging to the same process are

dependent.

 Two send transitions that use the same message queue are
dependent. Similarly, two receive transitions are dependent.
Note that a transition that involves handshaking or rendezvous
communication as in CSP or ADA can be treated as a joint
transition of both processes. Therefore, it depends on all of the
transitions of both processes.

98
 Heuristics for Ample Sets (Cont.)

An obvious candidate for ample(s) is the set Ti (s) of transitions

enabled in s for some process Pi.

 Since the transitions in Ti(s) are interdependent, an ample set

for s must include either all of the transitions or none of them.

 To construct an ample set for the current state s, we start with

6 ;.
some process Pi such that Ti(s) =

 We want to check whether ample(s) = Ti(s) satisfies

Condition C1.

 There are two cases in which this selection might violate C1.
 In both of these cases, some transitions independent of those
in Ti (s) are executed, eventually enabling a transition  that is
dependent on Ti(s).

 The independent transitions in the sequence cannot be in Ti,

since all the transitions of Pi are interdependent.

99
 Heuristics for Ample Sets (First Case)

In the first case,  belongs to some other process Pj .

A necessary condition for this to happen is that dep(Ti (s))

includes a transition of process Pj .

By examining the dependency relation, this condition can be

checked effectively.

100
 Heuristics for Ample Sets (Second Case)

In the second case,  belongs to Pi . Suppose that the transition

 2 Ti which violates C1 is executed from a state s0.
 The transitions executed on the path from s to s0 are
independent of Ti (s) and hence, are from other processes.

 Therefore, pci(s0) = pci(s). So  must be in currenti(s).

 In addition,  62 Ti(s), otherwise it does not violate C1.
 Thus,  2 currenti(s) n Ti(s).
 Since  is not in Ti(s), it is disabled in s.
 Therefore, a transition in pre() must be included in the
sequence from s to s0 .
Thus, a necessary condition is that pre(currenti (s) n Ti(s))
includes transitions of processes other than Pi.

This condition can also be checked effectively.

101
 If all else fails : : :

In both cases we discard Ti (s) and try the transitions Tj (s) of

another process j as a candidate for ample(s).

Note: We take a conservative approach discarding some ample

sets even though at run-time C1 might not actually be violated.

102
 Future Research Directions

Need to be able to handle systems with 5000 state variables

routinely. This is an order of magnitude over the number that we
can handle now.

 Develop automatic abstraction techniques. Abstraction is the

most important technique for dealing with the state explosion
problem.

 Devise more powerful compositional reasoning techniques.

Currently, this usually requires considerable manual
intervention.

 Find good techniques for reasoning about parameterized

systems (e.g. the number of processes is a parameter).

 Develop practical symmetry reduction techniques. Many

hardware and software systems are either symmetric or almost
symmetric.

 Combine Model Checking and Theorem Proving. Work is just

beginning on this problem. I think this will be necessary for
softare verification.
103
 Future Research Directions

 SAT procedures (like the Davis-Putnam procedure) have

many advantages over BDDs. Develop model checking
algorithms that use SAT procedures.

 Need practical model checking algorithms for reasoning about

real-time systems, hybrid systems, and security protocols.

 Develop techniques for extracting finite state synchronization

skeletons from concurrent programs. This seems feasible, but
there has been little work on it until recently.

 Develop model checking algorithms for infinite state systems

(including software with complex data types). This may
require several of the other techniques in the list.

104