Introduction to

Embedded Systems

Sanjit A. Seshia
UC Berkeley
EECS 149/249A
Fall 2015

© 2008-2015: E. A. Lee, A. L. Sangiovanni-Vincentelli, S. A. Seshia. All rights

reserved.

Chapter 15: Reachability Analysis and Model Checking

The Challenge of Dependable Software in

Embedded Systems
Today’s medical devices run on
software… software defects can have
life-threatening consequences.

[Journal of Pacing and Clinical

Electrophysiology, 2004]
“the patient collapsed while walking towards [different device]

the cashier after refueling his car […] A week later the
patient complained to his physician about an increasing
feeling of unwell-being since the fall.”
“In 1 of every 12,000 settings, the software can cause an
error in the programming resulting in the possibility of
producing paced rates up to 185 beats/min.”
EECS 149/249A, UC Berkeley: 2

1
A Robot delivery service, with obstacles
obstacles

Starting
position of robot
 = destination for robot
Specification:
The robot eventually reaches 

Suppose there are n destinations 1, 2, …, n

The new specification could be that
The robot visits 1, 2, …, n in that order
EECS 149/249A, UC Berkeley: 3

Reachability Analysis and Model Checking

Reachability analysis is the process of computing the set

of reachable states for a system.
 preceding problems can be solved using reachability
analysis

Model checking is an algorithmic method for determining

if a system satisfies a formal specification expressed in
temporal logic.

Model checking typically performs reachability analysis.

EECS 149/249A, UC Berkeley: 4

2
 Formal Verification

Property

System YES
[proof]
S M
Environment Compose Verify
E
NO
counterexample

EECS 149/249A, UC Berkeley: 5

Open vs. Closed Systems

A closed system is one with no inputs

For verification, we obtain a closed system by

composing the system and environment models
EECS 149/249A, UC Berkeley: 6

3
Model Checking G p
Consider an LTL formula of the form Gp where p is a
proposition (p is a property on a single state)
To verify Gp on a system M, one simply needs to
enumerate all the reachable states and check that they
all satisfy p.

EECS 149/249A, UC Berkeley: 7

Traffic Light Controller Example

Property:

EECS 149/249A, UC Berkeley: 8

4
Model Checking G p
Consider an LTL formula of the form Gp where p is a
proposition (p is a property on a single state)
To verify Gp on a system M, one simply needs to
enumerate all the reachable states and check that they
all satisfy p.
The state space found is typically represented as a
directed graph called a state graph.
When M is a finite-state machine, this reachability
analysis will terminate (in theory).
In practice, though, the number of states may be
prohibitively large consuming too much run-time or
memory (the state explosion problem).
EECS 149/249A, UC Berkeley: 9

Composed FSM for Traffic Light Controller

Property:
This FSM has 189 states
(accounting for different values of count)

EECS 149/249A, UC Berkeley: 10

5
Reachability Analysis Through Graph Traversal

Construct the state graph on the fly.

Start with initial state, and explore next states using a

suitable graph-traversal strategy.

EECS 149/249A, UC Berkeley: 11

Depth-First Search (DFS)

Maintain 2 data
structures:
1. Set of visited states R
2. Stack with current path
from the initial state

Potential problems
for a huge graph?

EECS 149/249A, UC Berkeley: 12

6
Generating counterexamples

If the DFS algorithm finds the target

(‘error’) state s, how can we generate a
trace from the initial state to that state?

EECS 149/249A, UC Berkeley: 13

Generating counterexamples

If the DFS algorithm finds the target (‘error’)

state s, how can we generate a trace from
the initial state to that state?

Simply read the trace

s0 off the stack

Stack:
s1
s
s1
s s0

EECS 149/249A, UC Berkeley: 14

7
Explicit State Model Checking Example
Property:

R = { (red, crossing, 0) }

EECS 149/249A, UC Berkeley: 15

Explicit State Model Checking Example

Property:

R = { (red, crossing, 0), (red, crossing, 1) }

EECS 149/249A, UC Berkeley: 16

8
Explicit State Model Checking Example
Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60) }

EECS 149/249A, UC Berkeley: 17

Explicit State Model Checking Example

Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60),

(green, none, 0) }

EECS 149/249A, UC Berkeley: 18

9
Explicit State Model Checking Example
Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60),

(green, none, 0), (green, none, 1) }

EECS 149/249A, UC Berkeley: 19

Explicit State Model Checking Example

Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60),

(green, none, 0), (green, none, 1), …, (green, none, 60) }

EECS 149/249A, UC Berkeley: 20

10
Explicit State Model Checking Example
Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60),

(green, none, 0), (green, none, 1), …, (green, none, 60),
(yellow, waiting, 0) }
EECS 149/249A, UC Berkeley: 21

Explicit State Model Checking Example

Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60),

(green, none, 0), (green, none, 1), …, (green, none, 60),
(yellow, waiting, 0), … (yellow, waiting, 5) }
EECS 149/249A, UC Berkeley: 22

11
Explicit State Model Checking Example
Property:

R = { (red, crossing, 0), (red, crossing, 1), … (red, crossing, 60),

(green, none, 0), (green, none, 1), …, (green, none, 60),
(yellow, waiting, 0), … (yellow, waiting, 5),
(pending, waiting, 1), …, (pending, waiting, 60) }
EECS 149/249A, UC Berkeley: 23

The Symbolic Approach

Rather than exploring new reachable states one at a
time, we can explore new sets of reachable states
 However, we only represent sets implicitly, as Boolean
functions

Set operations can be performed using Boolean algebra

Represent a finite set of states S by its characteristic
Boolean function fS
 fS (x) = 1 iff x  S

Similarly, the state transition function  yields a set (s) of

next states from current state s, and so can also be
represented using a characteristic Boolean function for
each s.
EECS 149/249A, UC Berkeley: 24

12
Symbolic Approach (Breadth First Search)

 Generate the state graph by repeated application of

transition function (
 If the goal state reached, stop & report success. Else,
continue until all states are seen.

 Qk
Q0 Q1 Q2 ...

EECS 149/249A, UC Berkeley: 25

The Symbolic Reachability Algorithm

\R
Two extremely useful techniques:
Binary Decision Diagrams (BDDs)
Boolean Satisfiability (SAT)
These are covered in EECS 144
EECS 149/249A, UC Berkeley: 26

13
Symbolic Model Checking Example
Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 27

Symbolic Model Checking Example

Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 28

14
Symbolic Model Checking Example
Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 29

Symbolic Model Checking Example

Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 30

15
Symbolic Model Checking Example
Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 31

Symbolic Model Checking Example

Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 32

16
Symbolic Model Checking Example
Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 33

Symbolic Model Checking Example

Property:

R, set of reachable states,

represented by:

EECS 149/249A, UC Berkeley: 34

17
Abstraction in Model Checking

• Should use simplest model of a system that provides

proof of safety.
• Simpler models have smaller state spaces and easier
to check.
• The challenge is to know what details can be
abstracted away.
• A simple and useful approach is called localization
abstraction.
• A localization abstraction hides state variables that
are irrelevant to the property being verified.

EECS 149/249A, UC Berkeley: 35

Abstract Model for Traffic Light Example

Property:
What’s the hidden variable?

EECS 149/249A, UC Berkeley: 36

18
Model Checking Liveness Properties

• A safety property (informally) states that “nothing bad

ever happens” and has finite-length counterexamples.

• A liveness property, on the other hand, states

“something good eventually happens”, and only has
infinite-length counterexamples.

• Model checking liveness properties is more involved

than simply doing a reachability analysis. See Section
15.4 for more information.

EECS 149/249A, UC Berkeley: 37

Suppose we have a Robot that must pick up

multiple things, in any order

How would you state this goal in temporal logic?

EECS 149/249A, UC Berkeley: 38

19
Suppose we have a Robot that must pick up
multiple things, in any order

Goal to be achieved is:

EECS 149/249A, UC Berkeley: 39

Variant: Suppose we have a Robot that must pick

up multiple things, in a specified order

How would you state this goal in temporal logic?

EECS 149/249A, UC Berkeley: 40

20
Controller Synthesis

Goal to be achieved is:

Consider the first part alone:

How can we use model checking to synthesize a control

strategy?
EECS 149/249A, UC Berkeley: 41

Controller Synthesis

Recall that:

Therefore, we can construct a counterexample to:

The counterexample is a trace that gets the robot to the

desired point.

EECS 149/249A, UC Berkeley: 42

21