CSE 814

Formal Methods in
Software Engineering
Software is complex
“One of the most complex man made artifacts” 
-- M. Dwyer, J. Hatcliff, R. Howell (KSU CIS 771)

“What were the lessons I learned from so many years of intensive

work on the practical problem of setting type by computer? One of
the most important lessons, perhaps, is the fact that SOFTWARE IS
HARD.... the writing of programs for TEX and for METAFONT proved
to be much more difficult than all the other things I had done (like
proving theorems or writing books). The creation of good software
demands a significantly higher standard of accuracy than those other
things do, and it requires a longer attention span than other
intellectual tasks.”

-- D. Knuth, Keynote address to 11th World Congress (IFIP ‘89)

MSU, CSE 814 INTRODUCTION 2

Software is a key infrastructure
!  Process control: oil, gas, water, power, …
!  Transportation: traffic control,auto pilot, …
!  Health care: patient monitoring, device control,
e-records,…
!  Finance: e-trading, on-line banking, security,
automated teller, …
!  Defense: weapons control, surveillance,
intelligence, …
!  Manufacturing: precision milling, assembly,
CAD, …
Failing software costs money and lives!
MSU, CSE 814 INTRODUCTION 3
Failing software costs money & lives
!  A collection of well-know software failures,
G. Tan (Lehigh University)
http://www.cse.lehigh.edu/~gtan/bug/softwarebug.html

!  Illustrative risks to the public in the use of

computer systems and related technology
P. Neumann (SRI)
http://www.csl.sri.com/users/neumann/illustrative.html

!  National Vulnerability Database:

NVD includes databases of security checklists, security
related software flaws, misconfigurations, product
names, and impact metrics.
http://nvd.nist.gov/

MSU, CSE 814 INTRODUCTION 4

Therac - 25
!  A software-controlled radiation machine
"  Responsible for at least 6 fatal accidents
"  Two modes of operation:
!  Low power electron beam

MSU, CSE 814 INTRODUCTION 5

Therac - 25
!  A software-controlled radiation machine
"  Responsible for at least 6 fatal accidents
"  Two modes of operation:
!  Low power electron beam
!  High power electron beam absorbed by metal target to produce an
X-ray beam

MSU, CSE 814 INTRODUCTION 6

Therac - 25
!  A software-controlled radiation machine
"  Responsible for at least 6 fatal accidents
"  Two modes of operation:
!  Low power electron beam
!  High power electron beam absorbed by metal target to produce an
X-ray beam
"  Race condition
!  Activate HP beam

MSU, CSE 814 INTRODUCTION 7

Therac - 25
!  A software-controlled radiation machine
"  Responsible for at least 6 fatal accidents
"  Two modes of operation:
!  Low power electron beam
!  High power electron beam absorbed by metal target to produce an
X-ray beam
"  Race condition
!  Activate HP beam
!  Within 8 seconds, activate LP beam

!  Some software engineering issues

"  Over confidence in software: Lack of hardware interlocks
"  Poor user interface w/ high incidence of false alarms
"  Failures were infrequent and hard to reproduce
!  Occasional arithmetic overflow
!  Unexpected input sequence

MSU, CSE 814 INTRODUCTION 8

MSU, CSE 814 INTRODUCTION 9
MSU, CSE 814 INTRODUCTION 10
MSU, CSE 814 INTRODUCTION 11
Priorities are changing
From: Bill Gates
Sent: Tuesday, January 15, 2002 2:22 PM
To: Microsoft and Subsidiaries: All FTE
Subject: Trustworthy computing
…

Trustworthy Computing is computing that is as available, reliable and
secure as electricity, water services and telephony.
…

In the past, we've made our software and services more compelling for
users by adding new features and functionality
…

We've done a terrific job at that, but all those great features won't
matter unless customers trust our software. So now, when we face a
choice between adding features and resolving security issues, we need to
choose security.
…

These principles should apply at every stage of the development cycle of
every kind of software we create
…

MSU, CSE 814 INTRODUCTION 12
Software engineering …
!  A collection of techniques and methodologies for
systematic development of complex software
systems
!  Essential for projects that
"  Involve large teams of developers
"  Are long-lived (supported, maintained, and evolved)
"  Are safety critical – failure could result in loss of life or
other disaster
"  Required to be of high integrity – failure could result in
unacceptable financial loss

MSU, CSE 814 INTRODUCTION 13

Software lifecycle

http://www.samsvb.co.uk/images/development_life_cycle.gif

MSU, CSE 814 INTRODUCTION 14

Software development phases
!  Requirements analysis
"  Identify requirements, possibly also architecture
!  Specification: What the system should do
"  Functional and non-functional properties
!  Design: How system will meet the specs
"  Modules and their interfaces
!  Coding …
!  Testing
"  Unit testing, integration testing, system testing
!  Maintenance
"  Debugging, support client, evolve system
MSU, CSE 814 INTRODUCTION 15
Waterfall model: classic

Reqt’s

Specification

Design

Coding

Testing

Maintenance

MSU, CSE 814 INTRODUCTION 16

V-shaped model

Reqt’s System test

Specification Integ. test

Design Unit test

Coding

MSU, CSE 814 INTRODUCTION 17

Incremental/iterative model
Reqt’s/spec.
Reqt’s/spec.
Reqt’s/spec.
Reqt’s/spec.
Reqt’s/spec. Design
Design
Design
Design
Design Code/u-test
Code/u-test
Code/u-test
Code/u-test
Code/u-test Int/sys-test
Int/sys-test
Int/sys-test
Int/sys-test
Int/sys-test Operation
Operation
Operation
Operation
Operation

Spiral model: incremental model emphasizing risk analysis

Basis for agile software development

MSU, CSE 814 INTRODUCTION 18

Necessary, but insufficient for safety-
critical systems
!  Testing
"  samples execution behavior, misses some
!  Systematic Inspections
"  don’t scale very well, misses some
!  Rigorous development processes
"  can help, but most organizations don’t apply them

Formal methods are increasingly being used

to provide stronger assurances, especially
in early phases.

MSU, CSE 814 INTRODUCTION 19

Formal methods approach
“The formal methods approach to software
construction is based on viewing a program and its
execution as mathematical objects and applying
mathematical and logical techniques to specify and
analyze the properties and behaviors of these
objects.”
-- R. Dewar and A. Pneuli

Formal specification: express properties that a

system must satisfy
"  Formal language: Syntax can be checked,
unambiguous semantics
"  Abstraction: Above the level of source code,
“what” (not “how”)
MSU, CSE 814 INTRODUCTION 20
Why use formal methods?
!  Forced to think systematically about issues
"  Leads to better design
"  Earlier detection of inconsistencies and flaws
!  Formal spec provides precise reference for
"  Checking that reqt’s are satisfied, conformance
"  Directing latter development phases
"  Documentation within a team of developers
"  Specification matching in support of reuse
"  ...
“Formal Specification: A Roadmap,” A. van Lamsweerds
Future of Software Engineering (Limerick, Ireland),
June 2000. ACM Press.

MSU, CSE 814 INTRODUCTION 21

The essence of software
“The essence of a software entity is a construct of
interlocking concepts: data sets, relationships among
data items, algorithms, and invocations of functions.
This essence is abstract … Nonetheless, it is highly
precise and richly detailed.

I believe the hard part of building software to be
the specification, design, and testing of this
conceptual construct, not the labor of representing
it and testing the fidelity of the representation.”

-- Fred Brookes. No Silver Bullet.

MSU, CSE 814 INTRODUCTION 22

Use formal specifications to
!  Precisely document the essence of software
!  Check syntactic well-formedness

!  Generate test cases

!  Check for/classify failures in test runs

!  Provide reqt’s for next iteration

!  Semantic analysis (design testing)

If my system satisfies these specs, can I infer that
it satisfies some additional properties or invariants?

Focus of this course is on “strong formal methods”:

formal methods with tool-supported semantic analysis

MSU, CSE 814 INTRODUCTION 23

In this course
!  Study representative formal spec languages
"  Predicate logic: Specification and automated proof
"  Alloy: Structural design models based on relations
"  SPARK Ada: Correct-by-construction using software
contracts
"  Possibly: Finite State Processes (FSP) - Linear-time
model of concurrent system behaviors
"  Possibly: Temporal logic – specification notation for
concurrent systems
"  Possibly: Model checking – verification that a model
satisfies a specification

MSU, CSE 814 INTRODUCTION 24

In this course

!  Learn about their associated methodologies and

analysis techniques
!  Use tools to check syntax, simulate behavior
(early prototyping), verify semantic properties

MSU, CSE 814 INTRODUCTION 25

Summary
!  Software is increasingly complex, increasingly
ubiquitous
!  Cannot trust safety critical software to careful
design, programming, and testing
!  Formal methods are not a panacea, but can
help, and are increasingly becoming necessary
to achieve high assurance
!  We will learn several formal methods—different
development stages, different methodologies,
different application domains
!  We will emphasize strong formal methods

MSU, CSE 814 INTRODUCTION 26

Acknowledgments
!  Slide 11 from a tutorial by R. Chapman and P. Amey (Praxis Critical Systems).
Posted at http://cs.nyu.edu/courses/fall01/G22.3033-007/tutorial.ppt
!  D. Knuth quote, slide 2, from “The Inevitable Pain of Software Development: Why
There is No Silver Bullet,” D. Berry (U. Waterloo). DOI 10.1007/b96009
!  Others borrow heavily from:
"  M. Dwyer, J. Hatcliff and R. Howell. Overheads for CIS 771: Software Specifications. Kansas
State University. 2001. Posted at http://santos.cis.ksu.edu/771-Distribution/
"  P. Dewar and A. Pneuli. Overheads for . New York University. 2001. Posted at
http://cs.nyu.edu/courses/fall01/
"  R. Lewallen. http://codebetter.com/blogs/raymond.lewallen/archive/2005/07/13/129114.aspx
"  J. Jacky. Safety-Critical Computing: Hazards, Practices, Standards, and Regulations.
http://staff.washington.edu/jon/pubs/safety-critical.html#fm

MSU, CSE 814 INTRODUCTION 27