UML Models

• Activity diagrams
– capture business processes involving concurrency and
synchronization
– good for analyzing dependencies between tasks
Lecture 10, Part 1: • Class Diagrams
– capture the structure of the information used by the

Verification and Validation system

– good for analysing the relationships between data items
used by the system
– good for helping you identify a modular structure for the
system
Jennifer Campbell • Statecharts
– capture all possible responses of an object to all uses
cases in which it is involved
CSC340 - Winter 2007 – good for modeling the dynamic behavior of a class of
objects
– good for analyzing event ordering, reachability, deadlock,
etc.

CSC340 University of Toronto 2

UML Models [2] Non-UML models

– Use Cases • Goal Models
• capture the view of the system from the view of its – Capture strategic goals of stakeholders
users – Good for exploring ‘how’ and ‘why’ questions with stakeholders
• good starting point for specification of functionality – Good for analysing trade-offs, especially over design choices
• good visual overview of the main functional • Fault Tree Models (as an example risk analysis technique)
requirements – Capture potential failures of a system and their root causes
– Sequence Diagrams • Entity-Relationship Models
– Capture the relational structure of information to be stored
• capture an individual scenario (one path through a – Good for analysing risk, especially in safety-critical applications
use case) – Good for understanding constraints and assumptions about the subject
• good for modelling dialog structure for a user domain
interface or a business process – Good basis for database design
• good for identifying which objects (classes) • Mode Class Tables, Event Tables and Condition Tables (SCR)
participate in each use case – Capture the dynamic behaviour of a real-time reactive system
• helps confirm that all the necessary classes and – Good for representing functional mapping of inputs to outputs
operations have been identified – Good for making behavioural models precise, for automated reasoning

CSC340 University of Toronto 3 CSC340 University of Toronto 4

 Objectives of V&V Verification and Validation
• Validation:
“The overall objective of V&V approaches – “Are we building the right
Problem
system?” Situation
is to insure that the project is free from – Does our problem statement
failures and meets its user’s expectations.” accurately capture the real
problem?
– Did we account for the needs of Problem

Validation
• Correctness all the stakeholders? Statement
– The product is free of errors. • Verification:
• Consistency – “Are we building the system

Verification
– The product is consistent (within itself and with other related products). right?”
• Necessity – Does our design meet the spec?
Implementation
– Everything in the product is necessary. – Does our implementation meet the
Statement
spec?
• Sufficiency
– Does the delivered system do
– The product is complete.
what we said it would do?
• Quality – Are our requirements models
– The product satisfies its quality requirements. [Col88] consistent with one another?
System [Blu92]
CSC340 University of Toronto 5 CSC340 University of Toronto 6

Inquiry Cycle Note similarity with

Shortcuts in the inquiry cycle
Prior Knowledge process of scientific Prior Knowledge
(e.g. customer feedback) investigation: (e.g. customer feedback)
Requirements models are
theories about the world;
Initial hypotheses
Designs are tests of those
Observe
theories
(what is wrong with
Observe thethe
current
the system?)
prototype?)
model?)
(what is wrong with
the current system?)
Look for anomalies - what can’t
the current theory explain? Check
Checkproperties
properties
Model of
ofthe
themodel
model
Intervene Intervene Model
(describe/explain the Get (describe/explain the
(replace the old system)
observed problems) Getusers
(replace the old system) users observed problems)
to
totry
tryitit
Carry out the Design experiments to Create/refine
Analyze
Analyze
experiments test the new theory a better theory
the
themodel
model
(manipulate
Build
Buildaa
the variables) Design Prototype
(invent a better system) Prototype
Design
(invent a better system)
CSC340 University of Toronto 7 CSC340 University of Toronto 8
 Refresher: V&V Criteria V&V Example
Application Domain Machine Domain
C - computers • Requirement R:
D - domain properties
R - requirements P - programs – “Reverse thrust shall only be enabled when the aircraft is moving on the
[Jac95] runway”
• Domain Properties D:
• Domain Properties: things in the application domain that are true
– Wheel pulses on if and only if wheels turning
anyway
– Wheels turning if and only if moving on runway
• Requirements: things in the application domain that we wish to be
made true • Specification S:
• Specification: a description of the behaviours the program must – Reverse thrust enabled if and only if wheel pulses on
have in order to meet the requirements • Validation
– Are our assumptions, D, about the domain correct? Did we miss any?
• Two verification criteria: • Two validation criteria:
– The Program running on a – Are the requirements, R, what is really needed? Did we miss any?
– Did we discover (and
particular Computer satisfies understand) all the important • Verification
the Specification Requirements? – Does the flight software, P, running on the aircraft flight computer, C, correctly
– The Specification, given the – Did we discover (and implement S?
Domain properties, satisfies – Does S, in the context of assumptions D, satisfy R?
understand) all the relevant
the Requirements
Domain properties?
CSC340 University of Toronto CSC340 University of Toronto 10

Verification &Verification
Validation

V&V Activities V&V Activities: Reviews

• Reviews (Fagan) Inspections Management reviews
– a process management tool – E.g. preliminary design review
– Walkthroughs, inspections, etc. (always formal) (PDR), critical design review
• Software Testing – used to improve quality of the (CDR), …
– Not applicable to RE. development process – Used to provide confidence
– collect defect data to analyze that the requirements are
• Formal Methods the quality of the process sound
– Use mathematics to prove that the requirements are consistent. – written output is important – Attended by management and
– major role in training junior sponsors (customers)
• Consistency checking (this can also be done formally) staff and transferring expertise – Often just a “dog-and-pony
– Verifying consistency between models show”
• Prototyping Walkthroughs
– Present a prototype to the stakeholder to confirm that it has the – developer technique (usually
expected behaviour. Review the SRS with
informal)
• Requirements Tracing – used by development teams stakeholders to validate.
– Trace each requirement back to its source. to improve quality of product
[Col88] – focus is on finding defects
CSC340 University of Toronto 11 CSC340 University of Toronto 12
 Verification Verification
V&V Activities:
V&V Activities: Formal Methods
Consistency Checking
Basic Cross-Checks for UML
Model Analysis Use Case Diagrams StateChart Diagrams
– Does each use case have a user? – Does each statechart diagram capture (the
• Animation of the model on small examples • Does each user have at least one use case? states of) a single class?
– Formal challenges: – Is each use case documented? • Is that class in the class diagram?
• “if the model is correct then the following property should hold...” • Using sequence diagrams or equivalent – Does each transition have a trigger event?
Class Diagrams • Is it clear which object initiates each event?
– ‘What if’ questions: • Is each event listed as an operation for that object’s
• reasoning about the consequences of particular requirements; – Does the class diagram capture all the class in the class diagram?
classes mentioned in other diagrams?
• reasoning about the effect of possible changes – Does each state represent a distinct
– Does every class have methods to get/set its combination of attribute values?
• “will the system ever do the following...” attributes? • Is it clear which combination of attribute values?
– State exploration Sequence Diagrams • Are all those attributes shown on the class
• E.g. use a model checking to find traces that satisfy some property – Is each class in the class diagram? diagram?
– Are there method calls in the class diagram for
• “Is the model well-formed?” – Can each message be sent?
each transition?
• Is there an association connecting sender and
– Are the parts of the model consistent with one another? receiver classes on the class diagram? • …a method call that will update attribute values for
the new state?
• Is there a method call in the sending class for
each sent message? • …method calls that will test any conditions on the
transition?
• Is there a method call in the receiving class for
each received message? • …method calls that will carry out any actions on the
transition?
CSC340 University of Toronto 13 CSC340 University of Toronto 14

Verification Verification
V&V Activities: V&V Activities:
Consistency Checking [2] Consistency Checking [3]
Use Case Diagrams Sequence Diagrams Sequence Diagrams Is each class in Class Diagrams
the class diagram?
Does each use
case have a user?

Is each use case Can each message be sent?

• Is there an association connecting
documented?
sender and receiver classes on the
Using class diagram?
Does each
sequence • Is there a method call in the sending
user have
diagrams or class for each sent message?
at least one
equivalent • Is there a method call in the
use case? receiving class for each received
message?

CSC340 University of Toronto 15 CSC340 University of Toronto 16

 Verification Validation
V&V Activities:
V&V Activities : Prototyping
Consistency Checking [4]
Class Diagrams Statechart Diagrams
Does each statechart “A software prototype is a partial
diagram capture (the implementation constructed primarily to
states of) a single
class? enable customers, users, or developers to
• Does each transition have a trigger event?
learn more about a problem or its solution.”
– Is it clear which object initiates each event? [Davis 1990]
– Is each event listed as an operation for that object’s class in the class diagram?
• Does each state represent a distinct combination of attribute values?
– Is it clear which combination of attribute values? “Prototyping is the process of building a
– Are all those attributes shown on the class diagram?
• Are there method calls in the class diagram for each transition?
working model of the system”
– …a method call that will update attribute values for the new state?
[Agresti 1986]
– …method calls that will test any conditions on the transition?
– …method calls that will carry out any actions on the transition?
CSC340 University of Toronto 17 CSC340 University of Toronto 18

Validation Validation

V&V Activities : Prototyping [2] V&V Activities : Prototyping [3]

“Plan to throw one away - you will anyway!”, Fred Brooks
Approaches to prototyping
– Presentation Prototypes
Throwaway Prototyping Evolutionary Prototyping
• Purpose: – Purpose
• explain, demonstrate and inform – then throw away
– to learn more about the problem or its • to learn more about the problem
• e.g. used for proof of concept; explaining design features; etc. solution… or its solution…
– Exploratory Prototypes – discard after desired knowledge is • …and reduce risk by building
• used to determine problems, elicit needs, clarify goals, compare gained. parts early
design options • Use: – Use:
• informal, unstructured and thrown away. – early or late • incremental; evolutionary

– Breadboards or Experimental Prototypes • Approach: – Approach:

– horizontal - build only one layer (e.g. • vertical - partial impl. of all layers;
• explore technical feasibility; test suitability of a technology
UI) • designed to be extended/adapted
• Typically no user/customer involvement – “quick and dirty”
– Evolutionary (e.g. “operational prototypes”, “pilot systems”)
• development seen as continuous process of adapting the system
• “prototype” is an early deliverable, to be continually improved.
CSC340 University of Toronto 19 CSC340 University of Toronto 20
 Validation Validation

V&V Activities : Prototyping [4] V&V Activities : Tracing

Throwaway Prototyping Evolutionary Prototyping Forward traceability: trace requirements from
• Advantages: – Advantages:
– Learning medium for better • Requirements not frozen stakeholders to requirements specification.
convergence • Return to last increment if error is
– Early delivery → early testing → less found Traceability matrix:
cost • Flexible(?)
– Successful even if it fails! – Disadvantages: ID User Requirements Forward
• Disadvantages: • Can end up with complex, Traceability
– Wasted effort if reqts change rapidly unstructured system which is
– Often replaces proper documentation hard to maintain S2 Users shall process R10, R11, R12
of the requirements • early architectural choice may be retirement claims.
– May set customers’ expectations too poor
S3 Users shall process R13
high • Optimal solutions not guaranteed
survivor claims
– Can get developed into final product • Lacks control and direction

[Lud05]
CSC340 University of Toronto 21 CSC340 University of Toronto 22

Validation

V&V Activities : Tracing [2] Independent V&V

Backward traceability: trace requirements from • V&V performed by a • Three types of
separate contractor independence:
req spec to stakeholder. Traceability matrix: – Independent V&V fulfills the – Managerial Independence:
need for an independent • separate responsibility from
ID User Requirements Backward that of developing the
technical opinion. software
Traceability
– Cost between 5% and 15% of • can decide when and where
R10 The system shall accept requirement S2 development costs to focus the V&V effort
data. – Studies show up to fivefold – Financial Independence:
return on investment: • Costed and funded separately
R11 The system shall calculate the amount S2
• Errors found earlier, cheaper • No risk of diverting resources
of retirement. when the going gets tough
to fix, cheaper to re-test
R12 The system shall calculate point-to- S2 • Clearer specifications
– Technical Independence:
point travel time. • Different personnel, to avoid
• Developer more likely to use analyst bias
R13 The system shall calculate the amount S3 best practices
• Use of different tools and
of survivor annuity. techniques

CSC340 University of Toronto 23 CSC340 University of Toronto 24

[Lud05]
 References
[Blu92] Blum, B. I. 1992. Software engineering: a
holistic view. Oxford University Press.
[Col88] Collofello, J. 1988. Introduction to Software
Verification and Validation. SEI-CM-12-1.1.
[Jac95] Jackson, Michael. Software Requirements
and Specifications. Addison-Wesley,
Reading, MA, 1995.
[Lud05] Ludwig Consulting Services.
http://www.jiludwig.com/Traceability_Matix_Structure.html

CSC340 University of Toronto 25