Agile Practices Proven in High

Assurance and Highly

Regulated Environments

2011 Rally Software Development and Leffingwell, LLC.

Define high assurance?

High assurance software systems are

unique because they must satisfy basic
functional service properties that the
system intends to deliver, as well as
guarantee desirable system properties
such as security, safety, timeliness,
and reliability.

2011 Rally Software Development and Leffingwell, LLC. 2

 2011 Rally Software Development and Leffingwell, LLC. 3
Regulating bodies

FDA Federal Drug Administration

ISO International Standards
European Union MEDDEV
Drug Controller General of India
Central Drugs Standard Control Organisation (CDSCO
Medical Devices Division
Health Canada
Ourselves (CMMI)

Global Harmonization Task Force guidance docs

IEC, although not a regulation is recognized as a
good standard when developing such things as
medical devices
2011 Rally Software Development and Leffingwell, LLC. 4
 2011 Rally Software Development and Leffingwell, LLC. 5
Although the waterfall model is a
useful tool for introducing design
controls, its usefulness in practice is
limited for more complex devices, a
concurrent engineering model is more
representative of the design
processes in use in the industry.

From [FDA CDRH 1997] Design Control Guidance for Medical Device
Manufacturers

2011 Rally Software Development and Leffingwell, LLC. 6

Surprise? FDA and IEC Guidance does
NOT recommend waterfall
[FDA CDRH 2002] It is important to note, that neither this document, nor
CFR820.30 itself, constrains development to single pass, stage-gated,
waterfall activities.
From General Principles of Software Validation.. [FDA CDRH 2002]:
While this guidance does not recommend any specific life cycle model or
any specific technique or method, it does recommend that software
validation and verification activities be conducted throughout the entire
software life cycle.
From [FDA CDRH 1997] Design Control Guidance for Medical Device
Manufacturers : Although the waterfall model is a useful tool for
introducing design controls, its usefulness in practice is limited for more
complex devices, a concurrent engineering model is more representative
of the design processes in use in the industry.
IEC 62304 medical device standard states: these activities and tasks
may overlap or interact and may be performed iteratively or recursively. It
is not the intent to imply that a waterfall model should be used.

2011 Rally Software Development and Leffingwell, LLC. 7

Industry myth perpetuated by our
own waterfall past?

2011 Rally Software Development and Leffingwell, LLC. 8

 Software
engineering
& SDLC
Lean / Agile

Craig Langenfeld PMP, CSM Regulated

[email protected] environment

2011 Rally Software Development and Leffingwell, LLC. 9

 Dean Leffingwell

Author Coach Executive

Agile Enterprise Coach Founder and CEO

and Mentor ProQuo, Inc., Internet identity
To some of the worlds larger
software enterprises
Senior VP
Rational Software
Responsible for Rational
Chief Methodologist Unified Process
Rally Software

Founder/CEO
Requisite, Inc.
Cofounder/Advisor Makers of RequisitePro
Ping Identity, Roving Planet,
Silver Creek Systems, Rally
Software Founder and CEO
RELA/Colorado Medtech
Medical Device Development

2011 Rally Software Development and Leffingwell, LLC. 10

Waterfall Story

2011 Rally Software Development and Leffingwell, LLC. 11

 2011 Rally Software Development and Leffingwell, LLC. 12
Where are we going?

Agile Proven within High Assurance

Healthcare Example

How?
Agile Framework for High Assurance
High Assurance Requirements Model
Artifact generation
Updated Quality Management Systems

2011 Rally Software Development and Leffingwell, LLC. 13

Agile is already in high assurance
Abbott Laboratories
20 30 % fewer defects were found
availability of working software early on was a significant factor
This experience has convinced us that an agile approach is the
approach best suited to development of FDA-regulated devices.
GE Healthcare Goes Agile Dr. Dobbs article 2010
we are making progress and feel that the benefits of our Agile adoption have
been worth the effort. Because of this we are rolling out Agile globally within
GE Healthcare
AFEI DoD Agile Development Conference
Agile Methods are in widespread use by the U.S. DoD, Prior the
commercial industry and DoD contractors believed the U.S. DoD was
not committed to Agile , an enormously incorrect assumption

Sources: Abbott Labs whitepaper: http://www.computer.org/portal/web/csdl/doi/10.1109/AGILE.2009.50.

AAMI report: See http://www.aami.org/applications/search/details.cfm?WebID=P1541_D6110
DoD Association for Enterprise Information (AFEI): See http://www.afei.org/Pages/default.aspx.
(See http://davidfrico.com/afei-2010.doc)
2011 Rally Software Development and Leffingwell, LLC. 14
Whitepapers and other references

Association for Advancement Medical

Instrumentation
TIR - Guidance on the use of AGILE practices in the
development of medical device software
Since AGILE is a highly
INCREMENTAL/EVOLUTIONARY approach, it can
therefore be mistakenly assumed that AGILE is
incompatible with the expectations for a medical device
software process.

Blogs
Scott Ambler Agile Scaling Model
Tom Grant Forrester Analyst
Dean Leffingwell Scaling Software Agility Blog
2011 Rally Software Development and Leffingwell, LLC. 15
Agile gets results

We
We experienced
experienced aa 20-50%
20-50%
increase
increase in
in productivity.
productivity.
BMC
BMC Case
Case Study
Study

makes
makes the
the work
work more
more
enjoyable,
enjoyable, helps
helps us
us work
work
together,
together, and
and is
is
empowering
empowering 37-50%
37-50% faster
faster to
to
Medtronic
Medtronic market
market
Quality QSM
QSM research
research

2011 Rally Software Development and Leffingwell, LLC. 16

Agile drives quality, safety, efficacy

fewer
fewer defects
defects were
were found
found
Abbott
Abbott Labs
Labs

Collective Coding
Ownership Standards
Test-Driven
Development

Pair Automated
Programming Testing

Simple
Continuous Design Refactoring
Integration

User Stories

of
of 131
131 respondents,
respondents, 88%88%
said
said quality
quality was
was better
better or
or
significantly
significantly better
better Helps
Helps us
us find
find bugs
bugs earlier
earlier
Shine
Shine Technology
Technology Survey
Survey Medtronic
Medtronic

2011 Rally Software Development and Leffingwell, LLC. 17

AN AGILE, HIGH ASSURANCE
LIFECYCLE FRAMEWORK

2011 Rally Software Development and Leffingwell, LLC. 18

But high assurance development has additional
requirements
Medical device exemplar: US FDA mandates
Software Verification and Validation

User Review
Needs

Design
Input

Design
Process

Design
Verification Output

Medical
device

Validation
Source: FDA CDRH 1997 Design Control
Guidance for Medical Device Manufacturers

2011 Rally Software Development and Leffingwell, LLC.

So we have additional mandates
Code of Federal Regulations CFR 21 Part 830, Subpart C Design Controls
mandates device design verification and validation.

Verification Validation
Provides objective evidence that the Confirmation that software
design outputs of a particular phase specifications conform to user needs
of the software development life cycle and intended uses, and that the
meet all of the specified requirements particular requirements implemented
for that phase. through software can be consistently
fulfilled.Since software is usually
part of a larger hardware system, the
validation includes evidence that
all software requirements have been
You built it implemented correctly and completely
right and are traceable to system
requirements.
Sources:
Regulation: Code of Federal Regulations 21 Part 830, Subpart C
Design Controls
Guidance: General Principles of Software Validation You built
2011 Rally Software Development and Leffingwell, LLC.
the right 20
And
Code of Federal Regulations CFR 21 Part 830, Subpart C Design Controls
mandates a requirements specification.

Requirements Specification Traceability

A documented software requirements FDA guidelines describe traceability and a
specification (SRS) provides a baseline for primary mechanism to assure that
both validation and verification. The verification and validation are complete
software validation process cannot be and consistent.
completed without an established software Traceability. The degree to which a
requirements specification relationship can be established between
(Ref: 21 CFR 820.3(z) and (aa) and 820.30(f) two or more products of the development
and (g process, especially products having a
predecessor-successor or master-
subordinate relationship to one another;
e.g., the degree to which the requirements
and design of a given software component
match [IEEE]

Sources:
Regulation: Code of Federal Regulations 21 Part 830, Subpart C
Design Controls
Guidance: General Principles of Software Validation

2011 Rally Software Development and Leffingwell, LLC. 21

 ITERATION
MECHANICS Daily
Backlog Standup
Grooming

Iteration Define Iteration

Planning Demo, Review
&
Build Retrospective

Verify

Product Iteration
Product
Backlog Backlog
Increment

2011 Rally Software Development and Leffingwell, LLC.

 PROJECT
AGIL

LIFECYCLE
E

System
Increment

Planning,
Analysis, Design
Architecture, QMS Transfer
Project Verification Verification Verification Validation
Inception Iteration Iteration Iteration Iteration N Production
Code
Set up Project
Infrastructure
Verification and Validation activities and artifacts driven by QMS

2011 Rally Software Development and Leffingwell, LLC.

High Assurance Scaled Agile Framework
The User Story
Acceptance
Definition of
Criteria
Done

As a <role>
I can <activity>
So that <business value>

As an EPAT (Extracorporeal Pulse Activation Technology) technician,

(<role>) I can adjust the energy delivered (<what I do with the system>)
in increments so as to deliver higher or lower energy pulses to the
patients treatment area (<value the patient receives from my action>).

2011 Rally Software Development and Leffingwell, LLC.

Traceability from User Story to Code and
Story Acceptance Test

SCM Repository
Software Implemented by
User Story Code
Requirements
1 1..*
Specification 1 1

Verified by Verified by

1..*
Test Repository
1..* Unit Test
Story
Acceptance Test

2011 Rally Software Development and Leffingwell, LLC.

Validating Product Claims

Feature
Product Requirements Pulse amplitude is adjustable
from 1-5 bar
Document

Traced to

As an operator, I can adjust the pulse As an operator, I always

amplitude in .1 bar increments so as to be see the current setting
able to make small changes to change on the display in .1 bar
energy delivered to patient area increments, so I can be
confident Im delivering
the right energy

Software Requirements
As an operator, rotating the energy knob past the
Specification point where the system is delivering 5 bar will
have no further effect

User User User

Story Story Story

2011 Rally Software Development and Leffingwell, LLC.

 High Assurance Agile Backlog Model

Source: Leffingwell. Agile Software

Requirements: Lean Requirements Practices
for Teams, Programs, and the Enterprise.
Addison-Wesley 2011.

2011 Rally Software Development

2011 and Leffingwell, LLC.
Leffingwell, LLC.
Validating Features and System Qualities

2011 Rally Software Development and Leffingwell, LLC. 29

Agile and Quality Management Systems
(QMS)
Continuous improvement or (re-)write from scratch

Establish cross-functional QMSscrum team

Run releases and sprints to refine / establish QMS

Design Controls needs to provide flexibility

Software Development Life Cycle (SDLC), Tools, etc. should be

specified in the Design and Development Plan (DDP), not in the
QMS

2011 Rally Software Development and Leffingwell, LLC. 30

Validation Sprint Activities

2011 Rally Software Development and Leffingwell, LLC. 31

Quality Management
Strategy

Matt Anderson
Director Program Management

9/4/17
MethodQ/SLIM Overview

Release
Release Plan
Plan
Initial Design Input Signature
Roadmap
User Stories/Capabilities (Epics)
Acceptance Criteria

Tasks to update for each User Story

Iterations
Iterations Solution Level Requirements
Design artifacts

Final Design Input Signature

Solution Level Requirement Document(s)

Release
Release Final Design Output Signature
Asset design artifacts, code, traceability

2011 Rally Software Development and Leffingwell, LLC.

Change Record Management

Release
Release CR
CR Initial Design Input Signature
Roadmap

Capability
Capability CR
CR

Final Design Input Signature

Current Solution Level Requirements
Release
Release CR
CR
Final Design Output Signature
Solution Level Test Scenarios
Test Evidence
Release
Release Solution Level Technical Artifacts

2011 Rally Software Development and Leffingwell, LLC.

Change Record Management

Release
Release CR
CR Initial Design Input Signature
User Stories
Acceptance Criteria
Initial Visual Design

Capability
Capability CR
CR Final Design Input Signature
Updated Solution Level Requirements
Visual Design

Final Design Output Signature

Test Scenarios
Test Evidence
Release
Release CR
CR Code/Code Review
Technical Artifacts as needed

Release
Release

2011 Rally Software Development and Leffingwell, LLC.

Parent/Child Relationships

Release
Release CR
CR

Capability
Capability CR
CR Capability
Capability CR
CR Capability
Capability CR
CR

Capability
Capability CR
CR Capability
Capability CR
CR
Capabilities cannot span releases, but can span iterations
CR can be both a Child and a Parent
Each CR must have completed Design Input and Output
Initial Design Input for Child can be covered by the Parent
2011 Rally Software Development and Leffingwell, LLC.
 Agile extremism does not help
(working software over documentation)

2011 Rally Software Development and Leffingwell, LLC. 37

Agile and most regulating bodies are not
at odds

2011 Rally Software Development and Leffingwell, LLC. 38

 Satisfy compliance
while preserving
Agility.

2011 Rally Software Development and Leffingwell, LLC.

Implement the appropriate degree of rigor

2011 Rally Software Development and Leffingwell, LLC.

 2011 Rally Software Development and Leffingwell, LLC. 41
Agile Perfect for High Assurance

Agile isnt just good

for High Assurance
development its
better than traditional
methods.
- Tom Grant, Forrester Group

2011 Rally Software Development and Leffingwell, LLC.

Live long and prosper!

Craig Langenfeld

[email protected]

cameo by Matt Anderson

2011 Rally Software Development and Leffingwell, LLC. 43