Winning Ways 2010 EMC Black Belt Summit

Using Scrum to Avoid Bad CMMI®

Implementations 29 October 2010
® CMMI is registered with the U.S. Patent and Trademark Office CMMI is an acronym for
Capability Maturity Model Integration.

Kent A. Johnson

Jeff Sutherland
© 2010
1
© 2010 Kent Johnson and Jeff Sutherland
 Kent A. Johnson
[email protected]
www.agiledigm.com
•  Chief Technical Officer of AgileDigm, Incorporated
•  AgileDigm is an international consulting company that was
formed in 2001 to support an agile paradigm for process
improvement.
•  SEI-certified SCAMPI High Maturity Lead Appraiser, an SEI-
authorized CMMI® instructor, and a Certified ScrumMaster
•  Has helped clients in over 25 countries to improve their ability
to develop products - including one of the world s only Agile
Organizations to achieve CMMI Maturity Level 5.
•  Co-author of Interpreting the CMMI: A Process Improvement
Approach, Second Edition, Auerbach (2008) and numerous
peer reviewed papers on Agile and CMMI.

© 2010
2
2
 Jeff Sutherland, Ph.D.
"   Chairman, Scrum Foundation (formerly Scrum Training Institute)
" Lithespeed – local partner Sanjiv Augustine
"   CEO Scrum, Inc. and Senior Advisor, OpenView Venture Partners
"   Agile coach for OpenView Venture Partners portfolio companies
"   CEO/CTO/VP Engineering for 11 software companies
"   Created first Scrum at Easel Corp. in 1993.
"   Achieved hyperproductive state in all companies. Signatory of Agile Manifesto
and founder of Agile Alliance

–  http://scrum.jeffsutherland.com
–  [email protected]
–  +1 617 606 3652

© 2010
3
3
Our Message
Great synergy comes from using Scrum with CMMI
–  Scrum brings value to CMMI
–  CMMI brings value to Scrum
Experience from 15+ organizations embracing
CMMI and Scrum
Summarized in three experiences
A. Initial Scrum and CMMI Implementation
B. Failed Scrum Back to Life
C. CMMI Level 5 using Scrum

© 2010
4
Agile and CMMI Misconceptions
•  CMMI organizations are bureaucratic and
inflexible. (misconception)
•  CMMI organizations are as flexible as the people
in the organization. CMMI supports
organizational change and improvement.
(reality)
•  Agile organizations are extremely flexible
because they have no real rules.
(misconception)
•  Agile organizations are as flexible as the people
in the organization. Agile methods support
change particularly within the team. (reality)
© 2010
5
 CMMI Does Not Require Waterfall
What the CMMI is not:
–  CMMI is not Waterfall .
CMMI does not require any particular lifecycle.

© 2010
6
Scrum CMMI Comparison (1/2)
Scrum Primer
•  22 pages double side 8 ½ by 11

CMMI 2nd Edition

•  676 page hardback book
•  Examples not friendly to Scrum

Interpreting the CMMI

•  404 page hardback book

© 2010
7
Scrum CMMI Comparison (2/2)
CMMI Scrum
•  A structured collection of •  A set of rules that
best practices constrains behavior
•  What you should do •  How you should do it
•  Focus is on improving •  Focus is on increasing
product quality through customer satisfaction
process performance

© 2010
8

© AgileDigm, Incorporated 2009

 Agile Manifesto
www.agilemanifesto.org
We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on

the right, we value the items on the left more.

Don't misread as there is no value to the things on the right.

© 2010
9
9 CSM v10.21 © Jeff Sutherland 1993-2009
9
Experience A: Initial Scrum & CMMI
•  Organizations that introduce Scrum and CMMI
together. Examples include a US defense
contractor, commercial consulting companies,
and product companies.
•  They had either a corporate directive, client
directive, or government directive to embrace
CMMI.
•  Initial goal: CMMI Level Two
•  Used Scrum to focus on the Common
Failures with CMMI (next slide).
© 2010
10
Common Failures with CMMI
1.  Too heavy a process definition
2.  Lack of management support
3.  Different understandings of mission and
goals
4.  Process adoption not well planned
5.  Process and procedure definition forced
on staff
6.  Pilots of process too limited
© 2010
11
Scrum Helped CMMI with
•  Clear customer focus
•  Lightweight process definition
•  Explicit team roles
•  Support from developers

© 2010
12
 What is Disciplined Scrum?
Sprint  
backlog  
Product  Backlog   Backlog  
items  selected  
for  release   S3  backlog  

S2  backlog  
Product
S1  backlog  
Backlog
Sprints  in  this  release  
Sprint Sprint
1 Sprint
2 3
Product   Daily  
Owner   Scrum  
Team   Retrospec?ve  

  Agenda   Demo  
Scrum   Burndown  

Master  
Agenda  

Sprint   Check   Check   Product  

backlog   Ready   Done   (increment)  
Ready   13 Done  
http://demo.callis.dk/scrum © 2010
checklist   checklist   13
Scrum Checklist Examples

Right Level of Process (1/2)

•  Practical processes and work aids contain
enough information to be useful.

© 2010 AgileDigm, Inc

14 14
© 2010
Scrum Status Report Example

Right Level of Process (2/2)

Anvil Project Weekly Status Report
User Stories Completed
–  As a User, I want to be able to use my anvil
24/7
–  As a User (all personas), I want to be able
to carry my anvil
–  …
Lessons Learned during the sprint
–  Breaking down stories into 20 points or less
is a big help
–  …
Impediments (not closed during week)
–  Testers assigned to multiple projects
–  …
Risks
–  User environment may not be ready on time
for final acceptance testing
–  ….
© 2010 AgileDigm, Inc© 2010 15
15
 Process Areas by Maturity Level
Maturity Level Process Areas
5 Optimizing Organizational Innovation and Deployment
Causal Analysis and Resolution
4 Quantitatively Managed Organizational Process Performance
Quantitative Project Management
3 Defined Requirements Development Technical Solution
Product Integration Verification
Decision Analysis and Resolution Validation
Organizational Process Focus
Organizational Process Definition +IPPD
Organizational Training
Integrated Project Management +IPPD
Risk Management
2 Managed Requirements Management
Project Planning
Project Monitoring and Control
ML 2 excluding SAM Supplier Agreement Management
Measurement and Analysis
Process and Product Quality Assurance
Configuration Management
1 Initial

© 2010
16
 ML2 Scrum Focus
Process Area Scrum Focus
Requirements Product Backlog (annotated)
Management Product Owner Interactions
Project Planning Product Backlog (annotated), Sprint Planning,
Release Planning, Scrum Definition
Project Monitoring Daily Stand Up Meetings, Sprint Reviews, Sprint
and Control Retrospective
Measurement and Measurement specifications outside of Scrum
Analysis Measures: # user stories, story point estimates,
burn down chart, velocity, audit results
Process and Product outside the scope of Scrum
These still
Quality Assurance need to be
Configuration outside the scope of Scrum done for ML2
Management 17
© 2010
17
Experience B: Failed Scrum Back to Life
•  This experience comes from organizations that
introduced Scrum across their projects. They
considered it a failure because:
–  Many of the projects reverted to doing ScrumBut.
(We are doing Scrum but…<pick from the list> no
daily meetings, no product owner, no burndown, etc.>
–  Projects did not get the great benefits that they had
heard about.
•  Initial goal: Use Real Scrum
•  Used CMMI principles to focus on the Common
Failures with Scrum.

© 2010
18
Benefits from Scrum
Excellent Scrum – annual revenue up 400%
•  Patient Keeper
•  Pegasystems (stock price up 400%)
Good Scrum – revenues up 300%
•  Some companies in Scandinavia
Pretty Good Scrum – revenues up 150-200%
•  Google 160%
•  Systematic A/S 200% (pre 2008)
ScrumBut – revenues up 0-35%
•  Yahoo, most companies
© 2010
19
 Common Failures with Scrum
1.  Misunderstanding what Scrum is
(and is not)
2.  Software not tested at end of sprint
(definition of Done)
3.  Backlog not ready at beginning of sprint
(definition of Ready)
4.  Lack of facilitation or bad facilitation
5.  Lack of management support
6.  Lack of client, customer, or end user
support © Jeff Sutherland and Kent Johnson 2010

© 2010
20
Consistent Performance
•  Difficulty: Agile often adopted on an ad hoc basis
–  Results in inconsistency across projects
–  Impacts large projects and programs
•  Approach: Define an Organizational Standard
–  Define organization s Agile processes along with
project tailoring guidelines
–  Train staff in defined Agile Processes
–  Using Organizational Process Definition and
Organizational Training Process Areas

© 2010
21
Key CMMI Attributes support Scrum
1.  Overall Planning and Tracking
2.  Training
3.  Risk Management
4.  Organizational Learning
5.  Organizational Visibility
6.  Measurement
7.  Institutionalization and Compliance

© 2010
22
Example CMMI Attribute

4. Organizational Learning
•  Vision
–  The organization obtains knowledge from the
use of processes by the team, project,
program, and enterprise. This knowledge is
shared across the organization.
•  CMMI Support for Agile includes:
–  Leveraging experience across the
organization
–  Knowledge sharing

© 2010
23
Example CMMI Attribute

7. Institutionalization (1/2)
•  Institutionalization is the ingrained way
of doing business that an organization
follows routinely as part of its culture.

•  Support for Agile includes:

–  Introducing new methods
–  Ensuring appropriate use of methods

This attribute is supported by CMMI generic practices.

© 2010
24
Example CMMI Attribute

7. Institutionalization (2/2)
Generic Goal 2 Institutionalize a Managed Process
GP 2.1 Establish an Organizational Policy
GP 2.2 Plan the Process
GP 2.3 Provide Resources
GP -Generic Practice
GP 2.4 Assign Responsibility
GP 2.5 Train People
GP 2.6 Manage Configurations
GP 2.7 Identify and Involve Relevant Stakeholders
GP 2.8 Monitor and Control the Process
GP 2.9 Objectively Evaluate Adherence
GP 2.10 Review Status with High Level Management
Generic Goal 3 Institutionalize a Defined Process
GP 3.1 Establish a Defined Process
GP 3.2 Collect Improvement Information

© 2010
25
Objectively Evaluate Adherence
•  A practical process shows what should be done.
•  An enterprise-wide function is usually created to
ensure that what should be done is done to
avoid the ScrumBut.
•  Such a function may be implemented in different
ways, for example:
–  Agile Coaches
–  Agile Program Management Office
•  The Nokia Test is one kind of evaluation that
could be done. (next two slides)

© 2010
26
 Nokia Test (1/2)
•  Created by Bas Vodde, Nokia Networks in Finland and
refined by Jeff Sutherland, co-creator of Scrum
•  Nine questions with answers between zero and ten (only
showing extreme answers in example)
Question 0 points 10 points
1 Iterations No iterations Fixed 4 weeks or less
2 Testing No dedicated Software is fully tested and
testers on team deployed each sprint
3 Specifications No Requirements Good user stories tied to
specifications
4 Product No Product Owner Product Owner who motivated
Owner team

© 2010
27
 Nokia Test (2/2)
Question 0 points 10 points
5 Product No Product Backlog Single Product Backlog, clearly
Backlog specified, prioritized, and
Product Owner can measure
Return on Investment
6 Estimates Product Backlog not Estimates produced by team
estimated and estimate error < 10%
7 Sprint No burndown chart Burndown based on DONE,
Burndown velocity known, Product Owner
Chart plans based on velocity
8 Team Manager or Project No one disrupting team, only
Disruptions Leader disrupt team Scrum roles
9 Team Tasks assigned to Team in hyperproductive state
team members

© 2010
28
Experience C: CMMI Maturity Level 5 Scrum
•  This experience comes from organization that
was CMMI Maturity Level 5. The organization
determined that their processes were too heavy
weight.
–  They embraced Lean Principles and determined that
Scrum was the best approach for them.
•  Goal: Become lean while maintaining a high
maturity CMMI rating
•  They introduced Scrum across all their projects
in a disciplined way.

© 2010
29
High Maturity Success Story
•  Systematic A/S is a success story using Scrum
and XP in a high maturity CMMI Implementation.
•  Addressed their business objectives with
–  doubled productivity
–  cutting defects by 40%
–  on top of years of other CMM and CMMI
improvements.
•  Reappraised at CMMI Maturity Level 5 in May
2009
•  One of the world s first Agile CMMI Maturity
Level 5 Companies

© 2010
30
 Systematic A/S Papers
•  Systematic A/S is a Danish company
•  Their improvement story is covered in detail in
three published papers:
–  Scrum and CMMI Level 5: the magic potion for Code
Warriors
•  Jeff Sutherland, Kent Johnson, & Carsten Jakobsen
–  Mature Agile with a Twist of CMMI
•  Carsten Jakobsen and Kent Johnson
–  Scrum and CMMI: Going from Good to Great
•  Carsten Jakobsen and Jeff Sutherland

© 2010
31
 Agile CMMI Performance Analysis
Project effort
100 % Rework
100%
CMMI Work
90%
Process focus
80%
50 % 69 %
70%
9%
60% Adding
50% SCRUM
40% 35 %
4%
30%
50 % 50 %
20% 25 %
10%
10 % 6%
CMMI 1 CMMI 5 CMMI 5
SCRUM
Source: Systematic A/S

© 2010
32
 Impediments
•  Data driven removal of impediments using
control charts
Examples on causes:
•  Special competences
•  Disk full
•  Setup misunderstood
•  COTS failed

Source: Systematic A/S

$ $ : :
$Revision

Root cause analysis of time to fix automatically generates

$Revision

ScrumMaster s impediment list. 33

© 2010
33
PagePage
 Story Process Efficiency
When work allocated to sprint is READY, flow and
stability are achieved
$
$Revision:

Objective: 60% Objective: 50h

34
Source: Systematic A/S © 2010
34
Page
Conclusion
•  Scrum + CMMI
–  Scrum avoids too much bureaucracy
–  CMMI avoids ScrumBut and other mistakes
–  support going from Good Scrum to Great
Scrum (Hyperproductive)
–  lead to organizational improvement
•  An agile implementation of CMMI can help
institutionalize Scrum in an organization

© 2010
35
Questions?
•  Time for some questions?

© 2010
36
 Certified ScrumMaster Course
•  Scrum Inc. provides a Certified
ScrumMaster Course with an additional
day to cover Scrum and CMMI
•  For this and other offerings see:
www.scrumfoundation.com

© 2010
37
More Detailed Information on the Subject
Features includes
•  Guides you through a proven approach
for designing the process improvement
effort, for monitoring and controlling
the effort, and measuring its success
•  Presents the pros and cons of
attempting process improvement and
implementing CMMI®
•  Offers advice for implementing several
process improvement projects
simultaneously to avoid added costs
and missteps
•  Discusses older methods of assessing
organizations and compares them to
the new SCAMPI method
•  Available from Auerbach / CRC Press
Publications and Amazon.com
•  Second Edition released in March 2008

© 2010
38