Software Project Management

Sixth Edition

Chapter 13.2

Software process
quality

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 1

 Product and Process Quality
A good process is usually required to
produce a good product.
For manufactured goods, process is the
principal quality determinant.
For design-based activity, other factors are
also involved:
For example, the capabilities of the
designers.
SPM (6e) Product quality© The McGraw-Hill Companies, 2017 2
 BS EN ISO 9001:2000 and quality
management systems
ISO 9001 is one of a family of standards that
specify the characteristics of a good quality
management system (QMS)
Can be applied to the creation of any type of
product or service, not just IT and software
Does NOT set universal product/service standards
DOES specify the way in which standards are
established and monitored
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 3
 ISO 9001:2000 principles
1. Understanding the requirements of the
customer
2. Leadership to provide unity of purpose and
direction to achieve quality
3. Involvement of staff at all levels
4. Focus on the individual which create
intermediate and deliverable products and
services
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 4
 ISO 9001:2000 principles

5. Focus on interrelation of processes that

deliver products and services
6. Continuous process improvement
7. Decision-making based on factual
evidence
8. Mutually beneficial relationships with
suppliers

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 5

 ISO 9001:2000 cycle
1
Establish requirements + acceptance
criteria + quality levels 2

Design product

Design production process 3

+ quality process
9
6

process build quality

improvement control
8 7
customer
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017

6
 ISO 9001 Requirements

Management Control of customer

responsibility supplied product
Quality system Product identification
and traceability
Contract review
Process control
Design Control
Inspection and testing
Document and data
Control of inspection,
control
measuring and test
Purchasing equipment

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 7

 The need to improve
can everything be improved at one?
no, must tackle the most important things first
‘poor companies are poor at changing’
some later improvements build on earlier ones
but there are problems
improvement takes up time and money
‘improvement’ may simply be more
bureaucracy!

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 8

 Capability maturity model (CMM)
Created by Software Engineering Institute, Carnegie
Mellon University
CMM developed by SEI for US government to help
procurement
The rationale was:
Include likely contractor performance as a factor
in contract awards.
Watts S. Humphrey ‘Managing the software
process’ Addison Wesley
Assessment is by questionnaire and interview
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 9
 Capability maturity model 2
Different versions have been developed for
different environments e.g. software
engineering
New version CMMI tries to set up a generic
model which can be used for different
environments

SPM (5e) Software process quality© The McGraw-Hill Companies, 2009 10

 SEI Capability Maturity Model

Can be used in two ways:

Capability evaluation
Software process assessment.

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 11

 Capability Evaluation

Provides a way to assess the software

process capability of an organization:
Helps in selecting a contractor
Indicates the likely contractor
performance.

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 12

 Software Process Assessment

Used by an organization to assess its

current process:
Suggests ways to improve the process
capability.
This type of assessment is for purely
internal use.

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 13

 What is CMM?
Describes an evolutionary improvement path for
software organizations from an ad hoc immature
process :
To a mature, disciplined one.
Provides guidance on:
How to control the process
How to evolve the process

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 14

 CMM Maturity Levels
Five maturity levels:
Stages are ordered so that improvements
at one stage provide foundations for the Optimizing (5)
next.
Based on the pioneering work of Philip
Crosby. Managed (4)

Defined (3)

Repeatable (2)

Initial (1)

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 15

 Process maturity levels 5. optimizing

A company is at level 1 by
default i.e. there is no level process
4. managed
zero control

3. defined process
management

2. repeatable process definition

basic management
control
1. initial
SPM (56) Software process quality© The McGraw-Hill Companies, 2017 16
 CMM Level 1 (Initial)
Organization operates Without any formalized
process or project plans
An organization at this level is characterized by
Ad hoc and chaotic activities.
Software development processes are not
defined,
Different developers follow their own process
The success of projects depend on individual
efforts and heroics.

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 17

 Level 2 (Repeatable)
Basic project management practices are followed
Size and cost estimation techniques:
Function point analysis, COCOMO, etc.
Tracking cost, schedule, and functionality.
Development process is ad hoc:
Not formally defined
Also not documented.

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 18

 Level 3 (Defined)

All management and development activities:

Defined and documented.
Common organization-wide understanding of
activities, roles, and responsibilities.
The process though defined:
Process and product qualities are not
measured.

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 19

 Level 4 (Managed)
Quantitative quality goals for products are set.
Software process and product quality are measured:
The measured values are used to control the product
quality.
Results of measurement used to evaluate project
performance:
Rather than improve process.
Detailed measures of the software process and product
quality are collected.
Both the software process and products are quantitatively
understood and controlled.
SPM (6e) Product quality© The McGraw-Hill Companies, 2017 20
 Level 5 (Optimizing)

Statistics collected from process and product

measurements are analyzed:
Continuous process improvement based on
the measurements.
Known types of defects are prevented from
recurring by tuning the process
Lessons learned from specific projects
incorporated into the process

SPM (6e) Product quality© The McGraw-Hill Companies, 2017 21

 Key process areas

The KPAs of a level indicate the areas that an

organization at the lower maturity level needs to focus
to reach this level.
KPAs provide a way for an organization to gradually
improve its quality of over several stages.
KPAs for each stage has been carefully designed such
that one stage enhances the capability already built
up.
Trying to focus on some higher level KPAs without
achieving the lower level KPAs would be
counterproductive.

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 22

 A repeatable model
inputs
(requirements)

control (budget, construct the outputs (code,

schedule, standards) system documentation)

resources
(staff, tools)

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 23

 Repeatable model KPAs

To move to this level concentrate on:

Configuration management
Quality assurance
Sub-contract management
Project planning
Project tracking and oversight
Measurement and analysis

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 24

 requirements
A defined process

design methods design &

define system design

tools, staff etc

inspection code & unit

test tested modules
criteria

tools, staff etc.

test plans integrate/
system test
tools, staff etc.
system
software 25
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017
 Repeatable to defined KPAs
Concentrate on
Requirements development and technical solution
Verification and validation
Product integration
Risk management
Organizational training
Organizational process focus (function)
Decision analysis and resolution
Process definition
Integrated project management

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 26

 a managed process
requirements
directives
design defects manage
design methods design &
define system design

tools, staff etc

ct iv es
dire

directives

system errors
er rors
ing
cod
inspection code & unit
test tested modules
criteria

tools, staff etc.

test plans integrate/
system test
tools, staff etc.
system
software 27
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017
 Defined to managed KPAs

Concentrate on:
Organizational process performance
Quantitative project management

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 28

 Optimizing
Optimize
old control/development system

directives
requirements
design defects manage
design methods design &
define system design

system errors
directives
tools, staff etc es
i r ec t iv rs
d
g e rr o
i n
inspection code & unit cod
test tested modules
criteria
tools, staff etc.
test plans integrate/
system test
tools, staff etc.
system
software

new control/development system

SPM (5e) Software process quality© The McGraw-Hill Companies, 2009 29
 Managing to optimizing: KPAs

Concentrate on:
Causal analysis and resolution
Organizational innovation and deployment

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 30

 CMMI (Capability Maturity Model
Integration)

CMMI is the successor of the Capability Maturity

Model (CMM).
After CMMI was first released in 1990:
It became popular in many other domains
Human Resource Management (HRM).: people
management (PCMM)
software acquisition (SA-CMM)
systems engineering (SE-CMM)

SPM (5e) Software process quality© The McGraw-Hill Companies, 2009 31

 Some questions about CMMI
Suitable only for large organizations?
e.g. need for special quality assurance and
process improvement groups
Defining processes may not be easy with new
technology
how can we plan when we’ve not used the
development method before?
Higher CMM levels easier with maintenance
environments?
Can you jump levels?

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 32

ISO/IEC 15504 IT process assessment
To provide guidance on the assessment of
software development processes
Process Reference Model: Needs a defined
set of processes that represent good practice to
be the benchmark
ISO 12207 is the default reference model
Could use others in specific environments

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 33

 ISO 15504 performance attributes
CMMI level ISO 15504

0. incomplete

initial 1.1.process performance – achieves defined

outcome

repeatable 2.1 process management – it is planned and

monitored

2.2 work product management – control of work

products

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 34

 ISO 15504 performance attributes -
contd

CMMI ISO 15504

Defined 3.1. Process definition

3.2. Process deployment

Managed 4.1. Process measurement

4.2. Process control

Optimizing 5.1. Process innovation

5.2. Process optimization

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 35

 Process Reference Model

A defined standard approach to development

Reflects recognized good practice
A benchmark against which the processes to
be assessed can be judged
ISO 12207 is the default model

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 36

 IS0 15504 Process Assessment

For each process in the relevant Process Reference

Model
For each set of attribute level criteria
Assess whether:
N: not achieved 0-15%
P: partially achieved >15%-50%
L: largely achieved >50%-85%
F: fully achieved >85%

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 37

 ISO 15504 performance indicators

This is just an example of how indicators for each level

might be identified
1.Performance
Descriptions of maximum and minimum expected input
values exist
2.1 Performance management
A plan of how expected input variable ranges are to be
obtained exists which is up to date

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 38

 ISO 15504 performance indicators 2
2.2 Work product management
There are minutes of a meeting where the input
requirements document was reviewed and corrections
were mandated

3.1 Process definition

A written procedure for input requirements gathering
exists

3.2 Process deployment

A control document exists that is signed as each part
of the procedure is completed

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 39

 ISO 15504 performance indicators 2

4.1. Process measurement

Collected measurement data can be collected
e.g. number of changes resulting from review
4.2. Process control
Memos relating to management actions taken in
the light of the above

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 40

 ISO 15504 performance indicators 3

5.1 Process innovation

Existence of some kind of ‘lessons learnt’ report at
the end of project
5.2. Process optimization
Existence of documents assessing the feasibility of
suggested process improvements and which
show consultation with relevant stakeholders

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 41

 Techniques to improve quality -Inspections

When a piece of work is completed, copies are

distributed to co-workers
Time is spent individually going through the work
noting defects
A meeting is held where the work is then discussed
A list of defects requiring re-work is produced

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 42

 Inspections - advantages of approach

An effective way of removing superficial errors

from a piece of software
Motivates the software developer to produce
better structured and self-descriptive code
Spreads good programming practice
enhances team-spirit
The main problem maintaining the commitment
of participants

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 43

 ‘Clean-room’ software development
Ideas associated with Harlan Mills at IBM
Three separate teams:
1. Specification team – documents user requirements
and usage profiles (how much use each function will
have)
2. Development team – develops code but does not test
it. Uses mathematical verification techniques
3. Certification team – tests code. Statistical model used
to decide when to stop

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 44

 Formal methods
Use of mathematical notations such as VDM and Z
to produce unambiguous specifications
Can prove correctness of software mathematically
(cf. geometric proofs of Pythagoras’ theorem)
Newer approach use Object Constraint Language
(OCL) to add detail to UML models
Aspiration is to be able to generate applications
directly from UML+OCL without manual coding –
Model Driven Architectures (MDA)
SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 45
 Quality plans

quality standards and procedures should be

documented in an organization’s quality
manual
for each separate project, the quality needs
should be assessed
select the level of quality assurance needed
for the project and document in a quality plan

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 46

Typical contents of a quality plan

scope of plan
references to other documents
quality management, including organization,
tasks, and responsibilities
documentation to be produced
standards, practices and conventions
reviews and audits

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 47

more contents of a quality plan

testing
problem reporting and corrective action
tools, techniques, and methodologies
code, media and supplier control
records collection, maintenance and retention
training
risk management

SPM (6e) Software process quality© The McGraw-Hill Companies, 2017 48