Software Quality Engineering

BS(SE)-VI

Dr. Assad Abbas

Department of Computer Science

12/21/20 2
Process Improvement
n A lot of organizations still face a multitude of problems
when it comes to creating high-quality software products.
n On a regular basis, the Standish Group publishes the so-
called “Chaos Report,” which documents these problems
in terms of IT project success and failure. The Chaos
Report regularly surveys a large set of IT projects and
distinguishes three categories of projects
5 Successful: Projects that have achieved the given target
on time and within budget.
5 Challenged: Projects that have been completed, but with
extensive additional effort and/or budget.
5 Failed: Projects that were cancelled without having
achieved the given target.

12/21/20 3
Process Improvement
n The results of various surveys have been quite stable over
many years: many projects failed or needed more effort and
budget than planned.
n There are many reasons for these problems and a lot of them
can be related to processes.
n Some are related to a lack of indispensable processes such
as sound project management, configuration and change
management, and validation and verification processes.
n These problems impose significant risks and consequently
potential damage to the organization.
n The resulting risks include financial risks, liability risks, the risk
of losing important customers, or the risk of losing the
organization’s good reputation etc.

12/21/20 4
Process Improvement
n Low process quality often leads to bad product
quality, as chaotic ad-hoc realization of software
projects is very defect-prone (quadrant 1) although
achieving good product quality with bad processes
can sometimes be seen in practice (quadrant 2)
n The institutionalization of processes can help to build
up and preserve organizational knowledge with
respect to software development (quadrant 3). Thus,
mature software development processes can be
seen as a prerequisite for high-quality software
products
n In consequence, without good process quality and an
understanding of the effects that processes have in a
specific development environment, success is not
easily repeatable. This is true in particular for
processes that rely solely on some sort of “high-
quality process documentation” without adapting and
improving the processes in the context of the
development environment (quadrant 4)
12/21/20 5
Software Process Improvement (SPI) Approaches
n Currently, mainly two types of SPI approaches are
being used in practice:
5 model-based SPI approaches (also referred to as
problem-oriented approaches or top-down
approaches)
5 continuous SPI approaches (also referred to as
solution oriented approaches or bottom-up
approaches).

12/21/20 6
Software Process Improvement (SPI) Approaches
n Model-based SPI approaches compare
organizational processes with a reference model and
can be used to identify coarse-grained problem
areas and potential improvement options.
n Continuous SPI approaches can be used to develop
company-specific solutions for important problems
and assess the effects of improvement actions.

12/21/20 7
Model-based Improvement
n Model-based SPI approaches such as ISO/IEC 15504 (SPICE) or
CMMI compare an organization’s processes or methods with a
reference model containing proven processes or methods
n Such a reference model often contains only requirements for such
processes or methods that result from experience collected in
successful organizations.
n Typically, the elements of such a reference model are associated
with different levels that are supposed to reflect an organization’s
different capability or maturity levels. Therefore, this type of model
is often called capability or maturity model
n An assessment (sometimes also called appraisal) in model based
approach determines to which degree an organization complies
with the demands of the respective model and is typically
performed by comparing the processes actually used against the
requirements for these processes as stated in the reference model

12/21/20 8
Benefits of Model-based Improvement Approaches
n Creating quality awareness
5 Model-based approaches can be easily used to create and enforce
awareness for quality issues in large organizations because many different
stakeholders (e.g., project managers, developers) are involved in the
improvement actions.
n Measurable goals
5 Improvement goals like “reach maturity level 3” can be easily understood,
independent of technical details, and are thus, easier to communicate to and
by managers. Additionally, from the management point of view, reaching a
specific capability level can be defined as a clear, measurable, and
assessable goal.
n Process areas and important base practices
5 The reference models contain relevant process areas and the maturity levels
prescribe a way to process improvement, which is perceived as being
reasonable. Furthermore, the models feature the early introduction of
important base practices such as project management practices.

12/21/20 9
Benefits of Model-based Improvement Approaches
n Focused improvement actions
5 Model-based approaches support the prioritization and
selection of the most important improvement measures.
Thus, improvement actions with high impact can be
performed first and the resulting benefits can be realized
within a short timeframe.
n Independent assessment
5 Assessments are usually performed by external experts
and thus, an independent assessment and evaluation of
the respective organization is facilitated.

12/21/20 10
ISO/IEC 15504 (SPICE)
n ISO/IEC 15504, often simply called SPICE, is an international
standard for evaluating and improving an organization’s software
processes.
n The international standard consists of five parts.
5 Part 1 (ISO/IEC 15504-1:2004) defines concepts and vocabulary,
5 Part 2 (ISO/IEC 15504-2:2003) the requirements for performing
process assessments, as a basis for process improvement and
capability level determination.
5 Part 3 (ISO/IEC 15504-3:2004) supports the users of the standard
in fulfilling the requirements for an assessment as stated in part 2.
5 Part 4 (ISO/IEC 15504-4:2004) assists with utilizing SPICE-
compliant assessments for process improvement and capability
level determination.
5 Part 5 provides an exemplar Process Assessment Model (PAM),
which fulfills the requirements stated in part 2

12/21/20 11
ISO/IEC 15504 (SPICE)
n A SPICE assessment evaluates a defined subset of the 49
SPICE processes, assigning each evaluated process a
Capability Level (CL), similar to CMMI’s continuous
representation.
n Evaluation is performed using predefined Process Attributes
(PA).
n CL 1 requires only one PA; all higher levels require two. A
SPICE process description states the Process Purpose,
expected Outcomes that should be produced by the process,
and Base Practices and Work Products, indicating the extent
of achievement of the process purpose and outcomes. In
addition to that, typical inputs and outputs are defined.

12/21/20 12
ISO/IEC 15504 (SPICE)
n Like CMMI, SPICE defines six capability levels:
5 CL 0 (Incomplete): No requirements (every organization
automatically achieves CL 0).
5 CL 1 (Performed): The process exists and achieves its specific
goals.
5 – CL 2 (Managed): CL 1 and the process is managed (planned,
monitored, and adjusted), and its work products are
established, controlled, and maintained.
5 CL 3 (Established): CL 2 and the process is implemented using
a defined process that is capable of achieving its process
outcomes.
5 CL 4 (Predictable): CL 3 and the process operates within
defined limits to achieve its outcomes.
5 CL 5 (Optimizing): CL 4 and the process is continuously
improved to meet relevant current and projected business
goals.
12/21/20 13
Continuous Improvement Approaches
n Continuous SPI approaches focus on the important problems of
a software development organization and usually involve
improvement cycles (like Plan-Do- Check-Act (PDCA) or Quality
Improvement Paradigm (QIP)) based on an initial baseline that
defines the respective starting point of each improvement action
n Continuous improvement approaches focus on solving a specific
problem by analyzing the problem, implementing and observing
problem-focused improvement actions, and measuring the
effects of the actions.
n The interpretation of the measurement data is used as input for
further optimization of the solution. In addition, solving one
problem typically reveals further improvement potential in related
areas. Thus, further improvement actions can be defined and an
improvement roadmap can be created “bottom up.”

12/21/20 14
Continuous Improvement Approaches
n PDCA Cycle
5 Continuous improvement approaches
concentrate on problems in an
organization’s development process.
5 Generally speaking, these approaches
consist of iterative improvement cycles
based on the PDCA cycle, which was
made popular by W. Edwards Deming
and is based on the work of Walter A.
Shewhart.
5 PDCA is not explicitly focused on software
development, but has traditionally been
used for industrial production and
business processes. It consists of a four-
step problem-solving process

12/21/20 15
Continuous Improvement Approaches
n Total Quality Management (TQM)
5 TQM is a holistic management approach toward
quality management and, in particular, continuous
improvement within an organization.
5 The TQM approach was mainly developed in the
context of industrial production processes. Although
classic production processes and software
development processes differ, it was possible to
transfer basic TQM concepts to the domain of
software development.

12/21/20 16
Continuous Improvement Approaches
n TQM

12/21/20 17
Goal-Question-Metric Paradigm
n Goals
5 What is the organization trying to achieve?
5 The objective of process improvement is to satisfy
these goals.
n Questions
5 Questions about areas of uncertainty related to the
goals. You need process knowledge to derive these.
n Metrics
5 Measurements to be collected to answer the
questions.

12/21/20 18
GQM Questions
n The GQM paradigm is used in process improvement
to help answer three critical questions:
5 Why are we introducing process improvement?
5 What information do we need to help identify and
assess improvements?
5 What process and product measurements are
required to provide this information?

12/21/20 19
Source
n Software Process Definition and Management,
Münch, J., Armbrust, O., Kowalczyk, M., & Soto, M.,
Springer Science & Business Media, 2012. (Chapter
5)

12/21/20 20