LECTURE 4: Quality Management

Topics

• Software quality
• Software standards
• Reviews and inspections
• Quality management and agile
development
• Software measurement

2
 Software quality management
• Software quality management is concerned
with ensuring that developed software
systems are “fit for purpose.”
• Systems should meet the needs of their
users, should perform efficiently and
reliably, and should be delivered on time and
within budget.
• Formalized quality management (QM) is
particularly important in teams that are
developing large, long-lifetime systems that
take several years to develop. 3
 Software quality management
• Concerned with ensuring that the required level of
quality is achieved in a software product.
• Three principal concerns:
– At the organizational level, quality management is concerned
with establishing a framework of organizational processes
and standards that will lead to high-quality software.
– At the project level, quality management involves the
application of specific quality processes and checking that
these planned processes have been followed.
– At the project level, quality management is also concerned
with establishing a quality plan for a project. The quality plan
should set out the quality goals for the project and define what
processes and standards are to be used. 4
 Quality management activities
• Quality management provides an independent
check on the software development process.
• The quality management process checks the
project deliverables to ensure that they are
consistent with organizational standards and goals
• The quality team should be independent from the
development team so that they can take an
objective view of the software.
• This allows them to report on software quality
without being influenced by software development
issues. 5
Quality management and software development

6
 Quality planning
• A quality plan sets out the desired product
qualities and how these are assessed and
defines the most significant quality
attributes.
• The quality plan should define the quality
assessment process.
• It should set out which organisational
standards should be applied and, where
necessary, define new standards to be used.
7
 Quality plans
• Quality plan structure (Humphrey 1989)
– Product introduction;
– Product plans;
– Process descriptions;
– Quality goals;
– Risks and risk management.
• Quality plans should be short, brief documents
– If they are too long, no-one will read them.

8
1. Product introduction - A description of the product, its intended
market, and the quality expectations for the product.
2. Product plans - The critical release dates and responsibilities
for the product, along with plans for distribution and product
servicing.
3. Process descriptions - The development and service processes
and standards that should be used for product development and
management.
4. Quality goals - The quality goals and plans for the product,
including an identification and justification of critical product
quality attributes.
5. Risks and risk management - The key risks that might affect
product quality and the actions to be taken to address these
risks.
9
 Scope of quality management
• Quality management is particularly important
for large, complex systems.
• The quality documentation is a record of
progress and supports continuity of
development as the development team
changes.
• For smaller systems, quality management
needs less documentation and should focus on
establishing a quality culture.
• Techniques have to evolve when agile
development is used. 10
Software quality

11
 Software quality
• Quality, simplistically, means that a product
should meet its specification.
• This is problematical for software systems
– There is a tension between customer quality requirements
(efficiency, reliability, etc.) and developer quality
requirements (maintainability, reusability, etc.);
– Some quality requirements are difficult to specify in an
unambiguous way;
– Software specifications are usually incomplete and often
inconsistent.
• The focus may be ‘fitness for purpose’ rather
than specification conformance. 12
 Software fitness for purpose
• Has the software been properly tested?
• Is the software sufficiently dependable to be put
into use?
• Is the performance of the software acceptable
for normal use?
• Is the software usable?
• Is the software well-structured and
understandable?
• Have programming and documentation standards
been followed in the development process?
13
 Non-functional characteristics
• The subjective quality of a software system is
largely based on its non-functional
characteristics.
• This reflects practical user experience – if the
software’s functionality is not what is expected,
then users will often just work around this and
find other ways to do what they want to do.
• However, if the software is unreliable or too slow,
then it is practically impossible for them to
achieve their goals.
14
 Software quality attributes
Safety Understandability Portability
Security Testability Usability
Reliability Adaptability Reusability
Resilience Modularity Efficiency
Robustness Complexity Learnability

• It is not possible for any system to be

optimized for all of these attributes.
• For example, improving security may lead to
loss of performance.
15
 Quality conflicts
• It is not possible for any system to be
optimized for all of these attributes
– For example, improving robustness may lead to loss of
performance.
• The quality plan should therefore define the
most important quality attributes for the
software that is being developed.
• The plan should also include a definition of the
quality assessment process, an agreed way of
assessing whether some quality, such as
maintainability or robustness, is present in the
product.
16
 Process and product quality
• The quality of a developed product is influenced by the
quality of the production process.
• This is important in software development as some
product quality attributes are hard to assess.
• However, there is a very complex and poorly
understood relationship between software processes
and product quality.
– The application of individual skills and experience is particularly
important in software development;
– External factors such as the novelty of an application or the need
for an accelerated development schedule may impair product
quality.

17
Process-based quality

18
 Quality culture
• Quality managers should aim to develop a
‘quality culture’ where everyone responsible for
software development is committed to
achieving a high level of product quality.
• They should encourage teams to take
responsibility for the quality of their work and
to develop new approaches to quality
improvement.
• They should support people who are interested
in the intangible aspects of quality and
encourage professional behavior in all team
members.
19
Software standards

20
 Software standards
• Standards define the required
attributes of a product or process.
• They play an important role in quality
management.
• Standards may be international, national,
organizational or project standards.

21
 Importance of standards

• Encapsulation of best practice avoids

repetition of past mistakes.
• They are a framework for defining what
quality means in a particular setting i.e. that
organization’s view of quality.
• They provide continuity - new staff can
understand the organisation by
understanding the standards that are used.

22
 Product and process standards
• Product standards
– Apply to the software product being developed. They
include document standards, such as the structure of
requirements documents, documentation standards, such
as a standard comment header for an object class
definition, and coding standards, which define how a
programming language should be used.
• Process standards
– These define the processes that should be followed during
software development. Process standards may include
definitions of specification, design and validation
processes, process support tools and a description of the
documents that should be written during these processes.23
 Product and process standards

• Examples of product and process standards

Product standards Process standards
Design review form Design review conduct
Requirements document Submission of new code for
structure system building
Method header format Version release process
Java programming style Project plan approval process
Project plan format Change control process
Change request form Test recording process

24
 Problems with standards
• They may not be seen as relevant and up-to-date
by software engineers.
• They often involve too much bureaucratic form
filling.
• If they are unsupported by software tools,
tedious form filling work is often involved to
maintain the documentation associated with the
standards.

25
 Standards development
• Involve practitioners in development. Engineers
should understand the rationale underlying a
standard.
• Review standards and their usage regularly.
Standards can quickly become outdated and
this reduces their credibility amongst
practitioners.
• Detailed standards should have specialized tool
support. Excessive clerical work is the most
significant complaint against standards.
– Web-based forms are not good enough. 26
 ISO 9001 standards framework

• An international set of standards that can be used

as a basis for developing quality management
systems.
• ISO 9001, the most general of these standards,
applies to organizations that design, develop and
maintain products, including software.
• The ISO 9001 standard is a framework for
developing software standards.
– It sets out general quality principles, describes quality
processes in general and lays out the organizational
standards and procedures that should be defined. These
27
should be documented in an organizational quality manual.
ISO 9001 core processes

28
ISO 9001 and quality management

29
 ISO 9001 certification
• Quality standards and procedures should
be documented in an organisational quality
manual.
• An external body may certify that an
organisation’s quality manual conforms to
ISO 9000 standards.
• Some customers require suppliers to be
ISO 9000 certified although the need
for flexibility here is increasingly
recognised. 30
 Software quality and ISO9001
• The ISO 9001 certification is inadequate because it
defines quality to be the conformance to standards.
• It takes no account of quality as experienced by
users of the software. For example, a company could
define test coverage standards specifying that all
methods in objects must be called at least once.
• Unfortunately, this standard can be met by
incomplete software testing that does not include
tests with different method parameters. So long as
the defined testing procedures are followed and
test records maintained, the company could be ISO
9001 certified. 31
Reviews and inspections

32
 Reviews and inspections
• A group examines part or all of a process or
system and its documentation to find potential
problems.
• Software or documents may be 'signed off' at a
review which signifies that progress to the next
development stage has been approved by
management.
• There are different types of review with
different objectives
– Inspections for defect removal (product);
– Reviews for progress assessment (product and process);
33
– Quality reviews (product and standards).
 Quality reviews
• A group of people carefully examine part or all
of a software system and its associated
documentation.
• Code, designs, specifications, test plans,
standards, etc. can all be reviewed.
• Software or documents may be 'signed off' at
a
review which signifies that progress to the
next
development stage has been approved by
management. 34
 Phases in the review process
• Pre-review activities
– Pre-review activities are concerned with review
planning and review preparation
• The review meeting
– During the review meeting, an author of the
document or program being reviewed should ‘walk
through’ the document with the review team.
• Post-review activities
– These address the problems and issues that have
been raised during the review meeting. 35
The software review process

36
 Distributed reviews
• The processes suggested for reviews assume that
the review team has a face-to-face meeting to
discuss the software or documents that they are
reviewing.
• However, project teams are now often
distributed, sometimes across countries or
continents, so it is impractical for team members
to meet face to face.
• Remote reviewing can be supported using shared
documents where each review team member can
annotate the document with their comments.
37
 Program inspections
• These are peer reviews where engineers examine
the source of a system with the aim of
discovering anomalies and defects.
• Inspections do not require execution of a system
so may be used before implementation.
• They may be applied to any representation of the
system (requirements, design, configuration data,
test data, etc.).
• They have been shown to be an effective
technique for discovering program errors.
38
 Inspection checklists
• Checklist of common errors should be used to
drive the inspection.
• Error checklists are programming language
dependent and reflect the characteristic
errors that are likely to arise in the language.
• In general, the 'weaker' the type checking, the
larger the checklist.
• Examples: Initialisation, Constant naming, loop
termination, array bounds, etc.

39
 An inspection checklist (a)
Fault class Inspection check
Data faults  Are all program variables initialized before their values are used?
 Have all constants been named?
 Should the upper bound of arrays be equal to the size of the array
or Size -1?
 If character strings are used, is a delimiter explicitly assigned?
 Is there any possibility of buffer overflow?

Control faults  For each conditional statement, is the condition correct?

 Is each loop certain to terminate?
 Are compound statements correctly bracketed?
 In case statements, are all possible cases accounted for?
 If a break is required after each case in case statements, has it
been included?

Input/output faults  Are all input variables used?

 Are all output variables assigned a value before they are output?
 Can unexpected inputs cause corruption?

40
 An inspection checklist (b)
Fault class Inspection check
Interface faults  Do all function and method calls have the correct number of
parameters?
 Do formal and actual parameter types match?
 Are the parameters in the right order?
 If components access shared memory, do they have the
same model of the shared memory structure?

Storage management faults  If a linked structure is modified, have all links been correctly
reassigned?
 If dynamic storage is used, has space been allocated
correctly?
 Is space explicitly deallocated after it is no longer required?

Exception management  Have all possible error conditions been taken into account?
faults

41
Quality management and agile development

42
 Quality management and agile development

• Quality management in agile development is

informal rather than document-based.
• It relies on establishing a quality culture, where
all team members feel responsible for software
quality and take actions to ensure that quality is
maintained.
• The agile community is fundamentally opposed to
what it sees as the bureaucratic overheads of
standards-based approaches and quality processes
as embodied in ISO 9001.
43
 Shared good practice
• Check before check-in
– Programmers are responsible for organizing their own
code reviews with other team members before the code is
checked in to the build system.
• Never break the build
– Team members should not check in code that causes the
system to fail. Developers have to test their code changes
against the whole system and be confident that these work
as expected.
• Fix problems when you see them
– If a programmer discovers problems or obscurities in code
developed by someone else, they can fix these directly 44
rather than referring them back to the original developer.
 Reviews and agile methods
• The review process in agile software
development is usually informal.
• In Scrum,, there is a review meeting after
each iteration of the software has been
completed (a sprint review), where quality
issues and problems may be discussed.
• In Extreme Programming, pair programming
ensures that code is constantly being
examined and reviewed by another team
member. 45
 Pair programming
• This is an approach where 2 people are responsible for
code development and work together to achieve this.
• Code developed by an individual is therefore
constantly being examined and reviewed by another
team member.
• Pair programming leads to a deep knowledge of a
program, as both programmers have to understand the
program in detail to continue development.
• This depth of knowledge is difficult to achieve in
inspection processes and pair programming can find
bugs that would not be discovered in formal
inspections. 46
 Pair programming weaknesses
• Mutual misunderstandings
– Both members of a pair may make the same mistake in
understanding the system requirements. Discussions
may reinforce these errors.
• Pair reputation
– Pairs may be reluctant to look for errors because they
do not want to slow down the progress of the project.
• Working relationships
– The pair’s ability to discover defects is likely to be
compromised by their close working relationship that
often leads to reluctance to criticize work partners.
47
 Agile QM and large systems
• When a large system is being developed for an
external customer, agile approaches to quality
management with minimal documentation may be
impractical.
– If the customer is a large company, it may have its own quality
management processes and may expect the software
development company to report on progress in a way that is
compatible with them.
– Where there are several geographically distributed teams involved
in development, perhaps from different companies, then informal
communications may be impractical.
– For long-lifetime systems, the team involved in development will
changeWithout documentation, new team members may find it
impossible to understand development.
48
 Key points

• Software quality management is concerned with

ensuring that software has a low number of defects
and that it reaches the required standards of
maintainability, reliability, portability etc. Software
standards are important for quality assurance as they
represent an identification of ‘best practice’. When
developing software, standards provide a solid
foundation for building good quality software.
• Reviews of the software process deliverables involve a
team of people who check that quality standards are
being followed. Reviews are the most widely used
technique for assessing quality.
74
 Key points

• In a program inspection or peer review, a small

team systematically checks the code. They read
the code in detail and look for possible errors and
omissions. The problems detected are discussed
at a code review meeting.
• Agile quality management relies on establishing a
quality culture where the development team works
together to improve software quality.

75
77