T Y

LI
UA MS-15
Q
R E G
A RIN Dr Muhammad
TW E Abbas
F I NE
SO NG E 2
E TURC
LE

CHAPTER-2
WHAT IS SOFTWARE QUALITY?
QUALITY
 OVERVIEW
Quality: Perspectives and Expectations
Quality Frameworks and ISO-9126
Correctness and Defects: Definitions,
Properties, and Measurements
A Historical Perspective of Quality
What Is Software Quality?

3
 OVERVIEW
What is software quality?, could be many different
answers
What are the characteristics for high-quality
software? An alternative question
We attempt to define software quality by defining
the expected characteristics of high-quality software
For this we need to:
examine different perspectives & expectations of users & other
people
examine the individual characteristics associated with quality
and their inter-relationships
focus our attention on the critical characteristics of functional
correctness
Examine the individual characteristics associated with quality (e.g.
reliability, functionality, efficiency) and their inter-relationships for
example how functionality is related to reliability or how reliability is
related to efficieny
4
SO WHAT IS QUALITY?
Software quality
may include many different attributes and
may be defined and perceived differently based on peoples
different roles and responsibilities
We adopt the correctness-centered view of
quality, that is,
high quality means none or few problems
of limited damage to customers

5
 CONT
Quality is defined as the products or services capability to
meet customer expectations.
Quality that is defined as a matter of products and services
whose measurable characteristics satisfy a fixed
specification.
Quality is defined as conformance to requirements.
Quality consists of those product features which meet the
need of customers and there by provide product
satisfaction.
Quality is multidimensional. some aspects of quality can be
measured. Example: Maximum speed, fuel, economy etc.
 CONT

Software was particularly problematical for the

following reasons:
Software has no physical existence.
The lack of knowledge of client needs at the start.
The change of client needs over time.
The rapid rate of change on both hardware and
software.
The high expectations of customers, particularly with
respect to adaptability.
Within the software quality area, the need to provide
a solution that matches user needs is often
considered as design quality, whilst ensuring a
match to the specification is considered as
manufacturing quality.
SOFTWARE QUALITY
Kitchen ham (1989 b) refers to software quality
fitness for needs and claims quality
involves matching expectations.
Two features of a piece of quality software:
Conformance to its specification
Fitness for its intended purpose.
The Department of Defense (DOD, 1985) in the
USA defines software quality as the degree
to which the attributes of the software enable
it to perform its intended end use.
 DIFFERENT VIEWS OF QUALITY
Examine the different views of quality based on the different roles
Researchers have divided it in to five major views

Transcendental view: some intangible properties that

cant be seen but delight the users (quality is hard
to define or describe in abstract terms) ( Transcendental
view e.g. efficient algorithm can do things with speed, cant be seen by user
but can delight him Product view: e.g. controlling faults per MLOC)

User view: fitness for purpose, meeting users needs

Manufacturing view: conformance to process standards
Product view: focus is on the inherent characteristics in the
product itself in the hope that controlling these internal
metrics will improve external behavior of the product
Value-based view: quality is the customers willingness to pay

9
 EXTERNAL CHARACTERISTICS
Correctness- Degree to which system is free from
faults in specification, design and implementation.
Usability- The Ease with which users can learn and
use the system.
Efficiency- Minimal use of system resource
including memory and execution time.
Reliability- The ability of a system to perform
whenever required without/with few failures.
Integrity- Prevention of unauthorized or improper
use.
Adaptability-Usability in other application than the
original one.
Accuracy- Degree of quantitative correctness.
Robustness- Functioning of system in presence of
invalid inputs, stress
environment.
 INTERNAL CHARACTERISTICS
Maintainability: Ease of modifying
software for changing/adding capabilities,
improving performance.
Flexibility: Extend of modifying system for
other uses/environments.
Portability: Ease of modifying system for
operating in different environment.
Reusability: Extend of using parts in other
systems.
CONT
External:
What a systems user is interested
in; typically properties of any single
particular system.
Internal:
What programmers/management
are interested in; properties of the
development of a collection of systems.
 TECHNIQUES FOR IMPROVING SQ
Explicit software quality objectives.
Explicit quality assurance activities.
Testing strategy.
Software Engineering guidelines.
Informal technical reviews.
Formal technical reviews.
External audits.
Development process.
CONT

Change control procedures.

Measurement of results.
Prototyping.
Mathematical proof.
Modular programming
techniques.
 VIEWS OF QUALITY
Quality is a multidimensional construct.
It may therefore be considered using a
polyhedron metaphor. Within this
metaphor, a three-dimensional solid
represents quality. Each face represents
a different aspect of quality such as
correctness, reliability, and efficiency.
It has been classified according to a
number of views or perspective. These
views are often diverse and may conflict
with each other. Each view comes from a
particular context.
THE SOFTWARE PROJECT ROLES
Project manager
Business analyst
Implementation programmer
Quality auditor
End user
Line manager
Project sponsor
 PEOPLES ROLES & RESPONSIBILITIES
Different people would have different
expectations based on their roles and
responsibilities
we can divide the people into two broad
groups:
Consumers of software products or services
customers, users
can be extended to non-human or invisible users
Producers of software products
anyone involved in managing, developing,
marketing and service of software products
can be extended to third party participants for
add on or testing.

1
7
 EXTERNAL/INTERNAL VIEW
External view Consumers perspective
who are more concerned with the external behavior
Internal view Producers perspective
because they are typically familiar with or at least aware
of various internal characteristic of the product
In other words:
External view mostly sees a software system as a black
box
Because one can observe its behavior but not see
through inside
Internal view mostly sees it as a white box
Because one can see what is inside and how it works

1
8
 PEOPLES ROLES AND
RESPONSIBILITIES
Consumers of software products or services,
including customers and users, either internally
or externally., who are responsible for the
acquisition of software products or services, and
who use the software products or services. (Dual
roles)
We can also extend the concept of users to include
such non-human or invisible users as other
software, embedded hardware, and the overall
operational environment that the software
operates under and interacts with (Whittaker,
2001).
Producers of software products, or anyone
involved with the development, management,
maintenance, marketing, and service of software
products.
 PEOPLES ROLES AND
ItRESPONSIBILITIES
can also include third-party
participants who may be involved in
add-on products and services, software
packaging, software certification,
fulfilling independent verification and
validation (IV&V) responsibilities etc.
External view mostly sees a software
system as a black box, where one can
observe its behaviour but not see
through inside; while the internal view
mostly sees it as a white box, or more
appropriately a clear box, where one
can see what is inside and how it works.
 CONSUMER EXPECTATIONS
FOR DIFFERENT SOFTWARE
General: functionality and reliability
Usability: GUI based products, plug-and-play
Interoperability: embedded systems (smooth
operation and interaction between the
software and these non-human users)
Safety: safety-critical systems
 QUALITY EXPECTATIONS ON THE
CONSUMER SIDE 1/2
a
Basic quality expectation of a user -
software system performs useful
functions as it is specified
Two basic elements to this expectation:
It performs the right functions as specified -
fits the users needs
It performs these specified functions
correctly - performs its functions reliably
Related concepts to these two basic elements:
Validation
Verification
Other expectations:
Usability, ease of installation inter-
operability and adaptability etc 2
3
 QUALITY EXPECTATIONS ON THE
CONSUMER SIDE 2/2
Difference in priority on expectations:
For many users especially novice users:
Usability, ease of installation
For non-human users: inter-operability and
adaptability, so that the software can work
well with others and within its surrounding
environment
Differing quality expectations of a
customer & users:
Basic expectations of customers are similar
to that of a user, with the additional
concerns what are those concerns?
2
4
QUALITY EXPECTATIONS ON THE
PRODUCER SIDE
For software producers
the most fundamental quality question is to fulfill their
contractual obligations
For product and service managers:
adherence to pre-selected software process and
relevant standards,
proper choice of software methodologies,
languages, and good design,
integrity of product components and reduce coupling
Tools, may be closely related to quality
For other people on the producer side:
their different concerns may also produce different
quality views and expectations, for example:
usability and modifiability may be paramount for people
involved with software service
maintainability for maintenance personnel
portability for third-party participants,
profitability and customer value for product marketing 2
5
 QUALITY EXPECTATIONS ON THE
PRODUCER SIDE
Contractual Obligations: conform to product
specifications or providing services that conform
to service agreement
Software methodologies, languages, and
tools: For product and service managers,
adherence to pre-selected software process and
relevant standards, proper choice of software
methodologies, languages, and tools, as well as
other factors, related to quality.
They are also interested in managing and satisfying
users quality expectations, by translating such
quality expectations into realistic quality goals
that can be defined and managed internally,
selecting appropriate and effective QA strategies
QUALITY EXPECTATIONS ON THE
PRODUCER SIDE
Producer quality views: people on the producer
side, their different concerns may also produce quality
views and expectations different
For example, usability and
modifiability may be paramount for
people involved with software service,
maintainability for maintenance
personnel, portability for third-party or
software packaging service providers,
and profitability and customer value
for product marketing
 QUALITY FRAMEWORKS
Various models or frameworks have been
proposed to accommodate these different
quality views and expectations,
define quality and related attributes,
features, characteristics, and
measurements.
ISO-9 126 (ISO, 2001), the mostly
influential one in the software engineering
community today, and discuss various
adaptations of such quality frameworks for
specific application environments.
Causal relationship from intangible quality
views to tangible software measures
28
QUALITY STANDARDS AND
FRAMEWORKS
Two approaches to software that can be
followed to ensure software quality:
Process based: assurance of the process by which a product is
developed (ISO 9001, ISO 9000-3 provides guidelines for the
application of the ISO 9001)
Product based: the evaluation of the quality of the end
product (ISO 9126).
Both approaches are important and both require
the presence of a system for managing quality.
 QUALITY FRAMEWORKS
Two main approaches:
Standard Models:
ISO 9126
McCall
Application or company specific quality
models (adaptations of standard models for
specific application environments)
FURPS
GQM Approach

30
 ISO-9126 QUALITY
FRAMEWORK
ISO 9126 is the software product evaluation
standard from International Organization for
Standardization (ISO)
ISO-9126 is the most influential one in the
Software Engineering community
ISO-9126 provides a hierarchical framework
for quality definition, organized into quality
characteristics and sub-characteristic
Characteristics of quality software product
(with minimal overlap).
There are six top-level quality characteristics,
with each associated with its own exclusive
(non-overlapping) sub-characteristics
 ISO-9126 QUALITY

FRAMEWORK
Functionality: A set of attributes that bear on the
existence of a set of functions and their specified
properties. The functions are those that satisfy
stated or implied needs. The sub-characteristics
include:
Suitability- to the users needs
Accuracy- of results
Interoperability- with other systems
Security- against unintended access
Reliability: A set of attributes that bear on the
capability of software to maintain its level of
performance under stated conditions for a stated
period of time. The sub-characteristics include:
Maturity- frequency of failures
Fault Tolerance- performance in case of faults
Recoverability- of functionality and data loss
 ISO-9126 QUALITY
FRAMEWORK
Usability: A set of attributes that bear on the
effort needed for use, and on the individual
assessment of such use, by a stated or implied
set of users. The sub characteristics include:
- Understandability
- Learnability
- Operability
Efficiency: A set of attributes that bear on the
relationship between the level of performance of
the software and the amount of resources used,
under stated conditions. The sub-characteristics
include:
- Time behavior
- Resource behavior
 ISO-9126 QUALITY
FRAMEWORK
Maintainability: A set of attributes that bear on
the effort needed to make specified
modifications. The sub-characteristics include:
- Analyzability- effort for diagnosis
Changeability- ease of modification
Stability- after change
Testability- effort required for testing after
change
Portability: A set of attributes that bear on the
ability of software to be transferred from one
environment to another. The sub-characteristics
include:
- Adaptability
- Installability
- Conformance
- Replaceability
ISO-9126: HOW IS IT USED?

definition of the quality

characteristics
associated quality evaluation
process
intended to be exhaustive.
ISO-9126 QUALITY
CHARACTERISTICS
ISO-9126: BUSINESS BENEFITS

For the purchaser:

understand clearly his/her
requirements for the product to be
developed, and
Communicate the same
For the supplier:
understand the requirement, and
assess with confidence whether it is
possible to provide the product with the
right level of software quality.
Eliminate any misunderstanding
between purchaser and supplier.
ISO-9126: GUIDELINES FOR USING
THE QUALITY CHARACTERISTICS
An evaluation process model with 3 stages:-
Quality requirements definition
takes as input a set of stated or implied needs,
relevant technical documentation and the ISO
Standard itself
produces a quality requirement specification.
Evaluation preparation
involves the selection of appropriate metrics, a
rating level definition and the definition of
assessment criteria.
Metrics typically give rise to quantifiable measures
mapped on to scales.
ISO-9126: GUIDELINES FOR USING
THE QUALITY CHARACTERISTICS
The rating levels definition determines what
ranges of values on those scales count as
satisfactory or unsatisfactory.
the assessment criteria definition involves
preparing a procedure for summarizing the results
of the evaluation and takes, as input, management
criteria specific to a particular environment.
Evaluation procedure
refined into three steps: measurement, rating and
assessment.
ISO-9126: GUIDELINES FOR USING
THE QUALITY CHARACTERISTICS
in measurement, the selected metrics are applied to
the software product and values on the scales of the
metrics obtained.
Subsequently, for each measured value, the rating level
is determined.
Assessment is the final step of the software evaluation
process, where a set of rated levels are summarized.
The result is a summary of the quality of the software
product.
The summarized quality is then compared with other
aspects such as time and cost, and the final managerial
decision is taken, based on managerial criteria.
ADAPTATIONS OF ISO-9126

Adapted to application domains

For the web-based applications a set of quality
attributes has been identified:
Reliability, usability, security as primary attributes
Availability, scalability, maintainability, time to market
as secondary attributes
Customized for companies
Many companies and communities have adapted
and customized existing quality frameworks to
define quality for themselves, taking into consideration
their specific business and market environment
E.g. IBM used CUPRIMDS (capability, usability,
performance, reliability, installation,
maintenance, documentation, service) for their
own software products
FURPS model by HP
GQM (Goal, Questions, Metric) Model 41
FURPS BY HP
Result of a statistical project survey at Hewlett Packard
1987 to improve its products.
Factors:
Functionality: functions it performs, their generality and security
Usability: aesthetics, consistency, documentation
Reliability: frequency and severity of failure, accuracy of output
Performance: response time, resource consumption
Supportability: can it be extended, adapted, corrected?
FURPS is originally a company specific quality model

42
GQM BY SOFTWARE ENGINEERING LAB AT
THE NASA
A measurement program can be more
successful if designed with the goals in
mind
GQM approach provides a framework
with 3 steps:
List the major goals of the development /
maintenance project
Derive from each goal the questions that
must be answered to determine if the goals
are being met
Decide what must be measured to answer
the questions adequately
43
 GQM BY SOFTWARE ENGINEERING LAB AT
THE NASA
Example:

GOAL Evaluate effectiveness of coding standard

Who is What is What is

QUESTIONS using coder code
Standard? productivity? quality?

Proportion of Experience of Code Effort Errors

Coders Coders size
METRICS -using standard -with standard
-using language -with language
-with environment 44
etc
GQM BY SOFTWARE ENGINEERING LAB AT
THE NASA
Benefits :
Generates only those measures relevant
to the goal
Several measurements may be needed to
answer a single question.
A single measurement may apply to more
than one question
The goal provides the purpose for
collecting the data
Disadvantages:
Additional efforts to derive the goals and
metrics
Error prone compared to standard
models 45
FOCUS ON CORRECTNESS 1/2

Among the software quality attributes,

some deal directly with the functional
correctness
Demonstrated by presence/absence of
problems
Other attributes deal with usability
portability etc
Correctness is typically related to
several quality characteristics or sub-
characteristics in quality frameworks
described before
For example, in ISO-9126 it is related to
both functionality and reliability
46
FOCUS ON CORRECTNESS 2/2
Correctness is typically the most important
aspect of quality for situations where daily
life or business depends on the software
Even in other cases, correctness remains a
fundamental part of the users expectations
Therefore, we adopt the correctness-
centered view of quality throughout this
course

To a user or a customer, the primary

concern is
that the software operates without failure, or with
as few failures as possible
When such failures or undesirable events do occur,
the impact should be as little as possible
47
CORRECTNESS & DEFECTS: DEFINITIONS

Failure: external behavior

Deviation from expected/specified behavior
The inability of a system or component to perform its required functions
within specified performance requirements.

Fault: internal characteristics - cause for failures

An incorrect step, process, or data definition in a computer
program
Error: refers to a missing or incorrect human
action such as human misconceptions,
misunderstandings, etc resulting in certain
fault(s) being injected into a software during
design, coding and data entry
Data entry error example: writing someones DOB a
century back. How to prevent it?
48
CORRECTNESS & DEFECTS: DEFINITIONS

Defect:
Generally refers to some problem, either
with external behavior or with internal
characteristics
error/fault/failure are collectively referred to
as defects
Bug/debug:
not good terms, avoid
People have raised moral or philosophical
objection to the use of bugs as evading
responsibility for something people
committed
Instead use defect detection & removal 49
EXAMPLE: ERROR, FAULT, FAILURE

Example: Software for finding

factorial
Error: factorial is the sum 1 to N
(instead of product!)
Fault: use + operator instead of *
Failure: wrong value produced

50
ERROR, FAULT, FAILURE: RELATION

Defect related concepts and relations

51
ERROR, FAULT, FAILURE: RELATION

A causal relation exists among the three

aspects of defects:
errors faults failures
However, this relationship is not
necessarily 1-to-1:
a single error may cause many faults
a single fault may cause many failures in repeated
executions
conversely, the same failure may be caused by
several faults
the same fault may be there due to different errors

52
CORRECTNESS-CENTERED PROPERTIES AND
MEASUREMENTS - EXTERNAL
With Correctness-centered approach, the
consumers concern is that the no failure, or few
failures with min impact
This concern can be captured by:
Failure Properties and Direct Failure Measurement
- Failure Properties include:
Information about specific failures, what they are,
how they occur etc.
Failure count, distribution, density etc.
Failure Likelihood and Reliability Measurement:
How often or how likely
Failure Severity Measurement and Safety
Assurance: Failure impact
All these will be discussed in detail in later
chapters 53
CORRECTNESS-CENTERED PROPERTIES AND
MEASUREMENTS - INTERNAL
Fault Properties and Related
Measurement
Individual faults: types, relation to specific
failures, their causes, time, and injection
circumstances
Collective faults: distribution and density
over development phases and different
software components

54
CORRECTNESS-CENTERED PROPERTIES AND
MEASUREMENTS

55
 THE FURPS MODEL (1987)
Hewlett-Packard developed a set of
software quality factors that make up its
name FURPS.
The FURPS model takes five characteristics
of quality attributes:

Functionality, Usability, Reliability,

Performance , and Supportability.
 DEFECT HANDLING IN QUALITY ENGINEERING

For most software organizations, ensuring

quality means dealing with defects
Three generic ways to deal with defects:
Defect prevention
Defect detection and removal (defect reduction)
Defect containment
Defect measurements are used to provide
feedback to software development process
Quality engineering can be viewed as defect
management

will be discussed in detail in later

chapters 57
 DEFECT HANDLING IN QUALITY ENGINEERING
Various QA alternatives and related techniques
can be used in a concerted effort to effectively and
efficiently deal with defects and assure software
quality.
In the process of dealing with defects, various
direct defect measurements and other indirect
quality measurements (used as quality indicators)
might be taken, often forming a multi-dimensional
measurement space referred to as quality profile
Execution of the planned QA activities, quality
engineering also includes:
quality planning before specific QA activities are
carried out,
measurement, analysis, and feedback to monitor and
control the QA activities. 58
 THE FURPS MODEL

When FURPS is used, two steps are

considered: setting priorities and
defining quality attributes that can
be measured.
One disadvantage of this model is
that it does not take into account
the software products portability.
 THE BOEHM MODEL (1978)
It represents a hierarchical
structure of characteristics.
Boehm model sees the view of
software with general utility.
Utility from various dimensions.
General utility is broken down
into portability, utility and
maintainability.
 THE BOEHM MODEL
Utility is further broken down into
reliability, efficiency and human
engineering.

Maintainability is in turn broken

down into testability,
understandability and modifiability.

This model is presented in levels

called primary uses, intermediate
construct and primitive constructs.
 THE BOEHM MODEL
Perspectives :-
Product revision (ability to change).
Product transition (adaptability to new
environments).
Product operations (basic operational
characteristics).
Product revision:-
Maintainability, the ability to find and fix a defect.
Flexibility, the ability to make changes required as
dictated by the business.
Testability, the ability to Validate the software
requirements.
 THE BOEHM MODEL
Product transition :- The product transition
perspective identifies quality factors that
influence the ability to adapt the
software to new environments:-
Portability, the ability to transfer the software
from one environment to another.
Reusability, the ease of using existing
software components in a different context.
Interoperability, the extent, or ease, to which
software components work together.
 THE BOEHM MODEL
Product operations :- The product
operations perspective identifies quality
factors that influence the extent to which
the software fulfils its specification:-
Correctness, the functionality matches the
specification.
Reliability, the extent to which the system fails.
Efficiency, system resource (including cpu, disk,
memory, network) usage.
Integrity, protection from unauthorized access.
Usability, ease of use.
THE BOEHM MODEL
THE BOEHM MODEL
 A HISTORICAL PERSPECTIVE OF
QUALITY
. Evolving perceptions of quality
Product specifications are specified in terms of ranges and
error tolerance, reducing variance in manufacturing has been
the focal point of statistical quality control.
Problems are due to non-conformance to specifications or
observed defects defined by the non-conformance
The average number of reported problems per 100 vehicle
by owners during the first three years (they used to count
only the first year) of their ownership based on actual survey
results.
Another commonly used quality measure for automobiles,
reliability, is measured by the number of problems over a
longer time for different stages of an automobiles lifetime
Customer loyalty due to their overall experience with the
service
software industry has incorporated both the conformance and service views
of quality, and high-quality software can be defined by three basic elements:
67
conformance, adaptability, and innovation.
 QUALITY IN SOFTWARE ENGINEERING
Within software engineering, quality has been one of the
several important factors, including cost, schedule, and
functionality, which have been studied by researchers
and practitioners (Blum, 1992; Humphrey, 1989; Ghezzi et
al., 2003; von Mayrhauser, 1990).
In the functional stage, the focus was on providing the
automated functions to replace
In the schedule stage, the focus was on introducing
important features and new sys-
In the cost stage, the focus was on reducing the price
to stay competitive accompanied by the widespread use
of personal computers.
In the reliability stage, the focus was managing users
quality expectations under the increased dependency
on software and high cost or severe damages
associated with software failures.
68
SO WHAT IS QUALITY?
Software quality
may include many different attributes and
may be defined and perceived differently
based on peoples different roles and
responsibilities

We adopt the correctness-centered

view of quality, that is,
high quality means none or few problems
of limited damage to customers
69
CONT
QUESTIONS

71