Module 3

Software Quality and

Relaibility
Quality characteristics
• What is software quality assurance?
• What strategies are applicable for software testing?
• What methods are used to design effective test cases?
• Are there realistic methods that will ensure that software is correct?
• How can we manage and control changes that always occur as
software is built?
• What measures and metrics can be used to assess the quality of
requirements and design models, source code, and test cases?
Quality concerns
• By the 1990s, major corporations recognized that billions of dollars each year were
being wasted on software that didn’t deliver the features and functionality that were
promised.
• Both government and industry became increasingly concerned that a major software
fault might cripple important infrastructure, costing tens of billions more
• Bad software difinitions – three of four defects per thoudand lines of code
• Most programmers inject about one error for every 10 lines of code they
write,multiplythatby the millions of lines of code in many commercial products
• Figure it costs software vendors atleast half their development budgets to fix errors
while testing
• Software engineers, managers, all stakeholders—involved in the software process is
responsible for quality.
Quality Concerns
• Quality reduces the amount of rework that it must do. That results in
lowercosts, and more importantly, improved time-to-market.
Steps to achieve quality software
• Proven software engineering process and practice
• Solid project management
• Comprehensive quality control
• Presence of a quality assurance infrastructure.
• Track quality by examining the results of all quality control activities
• Measure quality by examining errors before delivery and defects
released to the field.
Internal and external quality
• Internal quality has to do with the way that the system has been constructed.
• It is a much more granular measurement and considers elements like clean
code, complexity, duplication, component reuse. 
• This quality can be measured through predefined standards, linting tools, unit
tests etc.
• External qualities of a software product:
• Correctness.
• Reliability.
• Robustness.
• Efficiency.
• Usability.
Views on quality
• “quality is a complex and multifaceted concept” that can be described from five different points
of view.
• The transcendental view argues that quality is something that you immediately recognize, but
cannot explicitly define.
• The user view sees quality in terms of an end user’s specific goals. If a product meets those
goals, it exhibits quality.
• The manufacturer’s view defines quality in terms of the original specification of the product. If
the product conforms to the spec, it exhibits quality.
• The product view suggests that quality can be tied to inherent characteristics (e.g., functions
and features) of a product.
• Finally, the value-based view measures quality based on how much a customer is willing to pay
for a product.
• user satisfaction = compliant product + good quality + delivery within budget and schedule
 Garvin’s Quality Dimensions
• Quality should be considered by taking a multidimensional viewpoint
• Performance quality
• Feature quality
• Reliability
• Conformance
• Durability
• Serviceability
• Aesthetics
• (1) factors that can be directly measured (e.g., defects uncovered during testing)
and (2) factors that can be measured only indirectly (e.g., usability or
maintainability
McCall’s Quality Factor (Cont.)
• Quality factors focus on three important aspects of a software product: its operational
characteristics, its ability to undergo change, and its adaptability to new environments.
• Correctness. The extent to which a program satisfies its specification and fulfills the
customer’s mission objectives.
• Reliability. The extent to which a program can be expected to perform its intended function
with required precision. [It should be noted that other, more complete definitions of
reliability have been proposed (see Chapter 25).]
• Efficiency. The amount of computing resources and code required by a program to perform its
function.
• Integrity. Extent to which access to software or data by unauthorized persons can be
controlled.
• Usability. Effort required to learn, operate, prepare input for, and interpret output of a
program.
McCall’s Quality Factor (cont.)
• Maintainability. Effort required to locate and fix an error in a program. [This is
a very limited definition.]
• Flexibility. Effort required to modify an operational program. Testability. Effort
required to test a program to ensure that it performs its intended function.
• Portability. Effort required to transfer the program from one hardware and/or
software system environment to another.
• Reusability. Extent to which a program [or parts of a program] can be reused
in other applications—related to the packaging and scope of the functions
that the program performs.
• Interoperability. Effort required to couple one system to another.
McCall’s Quality Factor
It is difficult, and in some cases impossible, to develop direct measures of these quality
factors.

Many of the metrics defined by McCall et al. can be measured only indirectly.

Assessing the quality of an application using these factors will provide you with a solid
indication of software quality.
Boehms quality model
• The model represents a hierarchical quality model similar to McCall
Quality Model
• McCall’s model primarily focuses on precise measurement of high-
level characteristics, whereas Boehm’s quality model is based on a
wider range of characteristics.
• Ex: Characteristics of hardware performance, that are missing in
McCall’s model.
• Based on the characteristics, the Boehm’s model has three levels for
quality attributes.
• These levels are primary uses (high level characteristics), intermediate constructs (mid-level
characteristics) and primitive constructs (primitive characteristics).
• Highlevel characteristics
• As is utility –
Extent to which, we can use software as-is.
• Maintainability –
Effort required to detect and fix an error during maintenance.
• Portability –
Effort required to change software to fit in a new environment.
• Midlevel characteristics
• Reliability –
Extent to which software performs according to requirements.
• Efficiency –
Amount of hardware resources and code required to execute a function.
• Usability (Human Engineering) –
Extent of effort required to learn, operate and understand functions of the software.
• Testability –
Effort required to verify that software performs its intended functions.
• Understandability –
Effort required for a user to recognize logical concept and its applicability.
• Modifiability –
Effort required to modify a software during maintenance phase.
• Low level characteristics
• device independence, accuracy, completeness, consistency, device efficiency,
accessibility, communicativeness, self-descriptiveness, legibility, structuredness,
conciseness, augment-ability.
• Advantages :
• It focuses and tries to satisfy the needs of the user.
• It focuses on software maintenance cost effectiveness.
• Disadvantages :
• It doesn’t suggest, how to measure the quality characteristics.
• It is difficult to evaluate the quality of software using the top-down
approach.
• Boehm’s model is an improvised version of McCall’s model and it is used
extensively,.
• Because of the top-down approach to see quality of software, Boehm’s
model can’t be employed always.
FURPS
• FURPS is an acronym representing a model for classifying software quality attributes (functional and
non-functional requirements):
• Functionality - Capability (Size & Generality of Feature Set), Reusability (Compatibility,
Interoperability, Portability), Security (Safety & Exploitability)
• Usability (UX) - Human Factors, Aesthetics, Consistency, Documentation, Responsiveness
• Reliability - Availability (Failure Frequency (Robustness/Durability/Resilience), Failure Extent & Time-
Length (Recoverability/Survivability)), Predictability (Stability), Accuracy (Frequency/Severity of
Error)
• Performance - Speed, Efficiency, Resource Consumption (power, ram, cache, etc.), Throughput,
Capacity, Scalability
• Supportability (Serviceability, Maintainability, Sustainability, Repair Speed) - Testability, Flexibility
(Modifiability, Configurability, Adaptability, Extensibility, Modularity), Installability, Localizability
• The model, developed at Hewlett-Packard was first publicly elaborated by Grady and Caswell.
• FURPS+ is now widely used in the software industry. The + was later added to the model after
various campaigns at HP to extend the acronym to emphasize various attributes.
Furps +
• The "+" in FURPS+ also helps us to remember concerns such as:
• Funtional requirements – online help, email service, localization
• Design requirements - relational database
• Implementation requirements – implementation language
• Interface requirements – GUI interface , interaction
• Physical requirements – size of the hardware
ISO 9126 Quality Factors
• The ISO 9126 standard was developed in an attempt to identify the
key quality attributes for computer software.
• The standard identifies six key quality attributes:
• Functionality. The degree to which the software satisfies stated needs
as indicated by the following subattributes: suitability, accuracy,
interoperability, compliance, and security.
• Reliability. The amount of time that the software is available for use as
indicated by the following subattributes: maturity, fault tolerance,
recoverability.
ISO 9126 Quality Factors
• Usability. The degree to which the software is easy to use as indicated by the following
subattributes: understandability, learnability, operability.
• Efficiency. The degree to which the software makes optimal use of system resources
as indicated by the following subattributes: time behavior, resource behavior.
• Maintainability. The ease with which repair may be made to the software as indicated
by the following subattributes: analyzability, changeability, stability, testability.
• Portability. The ease with which the software can be transposed from one
environment to another as indicated by the following subattributes: adaptability,
installability, conformance, replaceability.
• ISO 9126 provides a worthwhile basis for indirect measures and an excellent checklist
for assessing the quality of a system.
Capability Maturity Models (CMM)
• CMM was developed by the Software Engineering Institute (SEI) at Carnegie Mellon
University in 1987. 
• It is not a software process model. It is a framework that is used to analyze the
approach and techniques followed by any organization to develop software products.
• It also provides guidelines to further enhance the maturity of the process used to
develop those software products.
• It is based on profound feedback and development practices adopted by the most
successful organizations worldwide.
• This model describes a strategy for software process improvement that should be
followed by moving through 5 different levels.
• Each level of maturity shows a process capability level. All the levels except level-1 are
further described by Key Process Areas (KPA’s).
CMM (Cont)
• Key Process Areas (KPA’s): 
Each of these KPA’s defines the basic requirements that should be met
by a software process in order to satisfy the KPA and achieve that
level of maturity. 
• Conceptually, key process areas form the basis for management
control of the software project and establish a context in which
technical methods are applied, work products like models,
documents, data, reports, etc. are produced, milestones are
established, quality is ensured and change is properly managed. 
CMM levels

• Level-1: Initial – 
• No KPA’s defined.
• Processes followed are Adhoc and immature and are not well defined.
• Unstable environment for software development.
• No basis for predicting product quality, time for completion, etc.
• Level-2: Repeatable – 
• Focuses on establishing basic project management policies.
• Experience with earlier projects is used for managing new similar natured projects.
• Project Planning- It includes defining resources required, goals, constraints, etc. for the project. It presents
a detailed plan to be followed systematically for the successful completion of good quality software.
• Configuration Management- The focus is on maintaining the performance of the software product,
including all its components, for the entire lifecycle.
• Requirements Management- It includes the management of customer reviews and feedback which result
in some changes in the requirement set. It also consists of accommodation of those modified
requirements.
• Subcontract Management- It focuses on the effective management of qualified software contractors i.e. it
manages the parts of the software which are developed by third parties.
• Software Quality Assurance- It guarantees a good quality software product by following certain rules and
quality standard guidelines while developing.
CMM levels
• Level-3: Defined – 
• At this level, documentation of the standard guidelines and procedures takes place.
• It is a well-defined integrated set of project-specific software engineering and management processes.
• Peer Reviews- In this method, defects are removed by using a number of review methods like walkthroughs,
inspections, buddy checks, etc.
• Intergroup Coordination- It consists of planned interactions between different development teams to ensure
efficient and proper fulfillment of customer needs.
• Organization Process Definition- Its key focus is on the development and maintenance of the standard
development processes.
• Organization Process Focus- It includes activities and practices that should be followed to improve the process
capabilities of an organization.
• Training Programs- It focuses on the enhancement of knowledge and skills of the team members including the
developers and ensuring an increase in work efficiency.
• Level-4: Managed – 
• At this stage, quantitative quality goals are set for the organization for software products as well as software
processes.
• The measurements made help the organization to predict the product and process quality within some limits
defined quantitatively.
• Software Quality Management- It includes the establishment of plans and strategies to develop quantitative
analysis and understanding of the product’s quality.
• Quantitative Management- It focuses on controlling the project performance in a quantitative manner.
CMM Levels
• Level-5: Optimizing – 
• This is the highest level of process maturity in CMM and focuses on continuous
process improvement in the organization using quantitative feedback.
• Use of new tools, techniques, and evaluation of software processes is done to
prevent recurrence of known defects.
• Process Change Management- Its focus is on the continuous improvement of the
organization’s software processes to improve productivity, quality, and cycle time for
the software product.
• Technology Change Management- It consists of the identification and use of new
technologies to improve product quality and decrease product development time.
• Defect Prevention- It focuses on the identification of causes of defects and prevents
them from recurring in future projects by improving project-defined processes.