UNIT 2

Software Quality Assurance:

It’s not enough to talk the talk by saying that software quality is important. You have to (1)
explicitly define what is meant when you say “software quality,” (2) create a set of activities
that will help ensure that every software engineering work product exhibits high quality, (3)
perform quality control and assurance activities on every software project, (4) use metrics to
develop strategies for improving your software process and, as a consequence, the quality of
the end product.
Everyone involved in the software engineering process is responsible for quality.
Some software developers continue to believe that software quality is something you begin to
worry about after code has been generated. Nothing could be further from the truth! Software
quality assurance (often called quality management) is an umbrella activity that is applied
throughout the software process.
Software quality assurance (SQA) encompasses: (1) an SQA process, (2) specific quality
assurance and quality control tasks (including technical reviews and a multi-tiered testing
strategy), (3) effective software engineering practice (methods and tools), (4) control of all
software work products and the changes made to them, (5) a procedure to ensure compliance
with software development standards (when applicable), and (6) measurement and reporting
mechanisms.

BACKGROUND ISSUES:

Quality control and assurance are essential activities for any business that produces products
to be used by others. Prior to the twentieth century, quality control was the sole responsibility
of the craftsperson who built a product. As time passed and mass production techniques
became commonplace, quality control became an activity performed by people other than the
ones who built the product.
The first formal quality assurance and control function was introduced at Bell Labs in 1916
and spread rapidly throughout the manufacturing world. During the 1940s, more formal
approaches to quality control were suggested. These relied on measurement and continuous
process improvement as key elements of quality management.
The history of quality assurance in software development parallels the history of quality in
hardware manufacturing. During the early days of computing(1950s and 1960s), quality was
the sole responsibility of the programmer. Standards for quality assurance for software were
introduced in military contract software development during the 1970s and have spread
rapidly into software development in the commercial world. Extending the definition
presented earlier, software quality assurance is a “planned and systematic pattern of actions”
that are required to ensure high quality in software. The scope of quality assurance
responsibility might best be characterized by paraphrasing a once-popular automobile
commercial: “Quality Is Job #1.” The implication for software is that many different
constituencies have software quality assurance responsibility — software engineers, project
managers, customers, sales people, and the individuals who serve within an SQA group.

Elements of Software Quality Assurance:

Software quality assurance encompasses a broad range of concerns and activities that focus
on the management of software quality. These can be summarized in the following manner
Standards. The IEEE, ISO, and other standards organizations have produced a broad array of
software engineering standards and related documents. Standards may be adopted voluntarily
by a software engineering organization or imposed by the customer or other stakeholders.
The job of SQA is to ensure that standards that have been adopted are followed and that all
work products conform to them.
Reviews and audits. Technical reviews are a quality control activity performed by software
engineers for software engineers. Their intent is to uncover errors. Audits are a type of review
performed by SQA personnel with the intent of ensuring that quality guidelines are being
followed for software engineering work. For example, an audit of the review process might
be conducted to ensure that reviews are being performed in a manner that will lead to the
highest likelihood of uncovering errors.
Testing. Software testing is a quality control function that has one primary goal—to find
errors. The job of SQA is to ensure that testing is properly planned and efficiently conducted
so that it has the highest likelihood of achieving its primary goal.
Error/defect collection and analysis. The only way to improve is to measure how you’re
doing. SQA collects and analyzes error and defect data to better understand how errors are
introduced and what software engineering activities are best suited to eliminating them.
Change management. Change is one of the most disruptive aspects of any software project.
If it is not properly managed, change can lead to confusion, and confusion almost always
leads to poor quality. SQA ensures that adequate change management practices have been
instituted.
Education. Every software organization wants to improve its software engineering practices.
A key contributor to improvement is education of software engineers, their managers, and
other stakeholders. The SQA organization takes the lead in software process improvement
and is a key proponent and sponsor of educational programs.
Vendor management. Three categories of software are acquired from external software
vendors— shrink-wrapped packages (e.g., Microsoft Office), a tailored shell that provides a
basic skeletal structure that is custom tailored to the needs of a purchaser, and contracted
software that is custom designed and constructed from specifications provided by the
customer organization. The job of the SQA organization is to ensure that high-quality
software results by suggesting specific quality practices that the vendor should follow (when
possible), and incorporating quality mandates as part of any contract with an external vendor.
Security management. With the increase in cyber crime and new government regulations
regarding privacy, every software organization should institute policies that protect data at all
levels, establish firewall protection for WebApps, and ensure that software has not been
tampered with internally. SQA ensures that appropriate process and technology are used to
achieve software security.
Safety. Because software is almost always a pivotal component of human-rated systems (e.g.,
automotive or aircraft applications), the impact of hidden defects can be catastrophic. SQA
may be responsible for assessing the impact of software failure and for initiating those steps
required to reduce risk.
Risk management. Although the analysis and mitigation of risk is the concern of software
engineers, the SQA organization ensures that risk management activities are properly
conducted and that risk-related contingency plans have been established.

SQA PROCESSES AND PRODUCT CHARACTERISTICS:

software quality assurance, it’s important to note that SQA procedures and approaches that
work in one software environment may not work as well in another. Even within a company
that adopts a consistent approach to software engineering, different software products may
exhibit different levels of quality.
The solution to this dilemma is to understand the specific quality requirements for a software
product and then select the process and specific SQA actions and tasks that will be used to
meet those requirements. The Software Engineering Institute’s CMMI and ISO 9000
standards are the most commonly used process frameworks. Each proposes “a syntax and
semantics” that will lead to the implementation of software engineering practices that
improve product quality. Rather than instantiating either framework in its entirety, a software
organization can “harmonize” the two models by selecting elements of both frameworks and
matching them to the quality requirements of an individual product.

SQA Tasks:

The charter of the SQA group is to assist the software team in achieving a high-quality end
product. The Software Engineering Institute recommends a set of SQA activities that address
quality assurance planning, oversight, record keeping, analysis, and reporting. These
activities are performed (or facilitated) by an independent SQA group that:

Prepares an SQA plan for a project. The plan is developed as part of project planning and
is reviewed by all stakeholders. Quality assurance activities performed by the software
engineering team and the SQA group are governed by the plan. The plan identifies
evaluations to be performed, audits and reviews to be conducted, standards that are applicable
to the project, procedures for error reporting and tracking, work products that are produced by
the SQA group, and feedback that will be provided to the software team.
Participates in the development of the project’s software process description. The
software team selects a process for the work to be performed. The SQA group reviews the
process description for compliance with organizational policy, internal software standards,
externally imposed standards (e.g., ISO-9001), and other parts of the software project plan.
Reviews software engineering activities to verify compliance with the defined software
process. The SQA group identifies, documents, and tracks deviations from the process and
verifies that corrections have been made.
Audits designated software work products to verify compliance with those defined as
part of the software process. The SQA group reviews selected work products; identifies,
documents, and tracks deviations; verifies that corrections have been made; and periodically
reports the results of its work to the project manager.
Ensures that deviations in software work and work products are documented and
handled according to a documented procedure. Deviations may be encountered in the
project plan, process description, applicable standards, or software engineering work
products.
Records any noncompliance and reports to senior management. Noncompliance items
are tracked until they are resolved.

SQA Goals and Metrics:

The SQA activities are performed to achieve a set of pragmatic goals:
Requirements quality. The correctness, completeness, and consistency of the requirements
model will have a strong influence on the quality of all work products that follow. SQA must
ensure that the software team has properly reviewed the requirements model to achieve a high
level of quality.
Design quality. Every element of the design model should be assessed by the software team
to ensure that it exhibits high quality and that the design itself conforms to requirements.
SQA looks for attributes of the design that are indicators of quality.
Code quality. Source code and related work products (e.g., other descriptive information)
must conform to local coding standards and exhibit characteristics that will facilitate
maintainability. SQA should isolate those attributes that allow a reasonable analysis of the
quality of code.
Quality control effectiveness. A software team should apply limited resources in a way that
has the highest likelihood of achieving a high-quality result. SQA analyzes the allocation of
resources for reviews and testing to assess whether they are being allocated in the most
effective manner.

STATISTICAL SOFTWARE QUALITY ASSURANCE:

Statistical quality assurance reflects a growing trend throughout the industry to become more
quantitative about quality. For software, statistical quality assurance implies the following
steps:
1. Information about software errors and defects is collected and categorized.
2. An attempt is made to trace each error and defect to its underlying cause (e.g., non
conformance to specifications, design error, violation of standards, poor communication with
the customer).
3. Using the Pareto principle (80 percent of the defects can be traced to 20 percent of all
possible causes), isolate the 20 percent (the vital few ).
4. Once the vital few causes have been identified, move to correct the problems that have
caused the errors and defects.
This relatively simple concept represents an important step toward the creation of an adaptive
software process in which changes are made to improve those elements of the process that
introduce error.

A Generic Example

To illustrate the use of statistical methods for software engineering work, assume that a
software engineering organization collects information on errors and defects for a period of
one year. Some of the errors are uncovered as software is being developed. Other defects are
encountered after the software has been released to its end users. Although hundreds of
different problems are uncovered, all can be tracked to one (or more) of the following causes:
• Incomplete or erroneous specifications (IES).
• Misinterpretation of customer communication (MCC).
• Intentional deviation from specifications (IDS).
• Violation of programming standards (VPS).
• Error in data representation (EDR).
• Inconsistent component interface (ICI).
• Error in design logic (EDL).
• Incomplete or erroneous testing (IET).
• Inaccurate or incomplete documentation (IID).
• Error in programming language translation of design (PLT).
• Ambiguous or inconsistent human/computer interface (HCI).
• Miscellaneous (MIS).
Six Sigma for Software Engineering:

Six Sigma is the most widely used strategy for statistical quality assurance in industry today.
Originally popularized by Motorola in the 1980s, the Six Sigma strategy “is a rigorous and
disciplined methodology that uses data and statistical analysis to measure and improve a
company’s operational performance by identifying and eliminating defects in manufacturing
and service-related processes”. The term Six Sigma is derived from six standard deviations—
3.4instances (defects) per million occurrences—implying an extremely high-quality standard.
The Six Sigma methodology defines three core steps:
• Define customer requirements and deliverables and project goals via well-defined methods
of customer communication.
• Measure the existing process and its output to determine current quality performance
(collect defect metrics).
• Analyze defect metrics and determine the vital few causes. If an existing software process is
in place, but improvement is required, Six Sigma suggests two additional steps:
• Improve the process by eliminating the root causes of defects.
• Control the process to ensure that future work does not reintroduce the causes of defects.
These core and additional steps are sometimes referred to as the DMAIC (define, measure,
analyze, improve, and control) method.
If an organization is developing a software process (rather than improving an existing
process), the core steps are augmented as follows:
• Design the process to (1) avoid the root causes of defects and
(2) to meet customer requirements.
• Verify that the process model will, in fact, avoid defects and meet customer requirements.
This variation is sometimes called the DMADV (define, measure, analyze, design, and
verify) method.
A comprehensive discussion of Six Sigma is best left to resources dedicated to the subject.

SOFTWARE RELIABILITY:

There is no doubt that the reliability of a computer program is an important element of its
overall quality. If a program repeatedly and frequently fails to perform, it matters little
whether other software quality factors are acceptable. Software reliability, unlike many other
quality factors, can be measured directly and estimated using historical and developmental
data. Software reliability is defined in statistical terms as “the probability of failure-free
operation of a computer program in a specified environment for a specified time” .
To illustrate, program X is estimated to have a reliability of 0.999 over eight elapsed
processing hours. In other words, if program X were to be executed 1000 times and require a
total of eight hours of elapsed processing time (execution time), it is likely to operate
correctly (without failure) 999 times.
Whenever software reliability is discussed, a pivotal question arises: What is meant by the
term failure ? In the context of any discussion of software quality and reliability, failure is
non conformance to software requirements. Yet, even within this definition, there are
gradations. Failures can be only annoying or catastrophic. One failure can be corrected within
seconds, while another requires weeks or even months to correct. Complicating the issue
even further, the correction of one failure may in fact result in the introduction of other errors
that ultimately result in other failures.
Software Safety:

Software safety is a software quality assurance activity that focuses on the identification and
assessment of potential hazards that may affect software negatively and cause an entire
system to fail. If hazards can be identified early in the software process, software design
features can be specified that will either eliminate or control potential hazards.
A modeling and analysis process is conducted as part of software safety. Initially, hazards are
identified and categorized by criticality and risk. For example, some of the hazards associated
with a computer-based cruise control for an automobile might be: (1) causes uncontrolled
acceleration that cannot be stopped, (2) does not respond to depression of brake pedal (by
turning off), (3) does not engage when switch is activated, and (4) slowly loses or gains
speed. Once these system-level hazards are identified, analysis techniques are used to assign
severity and probability of occurrence. 4 To be effective, software must be analyzed in the
context of the entire system. For example, a subtle user input error (people are system
components) may be magnified by a software fault to produce control data that improperly
positions a mechanical device. If and only if a set of external environmental conditions is
met, the improper position of the mechanical device will cause a disastrous failure. Analysis
techniques such as fault tree analysis, real-time logic, and Petri net models can be used to
predict the chain of events that can cause hazards and the probability that each of the events
will occur to create the chain.
Once hazards are identified and analyzed, safety-related requirements can be specified for the
software. That is, the specification can contain a list of undesirable events and the desired
system responses to these events. The role of software in managing undesirable events is then
indicated.

Although software reliability and software safety are closely related to one another, it is
important to understand the subtle difference between them. Software reliability uses
statistical analysis to determine the likelihood that a software failure will occur.
However, the occurrence of a failure does not necessarily result in a hazard or mishap.
Software safety examines the ways in which failures result in conditions that can lead to
a mishap. That is, failures are not considered in a vacuum, but are evaluated in the
context of an entire computer-based system and its environment.

THE ISO 9000 QUALITY STANDARDS

A quality assurance system may be defined as the organizational structure, responsibilities,

procedures, processes, and resources for implementing quality management. Quality
assurance systems are created to help organizations ensure their products and services satisfy
customer expectations by meeting their specifications. These systems cover a wide variety of
activities encompassing a product’s entire life cycle including planning, controlling,
measuring, testing and reporting, and improving quality levels throughout the development
and manufacturing process. ISO 9000 describes quality assurance elements in generic terms
that can be applied to any business regardless of the products or services offered.
To become registered to one of the quality assurance system models contained in ISO 9000, a
company’s quality system and operations are scrutinized by third party auditors for
compliance to the standard and for effective operation. Upon successful registration, a
company is issued a certificate from a registration body represented by the auditors. Semi
annual surveillance audits ensure continued compliance to the standard.
The requirements delineated by ISO 9001:2008 address topics such as management
responsibility, quality system, contract review, design control, document and data control,
product identification and traceability, process control, inspection and testing, corrective and
preventive action, control of quality records, internal quality audits, training, servicing, and
statistical techniques. In order for a software organization to become registered to ISO
9001:2008, it must establish policies and procedures to address each of the requirements just
noted (and others) and then be able to demonstrate that these policies and procedures are
being followed.

THE SQA PLAN:

The SQA Plan provides a road map for instituting software quality assurance. Developed by
the SQA group (or by the software team if an SQA group does not exist), the plan serves as a
template for SQA activities that are instituted for each software project.
A standard for SQA plans has been published by the IEEE. The standard recommends a
structure that identifies: (1) the purpose and scope of the plan, (2) a description of all
software engineering work products (e.g., models, documents, source code) that fall within
the purview of SQA, (3) all applicable standards and practices that are applied during the
software process, (4) SQA actions and tasks (including reviews and audits) and their
placement throughout the software process, (5) the tools and methods that support SQA
actions and tasks, (6) software configuration management procedures, (7) methods for
assembling, safeguarding, and maintaining all SQA-related records, and (8) organizational
roles and responsibilities relative to product quality.