Software Quality Engineering

BY

D R . R I Z WA N B I N FA I Z
Outline
Software Quality Engineering
Software Assurance
What is Software and what is Not Quality
Software Quality Perspective/Views
Why do need Quality Software
Software Quality Assurance Process
 My Self
Course Instructor
◦ Rizwan Bin Faiz
◦ PhD in CS (AI) from Loughborough University, UK
◦ It is rated in top 10% of all UK Universities

I am a motivated and committed student of computer science

and Software Engineering whose prime objective is to do
RE-SEARCH.
However teaching is my passion.
I have the ambition to impart high quality education in the field of
computer science and Software Engineering into the students.
 My Self

Email
◦ [email protected]
If I don’t reply in 48 hours (excluding week ends), send a complain
Email
Network sharing drive
◦ https://odoo.cust.edu.pk
◦ All Lecture notes will be uploaded before class.
◦ All students are advised to bring hard copy of lecture notes in the
class.
 What Do I Expect from You

• Reasonable reading and writing in English

• Analysis
• Computer Literacy: Web Search
Care more about what you learn than grades and degree
requirements
Challenging even for students with Software Engineering
courses

If I ever expect anything else, Stop and

Ask me!
 Rules to be Obeyed
Come prepared in each class.
Any question can be asked at any time in any class
◦ One person at one time
◦ Logical questions which initiate thinking are encouraged
◦ Avoid dull questioning like:
◦ Repeat,
◦ what,
◦ Explain etc

No one is allowed to look down at any one no matter what

level of question he/she has posed.
Comments are discouraged and questions of any level are
always encouraged at all time in and out side the class
 Rules to be Obeyed
I will talk and you will listen unless otherwise
◦ No whispering
Late comers are not welcomed for any reason under all
circumstances through out the semester.
Consequences
◦ You may be marked absent unless otherwise
Mobile phones should always be switched off
◦ I mean switched off not on silent mode

Sayings
◦ One person each day
 Course Grade
I confirm in the start of this course you all have 100%
marks and A Grade.
During the course you will be evaluated by assignments,
Mid Term, Project and Final papers each carrying some %.
You final grade is based upon you marks retained till the
end of the semester.
Lets see how many marks can you retain till the end of the
course?
 Why is software Quality a Concern

Software systems are becoming ubiquitous in modern society.

◦ Worldwide users rely on individual and interconnected computers, as well
as the global information infrastructure, such as the Internet and the
World Wide Web (WWW), to fulfill their needs for information processing,
storage, search, and retrieval.

This reliance requires the software to function correctly over a

long time, to be easy to use, and so on.
◦ Negative impact of software e.g. wrong functions, or performing intended
functions incorrectly, thus causing unintended consequences
Why is software Quality a Concern
-Achieving high level of product or service quality is the
objective of most organizations.
-It is no longer acceptable to deliver poor quality products, and fix
deficiencies after they have been shipped to the customer.
-However, software quality is a complex notion that cannot be
defined in a simple way, unlike for other manufactured products
such as cars, televisions, or computers.
- Quality product is expected to comply with customer requirements;
unfortunately such requirements are often incomplete for software.
- It is an ongoing challenge to precisely specify and check certain quality
requirements (e.g., usability, maintainability, reusability etc.)
 Why is software Quality a Concern
You would not like MS Office to occasionally save your documents
Competitiveness of International Market
◦ Cost of Quality

Software bugs, or errors, are so widespread and so damaging that

they cost the U.S. economy an estimated $59.5 billion annually.
Although all errors cannot be removed, more than a third of these
costs, or an estimated $22.2 billion, could be eliminated by an
improved testing infrastructure that enables earlier and more
effective identification and removal of software defects.
 Why is software Quality a Concern
Cancer treatment system
◦ A factor calculating radiation was applied twice
◦ 60% of patients had died by the time the fault was discovered
London Ambulance Service: computer aided dispatch software failed
◦ Breakdown for 36 hours
◦ Costing 20 lives
Intel Pentium bug:
◦ Formal verification of math routine would have avoided loss of $ 475 m
Ariane5
◦ destroyed 40 sec after take off since a software component in the controller
crashed costing around $ 500 m
 Why is software Quality a Concern
In 1947, computers were big, room‐sized machines operating on
mechanical relays and glowing vacuum tubes.
The state of the art at the time was the Mark II a behemoth being
built at Harvard University.
Technicians were running the new computer through its paces
when it suddenly stopped working.
They scrambled to figure out why and discovered, stuck between
a set of relay contacts deep in the bowels of the computer, was a
moth.
It had apparently flown into the system, attracted by the light and
heat, and was zapped by the high voltage when it landed on the
relay.
The computer “BUG” was born !
 Source of Bugs
The largest source of bugs is specification. Design comes second, Code
comes later and others.

Others
Code

Specification

Design
 Reasons for the existence of bugs …
Specification
‐ It keeps changing.
‐ There is no effective communication to the entire development
team.
‐ Software planning is not done accordingly.
Design
‐ The programmers lay out their plan for the software.
‐ Bugs occur here for the same reason they occur in the
specification.
‐ It’s rushed, changed, or not well communicated.
 Why SQA Activities
It is virtually impossible or practically infeasible to achieve
the complete prevention, elimination and contain of software
problems and related negative impact.
Consequently, various software QA activities are carried out
to prevent, eliminate and contain certain classes of problems
that lead to such negative impact, or to reduce the likelihood
or severity of such negative impact when it is unavoidable.
 Reasons why bugs exist …
Coding errors may be more familiar to you if you’re a programmer.
Typically, they can be traced to the software’s complexity, poor
documentation (especially in code that’s being updated and revised),
schedule pressure, or just plain dumb mistakes.
It’s important to note that many bugs that appear on the surface to be
programming errors can really be traced to specification and design
errors.
The other category is the catch‐all for what’s left.
Some bugs can be blamed on false positives, conditions that were
thought to be bugs but really weren’t.
They duplicate bugs, multiple ones that resulted from the same root
cause.
Some bugs can also be traced to testing errors.
 Why SQA Activities

 Increase in size and complexity

of software has unintended
consequences in terms of
causing quality problems.
 Due to increasing size and
complexity of today’s software
products, informal testing
without much planning and
preparation becomes
inadequate. *As the amount of project size/complexity
increases, the number of faults will increase
 The Cost of Bugs

It can cost a cent to fix a bug but, millions of dollars to

distribute a solution.
Bug is when a software doesn’t work properly.
The software didn’t operate as intended.
When to SQA Activities
 The Cost of Bugs
$1000

$100

$10

$1

Specification Design Code Test Release

The cost to fix bugs can increase dramatically over time

 Why is software Quality a Concern

Disney's Lion King, 1994‐1995

In the fall of 1994, the Disney company released its first
multimedia CD‐ROM game for children, The Lion King Animated
Storybook. On December 26th, the day after Christmas, Disney's
customer support received a huge amount of complaints from
the parents. The CD‐ROM was not working on the Personal
Computer.
Disney failed to test the software on a broad representation of
the many different PC Models available on the market. The
software worked on a few systems ‐ likely the ones that the
Disney programmers used to create the game ‐ but not on the
most common systems that the general public had.
 Why is software Quality a Concern
Intel Pentium Floating‐Point Division Bug, 1994

‐ Enter the following equation into your PC's calculator: (4195835/

3145727) * 3145727 – 4195835.
‐ If the answer is zero, your computer is just fine. If you get anything else,
you have an old Intel Pentium CPU with floating‐point division bug ‐
a software bug burned into a computer chip and reproduced over
and over in the manufacturing process.
‐ Intel management aware of the bugs before they released the chip set.
However, the management ignored the bug because they believed
that it is a small matter. A replacement of the faulty chips are
offered if a user could prove that he was affected by the bug.
‐ Intel apologized for the way it handled the bug and took a charge of
more than 400 millions to cover the costs of replacing bad chips.
Intel now reports known problems on its website and carefully
monitors customer feedback on Internet newsgroup.
 Why is software Quality a Concern
NASA Mars Polar Lander, 1999

‐ On December 3, 1999, NASA’s Mars Polar Lander disappeared

during its landing attempt on the Martian surface.
‐ A Failure Review Board investigated the failure and determined that
the most likely reason for the Malfunction was the unexpected
setting of a single data bit.
‐ Most alarming was why the problem wasn’t caught by internal
tests.
 Why is software Quality a Concern
Patriot Missile Defense System, 1991
‐ The U.S. Patriot Missile defense system is a scaled‐back
version of the Strategic Defense Initiative (“Star Wars”)
program proposed by President Ronald Reagan.
‐ It was first put to use in the Gulf War as a defense for Iraqi
Scud missiles.
‐ The software bug caused the system fail to defend against
several missiles, including one that killed 28 U.S. soldiers in
Dhahran, Saudi Arabia.
‐ In the Dhahran attack, the system had been operating for
more than 100 hours.
 Why is software Quality a Concern
The Y2K (Year 2000) Bug, Circa 1974

‐The payroll program relied heavily on date processing.

‐Due to a 2‐digit format for a date, the computations on year
2000 and 2001 will display as 00 and 01.
‐Several hundred billion dollars were spent, worldwide, to
replace or update computer programs to fix the Y2K
failures.
 Why is software Quality a Concern
Dangerous Viewing Ahead, 2004
‐On April 1, 1994, a message was posted to several Internet
user groups and then quickly circulated as an email that a
virus was discovered embedded in several JPEG format
pictures available on the Internet.
‐A proof‐of‐concept virus was created, proving that a JPEG
picture could be loaded with a virus that would infect the
system used to view it.
 What is Software

Programs

Software
System
Documentation
Data Documentation
User
Documentation
 What is software?
Computer programs, procedures, and possibly associated
documentation and data pertaining to the operation of a computer
system.

Software products may be developed for a particular customer or

may be developed for a general market
Software products may be
◦ Generic ‐ developed to be sold to a range of different customers
◦ Bespoke (custom) ‐ developed for a single customer according to their
specification
 Software Quality
Software quality is an abstract concept. Its presence can
be difficult to define, but its absence can be easy to see
Software quality is defined as a field of study and
practice that describes the desirable attributes
of software products
The degree to which a system or a component meets
specified requirements (customer’s need and
expectations)
 Quality Perspectives (Views)
Transcendental View
◦ In the transcendental view, quality is hard to define or describe in
abstract terms, but can be recognized if it is present
◦ It is generally associated with some intangible properties that delight
users Example: Very efficient algorithm
User View
◦ In the user view, quality is fitness for purpose or meeting user’s
needs
Manufacturing View
◦ In the manufacturing view, quality means conformance to process
standards
 Quality Perspectives (Views)
Product View
◦ In the product view, the focus is on inherent characteristics in
the product itself in the hope that controlling these internal
quality indicators will result in improved external product
behavior (quality in use) Example: Controlling faults per MLOC
Value‐Based View
◦ In the value‐based view, quality is the customers’ willingness to
pay for a software
 Software Quality
What is software quality?
Not a simple question to answer
Answer can be different for different stakeholders
Better question can be “What are the characteristics for
high‐quality software?”
To answer this question, first there is a need to discuss
perspective (views) and expectations of different users
involved in the software
 Software Quality
We need to know where are the stated or implied needs…
against which we test if they are met
We refer these quality features and characteristics as Quality
Attributes which are part of the Software Requirements
Specification (SRS) document
All SQA activities are design against Quality attributes in SRS.
 Software Quality Engineering (SQE)
Software Quality Engineering is the process of
implementing quality checks throughout the entire
development cycle.
The goal of quality engineering is to ensure the quality of
a product from the beginning, minimizing potential losses
caused by defects.
 Software Quality Engineering
QE spans the entire product lifecycle, not just the software
development process, whereas Quality Assurance has a more defined
focus in which QA looks to improve quality through testing.
Testing is an important subset of QA activities; and QA is an important
subset of Quality Engineering activities
 Software Quality Engineering (SQE) and
Software Quality Assurance (SQA)

SQE is about proactively ensuring quality is engineered into

every phase of development.
SQA is more reactive, focusing on identifying defects and verifying
that the software meets the defined quality standards.
Both are essential for ensuring the software is of high quality, but
SQE is more focused on improving the process, while SQA is about
confirming that the outcome meets expectations
 Software Quality Engineering (SQE) and
Software Quality Assurance (SQA)

Aspect Software Quality Engineering (SQE) Software Quality Assurance (SQA)

Focuses on process and quality Focuses on validation and
Scope
throughout the software lifecycle. verification of the product.
Ensure the product meets quality
Build quality into the product by
Goal standards through testing and
engineering processes.
reviews.
Process improvement, root cause
Testing, defect detection,
Activities analysis, defect prevention, and tool
documentation review, and audits.
integration.
Involves early stages of development Takes place after development to
Timing
and continuous involvement. ensure quality in the final product.
Aims for continuous quality Focuses on identifying and fixing
Outcome
improvement and better practices. defects in the final product.
 Software Quality Assurance (SQA) Activities
SQA activities are an attempt to detect defects upon various
software engineering artifacts e.g. requirement, design and
code specification.
SQA activities are driven by Software Requirement
Specification(SRS).
SQA set out the specific quality activities to particular product,
process, service, contract or project.
It sets out the desired product qualities, and define how these are to be
assessed.
 It select those organizational standards that the appropriate to a
particular process and development processes.
 Software Quality Assurance(SQA) Plan
Quality planning before specific QA activities are carried out,
in the so‐called pre‐QA activities in software quality
engineering.
We need to set the overall quality goal by managing
customer’s quality expectations under the project cost and
budgetary constraints.
We also need to select specific QA activities and techniques to
implement as well as measurement and models to provide
project monitoring and qualitative feedback.
 Software Quality Assurance (SQA) Plan

Quality planning includes forming a defect prevention, defect

detection and defect containment strategies.
Setting quality goals by matching customer’s quality
expectations with what can be economically achieved by the
software development organizations:
Identify quality sub attributes meaningful to target
customers and users.
 Software Quality Assurance (SQA)
In general, people’s quality expectations for software
systems they use and rely upon are two‐fold:
The software systems must do what they are supposed to
do. In other words, they must do the right things
For example, an airline reservation system is supposed to handle
reservations, not intended to fly airplanes automatically
They must perform these specific tasks correctly or
satisfactorily. In other words, they must do the things right
The system should help travel agents or individual travelers make
valid reservations within a pre‐specified time limit, instead of making
invalid ones
 Software Quality Assurance (SQA)
 Defect identification since they have been injected into the software systems
is in fact most traditional QA activities fall into this category. It is a process
of finding deviations of observed behaviour from the expected
behaviour.
Apply input Observe output
Software

Validate the observed output

 Is the observed output the same as the expected

output?
 Software Quality Assurance (SQA)
 Testing is the act of checking if a part or a product performs
as expected
Its goal is to maximize the number and severity of defects
found per dollar spent … thus: test early

We need to test our work, because we will make mistakes.

What is Not testing: Testing can only determine the
presence of defects, never their absence
Who should test: Someone other than the developer.
–Why?
 Verification
Software verification activities check the conformance of a
software system to its specifications.
In performing verification activities, we assume that we have a
well defined set of specifications.
A deviation from the specification is a fault, depending on
whether the behavior is specified or other software related
entities are specific, such as through coding standards, design
patterns, etc.
When a function or feature expected by the customers is
present, the activity to determine whether it performs or
behaves expectedly is then a verification activity.
 Verification
When we are checking specifications, non‐conformance
indicates the presence of faults or errors.
For example, a wrong algorithm or an inappropriate data
structure is used, some coding standard is violated, etc.
These problems are typically associated with various types of
software faults which if triggered may cause system failures.
Similarly, not following prescribed processes or selected
methodologies or misunderstanding of needed algorithms
and data structures, is associated with errors or error
sources that cause injection of faults.
 Validation
Validation checks the conformance to quality expectations
of customers and users in the form of whether the expected
functions or features are present or not.
Therefore, validation deals directly with users and their
requirements; while verification deals with internal product
specifications.
 Verification Vs Validation
Verification is a process performed by the developers to ensure
that the software is correctly developed.
Are we building the product right
Validation is a process performed by the users (Acceptance
Testing) to ensure that the software is in accordance to their
satisfaction.
Are we building the right product
 What is Software Quality

Quality is several attributes (portability, reliability, efficiency,

usability, understandability, modifiability) ...……………. Glass
 ‐ Quality is conformance to requirements …………………..Crosby
 ‐ Quality is fitness for use …………………………………......... .Deming
 ‐ Quality is value to some person ………………..…...…….
…Weinberg
 ‐ Quality is whatever the customer decides………….……. Ginac
 Quality is an attitude or state of mind ……………...…….. Juran
 Quality Perspective
One important fact in managing customer’s quality expectations is
that different quality attributes may have different levels of
importance to different customers and users.
Relevant quality views and attributes need to be identified first.
For example, reliability is typically the primary concern for various
business and commercial software systems because of people’s reliance on
such systems and the substantial financial loss if they are malfunctioning.
Similarly, if a software is used in various real‐time control situations, such
as air traffic control software and embedded software in automobile,
medical devices, etc., accidents due to failures may be catastrophic.
Therefore, safety is the major concern.
On the other hand, for mass market software packages, such as various
auxiliary utilities for personal computers, usability, instead of reliability or
safety, is the primary concern.
 Quality Perspective
Therefore, specific quality expectations by the customers
require us to identify relevant quality views and attributes prior
to setting appropriate quality goals.
This needs to be done in close consultation with the
customers and users, or those who represents their interests,
such as requirement analysts, marketing personnel, etc.
Once we obtained qualitative knowledge about customers’
quality expectations, we need to quantify these quality
expectations to set appropriate quality goals.
 Quality Perspective
A Quality has to be S M A R T E R
 Specific.… Measurable/Testable.... Attainable.… Relevant….Time‐
bound Evaluate and Readjust

 Danger of Lack of Quality

 Repair‐service behavior
 Without any clear idea of what the benchmarks are, we go in search of things
that are broken and our goal becomes fixing them
 Know‐how behavior
 We often don’t solve the problems that need to be solved but the ones we
know how to solve
 No justification for any quality action
 “I’m doing it for quality” anarchy , wasting a lot of time with no alignment
 Software Quality An Example
The system should be capable of processing 10
transactions per minute.
A software should be reliable.
The system should only allow authorized access.
The users should be forced to change their passwords?
The system should maintain logs of disk activity on one
disk to any other disk in the array in a fully redundant
manner.
 Software Quality An example
The Student Information System (SIS) is one of the most important
systems running in a university which manages almost all the
important tasks to be performed in order to run a campus,
smoothly and according to the university polices.
The system includes 7 subsystems within the system:
 Assessment Entry which manages assessment criteria, entry and reports;
Attendance Entry which manages record of attendance and provides
attendance sheets;
Course Offerings manages the process of offering courses to students at the
time of registration;
Assigning Course to Teachers;
Results Notification;
Transcript; and Semester Enrollments.

54
SIS Quality Goals

55
 SIS Quality
Stakeholders wants to Quality in terms of Reliability
and Security:
Reliability in terms of Accuracy which essentially means
that data recording should be accurate e.g. GPA, CGPA,
Attendance % etc.
Security in terms of Authorization which essentially means
that various stakeholders access to related data e.g.
Students assessment, attendance etc. should only be
viewable to students and editable to authorized course
instructor.
Unauthorized persons are not allowed to access any records.

56
SIS Quality Goals

Security
(Authorization)

Quality Goals

Reliability
(Accuracy)

57
 What is Software Not Quality

Failure: The inability of a system or component to perform its

required functions within specified performance requirements

Fault: An incorrect step, process, or data definition in a computer

program

Error: A human action that produces an incorrect result.

Error, fault and failure collectively refer to as defects

 What is Software Not Quality
errors  faults  failures

Errors may cause faults to be injected into the software, and faults may
cause failures when the software is executed.

A single error may cause many faults, such as in the case that a wrong
algorithm is applied in multiple modules and causes multiple faults, and a
single fault may cause many failures in repeated executions.

faults which do not cause a failure under the given scenarios are typically
called dormant or latent faults, which may still cause problems under a
different set of scenarios or circumstances
 What is Software Not Quality

 Bug / defect / fault consequence of a human error

 results in non‐conformance to requirements
 obvious as failure in running software
 What is Software Not Quality

The software doesn’t do something that the product

specification says it should do.
The software does something that the product specification says
it shouldn’t do.
The software does something that the product specification
doesn’t mention.
The software doesn’t do something that the product
specification doesn’t mention but should.
The software is difficult to understand, hard to use, slow, or – in
the software tester’s eyes – will be viewed by the end user as just
plain not right.
Software Quality Assurance Process