Software Quality Assurance

Pavithra Subashini
Senior lecturer
Faculty of Computing
Why you must study Quality
Assurance?
 Ground rules

• Interactivity is must
• Proactivity is expected
• No mobile phones during lecture
• Allow access to digital devices during activities
• Activity based learning
 Module set up

• Two-hour lectures per week – Activity based

learning
• One-hour tutorial
• At the end of the module you will be able to ensure
the quality of the software project.
• Encourage self study
– Research on SQA field
– Industrial changes/ adoption to the changes
 Reference book
https://www.istqb.org/downloads.html

Refer latest books from NSBM Library

 What is Software Quality Assurance?

“Set of activities to ensure the quality in software

engineering processes.”
– It ultimately result in quality software products.
– The activities establish and evaluate the processes that produce
products.
– It involves process-focused action.
 What is testing?

“Method to check whether the actual software product

matches expected requirements and to ensure that
software product is Defect free.”

– It involves execution of software/system components by

manual or automated tools
– The purpose of software testing is to identify errors, gaps
or missing requirements in contrast to actual
requirements
 Typical Objectives of Testing

• To prevent defects by evaluate work products such as

requirements, user stories, design, and code

• To verify whether all specified requirements have been

fulfilled

• To check whether the test object is complete and validate if

it works as the users and other stakeholders expect 

• To build confidence in the level of quality of the test object

 Typical Objectives of Testing
• To find defects and failures thus reduce the level of risk
of inadequate software quality

• To provide sufficient information to stakeholders to

allow them to make informed decisions, especially
regarding the level of quality of the test object

• To comply with contractual, legal, or regulatory

requirements or standards, and/or to verify the test
object’s compliance with such requirements or standards
 What is a “bug”?

• Error: a human action that produces an incorrect result

• Fault: a manifestation of an error in software
– also known as a defect or bug
– if executed, a fault may cause a failure
• Failure: deviation of the software from its expected
delivery or service
– (found defect)

Failure is an event; fault is a state of

the software, caused by an error
 Error - Fault - Failure
A person makes
an error ...

… that creates a
fault in the
software ...

… that can cause

a failure
in operation
 Reliability versus faults

• Reliability: the probability that software will not

cause the failure of the system for a specified
time under specified conditions
– Can a system be fault-free? (zero faults, right first
time)
– Can a software system be reliable but still have faults?
– Is a “fault-free” software application always reliable?
 Why do faults occur in software?

• software is written by human beings

– who know something, but not everything
– who have skills, but aren’t perfect
– who do make mistakes (errors)
• under increasing pressure to deliver to strict
deadlines
– no time to check but assumptions may be wrong
– systems may be incomplete
• if you have ever written software ...
 Do search ,you will find lots of
interesting facts

What do software faults cost?

• huge sums
– Ariane 5 ($7billion)
– Mariner space probe to Venus ($250m)
– American Airlines ($50m)
• very little or nothing at all
– minor inconvenience
– no visible or physical detrimental impact
• software is not “linear”:
– small input may have very large effect
 Safety-critical systems
• software faults can cause death or injury
– radiation treatment kills patients (Therac-25)
– train driver killed
– aircraft crashes (Airbus & Korean Airlines)
– bank system overdraft letters cause suicide
 So why is testing necessary?

– because software is likely to have faults

– to learn about the reliability of the software
– to fill the time between delivery of the software
and the release date
– to prove that the software has no faults
– because testing is included in the project plan
– because failures can be very expensive
– to avoid being sued by customers
– to stay in business
 Why not just "test everything"?

Total for 'exhaustive' testing:

20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests
If 1 second per test, 8000 mins, 133 hrs, 17.7 days
(not counting finger trouble, faults or retest)

10 secs = 34 wks, 1 min = 4 yrs, 10 min = 40 yrs

 Exhaustive testing?

• What is exhaustive testing?

– when all the testers are exhausted
– when all the planned tests have been executed
– exercising all combinations of inputs and
preconditions
• How much time will exhaustive testing take?
– infinite time
– not much time
– impractical amount of time
 How much testing is enough?
– it’s never enough
– when you have done what you planned
– when your customer/user is happy
– when you have proved that the system works
correctly
– when you are confident that the system works
correctly
– it depends on the risks for your system
 How much testing?

• It depends on RISK
– risk of missing important faults
– risk of incurring failure costs
– risk of releasing untested or under-tested software
– risk of losing credibility and market share
– risk of missing a market window
– risk of over-testing, ineffective testing
 So little time, so much to test ..

• test time will always be limited

• use RISK to determine:
– what to test first
– what to test most
– how thoroughly to test each item
- what not to test (this time)
} i.e. where to
place emphasis


use RISK to
- allocate the time available for testing by
prioritising testing ...
Most important principle

Prioritise tests
so that,
whenever you stop testing,
you have done the best testing
in the time available.
 Testing and quality

• testing measures software quality

• testing can find faults; when they are
removed, software quality (and possibly
reliability) is improved
• what does testing test?
– system function, correctness of operation
– non-functional qualities: reliability, usability,
maintainability, reusability, testability, etc.
 Other factors that influence testing
• contractual requirements
• legal requirements
• industry-specific requirements
– e.g. pharmaceutical industry (FDA), compiler
standard tests, safety-critical or safety-related
such as railroad switching, air traffic control

It is difficult to determine
how much testing is enough
but it is not impossible