SQA

Definition
• Software quality assurance is a means and practice of
monitoring all software engineering processes, methods, and
work products to ensure compliance against defined standards.
It may include ensuring conformance to standards or models,
such as ISO/IEC 9126, SPICE or CMMI.
• Ensuring software meets industry standards. SQA is usually carried
out by developers, an internal QA team or a third party, all of which
have their own benefits.
Definition
• Software quality assurance (SQA) is a means and practice of monitoring all software
engineering processes, methods, and work products to ensure compliance against defined
standards.[1] It may include ensuring conformance to standards or models, such as ISO or CMMI.
• It includes standards and procedures that managers, administrators or developers may use to
review and audit software products and activities to verify that the software meets quality criteria
which link to standards.
• SQA encompasses the entire software development process, including requirements
engineering, software design, coding, code reviews, source code control, software configuration
management, testing, release management and software integration.
Software Quality Assurance
• So the term software quality assurance would mean that the software guarantees high quality
• In this course, we’ll learn the different processes, techniques, and activities, which enables us –
the software professionals – to provide that guarantee to ourselves and our clients. It involoves
• Defect Prevention
• Formal Methods
• Education & Training
• Other activities
• Defect Removal/Reduction
• Testing
• Inspection

4
Defect Prevention and Removal
• Both defect prevention and removal techniques are used by the
“best-in-the-class” companies
• Both non-test and testing defect removal techniques must be applied

5
Typical Defect Removal
• Inspections
• Direct fault detection and removal
• Testing
• Failure observation and fault removal

6
Inspections
• Inspections is a critical analysis of software artifacts (such as SRS,
designs, codes, product specifications, test plans, etc) to catch and
remove defect directly.
• We will study it in detail in later lectures.

7
Inspection
• Requirement inspections
• Design inspections
• Code inspections
• Test plan reviews
• Test-case inspections
• User documentation editing or reviews

8
Testing
Testing Types
Typically Testing is classified into three categories.

 Functional Testing
 Non-Functional Testing or Performance Testing
 Maintenance (Regression and Maintenance)

11
Types
Functional Testing Non-Functional Maintenance Testing

Unit Testing Performance Regression

Integration Testing Endurance Maintenance

Smoke Load

UAT Volume

Localization Scalability

Globalization Usability

Interoperability
Defect Prevention

13
Defect Prevention
• We do not want defects or faults to enter our work products,
requirements, design, code, or other documents
• We try to eliminate the error sources in defect prevention

14
Philosophy of Defect Prevention - 1
• If human misconceptions are the error sources, education and
training can help us remove these error sources
• If imprecise designs and implementations that deviate from product
specifications or design intentions are the causes for faults, formal
methods can help us prevent such deviations

15
Philosophy of Defect Prevention - 2
• If non-conformance to selected processes or standards is the problem
that leads to fault injections, then process conformance or standard
enforcement can help use prevent the injection of related faults
• If certain tools or technologies can reduce fault injections under
similar environments, they should be adopted

16
Education and Training - 1
• Education and training provide people-based solutions for error
source elimination
• The people factor is the most important factor that determines the
quality and, ultimately, the success or failure of most software
projects

17
Education and Training - 2
• Education and training of software professionals can help them
control, manage, and improve the way they work

18
Focus of Education & Training
• Product and domain specific knowledge
• Software development knowledge and expertise
• Knowledge about Development methodology, technology, and tools
• Development process knowledge

19
Product and Domain Specific Knowledge
• If the people involved are not familiar with the product type or
application domain, there is a good chance that wrong solutions will
be implemented

20
Software Development Knowledge and Expertise

• Have good discussion on this

21
Knowledge about Development Methodology,
Technology, and Tools

• Have good discussion on this

22
Development Process Knowledge

• Have good discussion on this

23
Formal Methods
• Formal methods provide a way to eliminate certain error sources and
to verify the absence of related faults
• Formal methods include
• Formal specification
• Formal verification

24
Formal Specification
• Formal specification is concerned with producing an unambiguous set
of product specifications so that customer requirements, as well as
environmental constraints and design intentions, are correctly
reflected, thus reducing the chances of accidental fault injections

25
Formal Verifications
• Formal verification checks the conformance of software design or
code against these formal specifications, thus ensuring that the
software is fault free with respect to its formal specifications

26
• There are a number of formal method approaches. The oldest and
most influential formal method is the so-called axiomatic approach
• The research in this area is on-going and depending on the real need
of the software applications, formal methods are being used

27
• The biggest obstacle to formal methods is the high cost associated
with the difficult task of performing these human intensive activities
correctly without adequate automated support
• This fact also explains, to a degree, the increasing popularity of
limited scope and semi-formal approaches

28
 Software
Inspections

29
Inspections
• An inspection is a rigorous team review of a work product by peers of the
producer of the work product
• The size of the team will vary with the characteristics of the work product
being inspected; e.g., size, type
• The primary purpose is to find defects, recording as a basis for analysis on
the current project and for historical reference and for improvement for
future projects, analyzing them, and initiating rework to correct the defects
• Inspections are most effective when performed immediately after the work
product is complete, but they can be held any time the work product is
deemed ready for inspection
• Direct fault detection and removal
30
Inspections
• Faults are detected directly in inspection by human inspectors, either
during their individual inspections or various types of group sessions
• Identified faults need to be removed as a result of the inspection
process, and their removal also needs to be verified

31
Inspections
• Inspections remove software defects at reduced cost
• Inspections enable us to remove defects early in the software life
cycle, and it always cheaper to remove defects earlier in than later in
the software life cycle

32
• We know that defects are injected in every software life cycle activity
• We remove some of these defects in testing activities after code is
completed
• We also know that all defects are not removed at shipment time, and
these are known as latent defects

33
• We want to eliminate or at least minimize latent defects in the
shipped software product
• It is expensive to find and remove defects in the testing phase, and
even more expensive after shipment of the software
• We can use inspections to reduce these costs and improve the
timelines also

34
How Defect Removal is Cheaper for Inspections as
Compared to Software Testing

35
• During testing, defects are found, then the programmers are notified
of their presence, who will recreate the defects under the similar
circumstances, fix them, re-test the software and re-integrate the
software module, which were affected

36
• While in inspections, the inspection process is executed in the same
life cycle activity, and substantial amount of rework is avoided
• This results in the reduction of costs

37
• If and when defects are detected after the shipment of the software,
then these costs are even higher
• Many times, original development team is disbanded after the
completion of the project and new staff is looking after the
maintenance activity
• These people are usually not fully aware about the project
• This can result in unplanned expenses for the software development
company

38
• On the other hand, if an effective software inspections process is in
place, fewer defects enter the testing activity and the productivity of
tests improve
• The costs of tests are lower and the time to complete tests is reduced

39
• Several studies have confirmed the reduction in project costs when
defects were removed earlier

40
Defect Cost Relationship
Relative Costs to Remove a
Defect 10000

100 1000

Pre-Test Test Users

Major Defect Removal Area

41
• It is interesting to note that this relationship has remain consistent in
the last three decades – since the earliest studies when inspections
were being first reported
• In addition to the costs on project, there are additional costs to the
customer for downtime, lost opportunity, etc., when defects are
detected in maintenance

42
• Let’s look at the published data from different studies of companies in
which comparison of inspection costs and testing costs have been
made
• These were independent studies, and so they use different units to
report their results
• However, the pattern repeats that the cost of inspections is much
lower than that of software testing

43
Reported Cost Relationship - 1
Company Cost in Cost in Test Cost With
Inspections Customer
Discovery
IBM $48/defect $61-$1030 / $1770 /
defect defect

AT&T 1 unit 20 units --

ICL 1.2-1.6 8.47 --

hours/defect hours/defect

44
Reported Cost Relationship - 2
Company Cost in Cost in Cost With
Inspection Test Customer
s Discovery
AT&T 1.4 hours 8.5 hours --

JPL $105/ $1700/ --

defect defect

IBM 1 unit 9 times 117 times

more more
45
Reported Cost Relationship - 3
Company Cost in Cost in Cost With
Inspection Test Customer
s Discovery
Shell 1 unit 30 units --
Thorn EMI 1 unit 6.8-26 units 96 units

Applicon, 1 hour -- 30 hours

Inc.
Infosys 1 unit 3 – 6 units --
46
• These studies clearly report data from different companies that it is
cheaper to detect and remove data using software inspections as
compared to software testing
• There is evidence in the literature that inspection offer significant
return on investment even in their initial use

47
• Let’ now look at inspections from another point of view
• Relating defect origin points and defect discovery
• In a project with no software inspections, defects are typically
injected in the earlier activities and detected in later stages
• As a result, we get a chaos zone

48
Defect Origins and Discovery Points Without
Usage of Formal Inspections
Defect Origins
Requirements Design Coding Documentation Testing Maintenance

Requirements Design Coding Documentation Testing Maintenance

Defect Discovery
Chaos Zone

49
• This situation is a mess
• If only we were able to detect defects in the same life cycle activity,
we can eliminate the chaos zone, and bring some sanity back to the
project team and project management
• If we introduce software inspections, we can do that

50
Defect Origins and Discovery Points With
Usage of Formal Inspections
Defect Origins
Requirements Design Coding Documentation Testing Maintenance

Requirements Design Coding Documentation Testing Maintenance

Defect Discovery

51
• Here you can see that the chaos zone has been eliminated
• This is achieved by performing inspections on work products before
leaving that life cycle activity, and as a large number of requirements
defects will be detected and removed during the requirements
activity, design and coding defects will be detected and removed
during those activities, and so on

52
Why Isn’t Everyone Using Inspections?
• Now we are convinced that inspections have a clear value
independent of any model or standard for software development, so
why isn’t everyone using it?

53
Reasons for Not Using Inspections - 1
• There is resistance to Inspections because people view them as if they
are not easy to do well
• Management often views Inspections as an added cost, when in fact
Inspections will reduce cost during a project
• Development of new tools and environments

54
Reasons for Not Using Inspections - 2
• Inspections are not the most enjoyable engineering task compared to
designing and coding
• Inspections are labor intensive and low-tech
• Programmers/designers are possessive about the artifacts they create

55
Inspection Preconditions
• Clear and visible management support
• Defined policy
• Good training for all
• Effective procedures
• Proper planning
• Adequate resources

56