Scribd
Upload
248 views26 pages

Software Quality Engineering: Lecture 13 and 14: Verification & Validation + SQA Defect Removal Cost

The document discusses verification, validation, and qualification processes in software quality assurance. It then introduces a model for calculating the effectiveness and cost of defect removal at different stages of development. The model accounts for factors like the distribution of defects originating in each phase, the effectiveness of removal methods, and the varying costs of removing defects over time. Applying a comprehensive quality assurance plan with additional inspections and reviews is shown to remove more defects earlier at lower cost and reduce problems detected later by customers.

Uploaded by

Muhammad Shahbaz
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
248 views26 pages

Software Quality Engineering: Lecture 13 and 14: Verification & Validation + SQA Defect Removal Cost

The document discusses verification, validation, and qualification processes in software quality assurance. It then introduces a model for calculating the effectiveness and cost of defect removal at different stages of development. The model accounts for factors like the distribution of defects originating in each phase, the effectiveness of removal methods, and the varying costs of removing defects over time. Applying a comprehensive quality assurance plan with additional inspections and reviews is shown to remove more defects earlier at lower cost and reduce problems detected later by customers.

Uploaded by

Muhammad Shahbaz
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Software Quality Engineering

Lecture 13 and 14: Verification & Validation


+ SQA Defect Removal Cost

Muhammad Awais Gondal


Overview
• Verification, Validation and Qualification
• SQA Defect Removal Effectiveness and Cost
Verification, Validation & Qualification
(Three aspects of quality assurance of software
products)
Verification – The process of evaluating a system or component to
determine whether the products of a given development phase
satisfy the conditions imposed at the start of that phase

Validation - The process of evaluating a system or component


during or at the end of the development process to determine
whether it satisfies specified requirements

Qualification - The process used to determine whether a system


or component is suitable for operational use
Verification
• Are we building the product right?
• Verification is a static practice of verifying
documents, design, code and program.
• Verification uses methods like inspections,
reviews, walkthroughs etc.
Validation
• Are we building the right product?
• Validation is a dynamic mechanism of
validating and testing the actual product.
• Examining the extent of compliance with
customer’s needs.
• Validation uses methods like black box
(functional) testing, gray box testing, and
white box (structural) testing etc.
Qualification
• Focuses on operational aspects.
• Maintenance is the main concern.
A model for SQA defect removal
effectiveness and cost
• The model addresses two quantitative aspects
of SQA plan for defect detection:
– Total effectiveness in removing project defects
– Total cost of removing project defects
A model for SQA defect removal
effectiveness and cost
• The model is based on three types of data:
– Defect origin distribution (in which phase did the
defects occur)
– Defect removal effectiveness (how effective is the
removal of defects)
– Cost of defect removal (how much does it cost to
remove defects in different phases)
Defect Origin Distribution

No. Software development phase Average percentage of


defects originating in phase

1 Requirements Specification 15%

2 Design 35%

3 Coding (coding 30%, integration 10%) 40%

4 Documentation 10%
Defect Removal Effectiveness
• Percentage of removed defects is lower than
percentage of detected defects
WHY??

• Some corrections are ineffective


• Some defects are missed and passed on to
next phases
Defect Removal Effectiveness
• An assumption: an activity removes at least 40% of the
incoming defects
No. Quality assurance activity Average defect filtering
effectiveness rate
1 Requirements specification review 50%
2 Design inspection 60%
3 Design review 50%
4 Code inspection 65%
5 Unit test 50%
6 Unit test after code inspection 30%
7 Integration test 50%
8 System tests / acceptance tests 50%
9 Documentation review 50%
Defect Removal Cost
• Cost of removal of defects varies by
development phase
• Costs are much increased in later
development phases
Why??
Defect Removal Cost
No. Software development phase Average relative
defect cost (cost units)
1 Requirements Specification 1
2 Design 2.5
3 Unit tests 6.5
4 Integration tests 16
5 System tests / acceptance tests / system 40
documentation review

6 Operation by customer (after release) 110


Assumptions for the model
• Development process is linear, sequential following
waterfall model
• A number of new defects are introduced in each phase
• Review and test software quality assurance activities
serve as filters, removing a percentage of defects and
letting the rest pass to next development phase.
• At each phase, incoming defects are the sum of those not
removed plus new defects in current phase
• Cost of defect removal is calculated for each SQA activity
by multiplying the number of defects removed by the
relative cost of removing a defect.
• Remaining defects are passed to the customer. (heaviest
cost for defect removal)
Parameters for the model
• POD – Phase originated defects
• PD – Passed defects (from previous phases)
• %FE - % of filtering effectiveness
• RD – Removed defects
• CDR – Cost of defect removal
• TRC – Total Removal Costs

TRC = RD x CDR
A basic representation
The model applied on Standard
Quality Assurance Plan
• Having standard quality assurance
activities/filters
A process oriented illustration of
standard quality assurance plan
Using a comprehensive quality
assurance plan
• A comprehensive quality assurance plan with
more activities for filtering
• Adds two quality assurance activities to be
performed in phases of design and coding
– Design Inspection
– Code Inspection
• Improves the filtering effectiveness of other
quality assurance activities
Using a comprehensive quality
assurance plan
A process oriented illustration of
standard quality assurance plan
Conclusion
(Benefits of comprehensive plan)
• The comprehensive plan
– Removes more defects in earlier phases
– Saves resources … more economical
– Greater customer satisfaction … defects detected
in operation are less
Conclusion
(Benefits of comprehensive plan)
• The standard plan removes 57.6% of the defects in
requirements and design compared to 90.2% in the
comprehensive plan before coding begins.
• More economical - saves 41% of the total resources
investing in defect removal, compared with the
standard plan
• Compared to the standard plan, the comprehensive
plan makes a greater contribution to customer
satisfaction by reducing the rate of defects detected
during regular operations: from 6.9% to 2.6%
Next Lecture
• Review Objectives
• Formal Design Reviews

You might also like