Test Management

Incident or Bug Management

1
 Agenda

•Objectives of Defect Management

•Defect Management Process
•What is a Defect/Bug?
•What do software defects cost?
•How a defect/bug is produced?
•Why do software have faults?
•Who and when can you report a Defect?
•Defect Life Cycle and Contents of a defect
•Defect Reporting
•Defect Tracking Process
•Testing - Defect Correction (Defect Fix)
•Q&A

2
 Objective of Defect Management

Incident or Defect Management has the following objectives:

– The primary goal is to prevent defects.  Where this is not possible or practical, the
goals are to both find the defect as quickly as possible and minimize the impact of the
defect.

–  The defect management process should be risk driven -- i.e., strategies, priorities, and
resources should be based on the extent to which risk can be reduced.

–  Defect information should be used to improve the process.  This, in fact, is the primary
reason for gathering defect information

– Provide developers and other parties with feedback about the problem to enable
identification, isolation and correction as necessary.

– Provide test leaders a means of tracking the quality of the system under test and the
progress of the testing.

3
 Defect Management Process

Defect Management Process

4
 Defect Management Process
• Defect Prevention  -- Implementation of techniques,
methodology and standard processes to reduce the risk of
defects.

• Deliverable Baseline  -- Establishment of milestones where

deliverables will be considered complete and ready for further
development work.  

• Defect Discovery  -- Identification and reporting of defects for

development team acknowledgment.  

5
 Defect Management Process
• Defect Resolution  -- Work with the development team to prioritize,
schedule and fix a defect, and document the resolution.  This also
includes notification back to the tester to ensure that the resolution
is verified.

• Process Improvement -- Identification and analysis of the process in

which a defect originated to identify ways to improve the process to
prevent future occurrences of similar defects.

• Management Reporting  -- Analysis and reporting of defect

information to assist management with risk management, process
improvement and project management.

6
 What is a Defect/Bug?

• From the producer’s point of view: a deviation from

specifications, whether missing, wrong, or extra feature between the existing and
required conditions.

• From the customer’s point of view: anything that causes

customer dissatisfactions, whether in the requirements or not

• “A software error is present when the program does not do what its end user
reasonably expects it to do” (Myers, 1976.)

7
 What do Software Defects Cost?

The Intel Pentium Bug

Enter the following in your PC calculator: (4195835/3145727)* 3145727-4195835.
If the answer is not zero, you have an old Intel Pentium CPU with the floating-point
division bug
– Discovered in 1994
– Intel's test engineers had found the problem before release.
– When pressured, Intel offered to replace the faulty chip, but only if a user could prove
that he was affected
– Ultimately, there was a public outcry. Intel had to spend $400 mn to replace the faulty
chip
Software faults can cause death and injury also
– Radiation treatment kills patients due to UI misinformation w.r.t sensitivity between
Gamma and X-Ray radiations.
– Aircraft crashes due to failure of Auto-Pilot in the calculation of ground proximity signals

8
 How a defect/bug is produced?

Failure: Deviation of the Software

from its expected delivery or service

Mistake/Error: a human action that

produces an incorrect result Fault: An incorrect step, process,
or data definition in a program.
Also known as Defect or Bug

9
 Why do Software have faults?

• Software is written by human beings

– Who knows something, but not everything
– Who have skills, but aren’t perfect
– Who do make mistakes (Errors)
• Pressure to deliver on strict deadlines
– No time to check the assumptions made which may be wrong
– Systems may be incomplete
• Inexperience of the software professionals and complexity of design
– Professionals new to coding
– Professionals new to the technology used in spite of having experience
• Poor Communication

10
 Who and when can we report a defect?

Who?
• Anyone who finds that the software is not working as designed can report a defect
» Testers/QA Personnel
» Developers
» Technical Support
» Beta sites
» End users
» Sales and Marketing staff (those interacting with the customers)
When?
• When a difference is found between Expected and Actual Results
• When in doubt, write it up as a remark

11
 Defect Tracking (Defect Life Cycle)

QA Opens
Remark

QA Assigns Remark

Developer Reviews
Remark

Developer Developer Declines

Acknowledges the remark with stated
defect reasons

Developer Fixes Developer Assigns back to QA Reviews

defect QA remark/defect

If Fixed or reason for declination is appropriate If not fixed or QA finds reason for re-opening
QA Closes remark/defect declined defect, QA assigns it back to the
developer with stated reasons

12
 Defect Life Cycle

13
 Defect Life Cycle

• By default, all defects will be created as ‘Remark’. The Tech Lead/Project Manager
will “Open” the remark for review, a valid remark will then become a defect which
is assigned to a Software Engineer. The status of the defect becomes
‘Open/Assigned’

• The Software Engineer will analyze and reproduce the defect. The status of the
defect is then changed to Fixed or Not Reproducible or Not a defect etc depending
on the findings of the developer

• The respective QA Engineer will retest to verify the fix and change the status to
‘Verified Fixed’ if verified. On the other hand, if the fix is found to be unsuccessful,
then it will be re-opened

14
 Defect Life Cycle (Cont…)

• If the ‘Remark’ is declined it is reassigned to the person who has reported it with
any the following status assigned to the remark
– ‘Duplicate’ if the remark already exists
– ‘As Designed’ if the remark is as per the requirements
– ‘Deferred’ if it is not possible to fix the defect in the current release
– ‘Not Reproducible’ if the developer is not able to reproduce the remark
• Used to describe and quantify deviations from requirements.
• Defects are reported to:
– Correct the defect
– Avoid the cost incurred in post-release phase
– Report the status of the application
– Gather statistics used to develop defect expectations in future applications
– Improve the software development process

15
 Contents Of Defect - Eform – Defect Management
• ID :
• Name :
• Detail : Description about the defect
• Access :
– Public
– Internal
– Private
• Identified At :
• Type :
• Severity :
• Priority :
• Status :
• Defect Related To :

16
 Contents Of Defect - Eform – Defect Management
• Date Opened :
• Phase Injected :
• Defect Category :
• Component :
• Platform :
• Reported to Client :
• External Reference :
• Defect Cause :
• Resolution :
• Fixed Date
• Addressed in Release :
• Time Spend to Resolve :

17
 Contents Of Defect

• Found in Release :
• Over all Status :
• Work Flow Stage :
• Current Users :
• Create By :
• Date Closed :
===========================================================
==
Detail should be :
– Summary of the defect
– Steps to reproduce
– Excepted Results
– Actual Results

18
 Contents Of Defect (contd.)

• Severity should be predefined for consistency (defined in Test

Plan)
– Impact of the failure caused by this defect
• Critical – Involves data loss, data corruption or whole system failure.
Customer impact could be devastating
• High – Major functionality is missing in the key component of the product.
Some kind of workaround usually exists. Customer impact would be
significant
• Medium – Minor functionality issues. Easily reproducible work around
available
• Low – Cosmetic errors which does not have any impact on the
functionality

19
 Contents Of Defect (contd.)

• Priority determines the order defects get fixed

– Urgency to fix the defect

• High – This has major impact on the customer. This must be fixed
immediately
• Medium – This problem should be fixed before the release of the current
version
• Low – This has to be fixed if there is time or can be deferred to the next
release

20
 Contents of Defect (contd.)

Think of the following type of defects:

• Example: 1

• A spelling error on a user-interface screen. What severity/priority does this issue

deserve?

• Example: 2

• The anomalous server crash. We’ve all seen this type of issue. A server crash that
occurs on the first full moon of every leap year

21
 Defect/Bug Reporting

• Compare actual Outcome with expected outcome. Log discrepancies accordingly

based on the defect type
• At a high level defect can be identified in 3 main phases of software life cycle:
– Design Phase
• Specifications Incorrect
• Documentation not proper
• Design not correct
– Implementation/Coding Phase
• Database fault
• Coding Fault
• Suggestion
– Maintenance Phase
• Build/Package
• Installation
• Environment

22
 Defect/Bug Reporting (cont.)

Additional notes

– Any document that helps investigate a defect

– Screen shots

23
 Work Flow

Initiate

Review

Resolve

Verify

Close

24
 Characteristics of defect report

• Written with simple language

• Numbered
• Simple
• Understandable
• Reproducible
• Non-judgmental
• Analysis

25
 Effective Defect report

• Explain how to reproduce the problem

• Analyze the error so you can describe it in a minimum number of steps
• Write a report that is complete and easy to understand
• Write the problem and report immediately

26
 Defect Tracking Tools

• Simple: Excel spreadsheet, Microsoft word

• Specialized Products
– Bugzilla
– Clear Quest
– Quality Center
– JIRA
– Rational Test Manager
– Zero Defect

• Emails can also be a effective defect tracking tool

27
 Defect Tracking

• Monitoring defects from the time of recording until satisfactory resolution has
been determined such as:
– Closing the defect raised
– Deferring the defect for the next Build/Release
– Not a defect
– Not reproducible

28
 Testing defects correction or defect fix

• Re-testing: Re-run the same test (i.e. re-test)

– Must be exactly repeatable
– Same environment, versions (except for the software which has been
intentionally changed!)
– Same inputs and preconditions

• Regression Testing: Ensure unchanged functionality performs correctly

– Unit Regression Testing: single component
– Regional Regression Testing: modules / areas connected to changed
component
– Full Regression Testing: entire application

29
Q&A