Test Execution and Defect Management
Test Execution and Defect Management
Yujuan Dou
Identifier
HW_TC&TRPT_Group Number _yymmdd
Agenda
Test Execution Defect Management
Test Execution
TEST Execution
This phase involves actual test execution. The testers must execute the documented test cases and capture the results in a test report. The test leader should divide the test cases among the different testers and consolidate the results of testing. In case the application under test is expected to have multiple builds then a mechanism for identifying a correct build for testing should be decided.
Test Log
Each Test Log should be made up of a series of entries that present an audit trail for various aspects of the test execution including, but not limited to, the following:
the date and time stamp of when the event occurred a description (usually brief) of the event logged some indication of the observed status additional contextual information where relevant additional details relating to any anomalous or erroneous condition detected
Test Execution
Test Environment Test Data Test Tools
Test Execution
A Test Execution Run Plan could also be prepared depending upon the following criteria:
Impacted Functionality Unstable functionality Customer Priority
TC priority
Execution Order
Complexity
Execution rate
Functionality Dependency
TC Interdependency
Execution Order
RUP-Test Execution
Setup test environment to known state Set execution tool options Schedule Test Suite execution Execute Test Suite Evaluate execution of Test Suite Recover from halted tests Inspect the Test Logs for completeness and accuracy Restore test environment to known state Maintain traceability relationships Evaluate and verify your results
DEFECT MANAGEMENT
Defect Management
This process goes in parallel with the test execution phase. Defect reporting should be clear and concise and should make it easy for the developer to reproduce the failure scenario.
Defect Entry & Severity Classification: Typically, the bugs are logged into a defect database with the following information:
Synopsis - One line description of the bug. Bug environments - Info about OS, Application server, browser, Database, builds used with all the version information where the bug occurs are specified. Other environment on which the bug does not occur are also mentioned Detailed description with exact error message or the observed behavior
Defect Information
Exact steps to reproduce the bug Expected behavior Regression information if the bug was not occurring on any of the recent builds. This information is often used to determine whether the bug is to be fixed or not. Severity of the bug Priority of the bug
Defect Severity
Severity 0: Customers production use of the solution is stopped or so severely impacted that the customer cannot reasonably continue work. There is no workaround or the provided workaround is not acceptable to the customer because of its business impact. Severity 1: Implementation is operational but its functionality is seriously affected. If a workaround has been provided, the loss of functionality can only be sustained for a short time. Or there is a problem preventing roll-out / go-live / implementation.
Defect Severity
Severity 2: Implementation is operational but a problem has been identified and a specific portion of the system either provides incorrect results or is not operating as documented. A workaround is available and is acceptable to the customer. Severity 3: Customer has a question on the product or a request for a product modification or enhancement.
Severity in Bugzilla
This field describes the impact of a bug. Blocker Blocks development and/or testing work Critical crashes, loss of data, severe memory leak Major major loss of function Minor minor loss of function, or other problem where easy workaround is present Trivial cosmetic problem like misspelled words or misaligned text Enhancement Request for enhancement
Priority
This field describes the importance and order in which a bug should be fixed. This field is utilized by the programmers/engineers to prioritize their work to be done. The available priorities range from P1 (most important) to P5 (least important.)
Review of all the currently open product defects and determine the appropriate vehicle for delivery. Only the bugs that are needed to be fixed for the current release will be connected to this release for fixing. The typical criteria considered while deciding this are: Severity of the bug Whether it is a regression bug or not Criticality of the bug from the point of view of the endCustomer Verification of duplicate bugs, and to connect them accordingly through the bug tracking tool.
Defect Management
Start Is Bug Valid? No Reject the bug and update thru Bug tracker tool No
Yes
No End
Defect Management
Fixing of bugs and subsequent testing
The developer who is assigned the bug, fixes it, and adds the fix description through the bug tracking tool and also the build in which the fix would be available. The bug is then promoted to the Test state, and assigned to the QE manager for testing the fix. A new build containing a list of bug fixes is then released to SQA. The SQA team then verifies the fix by testing it.
Defect Management
If the bug still exists, then it is demoted to Open/Re-open state If the bug is fixed, then it is promoted to Closed state SQA also performs the regression testing to ensure that the fixes have no side-effects.