Chapter 03

TEST MANAGEMENT

14 Marks

Course Outcome: Prepare test plan for an application

(C 22518.3)

Content:
➢ Test Planning : Preparing a Test Plan, Deciding Test Approach
➢ Setting Up Criteria for Testing, Identifying Responsibilities
➢ Staffing, Resource Requirements
➢ Test Deliverables, Testing Tasks
➢ Test Management: Test Infrastructure Management
➢ Test People Management
➢ Test Process: Base Lining a Test Plan
➢ Test Case Specification
➢ Test Reporting: Executing Test Cases
➢ Preparing Test Summary Report
Test Planning:
• Test plan is a document that describes the scope, approach, resource and schedule
required for conducting testing activities.
• It identifies requirement of test items, which feature of application to be the different
testing task, who will perform the respective task and possible risk and their corrective
measures.
• Test planning is the first activity of test team. If a tester does not plan for testing then it
leads to failure.
• Test plans are defined in framework created by test strategy and established by
test policy Like any project, the testing also should be driven by a plan.
• The test plan acts as the anchor for the execution, tracking and reporting of the entire
testing project.
• The test plan acts as the anchor for the execution, tracking and reporting of the
entire testing project and covers.
• Test planning is the first activity of test team.
• If a tester does not plan for testing then it leads to failure.
• Test plans are defined in framework created by test strategy and established by test
policy like any project, the testing also should be driven by a plan.
• The test plan acts as the anchor for the execution, tracking and reporting of the entire
testing project.
• The test plan acts as the anchor for the execution, tracking and reporting of the entire
testing project and covers.

Activities of Test Plan

1. Scope Management.

2. Deciding Test Approach.

3. Setting up criteria for Testing.

4. Identifying Responsibilities.

5. Recourse Management.

6. Test Deliverables.

7. Testing Task (Size and Effort Estimation).

1. Scope Management.

Scope management performs to specifying the scope of a project .

For testing scope management includes:

a) Understanding what constitutes a release of product.

b) Breaking down the release into features.

c) Prioritizing the feature of testing.

d) Deciding which features will be tested & which will not be

Following factors derive the choice and prioritization of features to be tested.

a) Features those are new and critical for release.

b) Features whose failures can damage your software or application.

c) Features those are expected to be complex to test.

d) Features which are extensive of earlier features that have been defect prone.

2. Deciding Test Approach.

Test approach can be identifies by identifying:

a) What type of testing would use for testing functionality?

b) What are the configurations for testing features?

c) What integration testing would you do to ensure these features work together?

d) What “non-functional” tests would you need to do?

e) What localisation validations would be needed?

3. Setting up criteria for Testing.

There must be clear entry and exit criteria different phases of testing

a) Pass or Fail Criteria:

Specify the criteria that will be used to determine whether each
test item has passed or failed testing.

b) Suspend Criteria :
Specify criteria to be used to suspend the testing activity.

c) Resume Criteria :
Specify testing activities which must be redone when testing is
resumed.
4. Identifying Responsibilities.

The next aspect of planning is who part of it. Identifying responsibilities, staffing & training needs
addresses this aspect.
List the responsibilities of each team or individual.

Specify the number of testing staff and their roles.

Role Responsibility
Project Manager Mapping the total implementation.

1. Create test plan.

2. Identify test data.

Test Manager 3. Execute test condition and mark off results.

4. Prepare software error report.
5. Administrate error measurement system.

1. Ensure Phase 1 is delivered to schedule and quality.

2. Produce expected results.
Test Planner
3. Regular process at regular status reporting meetings.
4. Manage individual test cycles and resolve tester problems.

1. Designing the test.

2. Creating the test procedure.

Test Engineer 3. Creating the test data.

4. Execute tests.
5. Preparing Bug report.

1. Ensure delivered schedule and quality of the project.

2. Regularly review testing process.
SQA Project Leader
3. Manage risk issues relating to system test team.
4. Provide resources necessary for completing system test.

1. Ensure Unit testing.

Developer
2. Review each module before merging.
5. Resource Requirement.
As a part of planning for a testing project, the project manager should provide estimate
for the various hardware and software resources required.

Following are some factor to be considered.

a) Machine configuration.
b) Overhead required by the test automation tool.
c) Supporting tools such as Compiler, Test data generator and Configuration
management tool.
d) Special requirement for running machine intrusive tests.
e) Appropriate number of licenses of all the software.

6. Test Deliverables.
It includes test plan itself, test case design specification, test cases , test logs & test summary
report.

The deliverables include the following

The test plan Helpful for tester

Test case Specification Details needed for testing

Test design specification

Helpful in designing test
documents

Testing Strategy Approach to follow testing

Testing Scripts/ procedures Need to be followed
Test data Data useful during testing

Details of situation where

Test Incident report
testing performed

Test Traceability matrix Matrix to follow testing

Test results /Reports Entire report of testing
Install/Configuration guides Provides guidelines before testing
Test logs produced Useful for future testing

After completion of test this

Defect Report/ Release report
report is generated/prepared
7. Testing Task (Size and Effort Estimation).

Size and Effort estimation: This gives estimation in terms of size, effort & schedule of
testing project.
Estimation happens broadly in three phases.

a) Size Estimation:
i. Number of test cases.
ii. Number of test scenarios.
iii. Number of configurations to be tested.

b) Effort Estimation:
i. Productivity data.
ii. Reuse opportunities.
iii. Robustness of processes.

c) Schedule Estimation:

Advantages of test planning are:

1. Work involved in test planning and setup pays in the long term. It gives insight testing
activity completely. One knows scope and deliverables of test plan execution.

2. Test plan describes the way in which testing team will show whether software work
correctly as per requirements and the acceptance criteria as defined by customer or
development team with customer. It defines various objectives for testing to measure its
performance and coverage offered.

3. Test plan addresses various levels of testing such as unit testing module testing, System
testing, integration testing, black box testing as well as white box testing. Some time there
may be single master test plan with several child test plan at each level for a number of a
small plans or one monolithic test plan covering every aspect of testing.

4. Test plan explain who does testing. Why test are performed how test are conducted and
when tests are scheduled (calendar date and milestone). It defines various criteria such as
entry criteria, exit criteria, suspension criteria and resumption criteria at various stages of
testing.

5. Test plan must contain procedures, environment and tools necessary to implement an
orderly, controlled process for test execution, defect tracking, coordination of rework and
configuration, and change control.
 Test Management
Test management, process of managing the tests. A test management is also performed
using tools to manage both types of tests, automated and manual, that have been previously
specified by a test procedure.

Test Management tool often has multifunctional capabilities such as testware management,
test scheduling, the logging of results, test tracking, incident management and test reporting.

Test Management Responsibilities:

• Test Management has a clear set of roles and responsibilities for improving the quality of
the product.
• Test management helps the development and maintenance of product metrics during the
course of project.
• Test management enables developers to make sure that there are fewer design or coding
faults.

There are two types of management:

1. Test Infrastructure Management.
2. Test People Management.
1. Test Infrastructure Management.
Testing required a robust infrastructure to be planned.
Made up of three essential elements.
a) A test case database.
b) A defect repository.
c) Configuration management repository and tool.

a) A test case database:

• It capture all the relevant information about the test cases in an organisation.
• Some of the entities and the attributes in TCDB are given below.

Entity Purpose Attributes

1. Test Case ID
2. Test Case Name
Test Case Records all the “ static
information” about the test 3. Test Case Owner
4. Associated files for the test case

Provides a mapping between 1. Test case ID

Test case product
the tests and corresponding
cross reference 2. Module ID
product features.

1. Test case ID
Test case run Gives the history of when a 2. Run Date
test was run and what was
history 3. Time Taken
result.
4. Run Status

1. Test case ID
Gives details of test cases
Test case defect introduced to test certain 2. Defect Reference
cross reference specific defects detected in
3. Points to a record in defect
product
reference
b) Defect Repository:
Capture all the relevant details of defects reported for a product. It includes

Entity Purpose Attributes

1. Defect ID
2. Defect Priority
Records all the “ static
Defect details information” about the 3. Defect Description
defect 4. Affected Product
5. Any Relevant version information

Provide details of test 1. Defect ID

Defect test detail cases for a given defect
2. Test case ID
cross reference the TCDB

1. Defect ID
Fix detail Provide details of fixed for
a given defect 2. Fixed defect

Capture all the details of

1. Test case ID
the communication that
Communication 2. Defect reference.
transpired for this defect
3. Details of communication.
amount the various.
2. Test People Management.
Test People Management is an integral part of any project management
1. Developer
2. Tester
3. Sales Person
Ability to hire, motivate and retain the right people.

3. Integrating with product release:

The success of a product depends on the effectiveness of integration of the
development and testing activities.
The schedules of testing have to be linked directly to product release.
The following points to be decided for this planning.

a) Synchronisation point between development and testing as to when different

types of testing can commence.
b) Service level agreement between development and testing as to have long it
would take for testing team to complete the testing.
c) Consistent definitions of the various priorities severities of the defect.
d) Communication mechanisms documentation group to ensure that the
documentation is kept in synchronisation with the product in term known defect.
Test Process:
1. Base Lining a Test Plan:
• A test plan combines all the points in to a single document that acts as an anchor point for

the entire testing project.

• An organisation normally arrives at a template that is to be used across the board. Each

testing project puts together a test plan based on the template.

• The test plan is reviewed by a designated set of component people in the organisation.

• After this, the test plan is base lined into the configuration management repository.

• From then on the base lined test plan becomes the basis for running the testing project.

2. Test case specification

Using the test plan as the basis, the testing team designs test case specification, which
then becomes the basis for preparing individual test cases.
Hence a test case specification should clearly identify.

a) The purpose of the test.

b) Item being tested, along with their version / release number as appropriate.

c) Environment that need to be setup for running the test cases.

d) Input data to be used for the test cases.

e) Step to be followed to execute the test.

f) The expected result that are considered to be correct result.

g) A step to compare the actual result produced with the expected result.
Test Reporting

1. Recommending Product Release.

• Based on test summary report, an organisation can take decision on whether to release the
product or not.

• An organisation would like to realise a product with zero defect.

2. Matrix
• Matrix is a concise organiser to simple tests especially useful for functional tests.
• For example for most input fields you will do a series of the same tests, checking how the
field handles boundaries, unexpected character’s etc.

3. Executing Test Cases

• The prepared test cases have to be executed at the appropriate times during the project.

• As the test cases are executed during a test cycle, the defect repository is updated with-
a) Defect from the earlier test cycles that are fixed in current cycle.
b) New defect that get uncovered in the current run test of data.

• During test design and execution, the traceability matrix should keep current.

• As and when tests get designed and executed, the traceability matrix should be updated.

4. Collecting and analysing matrix.

• When tests are executed, information about test executed gets collected in the test logs
and other files.

• The basic measurement from the running the tests are then converted to meaningful
metrics by the use of appropriate transforms and formulate.
 5. Preparing Test Summary Report.
• At the completion of a test cycle, a test summary report is produced.

• This report gives insight to the senior management about the fitness of the product for
release.

• Testing required constant communication between the test team and other team (Like
the development team).

• There are two types of report-

a) Test Incident Report

b) Test summary report

a) Test Incident Report

• Test incident report is a document/report generated after the culmination of software

testing process, wherein the various incidents and defects are reported and logged by the
team members to maintain transparency among the team members and to take
important steps to resolve these issues.
• Test incident report is a communication that happens through the testing cycle as and
when defect are encountered.
• Test incident report is nothing but an entry made in the defect repository.
• Each defect has a unique ID and this is used to identify the incident.
Template of Test Incident Report

Incident ID: Date: Time:

Testers:

Incident Summary:

Incident Description:

a. Inputs: Describes the inputs actually used (e.g., files, keystrokes, etc.).

b. Expected Results: This comes from the test case that was running when the incident was discovered.

c. Actual Results: Actual results are recorded here

d. Anomalies: How do the actual results differ from the expected results? Also record other data (if it
appears to be significant) such as unusually light or heavy volume on the system, it's the last day of the
month, etc.

Impact: What is the potential impact on the user. The categories (and examples are) are:

Minor: Misspelled word on the screen.

Major: System degraded, but a workaround is available.

Critical: System crashes.

Investigation: This section of the incident report explains who found the incident and who the key
players are in its resolution. Also, what is the estimated amount of time required to isolate the
defect. Include your analysis of the defect in this section as well (questions from 2.1)

Disposition: Maintain a log of the incident as it goes through the analysis, debugging, correction,
re-testing, and implementation process. Be detailed and include dates and times of the work done
here.
b) Test Summary Report:
• A report that summarized the results of a test cycle is test summary report
• There are two types of Test Summary report (TSR)
a) Phase wise – Which is produced at the end of every phase.
b) Final Test summary Report.
• A summary report should present
1. A summary of the activities carried out during the test cycle.

2. Variance of the activities carried out from the activities planned.

3. Summary of result should include-

a) Tests that failed, with any root cause descriptions.
b) Severity of impact of the defect uncovered by the tests.

4. Comprehensive assessment and recommendation for release should include-

a) Fit for release

b) Recommendation of release.
What to Include in the Test Summary Report?

An informative test summary report should aim to be concise and relevant. If you check for
templates online, you will find several examples but the metrics might not be applicable in all cases,
such as for defect density or defect leakage, due to the nature of the project and work assignment.
Therefore, it is vital to customize your report after analysing precisely what is required and what is
not. The best way to do this is to use a template but remove sections from the document that will
be left blank if data is not available or applicable, so that the submitted report looks complete.

Here’s a list of what should be included:

1. The objective of the test – Mention the purpose of the testing, this is mainly to indicate that
the tester understood the expectation and the requirement.
2. Testing Approach – This is important because it indicates to the client what and how the
testing was performed. Provide details of the steps taken and the types of testing done to
achieve the task.
3. Areas Tested – This section must highlight all the features and functionalities tested. It
doesn’t have to be as detailed as a test scenario but be sure to include high level
information if, say, you are testing an eLearning system, and it must outline and verify the
flow to add a course, register, pay, and launch.
4. Areas Not Tested – Another significant part of any test summary report is information about
any features not tested. Any areas not tested will usually raise an alarm at the client’s end,
so make sure that anything has been left untested is noted and expectations are set
accordingly. Each point must have an associated reason, which might vary from the
limitation of access to the availability of the device, and so on.
5. Platform details – Gone are the days when applications used to be supported just on the
web. With the growing demand, testing likewise is not just limited to multiple browsers and
devices but different versions as well. So, it is recommended to include details of every
single platform and environment tested.
6. Defect Report – Even though this will already be in the bug report, having a link to the same
in the summary report can be a plus.
7. Testing Metrics (Optional) – Metrics, like total defects with a breakup of severity, can be
helpful. If applicable, include additional metrics regarding test cases.
8. Overall Summary – This section is mainly a place for you to provide your feedback on the
status of the application under test. It must inform the client of all critical issues so they can
estimate how close they are to launch and what remediation and time-scale they might
need in place for the fix.