Domain 16 - Improving The Testing Process

IMPROVING THE
TESTING PROCESS
CBOK Knowledge Domain 16
CSTE Study Group Presentation

By Cathtina M. Gayton
The Testing Process
 Understand the Problem

 Define the Strategy
 Create the Traceability Matrix
 Write the Scripts
 Run the Tests
 Create the Test Summary
 Conduct a Post Project Review
Understand the Problem
The test team may know little about the

application to be tested, so the first hurdle is to
understand the problem. This may take some
time, and involve reading all available
documentation, talking to business analysts and
end-users, and possibly the technical IT people.
Define the Strategy
When tackling large projects, it is essential to define the requirements for a
successful process. In your document, you want the following as a minimum:
 Cover page  Software Requirements
 Table of Contents  Personnel Requirements
 Introduction  Assumptions
 Scope  Initial Preparation Requirements
 Stakeholders  Functions to be Tested
 Version Control  Acceptance/Rejection Criteria
 Approvals  Schedule of Events
 Hardware Requirements
Create the Traceability Matrix
This is a simple matrix that defines all the test cases, with reference
back to the document which defined the requirement. This matrix
can also be used as a test log, if you add a column for “Date
Tested” and another for “Pass/Fail”, but this is optional.
If you do not have any requirements or specification documents, it
indicates sloppy project management. You will have to make a list
of test cases without the benefit of the business telling you what it
wants. As your matrix cannot refer back to any specifications, it is
simply called a “Test Matrix”.
Write the Scripts
There is usually one test script for each line in the test
matrix. Each test script has an objective or goal, which
is the purpose of the test. Each test script, or test case,
has one or more steps to be performed. Each test script
has at least two columns: the actions to be performed,
and the expected result. Test scripts also include
prerequisites and a column to indicate if the step passes
or fails.
Run the Tests
 Once the scripts have been

written, they can be “run”;
that is, followed step by
step. Each test case should
be marked with the name
of the tester, the date of the
test, and whether it passes
or fails.
Create the Test Summary
When the testing is finally completed, it is necessary to write a Test Summary

Report. In the test summary you want the following as a minimum:
 Cover page
 Table of Contents
 Introduction
 Stakeholders
 Version Control
 Approvals
 Methodology
 Summary of Results
 Statement of Risk
Conduct a Post Project
Review
The goal is to determine where problems

occurred, document them, and decide what
can be done to prevent that problem from
occurring on the next project.
Improving the Testing Process
 Module (Unit) testing involves detailed testing of the

functionality of a well defined (but not necessarily small) unit
of code
 Integration testing requires that code units are brought
together and that the data flows and other interfaces between
them are validated for correctness.
 System Testing confirms that the system as a whole delivers
the functionality originally required.
 Acceptance Testing allows the system to be proven to meet
the requirements of the user community.
Module (Unit) Testing
Module testing is the most detailed form of testing to be performed. The peer
process review can add great value to this project stage. Testing is often seen
as a destructive process; finding fault in a person’s work. In truth it adds value
to the code, forcing it to do the job required of it.
Test Specification Reviews and Code Coverage
Testing has to be performed against some form Test specification reviews continue the culture
of requirement specifications. A simple of peer review and continuous improvement, act
specification of the tests to be performed and the as a way of transferring good technique and
expected results and behaviors can be written experience, and check code coverage.
from these requirements. The test specification Test Logs can be used to learn from the
process itself is a form of testing, often finding
testing process and improve upon later
many bugs and omissions. Forcing the
development cycles. They report what bugs
developer to draw back from the intricacies of
have been found, which have been corrected and
how their code works helps to re-focus on the
which await retesting. They also serve as an
requirements and flushes out scenarios or
informal record of the testing that was
functionality that has been omitted.
performed and the changes that were made.
Integration Testing
This test begins after two or more application components have

been successfully unit tested. It is the first level of testing
which formally integrates a set of programs that
communicate among themselves via messages or files.
“Integration is a process of evolution which requires firm
foundations. Building layer upon layer requires unhurried
attention to detail. Rushing to throw a system together in
the hope that it will all hang together rarely works and is
certainly not reliable. Far better to cement each layer in
firmly before proceeding to add in the next.”
Jonathan Collins
During the system test phase the concern is not how the
functionality is achieved but that the required
functionality is correctly achieved. Knowledge of the
business and its requirements can be usefully combined
with knowledge of how to use the system. All system
functionality should be tested at this stage.
Acceptance
Testing
Building the confidence of the Allowing members of the
client and users is the role of
the acceptance test phase. user community to
Often a subset of the system tests can be actually perform the
used to demonstrate the key functionality
which provides 80% of the system value acceptance test builds
to the business. Many functions will be
used often and repeatedly by a variety of further confidence.
users, particular attention needs to be
paid to the usability and performance of
these functions. Focusing on these
functions can give maximum benefit from
the 80-20 rule. (E.g. Identify early on the
key 20% of the functionality that is used
80% of the time.
Regression
Testing
A simple system regression test specification would test
the core functionality in the quickest way possible. The
intention is not to test fine details but simply to prove
that the essential system functions are operational.
Ten Points for Improving
the Testing Process
1. Reinforce the need for testing
2. Minimize testing requirements
3. Allocate adequate resources
4. Plan for testing
5. Have clear objectives for testing
6. Use standardized test suites
7. Design tests for reusability
8. Use commercial tools for testing
9. Use standardized test languages
10. Use a specialized test language
Test Process
Improvement Models
Eight Step Generic Spiral
Model for Process
Improvement Programs
1. Examine the Organization’s Need and Business
Goals
2. Conduct Assessments
3. Initiate Process Improvement
4. Analyze Assessment Output and Derived Action Plan
5. Implement Improvement
6. Confirm Improvements
7. Sustain Improvement Gain
8. Monitor Performance
PROCESS IS PRODUCT
 PROBLEM:
How should a process improvement initiative be
organized and managed?
 SOLUTION:
Treat it like a development project! Recruit a corresponding
project team (often called a Process Improvement Team or PIT), select
a project leader, and establish a repository to store process
documentation and other process artifacts. Use appropriate planning,
tracking, configuration management, and other methods and tools,
just as they should be used for any other development project.
Ensure the visibility of the the project to upper management and the
rest of the organization.
SUMMARY
IMPROVING THE TESTING PROCESS
 The improvement of the testing process is an ever

evolving mechanism.
 Always look for ways to enhance the major testing
phases of module, integration, system acceptance
and regression testing.
 Develop your objective for desiring to implement
test process improvement and follow the process
improvement model steps.
 Remember to improve the testing process, the
process must the product!
RESOURCES
- Appleton, B. Patterns for Conducting Process Improvement

(1997) http://www.enteract.com/~brpdapp/
- Challis, J. Improving the Testing Process
http://www.logica.com/papers/testing.html
- Mats, L. Ten Points for Improving the Testing Process
http://www.csdmag.com/tech/tech9805.html
- Quality Assurance Institute, (2001). Certified Software Test
Engineer Examination Study Guide, Orlando, FL
- Story Bridge Creations Pty Ltd. The Testing Process (2000)
http://www.storybridge.com.au/Testing/TestProcessUnderstand.html

Domain 16 - Improving The Testing Process

Uploaded by

Copyright:

Available Formats

Domain 16 - Improving The Testing Process

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Domain 16 - Improving The Testing Process

Uploaded by

Copyright:

Available Formats

IMPROVING THE

CBOK Knowledge Domain 16

CSTE Study Group Presentation

 Understand the Problem

The test team may know little about the

 Cover page  Software Requirements

 Table of Contents  Personnel Requirements

 Scope  Initial Preparation Requirements

 Stakeholders  Functions to be Tested

 Version Control  Acceptance/Rejection Criteria

 Approvals  Schedule of Events

 Once the scripts have been

When the testing is finally completed, it is necessary to write a Test Summary

The goal is to determine where problems

 Module (Unit) testing involves detailed testing of the

This test begins after two or more application components have

 The improvement of the testing process is an ever

- Appleton, B. Patterns for Conducting Process Improvement

You might also like