Testing Process

Improvement
Goal with this Presentation
Need for process improvement.
Present an idea, TMM and TPI.
Describe the process which build up TMM
Difference between the TMM and TPI

2
Test Process Improvement: Why?

Increasing importance and size of software in society as a

whole
Testing often takes 30 - 40% of project costs.
How do we improve the efficiency of software testing, and
at the same time reduce the cost of testing? In other words,
how do we make software testing both better and cheaper?
The answer of course is process improvement.

3
Purpose of TMM

The purpose of the TMM is to assist software organizations

with assessing and improving their software testing
processes.

4
TMM’s development History

 Developed by IIT (Illinois Institute of Technology).

 Component CMM on areas of interest.
 Any organization that wishes to improve its testing process
by use of the TMM first commit to improving its overall
software development process by application of the CMM.
(Testing process is a subset of the overall software
development process; therefore, its maturity growth needs
support from the KPAs associated with general process
growth )

5
What is TMM

A maturity model with focus on following points

Describes the processes, which are the basic in order to
ensure a well planned and controlled test part of a project.
5 level of maturity in testing.
Inspired by CMM
The model aims at raising through the 5 levels of maturity.
The higher the level of maturity, higher is the quality of
software being developed.

6
TMM – usability

 The TMM can be used by:

 . Internal teams to evaluate the current testing maturity
 . Management to launch specific improvement initiatives
 . Development projects to improve a specific test
 . Users and contractors to define their roles in testing

7
Process Flow of Test Process
Improvement.

8
TMM’s 2 Main Component
A Set of Levels
TMM consist of 5 levels each level represents a stage in evaluation to a
mature testing process.
A set of maturity goals for each level and activities, task and
responsibilities needed to support them.
For each maturity level, a number of process areas are defined. A
process area is a cluster of related activities within the test process, e.g.
test planning or test training.

An Assessment Models
A set of maturity related goal questions. They will help determine to
what extent the organization has mechanisms in place to achieve those
goals and for resolving the maturity issues.
An assessment method
A training program design to select and instruct the evaluation team.

9
TMM Maturity Goals By Level

10
Maturity Goals Level 1- Initial

There are no maturity goals at this level.

Tests are developed in an ad hoc way after coding is done.
Testing and debugging activities go hand-in-hand to get the
defects out of the software.
The objective of testing is to show that the software works/
runs.

11
Maturity Goals Level 2-Phase
Definition
Develop Testing and Debugging goals
A life cycle phase that follows coding
Initiate a Test planning process
Basic testing techniques and methods are used during
testing.
No reviews to address early defects
The objective of testing is to show that the software meets
its specification

12
Maturity Goals Level 3- Integration

Software Test Organization

Technical Training Program
Integrate Testing into Software Lifecycle
Control and Monitor Testing Process.
Objective : to provide insight into the quality of the product,

13
 Monitoring and Control

Risk Project Data

Control de
via
Schedule g es tio
h an ns
c
Effort
Cost Plan Monitor
plan

Deliverables

Product quality
Exit criteria Test Project

14
Maturity Goals Level 4- Management
& Measurement
Establish Review Program
Establish a Test Measurement Program.
Software Quality Evaluation

15
Level 5- Optimization, Defect
Prevention and Quality Control
Application of process data for defect prevention.
Quality Control
Test Process Optimization
 The objective of testing is to prevent defects from re-
occurring in the future

16
Support for TMM maturity levels

TMM CMM
Supporting Key Process Areas

2 2 Requirements management, project planning and

software configuration management

3 2 Project tracking SQA

3 3 Organization process focus, organization process definition and training

programs

4 3 Intergroup coordination, Peer reviews

4 4 Software quality management and Quantitative process management

5 5 Process change management, technology change

management and defect prevention

17
TMM Summary

Baseline the current testing process level of maturity

Identify areas that can be improved identify testing
processes that can be adopted organization-wide
Provide a road map for implementing the improvements
Provide a method for measuring the improvement results

18
TPI (Test Process Improvement)

The Test Process Improvement (TPI®)-model offers insight

in the 'maturity' of test processes within your organization.
The TPI has been developed by Sogeti, a wholly owned
subsidiary of the international Capgemini organization. It
goes hand in hand with the Test Management Approach
(TMap) which is also a testing methodology from Sogeti.

This model has four basic components -

 1. Key Areas
 2. Levels
 3. Checkpoints
 4. Improvement Suggestions

19
Model Description

Each of the above key areas is assessed at various levels

like A, B, C and D.
The number of levels for all the key areas is not the same.
For e.g. ‘Static Testing Techniques’ key area has only two
levels – A and B. However, ‘Test Strategy’ has 4 levels – A,
B, C and D.
Each level has certain checkpoints for each of the key
areas.
The test process under assessment should satisfy these
checkpoints to be certified for that level.
The model also includes improvement suggestions to assist
the organizations in achieving higher levels of maturity.

20
TPI Model

21
Key Areas
Life cycle
 1. Test Strategy
 2. Lifecycle Model
 3. Moment of involvement
Techniques
 4. Estimating and Planning
 5. Test Specification Techniques
 6. Static Test Techniques
 7. Metrics
Infrastructure
 8. Test Automation
 9. Test Environment
 10. Office Environment
Organization
 11. Commitment and Motivation
 12. Testing Functions and Trainings
 13. Scope of Methodology
 14. Communication
 15. Reporting
 16. Defect Management
 17. Test ware Management
 18. Test Process Management
All Cornerstones
 19. Evaluation
 20. Low-level Testing

22
23
24
Test Maturity Matrix

The Test Maturity Matrix relates Levels to Key Areas.

It does so by expressing the internal priorities and
dependencies between Levels and Key Areas.
It illustrates the strong and weak points of the current test
process. This aids the prioritizing of actions for
improvement.
In the matrix each level is related to a certain scale of test
maturity. This results in 13 scales of test maturity.
The open cells between different levels have no meaning in
themselves, but indicate that achieving a higher maturity for
a key area is related to the maturity of other key areas

25
Test Maturity Matrix

26
Test Maturity Matrix

Indicates the level of maturity of each key area .

As we go from left to right in the matrix, our testing process
becomes more and more sophisticated from Controlled to
Efficient to Optimizing.
For example, to start with static testing techniques (at level
A), the moment of involvement must be at level A also. Due
to this dependency, the level A of moment of involvement is
earlier than level A of the Static testing techniques.
The blanks in the test matrix essentially mean nothing and
are present to take care of the dependencies.
Level 0 indicates that the key area does not exist in the
current testing process.

27
TPI Summary

 The TPI model offers objective procedures for classifying

the current situation of the test process. Additionally, the
model offers assistance for test process improvement in the
form of key areas, levels and improvement suggestions.
 means of checkpoints it is possible to determine the levels
of key areas that a test process is on.
 The different maturity levels and key areas and their
dependencies are presented in the Test Maturity Matrix.

28
Comparison of TMM and TPI

TMM TPI

Complements CMMi Associated closely with

TMap

Includes all levels of testing Focus on high level tests

Research Product Commercial Product

Contains 5 levels Contains 13 scales of

maturity

Follows staged progress Follows continuous process

29