Metrics Used in Testing: The Product Quality Measures
Metrics Used in Testing: The Product Quality Measures
This index is surveyed before product delivery and after product delivery
(and on-going on a periodic basis, using standard questionnaires).The following are
analyzed:
They are normalized per function point (or per LOC) at product delivery (first 3 months
or first year of operation) or Ongoing (per year of operation) by level of severity, by
category or cause, e.g.: requirements defect, design defect, code defect,
documentation/on-line help defect, defect introduced by fixes, etc.
4. Product volatility
• Ratio of maintenance fixes (to repair the system & bring it into compliance with
specifications), vs. enhancement requests (requests by users to enhance or change
functionality)
5. Defect ratios
8. Test coverage
9. Cost of defects
12. Reliability
• Availability (percentage of time a system is available, versus the time the system
is needed to be available)
• Mean time between failure (MTBF).
• Man time to repair (MTTR)
• Reliability ratio (MTBF / MTTR)
• Number of product recalls or fix releases
• Number of production re-runs as a ratio of production runs
Metric = Formula
Test Coverage = Number of units (KLOC/FP) tested / total size of the system. (LOC
represents Lines of Code)
Number of tests per unit size = Number of test cases per KLOC/FP (LOC represents
Lines of Code).
Source Code Analysis = Number of source code statements changed / total number of
tests.
Test Execution Productivity = No of Test cycles executed / Actual Effort for testing