 Defect Density should not exceed 40 defects per 100 PDs of estimated effort (i.e.

DD <= 40)

Defect Density = (Total Defects x 100) / Total estimated effort

Total Defects = Bugs from internal testing + UAT bugs + client bugs + document review
defects + code review defects.

o Defect Leakage should not exceed 3 client reported defects per 100 PDs of
estimated/billable effort (i.e. DL <= 3) for development/Enhancement/Maintenance
projects.

Defect Leakage = (Total Defects reported by client x 100) / Total estimated/billable

effort

 Defect Removal Efficiency (DRE) should be at least 93% (i.e. DRE >= 93%) for
development/Enhancement/Maintenance projects.

DRE = (Code review defects + Document review defects + Internal testing bugs)/ Total Defects

Total Defects = Bugs from internal testing + UAT bugs + client bugs + document review
defects + code review defects

 Review Effectiveness should be at least 40% (i.e. RE >= 40%)

Review Effectiveness = (Document Review Defects + Code Review Defects)/ Total Defects

Total Defects = Bugs from internal testing + UAT bugs + client bugs + document review
defects + code review defects

 Defect Age should not exceed 1.0 (i.e. Defect Age <= 1)

Average time (in phases) a defect remains undetected in the lifecycle of a project. Project
phases and their aging are as follows:

Phase Phase No.

Requirement 1

Design 2

Coding 3

Testing 4

Deployment 5

 Effort Variance should be within -5% and 5% (i.e. -5% <= Effort Variance <= 5%).

Effort Variance = (Actual Effort – Estimated Effort)/Estimated Effort

Note: Estimated Effort is the one arrived at after requirements are frozen
  Schedule Variance should not exceed 2% (i.e. Schedule Variance <=2%)

Schedule Variance = (Actual end date – Planned end date)/ (Planned end date – Planned start
date)

Note: Planned start and end dates are based on the schedule prepared after the requirements are
frozen

 Completed PBI Ratio in Agile Scrum Life Cycle projects should be at least 90% (i.e. Comp. PBI
Ratio >= 90%)

Completed PBI Ratio = No. of PBIs closed in a sprint/ No. of PBIs planned for the sprint

 Sprint Success Ratio in Agile Scrum Life Cycle projects should be at least 80% (i.e. Sprint
success ratio >= 80%)

Sprint Success Ratio = No. of successful sprints/ Total No. of sprints

Successful sprint is defined as one where the Completed PBI ratio is at least 90%

 Mean time to resolve NCs should not exceed 14 calendar days (MTTR NCs <= 14 cal days).

MTTR (Mean Time to Resolve) for NCs will be calculated based on the difference in days
between the time when an NC was first reported and the final time when it was marked as
resolved. Only NCs that have been closed in the reporting period will be considered for MTTR.

 Mean time to close NCs should not exceed 7 calendar days (i.e. MTTC NCs <=7 cal days )

Mean time to close NC will be calculated based on difference in days between the time when
an NC was resolved by projects and when it was verified and closed by SQAG team.

Severity based Weightage Assignment to defects will be as follows:

 Showstopper = 3.0

 High = 1.8

 Medium = 1.0

 Low = 0.5

 Trivial = 0.2

i.e. a Showstopper severity defect is equivalent to three Medium severity defects and so on

Note: The number of defects used for calculation of Defect Density, Defect Leakage, Defect Removal
Efficiency, Review Effectiveness, and Defect Age will be based on the severity of the defects. For
example, consider the following pattern of defects for a completed Release:

Severity Count
Defect Type
Trivial Low Medium High Showstopper Absolute Weighted
Doc Review 1 3 8 2 0 14 13.30
Code Review 2 4 16 4 1 27 28.60
Internal Testing 12 16 72 64 5 169 212.60
 UAT 0 4 2 2 1 9 10.60
Production 0 2 0 0 0 2 1.00
Total 15 29 98 72 7 221 266.10

Weighted count of defects is calculated based on the weightage for each severity type. Therefore:

Weighted Count of Document Review defects = (1*0.2) + (3*0.5) + (8*1) + (2*1.8) + (0*3) = 13.30

Once the Weighted Count of defects is calculated, the Metrics for Defect Density, Defect Leakage,
Defect Removal Efficiency, and Review Effectiveness are calculated using the formula for these Metrics.
The calculated value for each of these Metrics for the example above is:

Metric Formula Value

Defect Density = 266.10 / (300/100) 88.70
Defect Leakage = (10.6 +1.0)/(300/100) 3.87
Defect Removal Efficiency = (13.30 + 28.60 + 212.60) * 100 /266.10 95.64%
Review Effectiveness = (13.30 + 28.60) * 100 / 266.10 15.75%

Statistical Analysis Thresholds

These thresholds are defined for various statistical analysis techniques that are being used by projects
or metrics council to analyze metrics data.

 P-value in normality test output should be at least 0.05 (i.e. p-value >= 0.05)

 R-Sq (Adj) value in regression output should be at least 75% (i.e. R-sq (adj) >= 75%)

 P-value of individual parameter in regression output should not exceed 0.05 (i.e. p-value <=
0.05)

 P-value of regression equation should not exceed 0.05 (i.e. p-value <= 0.05).

 VIF (Variance Inflation Factor) value in regression output for each parameter should not
exceed 5 (i.e. VIF <= 5)

 P-value in F-test output should be at least 0.05 (i.e. p-value >= 0.05)

 P-value in two sample T-test output should be at least 0.05 (i.e. p-value >= 0.05)

 Probability threshold for simulation factor should be at least 75% (i.e. probability >= 75%).

Probability threshold is calculated based on the outputs (mean and standard deviation) obtained by
running the simulation on PCB and additional project data points and the project goals. This
probability figure shows the chances of successfully meeting the goal if project follows same process.