Audits are usually:

performed to demonstrate conformance to a defined set

of criteria

Conducted and moderated by a lead auditor

provide evidence of compliance collected through

interviews, witnessing and examining documents

have documented results

Managing reviews
The review strategy must be coordinated with the test

policy and the overall test strategy.

Reviews should be planned to take place at natural break

points or milestones:

typically after requirement and design specification

start with business objectives and work down to low

level design

often as part of a verification activity before, during,

and after test execution

Before establishing an overall review plan at the project

level, the review leader (may be a Test Manager) should

take into account:

What should be reviewed (product and processes)

Who should be involved in specific reviews

Which relevant risk factors to cover

The review leader should also:

identify the items to be reviewed

select the appropriate review type and level of

formality

identify a budget (time & resource) for the review

create a business case and include a risk evaluation and

return on investment calculation

The return on investment (ROI) for reviews is the

difference between the cost of onducting the review

and the cost of dealing with the same defects at a

later stage (or missing them altogether).

The optimal time to perform reviews depends on:

availability of the items to review in a sufficiently final

format

availability of the right personnel for the review

time when the final version of the item should be

available

time required for the review process of that specific

item

The objective of the review must be defined during test

planning and should also include:

conducting effective and efficient reviews

reaching consensus decisions regarding review

feedback
Adequate metrics also need to be established.

A good tip in case of audit or inspections on big projects

is to use brief inspections/audits conducted at the author's

request as document fragments are completed. This will

help have initial and iterative checks rather than a big

bang approach when all is taken in at once. There are also

options to have an advance audit prior the main

certification audit.

We should not forget about the project reviews which are

frequently held for the overall system and may also be

necessary for subsystems and even individual software

elements

Based on project complexity or product risks we should

adjust:

the number of reviews

the types of reviews

the organization of reviews

the people involved in reviews

Participants in reviews must:

Have the appropriate level of knowledge, both technical

and procedural

Be thorough and pay attention to details

Have clarity and the use of a correct prioritization

Understand their roles and responsibilities in the review

process
Review planning should address:

The risks associated with technical factors,

organizational factors and people issues

The availability of reviewers with sufficient technical

knowledge

Ensure that each team is committed to the success of

the review process

Ensure that each organization is allocating sufficient

time for required reviewers to prepare for and

participate in the reviews

Time allocation for required technical or process

training for the reviewers

Backup reviewers should be identified in case key

reviewers become unavailable due to changes in

personal or business plans

During execution, a review Leader must ensure:

Adequate measurements are provided by the

participants in the reviews to allow evaluation of

review efficiency

Checklists are created, and maintained to improve

future reviews

Defect severity and priority evaluation are defined for

use in defect management of issues found during

reviews
After each review, the review Leader should:

Collect the review metrics and ensure that the issues

identified are sufficiently resolved to meet the specific

test objectives for the review

Use the review metrics as input when determining the

return on investment (ROI) for reviews

Provide feedback information to the relevant

stakeholders

Provide feedback to review participants

A reviews effectiveness is evaluated by comparing results

from the review with the actual results found in

subsequent testing.

In case a work product is reviewed, but later found

defective, then the review leader should consider ways in

which the review process might have allowed the defects

to escape. Some likely causes:

include problems with the review process (like poor

entry or exit criteria

improper composition of the review team

inadequate review tools (checklists)

insufficient reviewer training and experience

too little preparation and review meeting time

It is possible that reviews lose their effectiveness over

time and this can be noticed in project retrospectives. In

such cases, the review leader should investigate the cause.

If a pattern of escaped defects is consistent across projects

this is another indicator that there are significant problems

with the review process which have to be assessed and

addressed.

By using metrics we must focus on the review process

and never use them to punish or reward individuals.

Managing formal reviews

The review leaders need to ensure that all steps in the

review process are followed.

Planning
Define the:

scope of the review

what documents or parts to review

the quality characteristics to be evaluated

Estimate effort and time frame

Identify review characteristics such as the:

review type with roles

activities

checklists