Reviews, Walkthroughs & Inspections

1. Formal Definitions

Quality Control (QC)

A set of techniques designed to verify and validate the quality of work products and observe whether
requirements are met.

Software Element

Every deliverable or in-process document produced or acquired during the Software Development Life
Cycle (SDLC) is a software element.

Verification and validation techniques

Verification - Is the task done correctly?

Validation - Is the correct task done?

Static Testing

V&V is done on any software element.

Dynamic Testing

V&V is done on executing the software with pre-defined test cases.

2. Importance of Static Testing

Why Static Testing?

The benefit is clear once you think about it. If you can find a problem in the requirements before it turns
into a problem in the system that will save time and money. The following statistics would be mind
boggling.

M.E. Fagan "Design and Code Inspections to Reduce Errors in Program Development", IBM Systems
Journal, March 1976.

Systems Product

67% of total defects during the development found in Inspection

Applications Product

82% of all the defects found during inspection of design and code

A.F. Ackerman, L. Buchwald, and F. Lewski, "Software Inspections: An Effective Verification Process," IEEE
Software, May 1989.
Operating System

Inspection decreased the cost of detecting a fault by 85%

Marilyn Bush, "Improving Software Quality: The Use of Formal Inspections at the Jet Propulsion
Laboratory", Proceedings of the 12th

International Conference on Software Engineering, pages 196-199, IEEE Computer Society Press, Nice,
France, March 1990.

Jet Propulsion Laboratory Project

Every two-hour inspection session results, on an average, in saving of $25,000

The following diagram of Fagan (Advances in Inspections, IEEE Transactions on Software Engineering)
captures the importance of Static Testing. The lesson learned could be summarized in one sentence -
Spend a little extra earlier or spend much more later.

The statistics, the above stories and Fagan’s diagram emphasizes the need for Static Testing. It is
appropriate to state that not all static testing involves people sitting at a table looking at a document.
Sometimes automated tools can help. For C programmers, the lint program can help find potential bugs in
programs. Java programmers can use tools like the JTest product to check their programs against a
coding standard.

When to start the Static Testing?

To get value from static testing, we have to start at the right time. For example, reviewing the
requirements after the programmers have finished coding the entire system may help testers design test
cases. However, the significant return on the static testing investment is no longer available, as testers
can't prevent bugs in code that's already written. For optimal returns, a static testing should happen as
soon as possible after the item to be tested has been created, while the assumptions and inspirations
remain fresh in the creator's mind and none of the errors in the item have caused negative consequences
in downstream processes. Effective reviews involve the right people. Business domain experts must attend
requirements reviews, system architects must attend design reviews, and expert programmers must
attend code reviews. As testers, we can also be valuable participants, because we're good at spotting
inconsistencies, vagueness, missing details, and the like. However, testers who attend review meetings do
need to bring sufficient knowledge of the business domain, system architecture, and programming to each
review. And everyone who attends a review, walkthrough or inspection should understand the basic
ground rules of such events.

The following diagram of Somerville (Software Engineering 6th Edition) communicates, where Static
Testing starts.

3. Reviews

IEEE classifies Static Testing under three broad categories:

• Reviews
• Walkthroughs
• Inspections

What is Reviews?

A meeting at which the software element is presented to project personnel, managers, users, customers
or other interested parties for comment or approval. The software element can be Project Plans, URS,
SRS, Design Documents, code, Test Plans, User Manual.

What are objectives of Reviews?

To ensure that:

• The software element conforms to its specifications.

• The development of the software element is being done as per plans, standards, and guidelines
applicable for the project.
• Changes to the software element are properly implemented and affect only those system areas
identified by the change specification.

Reviews - Input

• A statement of objectives for the technical reviews

• The software element being examined
• Software project management plan
• Current anomalies or issues list for the software product
 • Documented review procedures
• Earlier review report - when applicable
• Review team members should receive the review materials in advance and they come prepared for
the meeting
• Check list for defects

Reviews Meeting

• Examine the software element for adherence to specifications and standards

• Changes to software element are properly implemented and affect only the specified areas
• Record all deviations
• Assign responsibility for getting the issues resolved
• Review sessions are not expected to find solutions to the deviations.
• The areas of major concerns, status on previous feedback and review days utilized are also
recorded.
• The review leader shall verify, later, that the action items assigned in the meeting are closed

Reviews - Outputs

• List of review findings

• List of resolved and unresolved issues found during the later re-verification

4. Walkthrough

Walkthrough Definition

A technique in which a designer or programmer leads the members of the development team and other
interested parties through the segment of the documentation or code and the participants ask questions
and make comments about possible errors, violation of standards and other problems.

Walkthrough - Objectives

• To find defects
• To consider alternative implementations
• To ensure compliance to standards & specifications

Walkthrough Input

• A statement of objectives
• The software element for examination
• Standards for the development of the software
• Distribution of materials to the team members, before the meeting
• Team members shall examine and come prepared for the meeting
• Check list for defects

Walkthrough- Meeting

• Author presents the software element

• Members ask questions and raise issues regarding deviations
• Discuss concerns, perceived omissions or deviations from the specifications
• Document the above discussions
• Record the comments and decisions
• The walk-through leader shall verify, later, that the action items assigned in the meeting are closed
Walkthrough Outputs

• List of walk-through findings

• List of resolved and unresolved issues found during the later re-verification

5. Inspection

Inspection Definition

A visual examination of software element to detect errors, violations of development standards and other
problems. An inspection is very rigorous and the inspectors are trained in inspection techniques.
Determination of remedial or investigative action for a defect is a mandatory element of a software
inspection, although the solution should not be determined in the inspection meeting.

Inspection Objectives

• To verify that the software element satisfies the specifications & conforms to applicable standards
• To identify deviations
• To collect software engineering data like defect and effort
• To improve the checklists, as a spin-off

Inspection Input

• A statement of objectives for the inspection

• Having the software element ready
• Documented inspection procedure
• Current defect list
• A checklist of possible defects
• Arrange for all standards and guidelines
• Distribution of materials to the team members, before the meeting
• Team members shall examine and come prepared for the inspection

Inspection Meeting

• Introducing the participants and describing their role (by the Moderator)
• Presentation of the software element by non - author
• Inspectors raise questions to expose the defects
• Recording defects - a defect list details the location, description and severity of the defect
• Reviewing the defect list - specific questions to ensure completeness and accuracy
• Making exit decision

- Acceptance with no or minor rework, without further verification

- Accept with rework verification (by inspection team leader or a designated member of the
inspection team)
- Re-inspect

Inspection - Output

• Defect list, containing a defect location, description and classification

• An estimate of rework effort and rework completion date

6. Comparison of Reviews, Walk-Throughs and Inspections

Objectives:

• Reviews - Evaluate conformance to specifications; Ensure change integrity

 • Walkthroughs - Detect defects; Examine alternatives; Forum for learning
• Inspections - Detect and identify defects; Verify resolution

Group Dynamics:

• Reviews: 3 or more persons ; Technical experts and peer mix

• Walkthroughs: 2 to 7 persons Technical experts and peer mix
• Inspections: 3 to 6 persons; Documented attendance; with formally trained inspectors

Decision Making & Change Control:

• Reviews: Review Team requests Project Team leadership or management to act on

recommendations
• Walkthroughs: All decisions made by producer; Change is prerogative of the author
• Inspections: Team declares exit decision Acceptance, Rework & Verify or Rework & Re-inspect

Material Volume:

• Reviews: Moderate to high

• Walkthroughs: Relatively low
• Inspections: Relatively low

Presenter

• Reviews: Software Element Representative

• Walkthroughs: Author
• Inspections: Other than author

7. Advantages

Advantages of Static Methods over Dynamic Methods

• Early detection of software defects

• Static methods expose defects, whereas dynamic methods show only the symptom of the defect
• Static methods expose a batch of defects, whereas it is usually one by one in dynamic methods
• Some defects can be found only by Static Testing
o Code redundancy (when logic is not affected)
o Dead code
o Violations of coding standards

8. Metrics for Inspections

Fault Density

• Specification and Design

Faults per page
• Code
Faults per 1000 lines of code

Fault Detection Rate

Faults detected per hour

Fault Detection Efficiency

Faults detected per person - hour

Inspection Efficiency

(Number of faults found during inspection) / (Total number of faults during development)

Maintenance Vs Inspection

"Number of corrections during the first six months of operational phase" and "Number of defects found in
inspections" for different projects of comparable size

9. Common Issues for Reviews, Walk-throughs and Inspections

Responsibilities for Team Members

• Leader: To conduct / moderate the review / walkthrough / inspection process

• Reader: To present the relevant material in a logical fashion
• Recorder: To document defects and deviations
• Other Members: To critically question the material being presented

Communication Factors

• Discussion is Kept within bounds

• Not extreme in opinion
• Reasonable and calm
• Not directed at a personal level
• Concentrate on finding defects
• Not get bogged down with disagreement
• Not discuss trivial issues
• Not involve itself in solution-hunting
• The participants should be sensitive to each other by keeping the synergy of the meeting very high

Being aware of, and correcting any conditions, either physical or emotional, that are draining off the
participants attention, shall ensure that the meeting is fruitful i.e. Maximum number of defects is found
during the early stages of software development life cycle.