Code Reviews & Static Software

Analysis
Code Reviews & Static Software Analysis $ Testing Techniques

Topics

Introduction
• Types of Reviews along the software life cycle
• Reviews and testing
• Review planning
• Review roles, responsibilities and attendance

Types of reviews according to formality

Checklists

Reporting and follow-up

Other static software analysis techniques

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Types of reviews
Target / Review Item
(What)
Requirements review

Design review

Code review

User documentation review

[Proj. Man. | Config. Man. | QA | V&V | Test |...]

[plan | report] review
not the
focus Formality
here (How and
detect errors and problems
Who)

in rev
pe

au
de ec

sp
al
check conformity with specification

ch

er

di
sk k

kt

ec w
and fitness for purpose

t
-

hr

tio
ou
check quality attributes

ie

n
gh
and detect quality faults V&V and QA
check adherence to standards
check
not the progress
focus here Purpose / Goals
(Why)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Desk check

Also called self check

Informal review performed by the author of the artifact

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Peer reviews

“I show you mine and you show me yours”

The author of the reviewed item does not participate in
the review

Effective technique that can be applied when there is a
team (with two or more persons) for each role (analyst,
designer, programmer, technical writer, etc.)

The peer may be a senior colleague (senior/chief
analyst, senior/chief architect, senior/chief programmer,
senior/chief technical writer, etc.)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Walkthroughs

Type of technical review where the producer of the
reviewed material serves as the review leader and
actually guides the progression of the review (as a
review reader)

Traditionally applied to design and code

In the case of code walkthrough, test inputs may be
selected and review participants then literally walk
through the design or code

Checklist and preparation steps may be eliminated

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Inspections

A formal evaluation technique in which software requirements,
design, or code are examined in detail by a person or group other
than the author to detect faults, violations of development
standards, and other problems

Generally involve the author of a product

The inspector team may consist of different expertise, such as
domain expertise, or design method expertise, or language
expertise, etc. Inspections are usually conducted on a relatively
small section of the product.

Often the inspection team may have had a few hours to prepare,
perhaps by applying an analytic technique to a small section of the
product, or to the entire product with a focus only on one aspect,
e.g., interfaces.

A checklist, with questions germane to the issues of interest, is a
common tool used in inspections.

Inspection sessions can last a couple of hours or less, whereas
reviews and audits are usually broader in scope and take longer.

(source : SWEBOK)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Audits

An audit is an independent evaluation of conformance
of software products and processes to applicable regulations,
standards, plans, and procedures

An audit is a formally organized activity, with participants
having specific roles, such as lead auditor, other auditors, a
recorder, an initiator, and a representative of the audited
organization

Audits may examine plans like recovery, SQA, design
documentation, etc.

Audits can occur on almost any product at any stage of the
development or maintenance process
(source : SWEBOK)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Software reviews and the extended V-

model of software development Execute
acceptance
tests
Specify Execute
Requireme system
nts Requireme System/acceptance tests
nts review test plan & test
cases review/audit
Specify/Design
Code System/acceptance tests
Execute
Design integration
Integration tests
Design test plan & test
review cases
Specify/Design review/audit
revisit Code Integration tests
ed
Execute
Code unit tests
Unit
Code test plan & test
reviews cases review/audit
Specify/Design Code (Source: I. Burnstein, page
Unit tests 15)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Typical tests and reviews

(source: "Software Project
Survival Guide", Steve
revisit McConnell)
ed

high-level design

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Reviews and testing


A software system is more than the code; it is a set of related
artifacts; these may contain defects or problem areas that should be
reworked or removed; quality-related attributes of these artifacts
should be evaluated

Reviews allow us to detect and eliminate errors/defects early in the
software life cycle (even before any code is available for testing),
where they are less costly to repair

Most problems have their origin in requirements and design;
requirements and design artifacts can be reviewed but not executed
and tested
• Early prototyping is equally important to reveal problems in requirements
and high-level architectural design

A code review usually reveals directly the location of a bug, while
testing requires a debugging step to locate the origin of a bug

Adherence to coding standards cannot be checked by testing

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Technical and management

reviews

Technical Reviews - examine work products of the software project
(code, requirement specifications, software design documents, test
documentation, user documentation, installation procedures) for
V&V and QA purposes
• Multiple forms: Desk checking, Walkthroughs, Inspections, Peer Reviews,
Audits
• Covered here

Management Reviews - determine adequacy of and monitor
progress or inconsistencies against plans and schedules and
requirements
• Includes what Ian Somerville calls Progress Reviews
• May be exercised on plans and reports of many types (risk
management plans, project management plans, software configuration
management plans, audit reports, progress reports, V&V reports, etc.)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Components of a review plan


Review goals

Items being reviewed

Preconditions for the review

Roles, team size, participants

Training requirements

Review steps and procedures

Checklists and other related documents to be distributed to
participants

Time requirements

Nature of the review log and summary report

Rework and follow-up
(Source: I. Bursntein)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques
IEEE Standard for Software Reviews and
Audits (IEEE Std 1028-1988)

Specialize
d meaning

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Types of Reviews in IEEE Std 1028-

1988

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

 Code Reviews & Static Software Analysis $ Testing Techniques

A sample general checklist for

reviewing software documents

Coverage and completeness 
Clarity and Consistency (cont.)
• Are all essential items completed? • Are the diagrams, graphs and
• Have all irrelevant items been illustrations clear, correct, use the
proper notation, effective, in the
omitted? proper place?
• Is the technical level of each topic • Is the terminology clear and correct?
addressed properly for this • Is there a glossary of technical terms
document? that is complete and correct?
• Is there a clear statement of goals • Is the writing style clear
for this document? (nonambiguous)?
• (Don't forget: more documentation 
References and Aids to Document
does not mean better Comprehension
documentation) • Is there an abstract or introduction?

Correctness • Is there a well placed table of
• Are there incorrect items? contents?
• Are the topics or items broken down
• Are there any contradictions?
in a manner that is easy to follow and
• Are the any ambiguities? is understandable?
• Is there a bibliography that is clear,

Clarity and Consistency complete and correct?
• Are the material and statements in • Is there an index that is clear,
the document clear? complete and correct?
• Are the examples clear, useful, • Is the page and figure numbering
relevant
Adapted from Ilene and correct?
Burnstein, Practical Software Testing, page 327) correct and consistent?

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

A sample specification (or

requirements) attributes checklist
Attribute What to consider
Is anything missing or forgotten? Is it thorough? Does it include everything
Complete
necessary to make it stand alone?
Is the proposed solution correct? Does it properly define the goal? Are there
Accurate
any errors?
Precise,
Unambiguo Is the description exact and not vague? Is there a single interpretation? Is it
us and easy to read and understandable?
Clear
Is the description of the feature written so that it doesn't conflict with itself
Consistent
or other items in the specification?
Is the statement necessary to specify the feature? Is there extra information
Relevant that should be left out? Is the feature traceable to an original customer
need?
Can the feature be implemented with the available personnel, tools, and
Feasible
resources within the specified budget and schedule?
Does the specification stick with defining the product and not the underlying
Code-free
software design, architecture, and code?
Can the feature be tested? Is enough information provided that a tester
Testable
could create tests to verify its operation?
(Adapted from: Ron Patton, Software Testing)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques
A sample supplementary checklist for design
reviews
(for high-level architectural design and detailed design)

Are the high-level and detailed design consistent with requirements? Do they address
all the functional and quality requirements? Is detailed design consistent with high-level
design?

Are design decisions properly highlighted and justified and traced back to
requirements? Are design alternatives identified and evaluated?

Are design notations (ex: UML), methods (ex: OOD, ATAM) and standards chosen and
used adequately?

Are naming conventions being followed appropriately?

Is the system structuring (partitioning into sub-systems, modules, layers, etc.) well
defined and explained? Are the responsibilities of each module and the relationships
between modules well defined and explained? Do modules exhibit strong cohesion and
weak coupling?

Is there a clear and rigorous description of each module interface, both at the syntactic
and semantic level? Are dependencies identified?

Have user interface design issues, including standardization, been addressed properly?

Is there a clear description of the interfaces between this system and other software
and hardware systems?

Have reuse issues been properly addressed, namely the possible reuse of COTS
(commercial off the shelf) components (buy-or-build decision) and in-house reusable
components?
(Adapted from: Ilene Burnstein, page 328-329)

Is the system designedwww.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>>> so that it can be tested at various levels (unit, integration and
<<<<<<<<<<<<<<<<<<<<<<<
Code Reviews & Static Software Analysis $ Testing Techniques

A sample general code review

checklist (1)

Design Issues
• Does each unit implement a single function?
• Are there instances where the unit should he partitioned?
• Is code consistent with detailed design?
• Does the code cover detailed design?

Data Items
• Is there an input validity check?
• Arrays-check array dimensions, boundaries, indices.
• Variables - are they all defined, initiated? have correct types and scopes been
checked?
• Are all variables used?

Computations
• Are there computations using variables with inconsistent data types?
• Are there mixed-mode computations?
• Is the target value of an assignment smaller than the right-hand expression?
• Is over- or underflow a possibility (division by zero)?
• Are there invalid uses of integers or floating point arithmetic?
• Are there comparisons between floating point numbers?
• Are there assumptions about the evaluation order in Boolean expressions?
• Are the comparison operators correct?

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

A sample general code review

checklist (2)

Control Flow Issues
• Will the program, module or, unit eventually terminate?
• Is there a possibility of an infinite loop, a loop with a premature exit, a loop that
never executes?

Interface Issues
• Do the number and attributes of the parameters used by a caller match those of
the called routine? Is the order of parameters also correct and consistent in caller
and callee?
• Does a function or procedure alter a parameter that is only meant as an input
parameter?
• If there are global variables, do they have corresponding definitions and attributes
in all the modules that use them?

Input/output Issues
• Have all files been opened for use?
• Are all files properly closed at termination?
• If files are declared are their attributes correct?
• Are EOF or I/O errors conditions handed correctly?
• Is I/O buffer size and record size compatible?

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

A sample general code review

checklist (3)

Portability Issues
• Is there an assumed character set, and integer or floating point representation?
• Are their service calls that mar need to be modified?

Error Messages
• Have all warnings and informational messages been checked and used
appropriately?

Comments/Code Documentation
• Has the code been properly documented? Are there global, procedure, and line
comments where appropriate?
• Is the documentation clear, and correct, and does it support understanding?

Code Layout and White Space
• Has white space and indentation been used to support understanding of code logic
and code intent?

Maintenance
• Does each module have a single exit point?
• Are the modules easy to change (low coupling and high cohesion)?

(Adapted from: Ilene Burnstein, page 331)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

A sample code review checklist for C++

programs (1)

Data Items
• Are all variables lowercase?
• Are all variables initialized?
• Are variable names consistent, and do they reflect usage?
• Are all declarations documented (except for those that are very simple to
understand)?
• Is each name used for a singe function (except for loop variable names)?
• Is the scope of the variable as intended?

Constants
• Are all constants in uppercase?
• Are all constants defined with a "#define"?
• Are all constants used in multiple files defined in an INCLUDE header file?

Pointers
• Are pointers declared properly as pointers?
• Are the pointers initialized properly?

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

A sample code review checklist for C

programs (2)

Control
• Are if/then, else, and switch statements used clearly and properly?

Strings
• Strings should have proper pointers.
• Strings should end with a NULL.

Brackets
• All curly brackets should have appropriate indentations and be matched

Logic Operators
• Do all initializations use an " = " and not an " = ="?
• Check to see that all logic operators are correct, for example, use of = / = =, and ||

Computations
• Are parentheses used in complex expressions and are they used properly for
specifying precedences?
• Are shifts used properly?
(Adapted from: Ilene Burnstein, page. 331)

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Types of (end-user) software

documentation(1)

Packaging text and graphics. Box, carton, wrapping, and so on. Might
contain screen shots from the software, lists of features, system
requirements, and copyright information.

Marketing material, ads, and other inserts. These are all the pieces of
paper you usually throw away, but they are important tools used to promote
the sale of related software, add-on content, service contracts, and so on.
The information for them must be correct for a customer to take them
seriously.

Warranty/registration. This is the card that the customer fills out and
sends in to register the software. It can also be part of the software and
display onscreen for the user to read, acknowledge, and even complete
online.

EULA. Pronounced "you-la," it stands for End User License Agreement. This is
the legal document that the customer agrees to that says, among other
things, that he won't copy the software nor sue the manufacturer if he's
harmed by a bug. The EULA is sometimes printed on the envelope containing
the media-the floppy or CD. It also may pop up onscreen during the
software's installation.

Labels and stickers. These may appear on the media, on the box, or on the
printed material. There may also be serial number stickers and labels that
seal the EULA envelope. See in a following slide an example of a disk label
and all the information that needs to be checked.

Installation and setup instructions. Sometimes this information is
printed on the media, but it also can be included as a separate sheet of
paper or, if it's complex
>>>>>>>>>>>>>>>>>>>>>>> software, as an entire manual.
www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<
Code Reviews & Static Software Analysis $ Testing Techniques

Types of (end-user) software

documentation (2)

User's manual. The usefulness and flexibility of online manuals has made
printed manuals much less common than they once were. Most software now
comes with a small, concise "getting started"-type manual with the detailed
information moved to online format. The online manuals can be distributed on
the software's media, on a Web site, or a combination of both.

Online help. Online help often gets intertwined with the user's manual,
sometimes even replacing it. Online help is indexed and searchable, making it
much easier for users to find the information they're looking for. Many online
help systems allow natural language queries so users can type "Tell me how
to copy text from one program to another" and receive an appropriate
response.

Tutorials, wizards, and CBT (Computer Based Training). These tools
blend programming code and written documentation. They're often a mixture
of both content and high-level, macro-like programming and are often tied in
with the online help system. A user can ask a question and the software then
guides him through the steps to complete the task. Microsoft's Office
Assistant, sometimes referred to as the "paper clip guy" is an example of
such a system.

Samples, examples, and templates. An example of these would be a word
processor with forms or samples that a user can simply fill in to quickly create
professional-looking results. A compiler could have snippets of code that
demonstrate how to use certain aspects of the language.

Error messages. Often neglected; ultimately fall under the category of
documentation.
(Adapted from: Ron Patton, Software Testing, page 190-192)
>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<
Code Reviews & Static Software Analysis $ Testing Techniques
Quality attributes (or dimensions) to
check in technical information

Can be checked by asking

probing questions, like:
• Is the information
appropriate for the
intended audience?
• Is information presented
from a user’s point of
view?
• Is there a focus on real
tasks?
• Is the reason for the
information evident?
• Do titles and headings
reveal real tasks?

Build your own check

list! Adapt to your
needs!

Source: Developing Quality Technical Information (DQTI), Hargis, IBM, 1997

not only for software, not only for end-user
documentation (also documentation for developers
and maintainers)
>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<
Code Reviews & Static Software Analysis $ Testing Techniques

Contents of a formal review report

(1)

Checklist will all items covered (with a check mark) and
comments relating to each item

List of defects found, with
• description
• type
• frequency
• defect class, e.g.
- missing
- incorrect
- superfluous
• location
- cross-reference to the place or places in the reviewed document where the
defect occurs
• severity, e.g.
- major
- minor

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Contents of a formal review report

(2)

Summary report, with
• list of attendees
• review metrics, such as
- number of participants
- duration of the meeting
- size of the item being reviewed (usually LOC or number of pages)
- number of defects found
- total preparation time for the review team
- number of defects found per hour of review time
- number of defects found per page or LOC
- LOC or pages reviewed per hour
- ...
• status of the reviewed item (requirements document, etc.)
- accept – the item is accepted in its present form or with minor rework required
that does not need further verification
- conditional accept – the item needs rework and will be accepted after the
moderator has checked and verified the rework
- reinspect – considerable rework must be done to the item. The inspection
needs to be repeated when the rework is done.
• estimate of rework effort and the estimated date for completion of the
rework
• signatures and date
>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<
Code Reviews & Static Software Analysis $ Testing Techniques

Automated static software analysis

(1)

Static code analysis and audit tools
• rule based - perform checks that result in observations on coding
practices; look for constructs that "look dangerous"
• metric based – perform checks that result in observations on code quality
metrics values such as Cyclomatic Complexity and Nesting Depth

Formal proofs
• based on mathematics
• may be partially automated (or at least supported by tools that check the
internal consistency of the proof)

Model checking
• based on a finite state model of the system
• tools automate proof of properties such as reachability and absence of
cycles

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<

Code Reviews & Static Software Analysis $ Testing Techniques

Thank You

>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<<