The Norms and Standard of Software

Documentation

 Xiaoqiang Yan( 闫小强 )

 School of Information Engineering
 Email: [email protected]
 Wechat( 微信 ) ： 13592540976
Lesson 11 Software Testing Strategies

Software engineering standardization and software documentation 2

11.1 A strategic Approach to Software Testing
11.2 Strategic Issues
11.3 Unit Testing
11.4 Integration Testing
11.5 validation Testing
11.6 System Testing
11.7 The Art of Debugging

3
 Overview
 This chapter describes several approaches to testing software. Software
testing must be planned carefully to avoid wasting development time and
resources.
 Testing begins "in the small" and progresses "to the large". Initially
individual components are tested using white box and black box
techniques.
 After the individual components have been tested and added to the system,
integration testing takes place.
 Once the full software product is completed, system testing is performed.
 The Test Specification document should be reviewed like all other software
engineering work products.
 A sample Test Specification document should be submitted in this
semester. 

4
 11.1 Strategic Approach to software Testing

 Testing begins at the component level and works toward the integration of
the entire computer-based system.
 Different testing techniques are appropriate at different points in time.
 The developer of the software conducts testing and may be assisted by
independent test groups for large projects.
 The role of the independent tester is to remove the conflict of interest
inherent when the builder is testing his or her own product.
 Testing and debugging are different activities.
 Debugging must be accommodated in any testing strategy.
 Make a distinction between verification (are we building the product right?)
and validation (are we building the right product?)

5
 Testing Strategy

unit test integration

test

system validation
test test
6
 11.2 Strategic Issues

 Strategic Testing Issues

 Specify product requirements in a quantifiable manner before testing

starts.
 Specify testing objectives explicitly.
 Identify the user classes of the software.
 Develop a test plan that emphasizes rapid cycle testing.
 Build robust software that is designed to test itself.
 Use effective formal reviews as a filter prior to testing.
 Conduct formal technical reviews to assess the test strategy and test
cases

7
 11.3 Unit Testing

 Black box and white box testing.

 Module interfaces are tested for proper information flow.
 Local data are examined to ensure that integrity is
maintained.
 Boundary conditions are tested.
 Basis path testing should be used.
 All error handling paths should be tested.
 Drivers and/or stubs need to be developed to test
incomplete software. 

8
 Unit Testing

module
to be
tested

results
software
engineer test cases

9
 Unit Testing
module
to be
tested
interface
local data structures
boundary conditions
independent paths
error handling paths

test cases
10
 Unit Test Environment
driver
interface
local data structures

Module boundary conditions

independent paths
error handling paths

stub stub

test cases

RESULTS
11
 11.4 Integration Testing

Integration Testing Strategies

Options:
• the “big bang” approach
• an incremental construction strategy

12
 Top-down Integration Testing

1. Main control module used as a test driver and stubs are

substitutes for components directly subordinate to it.
2. Subordinate stubs are replaced one at a time with real
components (following the depth-first or breadth-first approach).
3. Tests are conducted as each component is integrated.
4. On completion of each set of tests and other stub is replaced with
a real component.
5. Regression testing may be used to ensure that new errors not
introduced.

13
 Top Down Integration

A
top module is tested with
stubs

B F G

stubs are replaced one at

a time, "depth first"
C
as new modules are integrated,
some subset of tests is re-run
D E

14
 Bottom-up Integration Testing

1. Low level components are combined in clusters that

perform a specific software function.
2. A driver (control program) is written to coordinate test
case input and output.
3. The cluster is tested.
4. Drivers are removed and clusters are combined
moving upward in the program structure.

15
 Bottom-Up Integration
A

B F G

drivers are replaced one at a

time, "depth first"
C

worker modules are grouped into

builds and integrated
D E

cluster
16
Let’s conclude some difference
between Stubs and Driver:

17
Regression Testing(check for defects propagated to other modules by
changes made to existing program

1. Representative sample of existing test cases is used

to exercise all software functions.

2. Additional test cases focusing software functions likely

to be affected by the change.

3. Tests cases that focus on the changed software

components.

18
 Smoke Testing

1. Software components already translated into code are

integrated into a build.
2. A series of tests designed to expose errors that will
keep the build from performing its functions are
created.
3. The build is integrated with the other builds and the
entire product is smoke tested daily (either top-down or
bottom-up integration may be used). 

19
 Sandwich Testing
A
Top modules are
tested with stubs

B F G

Worker modules are grouped into

builds and integrated
D E

cluster
20
 10.5 Validation Testing

 Ensure that each function or performance characteristic

conforms to its specification.
 Deviations (deficiencies) must be negotiated with the
customer to establish a means for resolving the errors.
 Configuration review or audit is used to ensure that all
elements of the software configuration have been
properly developed, cataloged, and documented to
allow its support during its maintenance phase.

21
 Acceptance Testing

 Making sure the software works correctly for intended

user in his or her normal work environment.
 Alpha test (version of the complete software is tested by
customer under the supervision of the developer at the
developer site)
 Beta test (version of the complete software is tested by
customer at his or her own site without the developer
being present)

22
 10.6 System Testing

 Recovery testing (checks the system ability to recover from

failures)

 Security testing (verifies that system protection mechanism prevent

improper penetration or data alteration)

 Stress testing (program is checked to see how well it deals with

abnormal resource demands ?quantity, frequency, or volume)

 Performance testing (designed to test the run-time performance of

software, especially real-time software
23
 10.7 The Art of Debugging

 Debugging

 Debugging (removal of a defect) occurs as a consequence of

successful testing.
 Some people are better at debugging than others.
 Common approaches:

1. Brute force (memory dumps and run-time traces are examined for clues to
error causes)
2. Backtracking (source code is examined by looking backwards from
symptom to potential causes of errors)
3. Cause elimination (uses binary partitioning to reduce the number of
locations potential where errors can exist)
24
Debugging: A Diagnostic Process

25
 The Debugging Process
test cases

new test results

regression cases
tests suspected
causes
corrections
Debugging
identified
causes

26
 Debugging Effort

time required
to diagnose the
time required symptom and
to correct the error determine the
and conduct cause
regression tests

27
 Symptoms & Causes
symptom and cause may be
geographically separated

symptom may disappear when

another problem is fixed
cause may be due to a
combination of non-errors

cause may be due to a system

or compiler error
cause may be due to
symptom assumptions that everyone
cause believes

symptom may be intermittent

28
 Consequences of Bugs

infectious
damage
catastrophic
extreme
serious
disturbing
annoying
mild
Bug Type
Bug Categories: function-related bugs, system-related bugs, data
coding bugs, design
bugs,bugs, documentation bugs, standards violations,
etc.
29
Debugging Techniques

brute force / testing

backtracking

induction

deduction

30
 Bug Removal Considerations

 Is the cause of the bug reproduced in another part of the

program?

 What "next bug" might be introduced by the fix that is

being proposed?

 What could have been done to prevent this bug in the

first place?

31
 Debugging vs. testing
 The purpose of testing is to identify what
happens when there is a mistake in a program's
source code.
 The purpose of debugging is to locate and fix the
mistake.
Thanks for coming !