Chapter 19

 SOFTWARE TESTING—COMPONENT
LEVEL
Slide Set to accompany
Software Engineering: A Practitioner’s Approach, 9/e
by Roger S. Pressman and Bruce R. Maxim

1
Software Testing

Testing is the process of exercising

a program with the specific intent of
finding errors prior to delivery to the
end user.

2
What Testing Shows
errors
requirements conformance

performance

an indication
of quality

3
Strategic Approach
 To perform effective testing, you should conduct
effective technical reviews. By doing this, many
errors will be eliminated before testing commences.
 Testing begins at the component level and works
"outward" toward the integration of the entire
computer-based system.
 Different testing techniques are appropriate for
different software engineering approaches and at
different points in time.
 Testing is conducted by the developer of the
software and (for large projects) an independent
test group (ITG).
 Testing and debugging are different activities, but
debugging must be accommodated in any testing
strategy.

4
V&V
 Verification refers to the set of tasks that
ensure that software correctly implements
a specific function.
 Validation refers to a different set of tasks
that ensure that the software that has been
built is traceable to customer
requirements. Boehm [Boe81] states this
another way:
 Verification: "Are we building the product
right?"
 Validation: "Are we building the right product?"

5
Who Tests the Software?

developer independent tester

Understands the system Must learn about the system,
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by quality

6
Testing Strategy

7
Software testing steps

8
Testing Strategy
 We begin by ‘testing-in-the-small’ and move
toward ‘testing-in-the-large’
 For conventional software
 The module (component) is our initial focus
 Integration of modules follows
 For OO software
 our focus when “testing in the small” changes from
an individual module (the conventional view) to an
OO class that encompasses attributes and
operations and implies communication and
collaboration

9
Strategic Issues
 Specify product requirements in a quantifiable manner
long before testing commences.
 State testing objectives explicitly.
 Understand the users of the software and develop a
profile for each user category.
 Develop a testing plan that emphasizes “rapid cycle
testing.”
 Build “robust” software that is designed to test itself
 Use effective technical reviews as a filter prior to testing
 Conduct technical reviews to assess the test strategy
and test cases themselves.
 Develop a continuous improvement approach for the
testing process.

10
Unit Testing

module
to be
tested

results

software
engineer
test cases

11
Unit test

12
Unit Test Environment
driver
interface
local data structures

Module boundary conditions

independent paths
error handling paths

stub stub

test cases

RESULTS
13
Test Case Design
"Bugs lurk in corners
and congregate at
boundaries ..."
Boris Beizer

OBJECTIVE to uncover errors

CRITERIA in a complete manner

CONSTRAINT with a minimum of effort and time

14
Software Testing

white-box black-box
methods methods

Methods

Strategies

15
White-Box Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...
16
White-Box Testing
Derive test cases that
guarantee that all independent paths within a
module have been exercised at least once,
exercise all logical decisions on their true and
false sides,
execute all loops at their boundaries and
within their operational bounds, and
exercise internal data structures to ensure
their validity.
Also called glass-box testing or
structural testing.
17
Why Cover?
logic errors and incorrect assumptions
are inversely proportional to a path's
execution probability

we often believe that a path is not

likely to be executed; in fact, reality is
often counter intuitive

typographical errors are random; it's

likely that untested paths will contain
some

18
Basis Path Testing
First, we compute the cyclomatic
complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

19
Basis Path Testing

20
Cyclomatic Complexity
A number of industry studies have indicated
that the higher V(G), the higher the probability
or errors.

modules

V(G)

modules in this range are

more error prone

21
Basis Path Testing
1
Next, we derive the
independent paths:

2 Since V(G) = 4,
there are four paths
3 Path 1: 1,2,3,6,7,8
4
5 6
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive test
7 cases to exercise these
paths.

22
Basis Path Testing Notes
you don't need a flow chart,
but the picture will help when
you trace program paths

count each simple logical test,

compound tests count as 2 or
more

basis path testing should be

applied to critical modules

23
Control Structure Testing
Condition testing > a test-case design method > logical.
Data flow testing > selects test paths of a program.
Loop testing > the validity of loop constructs. > simple
loops and nested loops.

24
Black-Box Testing/Behavioral o
functional testing
requirements

output

input events

25
Black-Box Testing
Black-box testing attempts to find errors
in the following categories:
incorrect or missing functions,
interface errors,
errors in data structures or external
database access,
behavior or performance errors, and
initialization and termination errors.

26
Black-Box Testing
 How is functional validity tested?
 How is system behavior and performance tested?
 What classes of input will make good test cases?
 Is the system particularly sensitive to certain input
values?
 How are the boundaries of a data class isolated?
 What data rates and data volume can the system
tolerate?
 What effect will specific combinations of data have on
system operation?

27
Equivalence Partitioning

user output FK
queries formats input
mouse
picks data
prompts

28
Sample Equivalence Classes
Valid data
user supplied commands
responses to system prompts
file names
computational data
physical parameters
bounding values
initiation values
output data formatting
responses to error messages
graphical data (e.g., mouse picks)

Invalid data
data outside bounds of the program
physically impossible data
proper value supplied in wrong place
29
Boundary Value Analysis
(BVA)

user output FK
queries formats input
mouse
picks data
prompts

output
input domain domain
30
Object-Oriented Testing
 begins by evaluating the correctness and
consistency of the analysis and design models
 testing strategy changes
 the concept of the ‘unit’ broadens due to
encapsulation
 integration focuses on classes and their execution
across a ‘thread’ or in the context of a usage
scenario
 validation uses conventional black box methods
 test case design draws on conventional
methods, but also encompasses special
features

31
OO Testing Strategy
 class testing is the equivalent of unit testing
 operations within the class are tested
 the state behavior of the class is examined
 integration applied three different strategies
 thread-based testing—integrates the set of
classes required to respond to one input or event
 use-based testing—integrates the set of classes
required to respond to one use case
 cluster testing—integrates the set of classes
required to demonstrate one collaboration

32
 OO Testing: Behavioral Testing
 State diagram for the Account class

The tests to be
designed
should achieve
all state
coverage
[KIR94]. That
is, the
operation
sequences
should cause
the Account
class to make
transition
through all
allowable
states
33
Key points:
 For conventional software >Unit and integration
tests
 Testing Object-oriented software > Test a class and
thread-based (sets of classes) testing for integration
 test-case design: white-box testing and black-box
testing.
 White-box tests > control structure > Basis path
testing > Condition and data flow testing
 Black-box tests > functional requirements > focus on
the information domain > Equivalence partitioning
and Boundary value analysis.

34
Chapter 20
 SOFTWARE TESTING— INTEGRATION
LEVEL
Slide Set to accompany
Software Engineering: A Practitioner’s Approach, 9/e
by Roger S. Pressman and Bruce R. Maxim

35
What is a “Good” Test?
 has a high probability of finding an
error
 is not redundant.
 should be “best of breed”
 should be neither too simple nor too
complex

36
Integration Testing Strategies
Options:
• the “big bang” approach (Chaos!)
• an incremental construction strategy

37
Top Down Integration
A
top module is tested with
stubs

B F G

stubs are replaced one at

a time, "depth first"
C
as new modules are integrated,
some subset of tests is re-run
D E

38
Top Down Integration
 The main control module > as a test driver; stubs
> substitutes for all subordinate components.
 Select integration approach > depth/breadth first,
subordinate stubs > replaces one at a time with
actual components.
 Conduct tests, as each component is integrated.
 Another stub is replaced with the real component.
 Regression testing may be conducted.

39
Bottom-Up Integration
A

B F G

drivers are replaced one at a

time, "depth first"
C

worker modules are grouped into

builds and integrated
D E

cluster

40
Bottom-Up Integration

41
Bottom-Up Integration
Eliminates the need for complex stubs.
Low-level components > clusters (builds).
A driver to coordinate test-case input and
output.
The cluster is tested.
Drivers are removed and clusters are
combined > moving upward.

42
Regression Testing
 the re-execution of some subset of tests that
have already been conducted to ensure that
changes have not propagated unintended side
effects
 Whenever software is corrected, some aspect
of the software configuration (the program, its
documentation, or the data that support it) is
changed.
 to ensure that changes (due to testing or for
other reasons) do not introduce unintended
behavior or additional errors.
 may be conducted manually, by re-executing a
subset of all test cases or using automated
capture/playback tools.

43
Smoke Testing/Continuous
Integration
common approach for creating “daily builds” for product
software
Smoke testing steps:
 Software components that have been translated
into code are integrated into a “build.”
 A series of tests is designed to expose errors that
will keep the build from properly performing its
function.
 The build is integrated with other builds and the
entire product (in its current form) is smoke tested
daily.
 The integration approach may be top down or
bottom up.

44
Object-Oriented Testing
 begins by evaluating the correctness and
consistency of the analysis and design models
 testing strategy changes
 the concept of the ‘unit’ broadens due to
encapsulation
 integration focuses on classes and their
execution across a ‘thread’ or in the context of
a usage scenario
 validation uses conventional black box
methods
 test case design draws on conventional methods,
but also encompasses special features

45
OO Testing Strategy
 class testing is the equivalent of unit testing
 operations within the class are tested
 the state behavior of the class is examined
 integration applied three different strategies
 thread-based testing—integrates the set of
classes required to respond to one input or event
 use-based testing—integrates the set of classes
required to respond to one use case
 cluster testing—integrates the set of classes
required to demonstrate one collaboration

46
Software Testing Patterns
 Testing patterns are described in much the same
way as design patterns.
 Example:
• Pattern name: ScenarioTesting
• Abstract: Once unit and integration tests
have been conducted, there is a need to
determine whether the software will
perform in a manner that satisfies users.
The ScenarioTesting pattern describes a
technique for exercising the software from
the user’s point of view. A failure at this
level indicates that the software has failed
to meet a user visible requirement. [Kan01]

47
Key points:
 Integration testing builds the software architecture >
interfacing between software components.
 white-box testing > “testing in the small.”
 Black-box testing > “testing in the large.”
 Integration testing of OO software > using a thread-
based or use-based strategy.
 Scenario-based testing dominates the validation of
OO systems,
 Regression testing > re-executing a selected test
case following any change
 Testing patterns describe common testing problems
and solutions
48