Chapter 13 & 14

Software Testing Strategies

and Techniques
Software Engineering: A Practitioner’s Approach, 6th edition
by Roger S. Pressman

1
 Software Testing
Testing is the process of exercising a
program with the specific intent of finding
errors prior to delivery to the end user.

2
What Testing Shows
errors
requirements conformance

performance

an indication
of quality

3
 Who Tests the Software?

developer independent tester

Understands the system Must learn about the system,
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by quality

4
Validation vs Verification
 Verification – Are we building the
product right?
 Is the code correct with respect to its
specification?

 Validation – Are we building the right

product?
 Does the specification reflect what it should?
5
 Testing Strategy
unit test integration
test

system validation
test test
6
Testing Strategy
 Begin with unit testing and work your way up to
system testing.
 Unit testing – test individual components (modules in
procedural languages; classes in OO languages)
 Integration testing – test collections of components
that must work together
 Validation testing – test the application as a whole
against user requirements
 System testing – test the application in the context of
an entire system

7
Unit Testing

module
to be
tested

results
software
engineer
test cases

8
Unit Testing
module
to be
tested
interface
local data structures
boundary conditions
independent paths
error handling paths

test cases 9
 Unit Test Environment
driver
interface
local data structures
Module boundary conditions
independent paths
error handling paths
stub stub

test cases
RESULTS 10
Integration Testing Strategies

Options:
• the “big bang” approach
• an incremental construction strategy

11
Top Down Integration
A
top module is tested with
stubs

B F G

stubs are replaced one at

a time, "depth first"
C
as new modules are integrated,
some subset of tests is re-run
D E

12
Bottom-Up Integration
A

B F G

drivers are replaced one at a

time, "depth first"
C

worker modules are grouped into

builds and integrated
D E

cluster
13
Regression Testing
 The selective retesting of a modified
system to help ensure that no bugs
have been introduced during
modification.

 Fixing one part of the code can break

another
14
High Order Testing
 Validation testing
 Focus is on software requirements
 System testing
 Focus is on system integration
 Alpha/Beta testing
 Focus is on customer usage
 Recovery testing
 forces the software to fail in a variety of ways and verifies that recovery is properly performed
 Security testing
 verifies that protection mechanisms built into a system will, in fact, protect it from improper
penetration
 Stress testing
 executes a system in a manner that demands resources in abnormal quantity, frequency, or volume
 Performance Testing
 test the run-time performance of software within the context of an integrated system

15
 What is a “Good” Test?
 A good test is one that has a high
probability of finding an error.

16
 Test Case Design

"Bugs lurk in corners

and congregate at
boundaries ..."
Boris Beizer

OBJECTIVE to uncover errors

CRITERIA in a complete manner

CONSTRAINT with a minimum of effort and time

17
Exhaustive Testing

loop < 20 X

14
There are 10 possible paths! If we execute one
test per millisecond, it would take 3,170 years to
test this program!! 18
Selective Testing
Selected path

loop < 20 X

19
Software Testing
white-box black-box
methods methods

Methods

Strategies

20
 White-Box Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...
21
 Why Cover?
logic errors and incorrect assumptions
are inversely proportional to a path's
execution probability

we often believe that a path is not

likely to be executed; in fact, reality is
often counter intuitive

typographical errors are random; it's

likely that untested paths will contain
some
22
Basis Path Testing
First, we compute the cyclomatic
complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

23
 Cyclomatic Complexity
A number of industry studies have indicated
that the higher V(G), the higher the probability
or errors.

modules

V(G)

modules in this range are

more error prone
24
 Basis Path Testing
Next, we derive the
independent paths:
1

Since V(G) = 4,
2 there are four paths

3 Path 1: 1,2,3,6,7,8
4
5 6 Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
7
Finally, we derive test
cases to exercise these
8
paths. 25
Basis Path Testing Notes
you don't need a flow chart,
but the picture will help when
you trace program paths

count each simple logical test,

compound tests count as 2 or
more

basis path testing should be

applied to critical modules

26
Black-Box Testing
requirements

output

input events

27
Equivalence Partitioning

user output FK
queries formats input
mouse data
picks prompts

28
Sample Equivalence
Classes
Valid data
user supplied commands
responses to system prompts
file names
computational data
physical parameters
bounding values
initiation values
output data formatting
responses to error messages
graphical data (e.g., mouse picks)
Invalid data
data outside bounds of the program
physically impossible data
proper value supplied in wrong place
29
 Boundary Value
Analysis

user output FK
queries formats input
mouse data
picks prompts

output
input domain domain

30
OOT Methods: Behavior Testing

The tests to be empty set up

designed should open acct setup Accnt acct

achieve all state deposit

coverage [KIR94]. (initial)

That is, the deposit

operation working
sequences should balance
acct
withdraw
cause the credit
accntInfo
Account class to withdrawal
make transition (final)

through all dead nonworking

allowable states acct close acct

Figure 1 4 .3 St at e diagram f or A ccount class ( adapt ed f rom [ KIR9 4 ] )

31