SOFTWARE TESTING

Lecture 9
INTRODUCTION

 Software Testing is the process of executing a program or

system with the intent of finding errors.

 It involves any activity aimed at evaluating an attribute or

capability of a program or system and determining that it
meets its required results
TESTING OBJECTIVES

1. Testing is a process of executing a program with the

intent of finding an error.
2. A good test case is one that has high probability of
finding an undiscovered error.
3. A successful test is one that uncovers an as-yet
undiscovered error.

 The major testing objective is to design tests that

systematically uncover types of errors with minimum
time and effort.
TEST CHARACTARISTICS

 A good test has a high probability of finding an error

 The tester must understand the software and how it might fail

 A good test is not redundant

 Testing time is limited; one test should not serve the same purpose as
another test

 A good test should be “best of breed”

 Tests that have the highest likelihood of uncovering a whole class of errors
should be used

 A good test should be neither too simple nor too complex

 Each test should be executed separately; combining a series of tests could
cause side effects and mask certain errors
CHARACTERISTICS OF TESTABLE
SOFTWARE
 Operable
 The better it works (i.e., better quality), the easier it is to test

 Observable
 Incorrect output is easily identified; internal errors are automatically
detected

 Controllable
 The states and variables of the software can be controlled directly by the
tester

 Decomposable
 The software is built from independent modules that can be tested
independently
CHARACTERISTICS OF TESTABLE
SOFTWARE
 Simple
 The program should exhibit functional, structural, and code simplicity

 Stable
 Changes to the software during testing are infrequent and do not invalidate
existing tests

 Understandable
 The architectural design is well understood; documentation is available and
organized
LEVELS OF TESTING

• Unit Testing
• Integration Testing
• Validation Testing
• Acceptance Testing
UNIT TESTING

• Algorithms and logic

• Data structures (global and local)
• Interfaces
• Independent paths
• Boundary conditions
• Error handling
WHY INTEGRATION TESTING IS
NECESSARY
• One module can have an adverse effect on another
• Subfunctions, when combined, may not produce the
desired major function
• Interfacing errors not detected in unit testing may appear
• Timing problems (in real-time systems) are not detectable
by unit testing
VALIDATION TESTING
• Determine if the software meets all of the requirements
defined in the SR
• Having written requirements is essential
• Regression testing involves selectively repeating existing
validation tests, not developing new tests
ACCEPTANCE TESTING
• Similar to validation testing except that customers are
present or directly involved.
• Usually the tests are developed by the customer
TEST TECHNIQUE

• White box or glass box testing

• Black box testing
 TWO UNIT TESTING
TECHNIQUES

 Black-box testing
 Knowing the specified function that a product has been designed to perform, test to see if
that function is fully operational and error free
 Not concerned with internal logical structure of the software

 White-box testing
 Knowing the internal workings of a product, test that all internal operations are
performed according to specifications and all internal components have been exercised
 Logical paths through the software are tested
 Test cases exercise specific sets of conditions and loops

13
WHITE-BOX TESTING
 WHITE BOX TESTING

Test cases are derived from the internal design specification or actual
code for the program.

Advantages
 Tests the internal details of the code;
 Checks all paths that a program can execute.

Limitations
 Wait until after designing and coding the program under test in order to select
test cases.
WHITE BOX TESTING
WHITE BOX TESTING

 White-box test design techniques include:

 Control flow testing
 Data flow testing
 Branch testing
 Path testing
 WHITE-BOX TESTING

 These test cases

 Guarantee that all independent paths within a module have been
exercised at least once
 Exercise all logical decisions on their true and false sides
 Execute all loops at their boundaries and within their operational
bounds
 Exercise internal data structures to ensure their validity

18
 BASIS PATH TESTING

• White-box testing technique proposed by Tom McCabe enables the test case
designer to derive a logical complexity measure of a procedural design

• Uses this measure as a guide for defining a basis set of execution paths

• Test cases derived to exercise the basis set are guaranteed to execute every
statement in the program at least one time during testing

19
 FLOW GRAPH NOTATION

 A circle in a graph represents a node, which stands for a sequence of one or more
procedural statements
 A node containing a simple conditional expression is referred to as a predicate node
 Each compound condition in a conditional expression containing one or more Boolean
operators (e.g., and, or) is represented by a separate predicate node
 A predicate node has two edges leading out from it (True and False)

 An edge, or a link, is a an arrow representing flow of control in a specific direction

 An edge must start and terminate at a node
 An edge does not intersect or cross over another edge

20
 FLOW GRAPH EXAMPLE
FLOW CHART FLOW GRAPH
0 0

1 1

2 2

3
3

6 4 6 4

7 8 5
7 8 5
9
9
11 10 11 10 21
 INDEPENDENT PROGRAM
PATHS
 Defined as a path through the program from the start node until the end node
that introduces at least one new set of processing statements or a new condition
(i.e., new nodes)
 Must move along at least one edge that has not been traversed before by a
previous path
 Basis set for flow graph on previous slide
 Path 1: 0-1-11
 Path 2: 0-1-2-3-4-5-10-1-11
 Path 3: 0-1-2-3-6-8-9-10-1-11
 Path 4: 0-1-2-3-6-7-9-10-1-11

 The number of paths in the basis set is determined by the cyclomatic complexity

22
CYCLOMATIC COMPLEXITY

• Provides a quantitative measure of the logical complexity of a program

• Defines the number of independent paths in the basis set
• Provides an upper bound for the number of tests that must be conducted to ensure
all statements have been executed at least once
 Can be computed two ways
 V(G) = E – N + 2, where E is the number of edges and N is the number of nodes in
graph G
 V(G) = P + 1, where P is the number of predicate nodes in the flow graph G

 Results in the following equations for the example flow graph

 V(G) = 14 edges – 12 nodes + 2 = 4
 V(G) = 3 predicate nodes + 1 = 4

23
DERIVING THE BASIS SET AND
TEST CASES

1. Using the design or code as a foundation, draw a corresponding flow

graph
2. Determine the cyclomatic complexity of the resultant flow graph
3. Determine a basis set of linearly independent paths
4. Prepare test cases that will force execution of each path in the basis
set

24
CYCLOMATIC COMPLEXITY

 Invented by Thomas McCabe (1974) to measure the

complexity of a program’s conditional logic
 Cyclomatic complexity of graph G equals #edges - #nodes
+2
 V(G) = e – n + 2

 Also corresponds to the number of linearly independent

paths in a program

25
CONVERTING CODE TO GRAPH

CODE FLOWCHART GRAPH

if expression1 then T F
expr1 n1
statement2 ?
else
(a) statement3 statm2 statm3 n2 n3
end if
statement4
statm4 n4

switch expr1
case 1: 1 3
expr1
statement2 ? n1
case 2: 2
(b) statm3 n2 n3 n4
statm2 statm3 statm4
case 3:
statm4
end switch
n5
statm5 statm5

statm1
do n1
statement1
while expr2 T
(c) expr2
n2
end do ?
statement3 F
n3
statm3 26
 EXAMPLE PATHS

if expression1
then Paths:
statement2 e1 n1
P1 = e1, e2, e4, e6, e7, e8
end if n2 e3
P2 = e1, e2, e4, e5, e4, e6, e7, e8
e2 n3
do P3 = e3, e4, e6, e7, e8, e10
e4 e5
statement3 n4
P4 = e6, e7, e8, e10, e3, e4
while expr4 e6 P5 = e1, e2, e4, e6, e9, e10
end do e7 n5 P6 = e4, e5
n6 e9
P7 = e3, e4, e6, e9, e10
if expression5 e8 n7 P8 = e1, e2, e4, e5, e4, e6, e9, e10
then
statement6
end if
V(G) = e – n + 2 = 9 – 7 + 2 = 4
statement7

27
EXAMPLE 1

28
V=e-n+2=11-9+2=4

29
30
V=e-n+2=11-9+2=4

31
BLACK-BOX TESTING
BLACK BOX TESTING

Test cases are derived from formal specification of the system.

Test case selection can be done without any reference to the
program design or code.
Only tests the functionality and features of the program.
Not the internal operation.
Advantages
 Test case selection is done before the implementation of a program.
 Help in getting the design and coding correct with respect to the
specification.
BLACK BOX
BLACK-BOX TESTING
CATEGORIES
• Incorrect or missing functions
• Interface errors
• Errors in data structures or external data base access
• Behavior or performance errors
• Initialization and termination errors

35
QUESTIONS ANSWERED BY
BLACK-BOX TESTING
 How is functional validity tested?
 How are system behavior and performance tested?
 What classes of input will make good test cases?
 Is the system particularly sensitive to certain input
values?
 How are the boundary values of a data class isolated?
 What data rates and data volume can the system
tolerate?
 What effect will specific combinations of data have on
system operation?