Software Testing

5/11/2024
 Objectives
• To explain the different types of testing techniques
used to discover program faults

• To introduce guidelines for interface testing

• To understand the principles of CASE tool support
for testing

5/11/2024
 The testing process
• Component (Unit) Testing:
needs source code (White-box)
Testing of individual program components
Usually the responsibility of the component developer
Tests are derived from the developer’s experience

• Integration testing (Acceptance testing, Black-box testing)

Testing of groups of components integrated to create a system or
sub-system
The responsibility of an independent testing team
Tests are based on a system specification (Acceptance testing)

5/11/2024
 Testing phases

Component Integration
(Unit) testing
testing

Software developer Independent testing team

5/11/2024
 Types of testing
• Defect testing
Exhaustive Testing
Black-box testing
Equivalence partitioning
White-box ‘Structural’ testing
Path testing
• Integration testing
• Interface testing
• Stress testing
• Object-oriented (OO) testing
• XP Automated testing
• Acceptance testing
• Testing workbenches
5/11/2024
Statistics:
Defect Distribution in SW Life Cycle
Defect Distributrion in Software Processes "SW Life Cycle"

Others
Requirements
10%
Design
Code
7% Code
Others

Requirements
56%
Design
27%

5/11/2024
Testing & Verification & Validation

• Testing = Verification + Validation

• Verification: Static Testing (no run)

• Validation: Dynamic Testing (Run code)

5/11/2024
 Use Cases & Functional Test Cases
• Write functional test case early

Happy path
Test
Result
Alternate flow

Exceptional flow
Functional
Tests

Output

5/11/2024
 Who Tests the Software
•

Developer Independent tester

- understands the system - must learn about the system
- has the source code - has no source code
- white-box ‘Unit’ testing - black-box ‘Acceptance’ testing
- will test "gently“ - will attempt to break the sys (ME!!)
- driven by delivery ‘schedule’ - driven by quality constraint
constraint

5/11/2024
 Test data and test cases
• Test data
Inputs to test the system

• Test case ‘Test scenario’ includes:

Inputs to test the system
and the predicted outputs

5/11/2024
 What is Defect testing?

Testing is:

• the process of exercising a program

• with the specific intent of finding errors

• prior to delivery to the end user.

5/11/2024
 The defect testing process

Deliverable
Specs of test input & predicted sys output

Test Test Test Test

cases data results reports

Design test Prepare test Run program Compare results

cases data with test data to test cases

Process

5/11/2024
 Black-box testing
• Program is considered as a ‘black-box’
• No need to know or access source code
• Functionality testing
• No implementation testing (implementation
testing needs source code)
• Test cases are based on the system specification
• Test planning can begin early in the software
process

5/11/2024
 Black-box Testing
“Functionality testing”
Input:
Test Case

Predicted sys output

Program
Sys Input Sys Output Compare

“No need
for source
code” Output: Test
Report

5/11/2024
 Black-box Testing
Inputs causing
anomalous
Input test data I behaviour
e

System / component Compare output with

predicted values

Outputs which reveal

the presence of
Output test results Oe defects

5/11/2024
 Equivalence partitioning
• Objective:

Reduce the number of test cases

5/11/2024
 Equivalence partitioning - cont.
• A way to derive test cases
• Is a black-box testing method
• Input data with common characteristics (positive numbers,
negative numbers, strings without blanks, etc)
• Each of these classes is an equivalence partition
• Program behaves in an equivalent way for each member of
an Equivalence partition
• Divides the input domain into classes of data form which
test cases can be derived

5/11/2024
 Equivalence partitioning – cont.

Input
Invalid inputs Valid inputs

System

Outputs

5/11/2024
 Equivalence partitioning - cont.
• Guidelines for test case selection for partitions
At the boundaries of the partition
Close to mid-point of the partition

Equivalence Partition

Left Boundary Value Right Boundary Value

Mid-point Value

5/11/2024
Equivalence partitioning - Example 1
• Program specs states:
Accepts 4 to 10 inputs
Each input is 5-digit integer greater > = than 10,000

• Partition system inputs and outputs into

‘equivalence sets’

5/11/2024
 Equivalence partitioning - cont.
Example
3 Valid 11
4 7 10

Invalid Invalid

Less than 4 Between 4 and 10 More than 10

Number of input values

BV
9999 Valid 100000
BV 10000 50000 99999
Mid-point

Less than 10000 Between 10000 and 99999 More than 99999

Invalid
Input values Invalid

5/11/2024
Equivalence partitioning - cont. Example 1
• If input is a 5-digit integer between 10,000 and 99,999, equivalence
partitions are
<10,000,
10,000 - 99, 999
and > 99, 999

• Test cases: 6 cases

00000, (invalid special value that may be checked)
09999, (invalid left boundary value)
10000, (valid left boundary value)
50000, (valid mid value)
99999, (valid right boundary value)
100000, (invalid right boundary value)

Remember: Bugs lurk in corners & congregate at boundaries

5/11/2024
 Structural testing ‘White-box
testing’
• Synonyms:
Glass-box, Clear-box, Transparent-box
• For small program units
• Needs source code

• Objective: is to exercise all program statements

• (not all path combinations)

5/11/2024
 White-box testing

Test data

Tests Derives

Component Test
code outputs

Starting point: known code

5/11/2024
 Path testing

• Each path through the program is executed

at least once

• For loops & conditions

• Used at unit testing and module testing

levels

5/11/2024
 Program flow graphs
• Flow Graph:
nodes representing program decisions
arcs representing the flow of control
Ignore sequential statements (assignments, procedures calls, I/O)

• Statements with conditions are therefore nodes in the flow

graph

• Cyclomatic complexity =
Number of edges - Number of nodes + 2

5/11/2024
 Cyclomatic complexity
• Cyclomatic complexity = number of tests to test
all control statements

• Cyclomatic complexity = number of conditions in

a program

• Although all paths are executed, all combinations

of paths are not executed

5/11/2024
 Example: Path testing
Binary search flow graph
1

while bottom <= top

bottom > top
2

3 if (elemArray [mid] == key

8 4
(if (elemArray [mid]< key

5 6
9

5/11/2024
 Independent paths
• Cyclomatic complexity = 11 - 9 + 2 = 4

• Thus 4 independent paths:

• 1, 2, 3, 8, 9
• 1, 2, 3, 4, 6, 7, 2
• 1, 2, 3, 4, 5, 7, 2
• 1, 2, 3, 4, 6, 7, 2, 8, 9

• Test cases should be derived so that all of these paths are

executed

5/11/2024
 Integration testing (black-box)
• Tests complete systems or subsystems composed
of integrated components

• Integration testing should be black-box testing

with tests derived from the specification

• Main difficulty is localising errors

• Incremental integration testing reduces this
problem

5/11/2024
 Incremental integration testing
Recall: XP Continuous Integration of: Stories + (Acceptance & Unit Tests)

A T1
Minimal system
T1
A
T1 T2
A B
T2
T2 B T3
T3
B C
T3 T4
C
T4
D T5

Test sequence Test sequence Test sequence

1 2 3
T1, T2, T3 are repeated to insure right interaction of C with A & B

5/11/2024
 Approaches to integration testing
• Top-down integration testing
Start with high-level system and integrate from the top-down
replacing individual components by stubs
Stubs have the same interface as the components but very limited
functionality

• Bottom-up integration testing (XP)

Integrate and test low-level components (or stories in XP), with
full functionality, before developing higher level components,
until the complete system is created

• In practice, combination of both

5/11/2024
 Top-down integration testing

Testing
Level 1 Level 1 . ..
sequence

Level 2 Level 2 Level 2 Level 2

Level 2
stubs

Level 3
stubs

5/11/2024
 Bottom-up integration testing

Test
drivers

Testing
Level N Level N Level N Level N Level N
sequence

Test
drivers
Level N–1 Level N–1 Level N–1

5/11/2024
 Testing approaches
• Architectural validation
Top-down integration testing is better at discovering errors in the
system architecture

• System demonstration
Top-down integration testing allows a limited demonstration at an
early stage in the development

• Test implementation
Often easier with bottom-up integration testing

5/11/2024
 Interface testing
• Takes place when modules or sub-systems are
integrated to create larger systems

• Objectives are to detect faults due to interface

errors or invalid assumptions about interfaces

• Particularly important for object-oriented

development as objects are defined by their
interfaces

5/11/2024
 Interface testing
Test
cases

A B

5/11/2024
 Interfaces types
• Parameter interfaces
Data passed from one procedure to another

• Shared memory interfaces

Block of memory is shared between procedures

• Message passing interfaces

Sub-systems request services from other sub-systems

5/11/2024
 Interface testing guidelines
• Design tests so that parameters to a called
procedure are at the extreme ends of their ranges

• Use stress testing in message passing systems

5/11/2024
 Stress testing
• Exercises the system beyond its maximum design load.
• Stressing the system often causes defects to come to light
• Stress testing failure behaviour..
Systems should not fail catastrophically.
Stress testing checks for unacceptable loss of service or data

• Particularly relevant to distributed systems which can

exhibit severe degradation as a network becomes
overloaded

5/11/2024
 Object-oriented (OO) testing

• The components to be tested are object classes that

are instantiated as objects

5/11/2024
 Object-oriented testing:
Testing levels
• Testing operations (methods)
associated with objects

Remember: XP Unit Testing by

developers on operations

Testing levels :
• Testing object classes
• Testing clusters of cooperating
objects
• Testing the complete OO
system

5/11/2024
 Object class testing
• Complete test coverage of a class involves:
Testing all operations (methods) associated with an
object
Setting and Getting all object attributes
(Recall: XP Unit testing for all methods and attributes
of a class)

• Inheritance makes it more difficult to design

object class tests
the information to be tested is not localised

5/11/2024
 Key Testing Issues
• Give high testing priority for parts of a system which are
commonly used
give less priority for parts which are rarely executed
• Equivalence partitions
are sets of test cases where the program should behave in an
equivalent way
aims at reducing the number of test cases
• Black-box testing is based on the system specification
• Structural testing identifies test cases which cause all paths
through the program to be executed

5/11/2024
 Key Testing Issues
• Test coverage:
measures ensure that all statements have been executed
at least once.

• For OO testing:
test object classes, test all operations, attributes and
states

5/11/2024