+

Topic 4
Software Testing
NOR IADAH YUSOP
[email protected]
+ 2

What is software testing?

n It is the process of executing a program/application under

positive and negative conditions by manual or automated
means.

n The program is executed using artificial data

n Exhaustive system testing is impossible – need a testing

policy that defines the required test coverage

n Software testing is part of a more general verification and

validation process.
+ 3

Verification vs validation (V & V)

n Verification:

“Are we building the product right ?”

n Validation:

"Are we building the right product?”

n Aim of V & V is to establish confidence that the

system is ‘fit for purpose’.
+ Why does the software need to be 4

tested?
§ To reveal the presence of errors NOT their absence in
the software
§ The aims are
§ to ensure that the software fulfills the clients’
requirements
§ to ensure the correctness, completeness and the
quality of the software
§ to discover program defects before it is put into use
+ 5

What do we need to do?

• Discover situations in which the behavior of the software

is incorrect, undesirable or does not conform to its
specification:
• Find as many errors (defects or bugs) as possible in a
given product.
• Validate the quality of a software testing using the
minimum cost and efforts.
• Generate high quality test cases, perform effective
tests, and issue correct and helpful problem reports
+ 6

Other concepts in software testing

n ErrorIt is a human action that produces the

incorrect result and thus produces a fault

n Bug The presence of error at the time of execution

of the software

n Fault State of software caused by an error

n Failure
An event when there is deviation of the
software from its expected result.
+ 7

Types of software testing

n Software inspections – analyse the static system

representation to discover problems (static
verification)
n May be supplemented by tool-based document
and code analysis.

n Softwaretesting – The system is executed with test

data and its operational behaviour is observed
(dynamic verification)
+ 8

Inspections and testing

Inspections

Requirements Software UML design Database

Program
specification architecture models schemas

System
prototype Testing
+ 9

Software testing process

contains a
specific
detailed
procedure of
understanding
testing a
of what the
particular
eventual testing
requirement
workflow will be

Test
Plan

Check for errors,

anomalies, and
information on non-
functional attributes
+ 10

Stages of testing

n Development testing
n the system is tested during development to discover bugs and
defects – unit testing, component/integration testing, system
testing

n Release testing
n a separate testing team test a complete version of the system
before it is released to users.

n User testing
n users or potential users of a system test the system in their own
environment.
+ 11

Release testing

n Theprocess of testing a particular release of a

system that is intended for use outside of the
development team.

n Theprimary goal - to convince the supplier of the

system that it is good enough for use.
n system delivers its specified functionality,
performance and dependability, and that it does
not fail during normal use.

nA black-box testing process where tests are only

derived from the system specification.
+ 12

Release testing and system testing

n Release testing is a form of system testing.

n Main differences:

Release testing System testing

Who Team that does NOT involve in Developer
the development

Aim • Check if the system meets • Discover bugs

the stipulated requirements in the system
and performance • Does defect
• Does validation testing – testing
check if it is good for
external use
+ 13

User testing
n User or customer testing is a stage in the testing process in
which users or customers provide input and advice on system
testing.

n Types of user testing:

Alpha testing Users of the software work with the development

team to test the software at the developer’s site.

Beta testing A release of the software is made available to

users to allow them to experiment and to raise
problems that they discover.

Acceptance testing Customers test a system to decide whether or

not it is ready to be accepted from the system
developers and deployed in the customer
environment.
+ 14

The acceptance testing process

Test Test Test Testing

Tests
criteria plan results report

Define Plan Derive Run Negotiate Accept or

acceptance acceptance acceptance acceptance test results reject
criteria testing tests tests system
+ 15

Testing strategies

n Partition testing - where you identify groups of inputs

that have common characteristics and should be
processed in the same way.
n Test cases should be chosen from each partition.
n More tutorial:
https://www.youtube.com/watch?v=uydAyjqTSiw

n Guideline-based testing - where you use testing

guidelines to choose test cases.
n These guidelines reflect previous experience of the
kinds of errors that programmers often make when
developing components.
+ 16

General testing guidelines

n Choose inputs that force the system to generate all error

messages

n Design inputs that cause input buffers to overflow

n Repeat the same input or series of inputs numerous times

n Force invalid outputs to be generated

n Force computation results to be too large or too small.

+ 17

Software Testing techniques

Black box testing White box testing

n No knowledge of n Knowledge of the
internal program internal program
design and code
design or code required.
required.
n Tests are based on
n Tests are based on coverage of code
requirements and statements, branches,
paths, conditions.
functionality.
n Example: Basic Path
n Example: Interface Testing Approach
Testing
+ 18

Black box testing

requirements

output

input

events
+ 19

White box testing

Test data

Tests Derives

Component Test
code outputs
+ An input-output model of program 20

testing

Input test data Inputs causing

Ie
anomalous
behaviour

System

Outputs which reveal

Output test results Oe the presence of
defects
+
White Box Testing
§ A hybrid of Basic Path Testing and Branch
Testing methods.
§ Based on Cyclomatic Complexity and uses
control flow to establish the path coverage
criteria.
§ Cyclomatic Complexity
§ developed by McCabe In 1976
§ measures the number of linearly independent
paths through a program
§ the higher the number the more complex the
code.
+
Basic Path Testing

n Basic path testing helps us to

reduce redundant tests.
n It suggests independent paths from which
we write test cases needed to ensure that
every statement and condition can be
executed at least one time.
+
Basic path testing approach

n Step 1: draw a control flow graph.

n Step 2: determine Cyclomatic Complexity.
n Step 3: find a basis set of paths.
n Step 4: generate test cases for each path.
+
Draw a control flow graph

u Basic control flow graph structures:

+ Draw a control flow graph

If – Then – Else Structure

Condition A
Arrows or edges represent flows of
True False control.
Circles or nodes represent actions.
Areas bounded by edges
and nodes are called
regions.
A predicate node is a node
containing a condition.
+ Control Flow Graph Example
FLOW CHART FLOW GRAPH
0 0

1 1

2 2

3
3

6 4 6 4

7 8 5
7 8 5

9
9
11 10 26
11 10
+ Draw a control flow graph

u 1: IF A = 100
u 2: THEN IF B > C
u 3: THEN A = B
u 4: ELSEA= C
u 5: ENDIF
u 6: ENDIF
u 7: Print A
+
Determine Cyclomatic complexity
There are several methods:
1. Cyclomatic complexity = edges - nodes + 2p
*p = the number of unconnected path

2. Cyclomatic complexity= Number of

Predicate Nodes + 1
Predicate node is a node that contain a
condition

3. Cyclomatic complexity =number of regions

in the control flow graph
+ Determine Cyclomatic complexity

Cyclomatic complexity = edges - nodes + 2p

u p = number of unconnected parts of the
graph.
Cyclomatic complexity
= 8-7+ 2*1= 3.
+
Determine Cyclomatic complexity

Cyclomatic complexity= Number of

Predicate Nodes + 1

Cyclomatic complexity
= 2+1= 3.
+ Determine Cyclomatic complexity

Cyclomatic complexity =number of regions in

the control flow graph

Cyclomatic complexity
=3
+ Determine Cyclomatic complexity

Cyclomatic complexity = edges – nodes + 2p

Cyclomatic complexity
= 7-8+ 2*2= 3.
+ 33

Cyclomatic Complexity Metrics

Cyclomatic Meaning
Number
1 - 10 • Structured and well-written code
• High testability
• Cost and effort are less
10 - 20 • Complex code
• Medium testability
• Cost and effort are medium
20 - 40 • Very complex code
• Low testability
• Cost and effort are high
>40 • Not at all testable
• Very high cost and effort
+
Find a basis set of paths

u Path 1: 1, 2, 3, 5, 6, 7.

u Path 2: 1, 2, 4, 5, 6, 7.

u Path 3: 1, 6, 7.
+
Generate test cases for each path

u We have 3 paths so we need at least one test

case to cover each path.
u Write test case for these paths.