Chapter 7

Software testing
Objectives

Understand testing
fundamentals.
Understand different types
of Black box and white box
testing methods.
Understand levels of
testing.
Understand different
quality assurance activities
to be taken.
Testing Fundamentals
What is software testing?
 Myers: The process of executing a program with
the intent of finding errors.
 Beizer: The act of designing and executing tests.
 Whittaker: The process of executing a software
system to determine whether it matches its
specification and executes in its intended
environment.
 IEEE : The process of operating a system or
component under specified conditions, observing
or recording the results, and making an evaluation
of some aspect of system or component.
 Testing Fundamentals
Why do we test?
 Testing often accounts for more project effort than any
other software engineering action.
 If it is conducted haphazardly, time is wasted,
unnecessary effort is expended, and even worse,
errors sneak through undetected.
 It would therefore seem reasonable to establish a
systematic strategy for testing a software.
We should test a software product
 To check if there are any errors in a part or a
product.
To gain confidence in the correctness of the product.
To ensure the quality and satisfaction of the product
 Testing Fundamentals
Who Tests the Software?
 Development engineers
 Understand the system, but test “gently”
 Driven by “delivery”
 Only perform unit tests and integration tests
 Test engineers
 Need to learn the system, but attempt to break it
 Driven by “quality”
 Define test cases, write test specifications, run tests,
analyze results
 Customers
 Driven by “requirements”
 Determine if the system satisfies the acceptance
criteria
 Black box testing
 This form of testing is also called functional or
behavioral testing.
 In black box testing the software to be tested is
treated as a block box i.e. the structure of the
program is not consider
 The test cases are decided solely on the basis of
the requirements or specifications of the
program or module.
The internals of the module or the program
are not considered for selection of test cases.
The tester only knows the inputs that can be
given to the system and what output the
system should give.

Cont’d
 The most obvious functional testing procedure
is exhaustive testing
 Software is designed to work for an input space
So exhaustive testing involves testing the
software with all elements in the input space
 However it is infeasible, because of very high a
cost
 So better method for selecting test cases is
needed
 Different approaches have been proposed
Requirements-based testing
 A general principle of requirements engineering is
that requirements should be testable.
 Requirements-based testing is a validation testing
technique where you consider each requirement and
derive a set of tests for that requirement.
 Example: LIBSYS requirements
The user shall be able to search either all of the
initial set of databases or select a subset from it.
The system shall provide appropriate viewers for
the user to read documents in the document store.
Every order shall be allocated a unique identifier
(ORDER_ID) that the user shall be able to copy to
the account’s permanent storage area.
 Initiate user search for searches for items that are
known to be present and known not to be present,
where the set of databases includes 1 database.
 Initiate user searches for items that are known to be
present and known not to be present, where the set of
databases includes 2 databases
 Initiate user searches for items that are known to be
present and known not to be present where the set of
databases includes more than 2 databases.
 Select one database from the set of databases and
initiate user searches for items that are known to be
present and known not to be present.
 Select more than one database from the set of
databases and initiate searches for items that are
known to be present and known not to be present.
Equivalence Class
partitioning
Divide the input space into equivalent
classes
If the software works for a test case from
a class then it is likely to work for all.
Two steps:
1.partitioning the values of input parameters
into equivalence groups
2.choosing the test input values
valid equivalence class

0 100

invalid equivalence classes

35
Cont’d
 Input data and output results often fall into
different classes where all members of a class
are related.
 Each of these classes is an equivalence
partition or domain where the program
behaves in an equivalent way for each class
member.
 Test cases should be chosen from each
partition.
 E.g. Compute rate of interest given loan
amount, monthly installment, and number of
months
Equivalent classes in output: + rate, rate =
0 ,-ve rate
Cont’d
 Once equivalence classes are selected for each of the
inputs, then test cases have to be selected
Select each test case covering as many valid
equivalence classes as possible
Or, have a test case that covers at most one valid
class for each input
Plus a separate test case for each invalid class
 Example
suppose the specification for a module says that an
input, the length of a Wire in millimeters, lies in the
range 1–499; then select
 One valid equivalence class that includes all
values from 1 to 499.
 Two invalid classes
 That consists of all values less than 1, and
 That consists of all values greater than 499.
 Boundary value analysis
 It has been observed that programs that work
correctly for a set of values in an equivalence
class fail on some special values.
These values often lie on the boundary of the
equivalence class.
 Test cases that have boundary values have
high yield
 These are also called extreme cases
 A boundary value test case is a set of input data
that lies on the edge of a equivalence class of
input/output
Cont’d
 For each equivalence class
choose values on the edges of the class
choose values just outside the edges
 E.g. if 0 <= x <= 1.0
0.0 , 1.0 are edges inside
-0.1,1.1 are just outside
 Similarly, if the input is a list, attention should
be focused on the first and last elements of the
list.
 Consider outputs also and have test cases
generate outputs on the boundary
Cause Effect graphing
 Equivalence classes and boundary value analysis
consider each input separately
 To handle multiple inputs, different combinations
of equivalent classes of inputs can be tried
 Number of combinations can be large – if n is
different input conditions such that each condition
is valid/invalid, then we will have 2n test cases.
 Cause effect graphing helps in selecting
combinations as input conditions
A cause is a distinct input condition, and an
effect is a distinct output condition (T/F).
 3 11
4 7 10

Less than 4 Between 4 and 0

1 More than 1
0

Number ofinput values

9999 100000
10000 50000 99999

Less than 0000

1 Between 10000 and 99999 More than 99999

Input values
Cont’d
 Identify causes and effects in the system
 Identify which causes can produce which
effects; can combine causes
 Causes/effects are nodes in the graph and arcs
are drawn to capture dependency; and/or are
allowed.
 From the cause effect graph, we can make a
decision table
Lists combination of conditions that set
different effects
Together they check for various effects
 Decision table can be used for forming the test
cases
Cause–Effect Graph Constructs
Figure below show constructs that can be
used in cause–effect graphing.
 Cause–Effect Graph example
E.g. A database should have each file listed in a 10 -section
master index by name and location. To display or list a
section in the inquiry mode the operator must enter a D
(display) or P (print), followed by a numeric character (0-9)
to represent the section number, followed by “return” key
Error handling
If the first character is other than D, or P the system
displays
INVALID COMMAND
If the second character is not in a range 0-9 the system
displays
INVALID INDEX
Cause–effect identification
Causes Effects
1. character 1 is a D
1. Index section is displayed
2. character 1 is a P
3. character 2 is a 2. Index section is printed
digit in the range 0- 3. INVALID COMMAND is
9
displayed
4. INVALID INDEX NUMBER
is displayed
A cause-and-effect graph
Example of a cause–effect decision chart
CE graphing: Example
 A bank database which allows two commands
Credit acc# amt
Debit acc# amt
 The Requirements are that
If credit and acc# valid, then credit
If debit and acc# valid and amt less than balance,
then debit
Invalid command – message
 Causes
C1: command is credit
C2: command is debit
C3: acc# is valid
C4: amt is valid
Cont’d
 Effects
Print “Invalid command”
Print “Invalid acct#”
Print “Debit amt not valid”
Debit account
Credit account
# 1 2 3 4 5

C1 0 1 x x x
C2 0 x 1 1 x
C3 x 0 1 1 1
C4 x x 0 1 1

E1 1
E2 1
E3 1
E4 1
E5 1
White box testing
 Black box testing focuses only on functionality
What the program does; not how it is
implemented
 White box testing focuses on implementation
Aim is to exercise different program
structures with the intent of uncovering
errors
 Is also called structural testing
 Various criteria exist for test case design
 Test cases have to be selected to satisfy
coverage criteria
 There are different approaches of structural
testing
Control flow based criteria
 Considers the program as control flow graph
 Any control flow graph has a start node and an
end node
 Control flow refers to flow of control from one
instruction to another.
 Control flow graph has a number of coverage
criteria
Statement coverage
Branch coverage
Path coverage and etc.
Statement Coverage Criterion
The simplest coverage criteria is statement
coverage;
Which requires that each statement of the
program be executed at least once during testing.
 This coverage criterion is not very strong, and can
leave errors undetected.
Because it has a limitation in that it does not
require a decision to evaluate to false if no else
clause
E.g. : abs (x) : if ( x>=0) x = -x; return(x)
The set of test cases {x = 0} achieves 100%
statement coverage, but error not detected
Guaranteeing 100% coverage not always possible
due to possibility of unreachable nodes
 Levels of Testing
 The code contains requirement defects, design
defects, and coding defects
 Nature of defects is different for different
stages
 One type of testing will be unable to detect the
different types of defects
 Different levels of testing are used to uncover
these defects

30
 Levels of Testing…
User needs Acceptance testing

Requirement System testing

specification

Design Integration testing

code Unit testing

3
 Levels of Testing…
 Unit testing
Is the process of testing individual components in
isolation.
Focuses on defects injected during coding: coding
phase sometimes called “coding and unit testing”
 Integration testing
Involves building a system from its components and
testing it for problems that arise from component
interactions.
 System testing
Testing of groups of components integrated to create a
system or sub-system
The responsibility of an independent testing team
Tests are based on a system specification
32 Most time consuming test phase
 Acceptance Testing
The focus of acceptance testing is to check if
the software satisfy user needs
Generally it is done by end users/customer in
customer environment, with real data
Only after successful acceptance testing the
software deployed
Any defects found are removed by developers

33
Testing and debugging
 Programmers carry out some testing of the code
they have developed. This often reveals program
defects (bugs) that must be removed from the
program. Finding and fixing program defects is
called debugging.
 Defect testing and debugging are different
processes.
 Testing establishes the existence of defects.
 Debugging is concerned with locating and
correcting these defects.
 Testing and debugging are different activities, but
debugging must be accommodated in any testing
strategy
Software Quality Assurance
To achieve a good quality software we have to
 Explicitly define what is meant when we say
“software quality,”
 Create a set of activities that will help ensure
that every software engineering work product
exhibits high quality.
 Perform quality control and assurance
activities on every software project.
 Use metrics to develop strategies for
improving your software process and, as a
consequence, the quality of the end product.
Elements of software quality
assurance
 Standards. The IEEE, ISO, and other standards
organizations have produced a broad array of
software engineering standards and related
documents.
 Reviews and audits. Technical reviews are a
quality control activity performed by software
engineers for software engineers . Their intent is
to uncover errors.
 Testing. Software testing is a quality control
function that has one primary goal to find errors.
 Error/defect collection and analysis. The only
way to improve is to measure how you’re doing.
Cont’d
 Change management. Change is one of the
most disruptive aspects of any software
project. If it is not properly managed, change
can lead to confusion, and confusion almost
always leads to poor quality.
 Education. Every software organization wants
to improve its software engineering practices.
A key contributor to improvement is education
of software engineers, their managers, and
other stakeholders.
 Vendor management. To ensure that high-
quality software results by suggesting specific
quality practices that the vendor should follow
(when possible), and incorporating quality
Cont’d
 Security management. With the increase in cyber
crime and new government regulations regarding
privacy, every software organization should institute
policies.
 Safety. Because software is almost always a pivotal
component of human-rated systems (e.g., automotive
or aircraft applications), the impact of hidden defects
can be catastrophic.
 Risk management. Although the analysis and
mitigation of risk is the concern of software engineers,
the SQA organization ensures that risk management
activities are properly conducted and that risk-related
contingency plans have been established.