Software Testing

Topics covered
 Development testing
 Test-driven development
 Release testing
 User testing
 Program testing
 Testing is intended to show that a program does
what it is intended to do and to discover program
defects before it is put into use.
 When you test software, you execute a program
using artificial data.
 You check the results of the test run for errors or
information about the program’s non-functional
attributes.
 Test can reveal the presence of errors NOT their
absence.
 Testing is part of a more general verification and
validation process, which also includes validation
techniques.
Program testing goals
 To demonstrate to the developer and the customer
that the software meets its requirements.
 For custom software, this means that there should be
at least one test for every requirement in the
requirements document.
 For generic software products, it means that there
should be tests for all of the system features, that will
be incorporated in the product release.
 To discover situations in which the behavior of the
software is incorrect, and undesirable
 Defect testing is concerned with rooting out
undesirable system behavior such as.
 system crashes, unwanted interactions with other
systems, incorrect computations and data corruption
 Validation and defect testing
 The first goal leads to validation testing
 You expect the system to perform correctly
using a given set of test cases that reflect the
system’s expected use.
 The second goal leads to verification
testing
 The test cases are designed to expose
defects. The test cases in defect testing can
be deliberately unclear (vague) and need not
reflect how the system is normally used.
 Testing process goals
 Validation testing
 To demonstrate to the developer and the system
customer that the software meets its requirements
 A successful test shows that the system operates
as intended.
 Defect testing
 To discover faults or defects in the software where
its behavior is incorrect
 A successful test is a test that makes the system
perform incorrectly and so exposes a defect in the
system.
Verification vs validation
 Verification:
"Are we building the product right”.
 The software behaviour should be
correct.
 Validation:
"Are we building the right product”.
 The software should do what the user
really requires.
 V & V confidence
 Aim of V & V is to establish confidence that the system is ‘fit
for purpose’.
 The system must be good enough for its intended use
 Depends on system’s purpose, user expectations and
marketing environment
 Software purpose
 The more critical the software, the more important that it

is reliable. For example, the level of confidence required

for software used to control a safety-critical system is
much higher as compared to general purpose system
 User expectations
 Users may have low expectations of certain kinds of

software.
 Marketing environment
 Getting a product to market early may be more important

than finding defects in the program.

Inspections and testing
 Software inspections Concerned with analysis of the
static system representation to discover problems
(static verification)
 Inspections mostly focus on the source code of a system
but any readable representation of the software, such as
its requirements or a design model, can be inspected.
 When you inspect a system, you use knowledge of the
system, its application domain, and the programming or
modeling language to discover errors.
 Software testing Concerned with exercising and
observing product behaviour (dynamic verification)
 The system is executed with test data and its
operational behaviour is observed.
Inspections and testing
 Advantages of inspections
 During testing, errors can mask (hide) other errors.
Because inspection is a static process, you don’t
have to be concerned with interactions between
errors.
 Incomplete versions of a system can be inspected
without additional costs. If a program is incomplete,
then you need to develop specialized test, to test the
parts that are available.
 As well as searching for program defects, an
inspection can also consider broader quality
attributes of a program, such as compliance with
standards, and maintainability.
Inspections and testing
 Inspections and testing are complementary
and not opposing verification techniques.
 Both should be used during the V & V
process.
 Inspections cannot check non-functional
characteristics such as performance,
usability, etc.
A model of the software testing
process
Stages of testing
 Development testing, where the system is
tested during development to discover bugs
and defects.
 Release testing, where a separate testing
team test a complete version of the system
before it is released to users.
 User testing, where users or potential users
of a system test the system in their own
environment.
 Development testing
 Development testing includes all testing activities that
are carried out by the team developing the system.
1. Unit testing, where individual program units or
object classes are tested. Unit testing should focus
on testing the functionality of objects or methods.
2. Component testing, where several individual units
are integrated to create composite components.
Component testing should focus on testing
component interfaces.
3. System testing, where some or all of the
components in a system are integrated and the
system is tested as a whole. System testing should
focus on testing component interactions.
Unit testing
 Unit testing is the process of testing individual
program in isolation.
 It is a defect testing process.
 Units may be:
 Individual functions or methods within an object
 Object classes with several attributes and
methods
 Object class testing
 Complete test coverage of all the features of
the class
 Testing all operations associated with an object
 set and check the value of all attributes associated
with the object;
 Inheritance makes it more difficult to design
object class tests as the information to be
tested is not localised.
Automated testing
 Whenever possible, unit testing should
be automated so that tests are run and
checked without manual involvement.
 In automated unit testing, you make use
of a test automation framework (such as
JUnit) to write and run your program
tests.
 Automated test components
 A setup part, where you initialize the system
with the test case, namely the inputs and
expected outputs.
 A call part, where you call the object or
method to be tested.
 An assertion part where you compare the
result of the call with the expected result. If
the assertion evaluates to true, the test has
been successful if false, then it has failed.
 Person class
//methods
public class Person {
public String getName() {
// fields
return name;
private String name; // name of the person
}
private int maximumBooks; // most books the
public void setName(String anyName) {
person can check out
name = anyName;
// constructors
}
public Person() {
public int getMaximumBooks() {
name = "unknown name";
maximumBooks = 3; return maximumBooks;
} }
} public void setMaximumBooks(int
maximumBooks) {
this.maximumBooks = maximumBooks;
}
}
 Test Case
import junit.framework.TestCase;
public class PersonTest extends TestCase {
public void testPerson() {
Person p1 = new Person();
assertEquals("unknown name", p1.getName());
assertEquals(3, p1.getMaximumBooks());
}
public void testSetName() {
Person p2 = new Person();
p2.setName("Fred");
assertEquals("Fred", p2.getName());
}
public void testSetMaximumBooks() {
Person p3 = new Person();
p3.setMaximumBooks(10);
assertEquals(10, p3.getMaximumBooks());
}
}
Unit test effectiveness
 If there are defects in the program, these should be
revealed by test cases.
 This leads to 2 types of unit test case:
 The first of these should reflect normal operation of a
program and should show that the program works as
expected.
 The other kind of test case should be based on testing
experience of where common problems arise. It should
use abnormal inputs to check that these are properly
processed and do not crash the component.
 Component Testing
 Software components are often composite
components that are made up of several interacting
objects.
 You access the functionality of these objects through
the defined component interface.
 Testing composite components should therefore
focus on showing that the component interface
behaves according to its specification.
 You can assume that unit tests on the individual objects
within the component have been completed.
Interface testing
 Interface errors
 Interface misuse
 A calling component calls another component and
makes an error in its use of its interface e.g.
parameters in the wrong order.
 Interface misunderstanding
 A calling component embeds assumptions about the
behaviour of the called component which are incorrect.
 Binary search method may be called with a parameter
that is an unordered array.
 Timing errors
 The called and the calling component operate at
different speeds and out-of-date information is
accessed.
Key points
 Testing can only show the presence of errors in a
program. It cannot demonstrate that there are no
remaining faults.
 Development testing is the responsibility of the
software development team. A separate team should
be responsible for testing a system before it is
released to customers.
 Development testing includes unit testing, in which
you test individual objects and methods , component
testing in which you test related groups of objects
and system testing, in which you test partial or
complete systems.