Program testing Program testing goals

 Testing is intended to show that a program does what it is  To demonstrate to the developer and the customer that
intended to do and to discover program defects before it is put the software meets its requirements.
into use.
▪ For custom software, this means that there should be at least
 When you test software, you execute a program using one test for every requirement in the requirements document.
artificial data. For generic software products, it means that there should be
tests for all of the system features, plus combinations of these
 You check the results of the test run for errors, anomalies or features, that will be incorporated in the product release.
information about the program’s non-functional attributes.
 To discover situations in which the behavior of the
 Can reveal the presence of errors NOT their software is incorrect, undesirable or does not conform to
absence. its specification.
 Testing is part of a more general verification and validation ▪ Defect testing is concerned with rooting out undesirable system
process, which also includes static validation techniques. behavior such as system crashes, unwanted interactions with
other systems, incorrect computations and data corruption.
30/10/2014 Chapter 8 Software Testing 3 30/10/2014 Chapter 8 Software Testing 4

Validation and defect testing Verification vs validation

 The first goal leads to validation testing  Verification:

▪ You expect the system to perform correctly using a given set of "Are we building the product right”.
test cases that reflect the system’s expected use.
 The software should conform to its specification.
 The second goal leads to defect testing
 Validation:
▪ The test cases are designed to expose defects. The test cases in
"Are we building the right product”.
defect testing can be deliberately obscure and need not reflect
how the system is normally used.  The software should do what the user really requires.

30/10/2014 Chapter 8 Software Testing 5 30/10/2014 Chapter 8 Software Testing 8

V & V confidence Inspections and testing

 Aim of V & V is to establish confidence that the system is

‘fit for purpose’.  Software inspections Concerned with analysis of
 Depends on system’s purpose, user expectations and the static system representation to discover problems
marketing environment (static verification)
▪ Software purpose ▪ May be supplement by tool-based document and code
• The level of confidence depends on how critical the software is to analysis.
an organisation.
▪ Discussed in Chapter 15.
▪ User expectations
• Users may have low expectations of certain kinds of software.  Software testing Concerned with exercising and
▪ Marketing environment observing product behaviour (dynamic verification)
• Getting a product to market early may be more important than ▪ The system is executed with test data and its operational
finding defects in the program.
behaviour is observed.

30/10/2014 Chapter 8 Software Testing 9 30/10/2014 Chapter 8 Software Testing 10

Software inspections Advantages of inspections

 These involve people examining the source  During testing, errors can mask (hide) other errors.
representation with the aim of discovering anomalies and Because inspection is a static process, you don’t have to
defects. be concerned with interactions between errors.
 Inspections not require execution of a system so may be  Incomplete versions of a system can be inspected
used before implementation. without additional costs. If a program is incomplete, then
 They may be applied to any representation of the system you need to develop specialized test harnesses to test
the parts that are available.
(requirements, design,configuration data, test data, etc.).
 As well as searching for program defects, an inspection
 They have been shown to be an effective technique for
can also consider broader quality attributes of a
discovering program errors.
program, such as compliance with standards, portability
and maintainability.

30/10/2014 Chapter 8 Software Testing 12 30/10/2014 Chapter 8 Software Testing 13

Inspections and testing A model of the software testing process

 Inspections and testing are complementary and not

opposing verification techniques.
 Both should be used during the V & V process.
 Inspections can check conformance with a specification
but not conformance with the customer’s real
requirements.
 Inspections cannot check non-functional characteristics
such as performance, usability, etc.

30/10/2014 Chapter 8 Software Testing 14 30/10/2014 Chapter 8 Software Testing 15

Stages of testing Development testing

 Development testing, where the system is tested during  Development testing includes all testing activities that
development to discover bugs and defects. are carried out by the team developing the system.
 Release testing, where a separate testing team test a ▪ Unit testing, where individual program units or object classes are
tested. Unit testing should focus on testing the functionality of
complete version of the system before it is released to
objects or methods.
users.
▪ Component testing, where several individual units are integrated
 User testing, where users or potential users of a system to create composite components. Component testing should
test the system in their own environment. focus on testing component interfaces.
▪ System testing, where some or all of the components in a
system are integrated and the system is tested as a whole.
System testing should focus on testing component interactions.

30/10/2014 Chapter 8 Software Testing 16 30/10/2014 Chapter 8 Software Testing 18

Unit testing Object class testing

 Unit testing is the process of testing individual  Complete test coverage of a class involves
components in isolation. ▪ Testing all operations associated with an object
 It is a defect testing process. ▪ Setting and interrogating all object attributes
▪ Exercising the object in all possible states.
 Units may be:
 Inheritance makes it more difficult to design object class
▪ Individual functions or methods within an object
tests as the information to be tested is not localised.
▪ Object classes with several attributes and methods
▪ Composite components with defined interfaces used to access
their functionality.

30/10/2014 Chapter 8 Software Testing 19 30/10/2014 Chapter 8 Software Testing 20

Automated testing Testing strategies

 Whenever possible, unit testing should be automated so  Partition testing, where you identify groups of inputs that
that tests are run and checked without manual have common characteristics and should be processed
intervention. in the same way.
 In automated unit testing, you make use of a test ▪ You should choose tests from within each of these groups.
automation framework (such as JUnit) to write and run  Guideline-based testing, where you use testing
your program tests. guidelines to choose test cases.
 Unit testing frameworks provide generic test classes that ▪ These guidelines reflect previous experience of the kinds of
errors that programmers often make when developing
you extend to create specific test cases. They can then
components.
run all of the tests that you have implemented and
report, often through some GUI, on the success of
otherwise of the tests.

30/10/2014 Chapter 8 Software Testing 23 30/10/2014 Chapter 8 Software Testing 26

Use-case testing Test cases derived from sequence diagram

 The use-cases developed to identify system interactions  An input of a request for a report should have an
can be used as a basis for system testing. associated acknowledgement. A report should ultimately
 Each use case usually involves several system be returned from the request.
components so testing the use case forces these ▪ You should create summarized data that can be used to check
that the report is correctly organized.
interactions to occur.
 An input request for a report to WeatherStation results in
 The sequence diagrams associated with the use case
a summarized report being generated.
documents the components and interactions that are
▪ Can be tested by creating raw data corresponding to the
being tested.
summary that you have prepared for the test of SatComms and
checking that the WeatherStation object correctly produces this
summary. This raw data is also used to test the WeatherData
object.

30/10/2014 Chapter 8 Software Testing 39 30/10/2014 Chapter 8 Software Testing 41

 Test-driven development TDD process activities

 Test-driven development (TDD) is an approach to  Start by identifying the increment of functionality that is
program development in which you inter-leave testing required. This should normally be small and
and code development. implementable in a few lines of code.
 Tests are written before code and ‘passing’ the tests is  Write a test for this functionality and implement this as
the critical driver of development. an automated test.
 You develop code incrementally, along with a test for that  Run the test, along with all other tests that have been
increment. You don’t move on to the next increment until implemented. Initially, you have not implemented the
the code that you have developed passes its test. functionality so the new test will fail.
 TDD was introduced as part of agile methods such as  Implement the functionality and re-run the test.
Extreme Programming. However, it can also be used in  Once all tests run successfully, you move on to
plan-driven development processes.
implementing the next chunk of functionality.
30/10/2014 Chapter 8 Software Testing 44 30/10/2014 Chapter 8 Software Testing 46

Release testing Requirements based testing

 Release testing is the process of testing a particular release  Requirements-based testing involves examining each
of a system that is intended for use outside of the requirement and developing a test or tests for it.
development team.  Mentcare system requirements:
 The primary goal of the release testing process is to ▪ If a patient is known to be allergic to any particular medication,
convince the supplier of the system that it is good enough then prescription of that medication shall result in a warning
for use. message being issued to the system user.
▪ Release testing, therefore, has to show that the system delivers its ▪ If a prescriber chooses to ignore an allergy warning, they shall
specified functionality, performance and dependability, and that it provide a reason why this has been ignored.
does not fail during normal use.
 Release testing is usually a black-box testing process
where tests are only derived from the system specification.

30/10/2014 Chapter 8 Software Testing 50 30/10/2014 Chapter 8 Software Testing 52

Requirements tests A usage scenario for the Mentcare system

George is a nurse who specializes in mental healthcare. One of his responsibilities is to visit patients
 Set up a patient record with no known allergies. Prescribe medication for
at home to check that their treatment is effective and that they are not suffering from medication
allergies that are known to exist. Check that a warning message is not side effects.
issued by the system.
On a day for home visits, George logs into the Mentcare system and uses it to print his schedule of
 Set up a patient record with a known allergy. Prescribe the medication to home visits for that day, along with summary information about the patients to be visited. He
that the patient is allergic to, and check that the warning is issued by the requests that the records for these patients be downloaded to his laptop. He is prompted for his key
phrase to encrypt the records on the laptop.
system.
One of the patients that he visits is Jim, who is being treated with medication for depression. Jim
 Set up a patient record in which allergies to two or more drugs are recorded. feels that the medication is helping him but believes that it has the side effect of keeping him awake
Prescribe both of these drugs separately and check that the correct warning at night. George looks up Jim’s record and is prompted for his key phrase to decrypt the record. He
for each drug is issued. checks the drug prescribed and queries its side effects. Sleeplessness is a known side effect so he
notes the problem in Jim’s record and suggests that he visits the clinic to have his medication
 Prescribe two drugs that the patient is allergic to. Check that two warnings changed. Jim agrees so George enters a prompt to call him when he gets back to the clinic to make
are correctly issued. an appointment with a physician. George ends the consultation and the system re-encrypts Jim’s
record.
 Prescribe a drug that issues a warning and overrule that warning. Check
After, finishing his consultations, George returns to the clinic and uploads the records of patients
that the system requires the user to provide information explaining why the visited to the database. The system generates a call list for George of those patients who He has to
warning was overruled. contact for follow-up information and make clinic appointments.

30/10/2014 Chapter 8 Software Testing 53 30/10/2014 Chapter 8 Software Testing 54

Features tested by scenario Performance testing

 Authentication by logging on to the system.  Part of release testing may involve testing the emergent
properties of a system, such as performance and
 Downloading and uploading of specified patient records
reliability.
to a laptop.
 Tests should reflect the profile of use of the system.
 Home visit scheduling.
 Performance tests usually involve planning a series of
 Encryption and decryption of patient records on a mobile
tests where the load is steadily increased until the
device.
system performance becomes unacceptable.
 Record retrieval and modification.
 Stress testing is a form of performance testing where the
 Links with the drugs database that maintains side-effect system is deliberately overloaded to test its failure
information. behaviour.
 The system for call prompting.
30/10/2014 Chapter 8 Software Testing 55 30/10/2014 Chapter 8 Software Testing 56

User testing Types of user testing

 User or customer testing is a stage in the testing process  Alpha testing

in which users or customers provide input and advice on ▪ Users of the software work with the development team to test the
system testing. software at the developer’s site.

 User testing is essential, even when comprehensive  Beta testing

system and release testing have been carried out. ▪ A release of the software is made available to users to allow
▪ The reason for this is that influences from the user’s working them to experiment and to raise problems that they discover with
environment have a major effect on the reliability, performance, the system developers.
usability and robustness of a system. These cannot be replicated  Acceptance testing
in a testing environment.
▪ Customers test a system to decide whether or not it is ready to
be accepted from the system developers and deployed in the
customer environment. Primarily for custom systems.

30/10/2014 Chapter 8 Software Testing 58 30/10/2014 Chapter 8 Software Testing 59