CSC 404:

1
Software Testing and validation
Week 10

Dr NYAMSI 4/22/2019
2 Generality
 Testing is intended to show that a program does what it is
intended

 Testing is also to discover program defects before it is put into

use.

 When you test software, you execute a program using

artificial data.

Dr NYAMSI 4/22/2019
3 Generality
 The testing process has two distinct goals:
1. To demonstrate to the developer and the customer that the
software meets its requirements.
2. To discover situations in which the behavior of the software is
incorrect, undesirable, or does not conform to its specification.

 Testing cannot demonstrate that the software is free of

defects or that it will behave as specified in every
circumstance.

Dr NYAMSI 4/22/2019
4 Generality
 Verification and Validation is the process of checking that a
software system meets specifications and that it fulfills its
intended purpose.

 Some call it software quality control

 So, Testing is simply for validation

 Testing can only show the presence of errors, not their

absence

Dr NYAMSI 4/22/2019
5 Objectives:
 Understand the stages of testing from development to acceptance
testing by system customers;

 Have been introduced to techniques that help you choose test cases
that are geared to discovering program defects;

 Understand test-first development, where you design tests before writing

code and run these tests automatically;

 Know the important differences between component, system, and

release testing and be aware of user testing processes and techniques.

Dr NYAMSI 4/22/2019
6 Topics
 Model of the software testing

 Development testing

Unit testing

Choosing unit test cases

Component testing

System testing

Dr NYAMSI 4/22/2019
7 Topics
 Test-driven development

 Release testing

Requirement based-testing

Scenario testing

Performance testing

 User testing

Dr NYAMSI 4/22/2019
8 Model of the software testing

Dr NYAMSI 4/22/2019
9 Development testing
 It is where the system is tested during development to discover
bugs and defects.

 System designers and programmers are likely to be involved in the

testing process.

 Development testing includes all testing activities that are carried

out by the team developing the system.

 The tester of the software is usually the programmer who

developed that software, although this is not always the case.

Dr NYAMSI 4/22/2019
10 Development testing: Unit testing
 It is where individual program units or object classes are tested.

 Unit testing should focus on testing the functionality of objects or

methods.

 Individual functions or methods are the simplest type of component.

 Your tests should be calls to these routines with different input

parameters.

 You can use “test case” to design the function or method tests.

Dr NYAMSI 4/22/2019
11 Development testing: Unit testing

 When you are testing object classes, you should design your tests
to provide coverage of all of the features of the object.

 Test all operations associated with the object;

 Set and check the value of all attributes associated with the object;

 Put the object into all possible states. This means that you should
simulate all events that cause a state change.

 Whenever possible, you should automate unit testing.

Dr NYAMSI 4/22/2019
12 Development testing: Unit testing

 An automated test has three parts:

1. A setup part, where you initialize the system with the test case,
namely the inputs and expected outputs.

2. A call part, where you call the object or method to be tested.

3. An assertion part where you compare the result of the call with the
expected result. If the assertion evaluates to true, the test has been
successful; if false, then it has failed.

Dr NYAMSI 4/22/2019
13 Development testing: Choosing unit test cases

 Testing is expensive and time consuming, so it is important that you

choose effective unit test cases.

 Effectiveness, in this case, means two things:

 The test cases should show that, when used as expected, the
component that you are testing does what it is supposed to do.

 If there are defects in the component, these should be revealed by

test cases.

Dr NYAMSI 4/22/2019
14 Development testing: Choosing unit test cases

 You should therefore write two kinds of test case.

The first of these should reflect normal operation of a program

and should show that the component works.

The other kind of test case should be based on testing

experience of where common problems arise. It should use
abnormal inputs to check that these are properly processed
and do not crash the component.

Dr NYAMSI 4/22/2019
15 Development testing: Choosing unit test cases
 We present two possible strategies that can be effective in helping
you choose test cases.
1. Partition testing, where you identify groups of inputs that have
common characteristics and should be processed in the same way.
You should choose tests from within each of these groups.

2. Guideline-based testing, where you use testing guidelines to choose

test cases. These guidelines reflect previous experience of the kinds
of errors that programmers often make when developing
components.

Dr NYAMSI 4/22/2019
16 Development testing: Component testing
 Software components are often composite components that are
made up of several interacting objects.

 Component testing is where several individual units are integrated

to create composite components.

 Component testing should focus on testing component interfaces.

 There are different types of interface between program

components and, consequently, different types of interface error
that can occur

Dr NYAMSI 4/22/2019
17 Development testing: Component testing

 Parameter interfaces: These are interfaces in which data or

sometimes function references are passed from one component
to another. Methods in an object have a parameter interface.

 Shared memory interfaces: These are interfaces in which a block

of memory is shared between components. Data is placed in the
memory by one subsystem and retrieved from there by other sub-
systems.

Dr NYAMSI 4/22/2019
18 Development testing: Component testing

 Procedural interfaces: These are interfaces in which one

component encapsulates a set of procedures that can be called
by other components. Objects and reusable components have
this form of interface.

 Message passing interfaces: These are interfaces in which one

component requests a service from another component by
passing a message to it. A return message includes the results of
executing the service.
Dr NYAMSI 4/22/2019
19 Development testing: Component testing
 Interface errors are one of the most common forms of error in complex
systems:

 Interface misuse: A calling component calls some other component and

makes an error in the use of its interface.

 Interface misunderstanding: A calling component misunderstands the

specification of the interface of the called component and makes
assumptions about its behavior.

 Timing errors: These occur in real-time systems that use a shared memory or
a message-passing interface.

Dr NYAMSI 4/22/2019
20 Development testing: Component testing
 Some general guidelines for interface testing are:

1. Examine the code to be tested and explicitly list each call to an external
component. Design a set of tests in which the values of the parameters to
the external components are at the extreme ends of their ranges.

2. Where pointers are passed across an interface, always test the interface
with null pointer parameters.

3. Where a component is called through a procedural interface, design tests

that deliberately cause the component to fail.

Dr NYAMSI 4/22/2019
21 Development testing: Component testing
 Some general guidelines for interface testing are:
4. Use stress testing in message passing systems. This means that you
should design tests that generate many more messages than are
likely to occur in practice.

5. Where several components interact through shared memory, design

tests that vary the order in which these components are activated.

 Remark: Inspections and reviews can sometimes be more cost

effective than testing for discovering interface errors.

Dr NYAMSI 4/22/2019
22 Development testing: System testing
 System testing during development involves integrating
components to create a version of the system and then testing
the integrated system.

 System testing checks that components are compatible, interact

correctly and transfer the right data at the right time across their
interfaces.

 It obviously overlaps with component testing but there are two

important differences.

Dr NYAMSI 4/22/2019
23 Development testing: System testing
1. During system testing, reusable components that have been separately
developed and off-the-shelf systems may be integrated with newly
developed components. The complete system is then tested.

2. Components developed by different team members or groups may be

integrated at this stage. System testing is a collective rather than an
individual process. In some companies, system testing may involve a
separate testing team with no involvement from designers and
programmers.

Dr NYAMSI 4/22/2019
24 Development testing: System testing
 When you integrate components to create a system, you get
emergent behavior.

 Therefore system testing should focus on testing the interactions

between the components and objects that make up a system.

 For most systems, it is difficult to know how much system testing is

essential and when you should to stop testing.

 Testing, therefore, has to be based on a subset of possible test

cases.

Dr NYAMSI 4/22/2019
25 Development testing: System testing

 System testing may be based on experience of system usage and

focus on testing the features of the operational system.

 All system functions that are accessed through menus should be

tested.

 Combinations of functions that are accessed through the same

menu must be tested.

 Where user input is provided, all functions must be tested with both
correct and incorrect input.

Dr NYAMSI 4/22/2019
26 Development testing: System testing

 Automated system testing is usually more difficult than automated

unit or component testing.

Dr NYAMSI 4/22/2019
27 Test-driven development
 Test-driven development (TDD) is an approach to program
development in which you interleave testing and code
development.

Dr NYAMSI 4/22/2019
28 Test-driven development
 Some benefits of test-driven development are:
1. Code coverage: Every code segment that you write should have at least
one associated test. Code is tested as it is written so defects are discovered
early in the development process.

2. Regression testing: A test suite is developed incrementally as a program is

developed.

3. Simplified debugging: When a test fails, it should be obvious where the

problem lies.

4. System documentation: The tests themselves act as a form of

documentation that describe what the code should be doing.

Dr NYAMSI 4/22/2019
29 Test-driven development
 One of the most important benefits of test-driven
development is that it reduces the costs of regression testing.

 Test-driven development has proved to be a successful approach

for small and medium-sized projects.

Dr NYAMSI 4/22/2019
30 Release testing
 Release testing is the process of testing a particular release of
a system that is intended for use outside of the development
team.

 A separate testing team tests a complete version of the

system before it is released to users.

 The aim of release testing is to check that the system meets

the requirements of system stakeholders

Dr NYAMSI 4/22/2019
31 Release testing
 There are two important distinctions between release testing and
system testing :

1. A separate team that has not been involved in the system

development should be responsible for release testing.

2. System testing by the development team should focus on

discovering bugs in the system (defect testing). The objective of
release testing is to check that the system meets its requirements
and is good enough for external use (validation testing).

Dr NYAMSI 4/22/2019
32 Release testing
 The primary goal of the release testing process is to convince
the supplier of the system that it is good enough for use.

 Release testing is usually a black-box testing process where tests

are derived from the system specification.

 The system is treated as a black box whose behavior can only

be determined by studying its inputs and the related outputs.

 Another name for this is ‘functional testing’.

Dr NYAMSI 4/22/2019
33 Release testing: Requirement based-testing
 A general principle of good requirements engineering practice is
that requirements should be testable.

 Testing a requirement does not mean just writing a single test.

 You normally have to write several tests to ensure that you have
coverage of the requirement.

 You should also maintain traceability records of your requirements-

based testing, which link the tests to the specific requirements that
are being tested.

Dr NYAMSI 4/22/2019
34 Release testing: Scenario testing

 Scenario testing is an approach to release testing where you

devise typical scenarios of use and use these to develop test
cases for the system.

 A scenario is a story that describes one way in which the

system might be used.

 Scenarios should be realistic and real system users should be

able to relate to them.

Dr NYAMSI 4/22/2019
35 Release testing: Scenario testing

 When you use a scenario-based approach, you are normally

testing several requirements within the same scenario.

 As well as checking individual requirements, you are also

checking that combinations of requirements do not cause
problems.

Dr NYAMSI 4/22/2019
36 Release testing: Performance testing

 Once a system has been completely integrated, it is possible to

test for emergent properties, such as performance and reliability.

 Performance tests have to be designed to ensure that the system

can process its intended load.

 This usually involves running a series of tests where you increase the
load until the system performance becomes unacceptable.

Dr NYAMSI 4/22/2019
37 Release testing: Performance testing

 This type of testing has two functions:

1. It tests the failure behavior of the system.

2. It stresses the system and may cause defects to come to light

that would not normally be discovered.

Dr NYAMSI 4/22/2019
38 User testing

 User or customer testing is a stage in the testing process in which

users or customers provide input and advice on system testing.

 Users or potential users of a system test the system in their own

environment.

 For software products, the ‘user’ may be an internal marketing

group who decide if the software can be marketed, released,
and sold.

Dr NYAMSI 4/22/2019
39 User testing
 In practice, there are three different types of user testing:
1. Alpha testing, where users of the software work with the
development team to test the software at the developer’s site.

2. Beta testing, where a release of the software is made available to

users to allow them to experiment and to raise problems that they
discover with the system developers.

3. Acceptance testing, where customers test a system to decide

whether or not it is ready to be accepted from the system
developers and deployed in the customer environment.

Dr NYAMSI 4/22/2019
40 User testing
 Alpha testing is often used when developing software products that are
sold as shrink-wrapped systems.

 Beta testing takes place when an early, sometimes unfinished, release of a

software system is made available to customers and users for evaluation.

 Acceptance testing is an inherent part of custom systems development. It

takes place after release testing. It involves a customer formally testing a
system to decide whether or not it should be accepted from the system
developer.

 Acceptance implies that payment should be made for the system.

Dr NYAMSI 4/22/2019
41 User testing
 Acceptance testing process.

Dr NYAMSI 4/22/2019
42 Summary
 Testing can only show the presence of errors in a program. It cannot
demonstrate that there are no remaining faults.

 Development testing is the responsibility of the software development team.

A separate team should be responsible for testing a system before it is
released to customers. In the user testing process, customers or system users
provide test data and check that tests are successful.

 Development testing includes unit testing, in which you test individual

objects and methods; component testing, in which you test related groups
of objects; and system testing, in which you test partial or complete systems.

Dr NYAMSI 4/22/2019
43 Summary
 When testing software, you should try to ‘break’ the software by using
experience and guidelines to choose types of test cases that have been
effective in discovering defects in other systems.

 Wherever possible, you should write automated tests. The tests are
embedded in a program that can be run every time a change is made to
a system.

 Test-first development is an approach to development where tests are

written before the code to be tested. Small code changes are made and
the code is refactored until all tests execute successfully.

Dr NYAMSI 4/22/2019
44 Summary
 Scenario testing is useful because it replicates the practical use of the
system. It involves inventing a typical usage scenario and using this to derive
test cases.

 Acceptance testing is a user testing process where the aim is to decide if

the software is good enough to be deployed and used in its operational
environment.

Dr NYAMSI 4/22/2019