Testing

Software Engineering
Sommerville
-Chapter 4,27 and 28
(some components also based on
Information Technology Project Management
Fourth Edition, K. Schwalbe- Ch8)

INFO5990 – Prof. Practice in IT

 Quality Management

A recap

INFO5990 – Prof. Practice in IT

 Quality Management (page 85)
• Software quality management can be structured into
three main activities :
– Quality assurance: The establishment of organisational
procedures and standards that lead to high-quality
software.
– Quality planning: The selection of appropriate procedures
and standards from this framework, adapted for a specific
quality software project.
– Quality control: The definition and enactment of processes
that ensure the software development team have followed
project quality procedures and standards.

Page 2
INFO5990 – Prof. Practice in IT
Quality Management (page 85)
• Quality management
– provides an independent check on the
software development process.
– Is based on the assumption (from
manufacturing) that the quality of the
development process directly affects the
quality of delivered products.
• In software development the relationship
between software processes and the
quality of the product is more complex but
still significant.
Page 3
INFO5990 – Prof. Practice in IT
Modern Quality Management

• Modern quality management:

– Requires customer satisfaction.

– Prefers prevention to inspection.

– Recognizes management responsibility for

quality.

Page 4
INFO5990 – Prof. Practice in IT
 Quality of Product
IDEA (conforms to
Finished
Product
Requirements)

test against Review

Feasibility Study

test against User Acceptance

User Requirements

test against System Test

System Design

Program Design test against Program Test

Coding

Quality Assurance Process

Page 5
INFO5990 – Prof. Practice in IT
Testing

• Many IT professionals think of testing as

a stage that comes near the end of IT
product development.

• Testing should be done during almost

every phase of the IT product
development life cycle

Page 6
INFO5990 – Prof. Practice in IT
Figure 8-4. Testing Tasks in the Software
Development Life Cycle

Page 7
INFO5990 – Prof. Practice in IT
Types of Tests (ch 4 – page 18)
• Component/Unit testing - individual component (often
a program) are tested to ensure they operate correctly.
– (Integration testing tests functionally grouped components).

• System testing components are integrated to make up

‘the system’. Tests are directed at finding errors from the
unanticipated interaction between components.

• User acceptance testing – final test before it moved to

operation. Tested with data supplied by the customer.
Test it meets requirements.

Page 8
INFO5990 – Prof. Practice in IT
Specialist type testing
• Usability testing
– (eg. Is it the best design to do the job for which it is meant?)
• Capacity testing
– (eg. how many people can use this system before it breaks?)
• Performance testing
– (eg. Does it meet the response time requirements?)
• Tuning..

These require more specialist technical skills and

specific tools

Page 9
INFO5990 – Prof. Practice in IT
Eliciting requirements – Validation (from prev
lecture)
• Manual requirements reviews:
– Consistency: Defined requirements do not
contradict each other
– Completeness: Degree to which all stated, unstated
and implied requirements have been recorded and
described
– Testability/Verifiability: The degree to which
tests can be developed that would prove the
requirements have been implemented correctly
• Prototyping- provides an executable model
that can be experimented with
• Test Cases generated
Page 10
INFO5990 – Prof. Practice in IT
Figure 8-5. Gantt Chart for Building Testing into a
Systems Development Project Plan

Page 11
INFO5990 – Prof. Practice in IT
CLOS-TEL - Software Test Plan

Process and Templates

INFO5990 – Prof. Practice in IT

 Test management
• Describes the testing to be conducted across the
full project lifecycle
– Unit/component
– System
– UAT
– Other
• Describes the tools and techniques including
– Managing test cases – creation, execution
– Test data generation
– Error tracking
• Test environment to be used.
Page 13
INFO5990 – Prof. Practice in IT
Test case identification
• It is essential for testing management
that every test case should have a
unique identification that can be tracked
throughout the process.
• Test cases must also be tracked back to
the requirement they relate to:
– Nothing should be tested that is not related
to a requirement
– No requirement should be ‘untested’.
Page 14
INFO5990 – Prof. Practice in IT
Software Test Plan

• Refer to the STP template

Page 15
INFO5990 – Prof. Practice in IT
 Test management - STP
• Simplified Test Management Plan
• Ch2 - Describes the testing to be conducted
across the full project lifecycle
• Ch 2.1 - Uses unique test case number for each
type of testing (you can define these)
• Ch 2.2 - Defines when test case will be executed
• Ch 3 & 4 – Technical and other testing (TBC later)
• Ch 5.1 - Maps the UAT test cases to requirements
(or change request)
• Ch 5.2 – Describes each test case in detail for
UAT
• Ch 6 - Describes how errors / bugs will be tracked
Page 16
INFO5990 – Prof. Practice in IT
The PCI System
• Asset Management group
– Register the new PCs
– Allocate a PC to a user
– Allocate a temporary PC if one is being fixed
• Helpdesk / Maintenance
– Search for current location, PC history and description
• PC’s user
– Update the current location
• Auditors
– Full list of all PCs and locations for verification
• Management
– Adhoc reporting

Page 17
INFO5990 – Prof. Practice in IT
Scenario definition for PCI

PC Registration

Asset
Management
PC Allocation

PC Search and Maintenance

update
PC user

PC audit list

Asset Auditor

Ad hoc reporting
Manager
Page 18
INFO5990 – Prof. Practice in IT
Use Case Example: PC Registration
Summary The asset manger registers a new PC in the asset
management system

Actors Asset manager

Preconditions New PC arrives in the store room.

Description A new PC is unpacked from the storeroom. The

make, model, serial number are to be noted and the
next asset number is to be automatically assigned
Exceptions PC already exists or PC incomplete or broken

Post conditions: Asset recorded is available for allocation to use

Page 19
INFO5990 – Prof. Practice in IT
Requirement Definition – example for PCI
PCIR001: Registering an asset
Description: The asset manger registers a new PC in the asset management
system see ‘PC Registration Use Case’ for more details and the proposed
screen layouts Prototype ‘PC Registration’.
Status: Approved
Benefit: Crucial

Summary The asset manger registers a new PC in the asset

management system

Actors Asset manager PC Make

PC Model
Preconditions None

PC Serial
Description A new PC is unpacked from the storeroom. The
make, model, serial number are to be provided
and the next asset number is to be automatically Number
assigned
Exceptions PC already exists

Post Asset recorded is available for allocation to use

conditions:

Page 20
INFO5990 – Prof. Practice in IT
 Test Case
Identifier of Test Case WEBU0001

Brief Description (of what is being tested): Asset

Registration - A new PC is
registered in the asset management system by the asset manager
Type of test (Functional, End-to-End) Functional
Configuration - i.e. what other software must be present for this to work
The asset manager must have a valid login
Asset table must be setup so next number to be allocated is A0001
Input data
Test 1: Enter Make = Compaq, Model = Presario, Serial Number = 0123456
Test 2: Enter the same details again
Expected output
Test1 : Asset number A0001 is assigned to this PC
Test 2: Error message ‘Asset is already registered’ is returned, asset is not registered again.
Execution History:
Date/time Who Result Comments

Page 21
INFO5990 – Prof. Practice in IT
CLOS-TEL - Software Test Plan

Defect or Bug Tracking

INFO5990 – Prof. Practice in IT

Defect or bug tracking
• This process is required to ensure that
bugs/defect found during testing are:
– Captured
– Allocated to someone to be fixed
– Fixed and passed through the testing
process again
• The test case must then be scheduled
for re-testing (in the current test stage)
• Closed (if now fixed) or ‘fixed again’

Page 23
INFO5990 – Prof. Practice in IT
Bug severity
• All bugs are not created ‘even’ ie. Some are more
severe than others
– Some will prevent any further testing occurring (eg. Inability to
logon)
– Some may prevent sections of the system being tested.
– Some may be minor formatting or spelling errors on a screen
or report
• Bugs should be treated in the same way (using the
same process) but some move through the process
with more urgency than others
• If you categorise the bugs then even if some are
outstanding a decision can be made on whether to
progress to the ‘next stage’ (even if that next stage is
operation)

Page 24
INFO5990 – Prof. Practice in IT
Testing Alone Is Not Enough
• Watts S. Humphrey, a renowned expert on
software quality, defines a software defect as
anything that must be changed before delivery of
the program.
• Testing does not sufficiently prevent software
defects because:
– The number of ways to test a complex system
is huge.
– Users will continue to invent new ways to use a
system that its developers never considered.

Page 25
INFO5990 – Prof. Practice in IT
 Quality Management - Stage Containment
Stage Containment is a development concept that seeks to detect, contain and
correct errors in the stage where they are created. Stage containment helps to
ensure a higher quality result throughout the process, as errors created in on
stage have a smaller change of reaching the latter stages.

Requirements Design Build & Test

ENTRY CRITERIA:
EXIT CRITERIA: • Availability of development
• Signed-off environment
ENTRY CRITERIA: EXIT CRITERIA:
Requirements
• Signed off Detailed • Signed-off Detailed Design
Work Plan
• Signed-off Test Strategy

Entry Criteria = Exit Criteria + Other Criteria

Page 26
INFO5990 – Prof. Practice in IT
Who does the testing?
• If a developer tests a module they coded they
are testing to prove it is correct
• If a tester tests the same module their aim is
to find ways it doesn’t work

• Who therefore is more likely to find the bugs?

• Is it practical to have someone else test when
the knowledge of the code is with the
developer?
• Are the same skills used when developing
and testing?
Page 27
INFO5990 – Prof. Practice in IT
Summary – Testing
• Testing is a full lifecycle process
– Start by planning all testing upfront – Test Management Plan (or
STP)
– Testing can start at the requirements stage - can do ‘paper tests’ on
requirements to ensure that they can be tested
• Test case must be:
– Uniquely identified
– Tracked directly to one or more requirements
– Tracked throughout the testing process
• Bugs must be:
– Tracked from creation through to retesting
– Should be allocated severity levels to enable the more severe ones
to be given higher priority
• Tools are available to assist with all these processes.

Page 28
INFO5990 – Prof. Practice in IT