SOFTWARE

ENGINEERING
CO3001

CHAPTER 9 – SOFTWARE QUALITY & Anh Nguyen-Duc

QUALITY ASSURANCE Quan Thanh Tho

WEEK 9
Adapted from https://iansommerville.com/software-engineering-book/slides/
TOPICS COVERED

 Software Quality & its importance

 Software testing
 Test planning
 Test driven development

Sep 2019 CHAPTER 8. SOFTWARE TESTING 2

 THE IMPORTANCE OF
SOFTWARE QUALITY

Sep 2019 CHAPTER 8. SOFTWARE TESTING 3

WHY IS SOFTWARE QUALITY IMPORTANT?
DEFINITION OF QUALITY

 (ISO) defines quality as the totality of characteristics of

an entity that bear on its ability to satisfy stated or
implied needs (ISO8042:1994) or the degree to which a
set of inherent characteristics fulfils requirements.
(ISO9000:2000).
 Conformance to requirements means the project s
processes and products meet written specifications.
 Fitness for use means a product can be used as it was
intended.
 Quality aspects:
 product: delivered to the customer
 process: produces the software product
 resources: (both the product and the process require
PROCESS QUALITY VS. PRODUCT QUALITY

 Quality can mean the difference between

excellence and disaster
 Airbus A400M Atlas crash in 2015, 4 killed
PROCESS QUALITY VS. PRODUCT QUALITY

“The black boxes attest to that

there are no structural defects
[with the aircraft], but we have a
serious quality problem in the
final assembly.”

“…either a weakness in the test

procedure of planes before they
fly, or a problem that results
from the implementation of
these procedures.”
PRODUCT OR PROCESS ISSUE?

 8/2016: Security breach with Bitcoin cost 72 mil.

Usd lost in market
SOFTWARE QUALITY ATTRIBUTES (1)
SOFTWARE QUALITY MODEL
 SOFTWARE TESTING

Sep 2019 CHAPTER 8. SOFTWARE TESTING 11

PROGRAM TESTING

 Testing is intended to show that a program does

what it is intended to do and to discover program
defects before it is put into use.
 Can reveal the presence of errors NOT their
absence.
 Testing is part of a more general verification and
validation process, which also includes static
validation techniques.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 12

PROGRAM TESTING GOALS

 To demonstrate to the developer and the customer

that the software meets its requirements.
 validation testing

 To discover situations in which the behavior of the

software is incorrect, undesirable or does not
conform to its specification.
 defect testing

Sep 2019 CHAPTER 8. SOFTWARE TESTING 13

QUALITY ASSURANCE

Sep 2019 CHAPTER 8. SOFTWARE TESTING 14

PSYCHOLOGY OF TESTING (1)

 A program is its programmer’s baby!

 Trying to find errors in one’s own program is like trying
to find defects in one’s own baby.
 It is best to have someone other than the programmer
doing the testing.
 Tester must be highly skilled, experienced
professional.
 It helps if he or she possesses a diabolical mind.
PSYCHOLOGY OF TESTING (2)

 Testing achievements depend a lot on what are the

goals.
 Myers says (79):
 If your goal is to show absence of errors, you will not
discover many.
 If you are trying to show the program correct, your
subconscious will manufacture safe test cases.
 If your goal is to show presence of errors, you will
discover large percentage of them.

Testing is the process of executing a program with the

intention of finding errors (G. Myers)
AN INPUT-OUTPUT MODEL OF PROGRAM TESTING

Sep 2019 CHAPTER 8. SOFTWARE TESTING 17

A MODEL OF THE SOFTWARE TESTING PROCESS

Sep 2019 CHAPTER 8. SOFTWARE TESTING 18

STAGES OF TESTING

 Development testing
 the system is tested during development to discover bugs
and defects.
 Release testing
 a separate testing team test a complete version of the
system before it is released to users.
 User testing
 users or potential users of a system test the system in
their own environment.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 19

DEVELOPMENT TESTING

carried out by the team developing the

 Unit testing: system.
 for individual program units or object classes
 focus on testing the functionality of objects or methods.
 Component testing:
 several individual units are integrated to create composite
components
 focus on testing component interfaces.
 System testing:
 some or all of the components in a system are integrated and
the system is tested as a whole
 focus on testing component interactions.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 20

UNIT TESTING

 Unit testing is the process of testing individual

components in isolation.
 It is a defect testing process.
 Units may be:
 Individual functions or methods within an object
 Object classes with several attributes and methods
 Composite components with defined interfaces used to
access their functionality.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 21

UNIT TESTING: BLACK-/WHITE-BOX TEST
from
requirement Compare actual
s output with require
output

from
requirement
s & key Confirm expected
design behaviour
elements

Gray-box: mix of black- and white-box

Sep 2019 testing CHAPTER 8. SOFTWARE TESTING 22
for (i=0; i<numrows; i++)
for (j=0; j<numcols; j++);
pixels++;

int minval(int *A, int n) {

int currmin;

for (int i=0; i<n; i++)

if (A[i] < currmin)
currmin = A[i];
return currmin;
}

Sep 2019 CHAPTER 8. SOFTWARE TESTING 23

switch (i) {
case 1:
do_something(1); break;
case 2:
do_something(2); break;
case 3:
do_something(1); break;
case 4:
do_something(4); break;
default:
break;
}

Sep 2019 CHAPTER 8. SOFTWARE TESTING 24

EQUIVALENCE PARTITIONING

Sep 2019 CHAPTER 8. SOFTWARE TESTING 25

INTERFACE TESTING

 Detect faults due to

 interface errors
 or invalid assumptions about interfaces.
 Interface types
 Parameter interfaces
 Shared memory interfaces
 Procedural interfaces
 Message passing interfaces

Sep 2019 CHAPTER 8. SOFTWARE TESTING 26

WHITE-BOX TESTING

 Statement
coverage
 Branch coverage
 Path coverage

Sep 2019 CHAPTER 8. SOFTWARE TESTING 27

REGRESSION TESTING
Test the system to check that changes
have not ‘broken’ previously working
code.
 Better with automated testing
 All tests are re-run every time a change is made to
the program.

 Tests must run ‘successfully’ before the change is

committed.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 28

AUTOMATED TESTING

 Whenever possible, unit testing should be

automated
 Use of a test automation framework (such as JUnit)

Sep 2019 CHAPTER 8. SOFTWARE TESTING 29

SYSTEM TESTING System testing during development = to
create a version of the system and then
testing the integrated system.
 Focus on testing the interactions between
components.
 System testing checks that components are compatible,
interact correctly and transfer the right data at the right
time across their interfaces.

 And tests the emergent behaviour of a system.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 30

TYPES OF SYSTEM TESTS
 Volume  Security
 Subject product to large amounts  Subject to compromise attempts.
of input.
 Resource usage
 Usability  Measure usage of RAM and disk
 Measure user reaction (e.g., score space etc.
1-10).
 Install-ability
 Performance  Install under various circumstances.
 Measure speed under various
circumstances.  Recoverability
 Force activities that take the
 Configuration application down.
 Configure to various hardware /
software  Serviceability
 Service application under various
 Compatibility situations.
 with other designated
applications  Load / Stress
 Subject to extreme data & event
 Reliability / Availability traffic
 Measure up-time over extended
period.
Sep 2019 CHAPTER 8. SOFTWARE TESTING 31
USE-CASE TESTING – TEST CASE
Test case SUC3 Correct PIN entry on first try
Test description A customer enters the PIN number correctly on the
first attempt.
Related screens
Pre-conditions 1. The expected PIN is known
2. Screen 2 is displayed
Actions 1. Screen 2 shows ‘- - - - ’
2. Customer touches 1st digit
3. Screen 2 shows ‘- - - * ’
4. Customer touches 2nd digit
5. Screen 2 shows ‘- - * * ’
6. Customer touches 3rd digit
7. Screen 2 shows ‘- * * * ’
8. Customer touches 4th digit
9. Screen 2 shows ‘* * * * ’
10. Customer touches Enter
11. Screen 5 is displayed
Inputs Correct PIN number: 1357
Expected Outputs Select Transaction screen is active
Sep 2019 Testing Software interface run in Windows 10 TESTING
CHAPTER 8. SOFTWARE 32
USE-CASE TESTING
The use-cases developed to identify system
interactions can be used as a basis for system
testing.

 Each use case usually involves several system

components so testing the use case forces these
interactions to occur.
 The sequence diagrams associated with the use case
documents the components and interactions that are
being tested.

Sep 2019 CHAPTER 8. SOFTWARE TESTING 33

PERFORMANCE TESTING
Part of release testing may involve testing
the emergent properties of a system, such
as performance and reliability.

 Tests at the system level

 Tests should reflect the profile of use of the system.
 Is usually a series of tests
 the load is steadily increased until the system performance
becomes unacceptable.
 Stress testing
 is a form of performance testing where the system is
deliberately overloaded to test its failure behaviour.
Sep 2019 CHAPTER 8. SOFTWARE TESTING 34
PERFORMANCE TESTING – TEST CASE

 Tests at the system level

 Tests should reflect the profile of use of the system.
 Is usually a series of tests
 the load is steadily increased until the system performance
becomes unacceptable.
 Stress testing
 is a form of performance testing where the system is
deliberately overloaded to test its failure behaviour.
Sep 2019 CHAPTER 8. SOFTWARE TESTING 35
 USABILITY TESTING

 Evaluating the usability aspect of the software by

testing it with representative users
 Where do I click next?
 Which page needs to be
navigated?
 Which Icon or Jargon represents
what?
 Error messages are not
consistent or effectively
displayed
 Session time not sufficient

Sep 2019 CHAPTER 8. SOFTWARE TESTING 36

 USABILITY TESTING – TEST CASE

 Evaluating the usability aspect of the software by

testing it with representative users

Sep 2019 CHAPTER 8. SOFTWARE TESTING 37

 TEST PLANNING

Sep 2019 CHAPTER 8. SOFTWARE TESTING 38

TEST PLAN

 Testing requires meticulous planning on an operational level and on a strategic

level
 The test manager needs to transpose such generic guidelines to create a concrete
testing strategy for the project at hand
 Test plan should state:
 Test object: which components, modules, neighboring systems, and interfaces (will) make up the system to
be tested
 test objectives: specific testing objectives and the criteria you are testing against for each test object and the
entire system
 Customize the testing process
 testing methods and techniques: overall testing approach and the testing techniques
 required infrastructure
 Success/ failure criteria: test metrics and threshold

Sep 2019 CHAPTER 8. SOFTWARE TESTING 39

A CASE – ATM TERMINAL SOFTWARE TESTING

 A simple version of ATM

machine
 Insert your card, type
PIN code and withdraw
cash from the machine

Sep 2019 CHAPTER 8. SOFTWARE TESTING 40

A CASE – ATM TERMINAL SOFTWARE TESTING

 List of designed
screens

Sep 2019 CHAPTER 8. SOFTWARE TESTING 41

A CASE – ATM TERMINAL SOFTWARE TESTING

 State diagram
from the Idle
state to the
Closed ATM
session

Sep 2019 CHAPTER 8. SOFTWARE TESTING 42

NOW ABOUT TESTING …

 We are developing and integrating all of these

modules/ screens
 Which test levels we can do`?
 What types of tests we can do?

Sep 2019 CHAPTER 8. SOFTWARE TESTING 43

TEST PLAN - EXAMPLE

Type of Description Number of tests Test Test data Test

tests types metric
Unit test Testing each screen to make At least 15 Whitebox Pin code, Path
coverage
sure each screen displays assertation:
blackbox Amount Line
according to its specification S1 – S15 coverage
to
Branch
withdraw coverage
System Testing possible different use At least 25 test Blackbox Pin code, Requireme
nt coverage
test/ Use case scenarios with successful cases as below
Amount
case test and unsuccessful ATM
to
transactions
withdraw
Usability Evaluate the user experience At least 2 test cases Blackbox Tasks to Usability
test with the ATM interfaces with the perform score
two scenarios with successful
Pin code,
and unsuccessful ATM
transactions Amount
to
withdraw
Performanc Simulating POS system’s 1 test with 100 Blackbox Number Transaction
time
e test transaction processing under transactions and of
Delayed
extreme conditions another test with transacti time
Sep 2019
Internet-based transactions 1000 transactions onsCHAPTER 8. SOFTWARE TESTING 44
USER ACCEPTANCE TEST (UAT)

 acceptance testing is a test conducted to determine

if the requirements of a specification or contract
are met
 It can look like a system test … with customer
involvement
 It is blackbox testing
 It may involve performance test, stress test,
usability test, etc
 Testers should be given real-life scenarios such as
the three most common or difficult tasks that the
Sep 2019 CHAPTER 8. SOFTWARE TESTING 45
STAGES IN THE ACCEPTANCE TESTING PROCESS

 Define acceptance criteria

 Plan acceptance testing
 Derive acceptance tests
 Run acceptance tests
 Negotiate test results
 Reject/accept system

Sep 2019 CHAPTER 8. SOFTWARE TESTING 46

TEST DRIVEN DEVELOPMENT

Sep 2019 CHAPTER 8. SOFTWARE TESTING 47

BUG TRIAGING

Sep 2019 CHAPTER 8. SOFTWARE TESTING 48

TEST-DRIVEN DEVELOPMENT

inter-leave testing and code development

Benefits of test-driven development

• Code coverage
• Regression testing
• Simplified debugging
Sep 2019 • System documentation CHAPTER 8. SOFTWARE TESTING 49