1

Software Quality Assurance

Đảm bảo chất lượng phần mềm
Lecture 4: Black Box Approach
for Test Case Designing

2
Contents
Những nội dung chính

• Motivation of Blackbox Testing

• Equivalence class partitioning

• Boundary value analysis

• Decision Table

• State Transition Testing

3
4.1. Blackbox Testing

4
What is Black box testing
• One of testing techniques allowing to design good test
cases
• Black box testing is a strategy in which testing is based on
the requirements and specifications
• Verify the output without inspecting the internal workings
• We do not know:
• How the box handles errors
• Whether the inputs are executing all paths of code

5
 Black Box vs. White Box
Black-Box vs. White-Box
• Black Box: functional testing
• Unit test • External/user view: • Internal/developer view:
– Check conformance with – Allows tester to be confident about
• Integration test specification test coverage

• System test • Abstraction from details: • Based on control or data flow:

– Source code not needed – Easier debugging
• Programmers & Test Engineers • Scales up: • Does not scale up:
& Quality Assurance Engineers – Different techniques at – Mostly applicable at unit and
different levels of granularity integration testing levels
• White Box: structural testing
• Unit test USE BOTH!
• Integration test
• Programmers & Test Engineers Lund University / Faculty of Engineering/ Department of Computer Science / Software Engineering Research Group

6
Black box testing techniques
• Equivalence class partitioning
• Boundary value analysis
• Decision Table
• State Transition Testing

7
4.2. Equivalence class
partitioning (ECP)

8
Equivalence Class Partitioning
Phân chia lớp tương đương
• A method testing that divides the input domain into
classes of data
• An Equivalence Class is
• Consist of a set of data that acts the same way
• Any data value within class is equivalent
• 2 classes
• Valid class
• Invalid class

9
Equivalent Class Testing
Kiểm thử lớp tương đương
• Step 1: Identify all equivalent classes from the requirement
and specification of the function
• Step 2: Pick up at least 1 value in each equivalence class to
create test case

Negative 0 Positive Negative 0 Positive

integer integer integer integer

Select -10
Select 100

Step 1 Step 2

10
Example: Insurance System
• Specification
• System shall reject over-age insurance applicants
• Reject male insurance applicants over the age of 80 years on day of
application
• Reject female insurance applicants over the age of 85 years on day
of application
• Identify equivalent classes?

11
Answer
• Input: Gender & Age
• Output: accept/reject
Classes Test Cases
C1: Input: Males over 80 T1: male, 83, reject
C2: Input: Males 80 or under T2: male, 56, accept
C3: Input: Females 85 or under T3: female, 83, accept
C4: Input: Females over 85 T4: female, 87, reject

• What’s about the invalid data?

12
Equivalence Class Testing Guidelines
Một số chú ý khi sử dụng phân lớp tương đương
If input condition
• is a range, e.g., x = [0..9]
• is an ordered list of values, e.g., owner = {1, 2, 3, 4}
• ==> one valid and two invalid classes are defined
• is a set, e.g., vehicle = {car, motorcycle, truck}
• is a "must be" condition (boolean), e.g., "first character of the identifier
must be a letter"
• ==> one valid and one invalid class are defined
• is anything else
• ==> partition further

13
Exercise 1: Program to determine employability

Age Employment Status

0-15 Don't hire

16-17 Can hire part-time

18-55 Can hire full-time

56-99 Don't hire

How many equivalent classes? How many test case should be designed?

14
Exercise 2: Examination Judgment Program
• Specification
• Two subjects: Maths and Physics
• Student passed if:
• Scores of both mathematics and physics are greater than or
equal to 70 out of 100
• average of mathematics and physics is greater than or equal to
80 out of 100
• Failed if otherwise

15
Exercise 2
Score of Average
Physics. Score is 80.

(2)
100
(5)
(1)
80 Score Math. Physics Result
(4)
70
(1) 55 85 Failed
(3)
(2) 67 97 Passed
60
Average
Score
(3) 96 68 Passed
is 80. (4) 77 80 Passed
40
(7) (6) (5) 85 92 Passed
(6) 79 58 Failed
20
(7) 52 58 Failed
Score of
Math.

0 20 40 60 70 80 100

16
Advantage of Equivalence Partitioning

Reduce the number of test case Ensure each class is tested

17
Disadvantages of Equivalence Partitioning
• The identification of equivalence classes relies on the
experiences of tester
• Not consider boundary values

18
4.3. Boundary Value
Analysis (BVA)

19
Boundary Value Analysis
Phân tích giá trị biên
• A supplementary method for the equivalence partitioning
method
• Allow to select test cases to represent each side of the class
boundaries
• Steps:
• Identify the equivalence classes
• Identify the boundary of each class
• Create test case for each boundary value by choosing one point on
the boundary, one point just below the boundary, and one point
just above the boundary

20
Example 14

Example – boundary value testing

• Employability Module Age Employment status
• 6 equivalence 0-15 Don’t hire

classes (4 valid 16-17 Can hire part-time

18-54 Can hire full-time
classes)
55-99 Don’t hire
• For each
boundary value,
pick up 3 values -1 0 1 15 16 17 17 18 19 54 55 56 98 99 100
on/below/upper
• 15 Test cases
Other test cases?

9/27/21 21

21
Example: Human’s age
• Another way to choose values
• One point on the boundary
• One point beside the boundary in invalid class

70 years old

18 60
Children Adult Elderly

17 61
30 years old

22
Advantage of Boundary Value Analysis
• Easy to understand and control
• Allow to complete the equivalence class method
• Achieve quality test cases

23
Limitations
• Does not work well with Boolean variables
• Very complex if variables are not independent
• Most suited to systems in which much of the input data
takes on values within ranges or within sets

24
 Example: The nextDate function
• Problem statement:
• nextDate is a function of three variables: month(M), day(D),
year(Y). It returns the date of the day after the input date. The
month, day and year variables have integer values subject to
these conditions:
• C1: 1 <= M <= 12
• C2: 1 <= D <= 31
• C3: 1850 <= Y <= 2050
• How many equivalent classes there are?

25
variables have integer values subject to these conditions:
• C1:1 <= month <=12;
•
Equivalent Classes
C2: 1<= day <= 31;
• C3: 1850 <= year <= 2050
Valid ECs
M1 = {month: 1 <= month <= 12} M2 = {month: month < 1}

D1={day:1<=day<=31} M2 = {month: month > 12}

Y1 = {year: 1850 <= year <= 2050} D2={day:day<1}

Can we define better equivalence classes?

What about combinations of M,D & Y?

26
 15

NextDate – multiple variables

Equivalent Classes
Remember?
• C1:1 <= month <=12;
• Which months have 30 days? 31 days?
• C2: 1<= day <= 31;
• February has 28
• C3: 1850 <=or 29<=
year days depend on whether year is leap
2050
yearCan
or we
notdefine better equivalence classes?
Valid ECs
M1 = {month has 30 days}
M2 = {month has 31 days}
M3 ={February}

D1={1<=day<=27}, D2={day=28}
D3= {day = 29}, D4= {day = 30}, D5 {day = 31}
Y1 = {year: year is a non-leap year}
Y2 = {year: year is a leap year}

What about combinations of M,D & Y?

27
Design Test cases
16

Case ID Month Day Year Expected

Output
C1 -1 15 1902 Value of
month not in
range
C2
C3
C4
…

There are too many combinations of M, D and Y è a lot of test case

28
4.4. Decision Table
Technique

29
Decision Table
Bảng quyết định
• Is an exellent tool to capture certain kinds of system
requirements and to document internal system design.
• Often used to record complex business rules that a system
must implement
• In addition, they can serve as a guide to creating test cases

30
General Form
General form:
Rule 1 Rule 2 … Rule p
Condition 1
Condition 2
Conditions
…
Condition n
Action 1
Action 2
Actions
…
Action m

31
Example: Car Insurance Discount
• Specification
• If married and number of employed years >=3 then discount 70%
• If married and number of employed years < 3 then
• If good student, discount 50%
• Otherwise, 20%
• If not married and number of employed years >=3, discount 60%
• In case of not married and number of employed years < 3
• If good student, discount 40%
• Otherwise 0%

32
 21

Answer
Example: car insurance discount
Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8

Married? Yes Yes Yes Yes No No No No

# years >= 3 >= 3 <3 <3 >= 3 >= 3 <3 <3

C employed

Good student? Yes No Yes No Yes No yes No

A Discount 70 70 50 20 60 60 40 0

8 test cases

33
 Answer 22

Example - simplified

Rule Rule 3 Rule 4 Rule Rule 7 Rule 8

1-2 5-6

Married? Yes Yes Yes No No No

# years >= 3 <3 <3 >= 3 <3 <3

employed 6 test cases
C
Good student? - Yes No - Yes No

A Discount 70 50 20 60 40 0

34
Some terms
Một số thuật ngữ 23

Definitions
Rule Rule 3 Rule 4 Rule Rule 7 Rule 8
1-2 5-6 Limited entry
table –
conditions are
Married? Yes Yes Yes No No No binary only
# years >= 3 <3 <3 >= 3 <3 <3
C employed Extended entry
table – conditions
Good student? - Yes No - Yes No
may take on
Discount 70 50 20 60 40 0 multiple values
A

Don’t-care-entry – condition is irrelevant

or not applicable; any value is OK

Decision tables are declarative – order between entries does not matter

35
Using Table Decision for Designing Test Cases
Sử dụng bảng Using decision tables for testing
quyết định để thiết kế ca kiểm 24

thử • Rules become test cases

• Conditions become inputs
• Actions become expected results:
Test Case 1 Test Case 2 … Test Case p
• Rules becomes Condition 1

Test Cases Inputs

Condition 2
…
• Conditions Condition n
become inputs Action 1

• Actions become Expected Action 2

Results …
expected results
Action m

36
Exercise 1: Triangle Program
• The program accepts three integers a,b,c as inputs
• These integers are interpreted as representing the lengths
of sides of a triangle
• These variables a,b,c must satisfy the following conditions
• a < b + c, b < a + c, c < a + b
• The program must determine whether the inputs are not a
triangle, an isosceles or an equilateral triangle

37
 26

Decision table
Decision table for the triangle program
Rule Rule Rule Rule Rule Rule Rule Rule Rule
1 2 3 4 5 6 7 8 9
a,b,c forms a
triangle
a=b
C
a=c
b=c
NaT
Isosceles
A
Scalene
Equilateral

38
Decision Table 27

Decision table for the triangle program

Rule Rule Rule Rule Rule Rule Rule Rule Rule
1 2 3 4 5 6 7 8 9
a,b,c forms a N Y Y Y Y Y Y Y Y
triangle
a=b _ Y Y Y Y N N N N
C
a=c _ Y Y N N Y Y N N
b=c _ Y N Y N Y N Y N
NaT
Isosceles
A
Scalene
Equilateral

39
 28

Decision table for the triangle program

Decision Table
Rule Rule Rule Rule Rule Rule Rule Rule Rule
1 2 3 4 5 6 7 8 9
a,b,c forms a N Y Y Y Y Y Y Y Y
triangle
a=b _ Y Y Y Y N N N N
C
a=c _ Y Y N N Y Y N N
b=c _ Y N Y N Y N Y N
NaT X

Impossible

Impossible

Impossible
Isosceles X X X
A
Scalene X
Equilateral X

40
Design Test Case
Testing the triangle program 29

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c

b<c+a

c<b+a
C
a=b

a=c

b=c

NaT

Isosceles
A
Scalene

Equilateral

41
Design Test Case
Testing the triangle program 30

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N
b<c+a -
a<b+a -
C
a=b -
a=c -
b=c -
NaT X
Isosceles
A
Scalene

Equilateral

42
 Design Test Case
Testing the triangle program 31

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N 4 2 1 NaT
b<c+a -
c<b+a -
C
a=b -
a=c -
b=c -
NaT X
Isosceles
A
Scalene

Equilateral

43
Design Test Case
Testing the triangle program 32

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N Y Y T 4 2 1 NaT
b<c+a - N Y T
c<b+a - - N T
C
a=b - - - T
a=c - - - T
b=c - - - T
NaT X
Isosceles
A
Scalene

Equilateral

44
Design Test Case
Testing the triangle program 33

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N Y Y T 4 2 1 NaT
b<c+a - N Y T 2 4 1 NaT
c<b+a - - N T 1 4 2 NaT
C
a=b - - - T 3 3 3 Eq
a=c - - - T
b=c - - - T
NaT X X X
Isosceles
A
Scalene

Equilateral X

45
Design Test Case
Testing the triangle program 34

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N Y Y T T T 4 2 1 NaT
b<c+a - N Y T T T 2 4 1 NaT
c<b+a - - N T T T 1 4 2 NaT
C
a=b - - - T T T 3 3 3 Eq
a=c - - - T F T
b=c - - - T T F
NaT X X X
Isosceles
A
Scalene

Equilateral X

46
Design Test Case
Testing the triangle program 35

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N Y Y T T T 4 2 1 NaT
b<c+a - N Y T T T 2 4 1 NaT
c<b+a - - N T T T 1 4 2 NaT
C
a=b - - - T T T 3 3 3 Eq
a=c - - - T F T
b=c - - - T T F
NaT X X X
Isosceles
A
Scalene

Equilateral X

47
Design Test Case
Testing the triangle program 36

1 2 3 4 5 6 7 8 9 10 11 a b c Exp
a<b+c N Y Y Y Y Y Y Y Y Y Y 4 2 1 NaT
b<c+a - N Y Y Y Y Y Y Y Y Y 2 4 1 NaT
c<b+a - - N Y Y Y Y Y Y Y Y 1 4 2 NaT
C
a=b - - - Y Y Y Y N N N N 3 3 3 Eq
a=c - - - Y N Y N Y Y N N
b=c - - - Y Y N N Y N Y N
NaT X X X 5 5 3 Iso
Isosceles X X X
A
Scalene X 5 3 5 Iso
Equilateral X 3 5 5 Iso
3 4 5 Sca

48
• C1:1 <= month <=12;
• C2: 1<= day <= 31;
Exercise 2: the nextDate function
• C3: 1850 <= year <= 2050
Can we define better equivalence classes?
Valid ECs
M1 = {month has 30 days}
M2 = {month has 31 days}
M3 ={February}

D1={1<=day<=27}, D2={day=28}
D3= {day = 29}, D4= {day = 30}, D5 {day = 31}
Y1 = {year: year is a non-leap year}
Y2 = {year: year is a leap year}

What about combinations of M,D & Y?

49
Decision Table for the
The NextDate Function
nextDate function
40

1 2 3 4
month in M30
month in M31
February
December
1<=day<28
C
Day=28
Day=29
Day=30
Day=31
Leap year
Increment Year
Increment Month
Increment Day
A
Reset year
Reset month
Reset day

50
 Decision Table for the nextDate function
The NextDate Function 41

1 2 3 4
month in M30 T - - -
month in M31 - T - -
February - - T -
December - - - T
1<=day<28 T T - T
C
Day=28 - - T -
Day=29 - - - -
Day=30 - - - - Can we simplify the conditions?
Day=31 - - - -
Leap year F F F F
Increment Year
Increment Month X
Increment Day X X X
A
Reset year
Reset month
Reset day X

51
Decision Table Function
The NextDate for the nextDate function 43

1 2 3 4
month

C day

year

Increment Year
Increment Month
Increment Day
A Reset year
Reset month
Reset day
Error

M31 = { 1,3,5,7,8,10 }; M30 = { 4,6,9,11 }; M2 = { 2 }; M12 = { 12 };D27={1,2,...,27};

D28={28},D29={29 },D30={30},D31={31} YN = not a leap year; YL = leap year

52
Decision Table for the nextDate function
44
1 2 3 4 5 6 7 8 9
month M30 M30 M30 M31 M31 M12 M12 M2 M2 M2 M2 M2 M2

day D27 D30 D31 D27 D31 D27 D31 D30 D27 D28 D28 D29 D29
D28 D28 D28 D31
C
D29 D29 D29
D30 D30
year - - - - - - - - - YL YN YL YN

Increment X
Year
Increment X X X X
Month
Increment X X X X X
Day
A
Reset year
Reset X X X
month
Reset day X X X
Error X X X

53
Design Test Cases
Test cases for The NextDate Function
45

Case ID Month Day Year Expected output

1-3 April 15 2001 April 16, 2001

4 April 30 2001 May 1, 2001

5 April 31 2001 Error

6-9 January 15 2001 January 16, 2001

10 January 31 2001 February 1, 2001

11-14 December 15 2001 December 16, 2001

15 December 31 2001 January 1, 2002

16 February 15 2001 February 16, 2001

17 February 28 2004 February 29, 2004

18 February 28 2001 March 1, 2001

19 February 29 2004 March 1, 2004

20 February 29 2001 Error

21, 22 February 30 2001 Error

54
Advantages
• This technique is useful for applications characterized by
any of the following:
• Prominent if-then-else logic
• Logical relationships among input variables
• Calculations involving subsets of the input variables
• Cause-and-effect relationships between inputs and outputs

55
 Disadvantages

• Decision table
may become very
large if we have Condition Rule
1
Rule 2 Rule 3 Rule 4 Rule 5 Rule 6

many conditions
• We need to factor
large tables into
smaller ones to
remove
redundancy Action

56
Exercise 3: Road charge
• A program that calculates road user charges for foreign
heavy vehicles
• Inputs:
• distance that the driver are intending to drive on a road
• the emission category of the vehicle (0 or 1 or 2)
• the number of axles of the truck
• Driver can pay for day or week
• Chargeable period for day is 00.00-24.00. For example, if
the journey begins at 21h30 on one day and finishes at
02.20 next day, payment for two days is required

57
Note that the chargeable period for day is 00.00-24.00. So, if a journey on a toll road begins, for
example, at 21.30 on one day and finishes at 02.30 next day, payment for two days is required.

Exercise 3: Tarif Tables

Tolls are paid in SEK.”

Toll Tariff 2014

Max
3 3 3 4 4 4
axles

Emission
0 1 2 or cleaner 0 1 2 or cleaner
Class

1 day 67 67 67 67 67 67

1 week 220 194 169 347 313 279

58
Exercise 4: Bus tarif
• A program allows to calculate automatically the monthly
bus tarif
• Inputs: age and distance
Distance <=10 km Distance > 10 km
• Age Infant charge 0 0
• 0-3 years old: infant charge Child charge 100 130
Adult charge 200 250
• 4-14 years old: child charge
Silver charge 160 200
• 15-59 years old: adult charge
• 60-99 years old: silver charge
• Do not treat at the age of 100 or more
• Distance: <=10 or >10 km

59
Exercise 4: Bus tarif (in EUR)

Distance <=10 km Distance > 10 km

Infant charge 0 0

Child charge 100 130

Adult charge 200 250

Silver charge 160 200

IT4501 - Software Engineering Department -

9/27/21 60
SoICT/HUST
Exercise 4:
• With input distance, list test cases which using Equivalent
and Boundary (if possible)
• With input age, list test cases which using Equivalent and
Boundary if possible
• List test cases combination of distance and age by
Equivalent method
• List test cases combination of distance and age by
Boundary method
• Create decision table base on input distance and age

61
Exercise 5: Register Form
• A register form allows to create a username and a password
and add the account to the database
• Specification:
• Username: from 3 to 15 characters, no space and special characters
including #,$,@
• Password: should be 8 digits, first character is not zero
• Using Equivalent Class and Boundary Value Analysis to
design test cases

62
4.5. State Transition
Testing

63
Basic ideas
• Applied when an application gives a different output for the
same input, depending on what has happened in the earlier
state
• The system can be in a (finite) number of different states
and the transitions from one state to another are
determined by rules
• Have to determine states, events, actions and transitions
that should be tested

64
Basic Terms
Thuật ngữ cơ bản
• The states that the system may occupy
• ex: a document is closed or opened
• The transitions from one state to another
• The events that cause a transition
• ex: the action “closing” a document cause the transition form the
state “opened” to “closed”
• The actions that result from a transition
• ex: an error message, a warning ...

65
Design Test Cases
• Create a set of test cases such that all states are visited at
least once. May miss important transitions.
• Create a set of test cases such that all states are triggered
at least once. May miss both states and transitions
• Create a set of test cases such that all transitions are
exercised at least once. Subsumes (includes) all-states and
all-events
• Stronger: cover all possible pairs of transitions (1-switch coverage)
• Even stronger: cover all possible triplets of transitions (2-switch
coverage)
• Create a set of test cases such that all paths are executed
at least once. Subsumes all others. Can be infeasible -
consider loops

66
Example 1: ATM machine
• Go to the ATM
• Insert card – Verify PIN
• Withdraw money
• Successful if sufficient balance
• Refused if insufficient balance
• User can try to type PIN 3 times – If not valid, card is
rejected
• How many states the system has?
• Draw the state-transition diagram

67
State Transition Diagram

• S1: Start State

• S2: Wait for PIN c
• S3: 1st try invalid
• S4: 2nd try invalid
• S5: 3rd try invalid
• S6: Card rejected
• S7: Access account

9/27/21 68

68
All states coverage
• TC1: Start State –
Valid PIN – Access
Account
• TC2: Start State – 1st
try invalid – 2nd try
invalid – 3rd try invalid
– Card Rejected

69

69
All transitions coverage

• Which test cases

will coverage all
transitions of the
system?

9/27/21 70

70
Example 2: Flight Booking System
52

Flight booking system

• The customer provides some information and makes a
booking.
The customer provides some information and makes a booking. He then
• He then has a certain amount of time to make the
has a certain amount of time to make the reservation.
reservation.
Event

giveInfo/ Booking
startPayTimer Made payMoney

Action Transition Paid

Entry point
State

71
 54

Flight Booking System Complete

Flight booking system complete

payMoney
giveInfo/ Booking
startPayTimer Made checkIn
Paid
cancel
cancel/refund
time Ticketed
Expires Cancelled/
Customer
Cancelled
/Unpaid fly
Used

72
Design Test Case for Flight Booking System
• Draw a table of states and events
• Identify test cases to satisfy all states coverage
• Identify test cases to satisfy all transitions coverage

73
 Thank you
for your
attention!!!

74