Software Testing and Quality

Assurance
Part 6
Input Space Partitioning

1
 Outline
Input Space Partitioning (ISP)
Input Domains
Partitioning Domains
Partition Selection
Modeling the Input Domain
Interface-based approach
Functionality-based approach
Combinations Strategies Criteria

2
 Input Domains
The input domain to a program contains all
the possible inputs to that program
For even small programs, the input domain
can be so large that it might as well be
infinite
Testing is fundamentally about choosing
finite sets of values from the input domain

3
 Input Domains
Input parameters define the scope of
the input domain
Parameters to a method
Data read from a file
Global variables
User level inputs
Domain for each input parameter is
partitioned into regions
At least one value is chosen from each
region

4
 Benefits of ISP
Can be equally applied at several levels
of testing
Unit
Integration
System
Relatively easy to apply with no
automation
Easy to adjust the procedure to get
more or fewer tests
 No implementation knowledge is
needed
just the input space

5
 Partitioning Domains
Domain D
Partition scheme q of D
The partition q defines a set of blocks,
Bq = b1 , b2 , … bn
The partition must satisfy two properties
:
1.blocks must be pairwise disjoint (no overlap)
2.together the blocks cover the domain D
(complete)

6
 Partitioning Domains

b1 b2 bi  bj = ,  i  j, bi, bj

b3  bB= D
q

b  Bq

7
 Properties of Blocks
If the blocks are not complete or disjoint,
that means the partition scheme have not
been considered carefully enough

They should be reviewed carefully, like any

design attempt

Different alternatives should be considered

We model the input space in five steps…

8
 Choosing a Partition Scheme
Choosing (or defining) partitions seems
easy, but is easy to get wrong
Consider the “order of files F”
So we have a set of files F that will be used
as input and we will divide the files based
on their order characteristic and coming up
with the following three blocks
b1 = sorted in
ascending order
b2 = sorted in
descending order
b3 = arbitrary order
9
 Choosing Partitions

b1 = sorted in ascending order Solution:

b2 = sorted in descending order Each characteristic should
address just one property
b3 = arbitrary order
But, What if the number of files is File F sorted ascending
of length 1? - b1 = true
- b2 = false
Then that file will be in all three
blocks … File F sorted descending
- b1 = true
That is, disjointness is not satisfied - b2 = false

10
 Using Blocks – Assumptions
Choose a value from each block
Each value is assumed to be equally
useful for testing
Application to testing
Find characteristics in the inputs :
parameters, semantic descriptions, …
Partition each characteristics, i.e. break them
down into blocks
Choose tests by combining values from
characteristics

11
 Modeling the Input Domain
Step 1 : Identify testable functions to
identify characteristics
Individual methods have one testable
function
In a class, the characteristics are often the
same for several methods. So it is useful to
come up with a set of characteristics for the
entire class.
Programs have more complicated
characteristics—modeling documents such as
UML use cases can be used to design
characteristics

12
 Modeling the Input Domain
Step 2 : Find all the parameters
Often fairly straightforward, even mechanical
Important to be complete
Methods : Parameters and state (non-local)
variables used
Components : Parameters to methods and
state variables
System : All inputs, including files and
databases

13
 Modeling the Input Domain
Step 3 : Model the input domain
This is the most creative design step in
applying ISP
The domain is scoped by the parameters
The structure is defined in terms of
characteristics
Each characteristic is partitioned into sets of
blocks
Each block represents a set of values of the
parameters
From the perspective of that particular
characteristic, the values in each block are
considered equivalent.
14
 Modeling the Input Domain
Step 5 : Refine combinations of blocks into
test inputs
Choose appropriate values from each block

15
 Two Approaches to Input
Domain Modeling
1. Interface-based approach
 Develops characteristics directly from
individual input parameters
 Simplest application
 Can be partially automated in some
situations

16
 1. Interface-Based Approach
Mechanically consider each parameter in
isolation
Some domain and semantic information won’t
be used
Could lead to an incomplete IDM
Ignores relationships among parameters.
For example there are 2 parameters a and b,
each which are Boolean (binary). The
interface approach would require we create a
partition with 4 blocks as it assumes that each
values combination is a block on its own.

17
 2. Functionality-Based
Approach
 Develops characteristics from a behavioral view of
the program under test
Identify characteristics that correspond to the
intended functionality
Can incorporate domain and semantic knowledge
Can use relationships among parameters
Using the previous example, it can be
determined based on the domain knowledge (for
example) that a and b cannot be both equal to
false or true at the same time. This removes two
blocks from the total of 4 in the Interface based
approach. Less blocks = less testing.

18
 2. Functionality-Based
Approach
Modeling can be based on requirements,
not implementation

19
 2. Functionality-Based
Approach
Functionality-based approach
 Harder to develop—requires more design
effort
 May result in better tests, or fewer tests
that are as effective

20
 Identifying Functionalities,
Parameters and Characteristics
A creative engineering step
More characteristics means more tests
More blocks in each characteristics
means more tests

21
 Modeling the Input Domain
Strategies for identifying values :
Include valid, invalid and special values
Sub-partition some blocks
Explore boundaries of domains
Include values that represent “normal use”
Try to balance the number of blocks in each
characteristic
Check for completeness and disjointness

22
 Modeling the Input Domain
Step 4 : Apply a test criterion to choose
combinations of values
A test input has a value for each parameter
A test input belongs to one block for each
characteristic
Choosing all combinations is usually
infeasible
Coverage criteria allow subsets to be chosen
to control the number of combinations

23
 Choosing Combinations of
Values
Once characteristics and partitions are
defined, the next step is to choose test
values
We use criteria – to choose effective
subsets
The most obvious criterion is to choose all
combinations …

24
 Choosing Combinations of
Values

All Combinations (ACoC) : All combinations of

blocks from all characteristics must be used.

• Number of tests is the product of the number of blocks in each

characteristic : Q
 (B )
i=1 i
• The can be way too many in some cases. For example if there is 5
characteristics with 3 blocks each that would be 3x3x3x3x3 = 243
tests!!
25
 ISP Criteria – Each Choice
 64 tests for TriTyp is almost certainly way too
many
 One criterion comes from the idea that we
should try at least one value from each block
Each Choice (EC) : One value from each block for each
characteristic must be used in at least one test case.

Number of tests is the number of blocks in

the largest characteristic

26
 ISP Criteria – Pair-Wise
 Each choice yields few tests – cheap but
perhaps ineffective
 Another approach asks values to be combined
with other values

Pair-Wise (PW) : A value from each block for each

characteristic must be combined with a value from every block
for each other characteristic.

27
 ISP Criteria – Pair-Wise
Number of tests is at least the product of
two largest characteristics

Q Q
(Max (B )
i=1 i ) * (Max (B )
j=1, j!=i j )

28
 ISP Criteria –T-Wise
A natural extension is to require
combinations of t values instead of 2

t-Wise (TW) : A value from each block for each group of t

characteristics must be combined.

29
 ISP Criteria –T-Wise
 Number of tests is at least the product of t
largest characteristics
 If all characteristics are the same size, the
formula is

Q t
(Max (B
i=1 i
))
• If t is the number of characteristics Q, then all combinations
• That is … Q-wise = AC (All Combinations)
• t-wise is expensive and benefits are not clear

30
 ISP Criteria – Base Choice
 Testers sometimes recognize that certain values are
important
 This uses domain knowledge of the program

Base Choice (BC) : A base choice block is chosen for each

characteristic, and a base test is formed by using the base choice
for each characteristic. Subsequent tests are chosen by holding all
but one base choice constant and using each non-base choice in
each other characteristic.

31
 ISP Criteria – Base Choice
Number of tests is one base test + one test
for each other block.

32
 ISP Criteria – Multiple Base
Choice
Testers sometimes have more than one logical base choice

Multiple Base Choice (MBC) : One or more base choice blocks are
chosen for each characteristic, and base tests are formed by using
each base choice for each characteristic. Subsequent tests are
chosen by holding all but one base choice constant for each base
test and using each non-base choices in each other characteristic.

33
 ISP Criteria – Multiple Base
Choice
If there are M base tests and mi base
choices for each characteristics:

For TriTyp: Base

2, 2, 2 2, 2, 0 2, 0, 2 0, 2, 2
2, 2, -1 2, -1, 2 -1, 2, 2
1, 1, 1 1, 1, 0 1, 0, 1 0, 1, 1
1, 1, -1 1, -1, 1 -1, 1, 1
34
 ISP Coverage Criteria
Subsumption
All Combinations
Coverage
AC

T-Wise Multiple Base

Coverage Choice Coverage
TW MBC

Pair-Wise Base Choice

Coverage Coverage
PW BC

Each Choice
Coverage
EC
35
 Constraints Among
Characteristics
Some combinations of blocks are
infeasible
“less than zero” and “scalene” … not
possible at the same time
These are represented as constraints
among blocks
Two general types of constraints
A block from one characteristic cannot be
combined with a specific block from another
A block from one characteristic can ONLY BE
combined with a specific block form another
characteristic

36
 Constraints Among
Characteristics
Handling constraints depends on the
criterion used
AC, PW, TW : Drop the infeasible pairs
BC, MBC : Change a value to another non-
base choice to find a feasible combination

37
 Example Handling
Constraints
Sorting an array
Input : variable length array of arbitrary type
Outputs : sorted array, largest value, smallest value

38
 Characteristics:
• Length of array
• Type of elements
• Max value
• Min value
Blocks from other
• Position of max value characteristics are
• Position of min value irrelevant

Partitions: Blocks must be

• Len { 0, 1, 2..100, 101..MAXINT } combined

• Type { int, char, string, other }

• Max { 0, 1, >1, ‘a’, ‘Z’, ‘b’, …,
‘Y’ }
Blocks must be
• Min {…} combined

39
• Max Pos { 1, 2 .. Len-1, Len }