SOFTWARE QUALITY ENGINEERING

Ayesha
Kanwal
Lecture No. 3
 Review of Previous Lecture
2

 Testing: process of comparing "what is" with

"what ought to be."
 Levels of Testing:
 Unit Testing
 Integration Testing
 System Testing
 Acceptance Testing
 Regression Testing
 Types of Testing
 Black Box Testing
 White Box Testing
 Grey Box Testing
3 Black Box Testing
Also called Functional Testing
Black box testing is a strategy in

which testing is based solely on the

requirements and specifications.
Black box testing requires no

knowledge of the internal paths,

structure, or implementation of the
software under test (SUT).
 Applicability
4

 Black box testing can be applied at all

levels of system development—unit,
integration, system, and acceptance.
 Use of Black Box Testing
5

 Can’t look at particular implementation

 Sub-factors testable
 Output correctness, Documentation,
Availability
 Reliability
 Stress/load
 Security
 Training, Operational
 Maintainability
 Testability
 Portability
 Interoperability: software, equipment
 Black Box Testing Requirements (Inputs)
6

 What the system is supposed to do

 How do we know this?
 How is it described
 Need a model of the system
 Need a model of the tests
 Need a working component or
(sub)system
7
Techniques of Black Box
Testing
 Check list based testing
 Equivalence Class Testing
 Boundary Value Testing
 Decision Table Testing
 Pair-wise Testing
 State-Transition Testing
 Domain Analysis Testing
 Use Case Testing
Checklist based testing
 The use of informal lists forms the
rudimentary and implicit checklists

 Each item can be checked off when

corresponding testing was performed,
until every item on the lists is checked
off.
checklists for black-box or white-box
coverage testing
 Functional (black-box) checklists at different levels of abstraction
and granularity, ranging from major high-level functions for the
overall system to specific low-level functions of an individual unit
or components.

 Checklists of system elements (white-box) at different levels of

granularity, ranging from sub-systems and modules at the high
level to individual statements or data items at the low level.

 Checklists of various structures or features that cut through

different elements, such as list of call-pairs, consumers-and-
producers for certain resources, modules sharing some common
data, etc. These lists are concerned with implementation
structures, therefore can be classified as white-box checklists as
well
A high-level functional checklist for some
relational database products
hierarchical and combined checklists

 The most commonly used form is the

hierarchical checklist, with each item in
a higher-level checklist expandable to a
full checklist at a lower level until we
stop at a level of detail deemed enough
by some criteria.
limitations of general
checklists
 It would be hard to come up with checklists
that cover all the functional (black-box) or
structural (white-box) components

 Resulting in missed areas of coverage know

as “holes” in coverage

 To provide good coverage, a lot of overlaps

between different items in the checklists
may be introduced, resulting in redundant
 Partitions: Formal Definitions
 Partition of set S into subsets
G1, G2, …, Gn (Gi e S):

 Gi 's are mutually exclusive:

i, j , i  j  Gi  G j {}

 Gi 's are collectively

n exhaustive:

G
i 1
i S

 Each Gi forms an equivalent class.

 Formal conditions sometimes possible
 Often implicit by membership to Gi
Equivalence partitioning
Equivalence Partitioning
 Typically the universe of all possible test cases
is so large that you cannot try them all
 You have to select a relatively small number of
test cases to actually run
 Which test cases should you choose?
 Equivalence partitioning helps answer this
question
 It is a software test design technique that involves
dividing input values into valid and invalid
partitions and selecting representative values from
each partition as test data.
Equivalence Partitioning
 Partition the test cases into "equivalence classes"

 Each equivalence class contains a set of "equivalent"

test cases

 Two test cases are considered to be equivalent if we

expect the program to process them both in the same
way (i.e., follow the same path through the code)

 If you expect the program to process two test cases in

the same way, only test one of them, thus reducing
the number of test cases you have to run
Equivalence Partitioning

 First-level partitioning: Valid vs. Invalid test

cases

Valid Invalid
Equivalence Partitioning
 Partition valid and invalid test cases into
equivalence classes
Equivalence Partitioning
 Create a test case for at least one value from
each equivalence class
Example I (1)
 Consider an application that takes an
integer as input
 Let us suppose that the only legal
values are in the range [1..100]
 Which input value(s) will you use to
test this application?
Example I (2)
 The set of input values can be divided into
 A set of expected, or legal, inputs (E) containing all
integers in the range [1..100]
 A set of unexpected, or illegal, inputs (U ) containing
the remaining integers
All integers

U: Other integers

E: [1..100]
Example I (3)
 Assume that the application is required to
process all values in the range [1..50] in
accordance with requirement R1 and those in the
range [51..100] according to requirement R2.
 E is divided into two regions depending on the
expected behavior.
 Also assume that all invalid inputs less than 1
are to be treated in one way while all greater
than 100 are to be treated differently.
 This leads to a subdivision of U into two categories.
 All integers
<1

[51..100] >100

[1..50]
Example I (4)
 How many input values should we
use for testing the application?
Equivalence Partitioning
 Test selection using equivalence partitioning
allows a tester to divide the input domain into a
relatively small number of sub-domains.
 The sub-domains are disjoint.
 Each subset is known as an equivalence class.
 The four subsets shown in (a) constitute a partition
of the input domain while the subsets in (b) are
not.
 What if there is more than one
input variable?
Unidimensional Partitioning
 One way to partition the input domain is to
consider one input variable at a time. Thus each
input variable leads to a partition of the input
domain.

 We refer to this style of partitioning as

unidimensional equivalence partitioning or simply
unidimensional partitioning.
Multidimensional
Partitioning
 Another way is to consider the input domain I as the set product of the
input variables and define a relation on I. This procedure creates one
partition consisting of several equivalence classes.

 We refer to this method as multidimensional equivalence partitioning or

simply multidimensional partitioning.

 Multidimensional equivalence partitioning

Guarantees completeness of testing
A x B x C given domains A, B, C

 Multidimensional partitioning leads to a large number of

equivalence classes that are difficult to manage
manually.
 Many classes so created might be infeasible.
 Equivalence classes offer completeness of tests
Example II (1)
 Consider an application that requires two integer
inputs x and y. Each of these inputs is expected to
lie in the following ranges: 3 x7 and 5y9.
 How many pairs of (x, y) should we use to test this
application?
Example II (2)
 Using Unidimensional Partitioning

E1: x<3 E2: 3x7 E3: x>7 y ignored.

E4: y<5 E5: 5y9 E6: y>9 x ignored.

Example II (3)
 Using Multidimensional Partitioning

E1: x<3, y<5 E2: x<3, 5y9 E3: x<3, y>9

E4: 3x7, y<5 E5: 3x7, 5y9 E6: 3x7, y>9

E7: x>7, y<5 E8: x>7, 5y9 E9: x>7, y>9

Equivalence Partitioning
 Basic idea: consider input/output
domains and partition them into equiv.
classes
 For different values from the same class,
the software should behave equivalently
 Use test values from each class
 Example: if the range for input x is 2..5,
there are three classes: “<2”, “between
2..5”, “>5”
 Testing with values from different classes
is more likely to uncover errors than
testing with values from the same class
32
Equivalence Partitioning: Example 1

 When designing test cases, you may use

different definitions of “equivalence”, each of
which will partition the test case space
differently

For example: int Add(n1, n2, n3, …)
 Equivalence Definition 1: partition test

cases by the number of inputs (1, 2, 3, etc.)

 Equivalence Definition 2: partition test

cases by the number signs they contain

(positive, negative, both)
 Equivalence Definition 3: partition test

cases by the magnitude of operands (large

numbers, small numbers, both)
Example 2
 When designing test cases, you may use
different definitions of “equivalence”, each of
which will partition the test case space
differently
 Example: string Fetch(URL)
 Equivalence Definition 1: partition test cases by URL
protocol (“http”, “https”, “ftp”, “file”, etc.)
 Equivalence Definition 2: partition test cases by type of
file being retrieved (HTML, GIF, JPEG, Plain Text, etc.)
 Equivalence Definition 3: partition test cases by length
of URL (very short, short, medium, long, very long,
etc.)
 Etc.
Example 3
 Examples of equivalence classes
 Input x in a certain range [a..b]: this defines
three classes
 “x<a”,
 “a<=x<=b”,
 “b<x”
 Input x is boolean: classes “true” and “false”
 Choosing test values
 Choose a typical value in the middle of the
class(es) that represent valid input
 Also choose values at the boundaries of all
classes: e.g., if the range is [a..b], use a-1,a,
a+1, b-1,b,b+1
35
Example 4
 Suppose our specification says that the code
accepts input values between 4 and 24, and
also 3-digit integers
 What partitions can you have for this?
 One partition: number of inputs
 Classes are “x<4”, “4<=x<=24”, “24<x”
 Chosen values from the classes: 3,4,5, 14,
23,24,25
 Another partition: integer values
 Classes are “x<100”, “100<=x<=999”, “999<x”
 Chosen values from the classes: 99,100,101, 500,
998,999,1000
36
Example 5
Input Valid Equivalence Classes Invalid Equivalence Classes
A integer N such that:
-99 <= N <= 99
? ?
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
? ?
Example 5
Input Valid Equivalence Classes Invalid Equivalence Classes
A integer N such that: [-99, -10]
-99 <= N <= 99 [-9, -1]
0
[1, 9]
?
[10, 99]
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
? ?
Example 5
Input Valid Equivalence Classes Invalid Equivalence Classes
A integer N such that: [-99, -10] < -99
-99 <= N <= 99 [-9, -1] > 99
0 Malformed numbers
[1, 9] {12-, 1-2-3, …}
[10, 99] Non-numeric strings
{junk, 1E2, $13}
Empty value
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
? ?
Example 5
Input Valid Equivalence Classes Invalid Equivalence Classes
A integer N such that: [-99, -10] < -99
-99 <= N <= 99 [-9, -1] > 99
0 Malformed numbers
[1, 9] {12-, 1-2-3, …}
[10, 99] Non-numeric strings
{junk, 1E2, $13}
Empty value
Phone Number 555-5555
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
(555)555-5555
555-555-5555
200 <= Area code <= 999
?
200 < Prefix <= 999
Example 5
Test data Valid Equivalence Classes Invalid Equivalence Classes
A integer N such that: [-99, -10] < -99
-99 <= N <= 99 [-9, -1] > 99
0 Malformed numbers
[1, 9] {12-, 1-2-3, …}
[10, 99] Non-numeric strings
{junk, 1E2, $13}
Empty value
Phone Number 555-5555 Invalid format 5555555,
Area code: [200, 999] (555)555-5555 (555)(555)5555, etc.
Prefix: (200, 999] 555-555-5555 Area code < 200 or > 999
Suffix: Any 4 digits 200 <= Area code <= 999 Area code with non-numeric
200 < Prefix <= 999 characters
Similar for Prefix and Suffix
Class Activity
Group (2 Students)
 Equivalence Partitioning
Example
 Grocery Store software module
 Accepts the name of a grocery item and a list of the different sizes the item
comes in, specified in ounces.
 Name is to be alphabetic characters 2 to 15 characters in length. Each size may
be a value in the range of 1 to 48, whole numbers only.
 The sizes are to be entered in ascending order (smaller sizes first).
 A maximum of five sizes may be entered for each item. The item name is to be
entered first, followed by a comma, then followed by a list of sizes.
 A comma will be used to separate each size. Spaces (blanks) are to be ignored
anywhere in the input.

 Task: Drive all valid and invalid classes and fill the below
table
 Expected
# Test Data Classes Covered
Outcome
Any test data e.g. Name of your
1 T
xy,1 classes
 A ticketing system where children under
age 6 are allowed to travel for free, people
under 18 as well as senior people older
than 64 pay $10 while adults need to pay
$20. For testing this system, we can use
equivalence class partitioning to design our
tests.
 Derive equivalence classes for this system
 Think of the boundaries while deriving the
classes
Boundary value analysis and equivalence
classes
Boundary Value Analysis
 When choosing values from an equivalence class to test, use
the values that are most likely to cause the program to fail

 Errors tend to occur at the boundaries of equivalence classes

rather than at the "center"
 If (200 < areaCode && areaCode < 999) { // valid area code }
 Wrong!
 If (200 <= areaCode && areaCode <= 999) { // valid area
code }
 Testing area codes 200 and 999 would catch this error, but a
center value like 770 would not

 In addition to testing center values, we should also test

boundary values
 Right on a boundary
 Very close to a boundary on either side
Boundary Value Analysis -
examples
Input Boundary Cases
A number N such that:
-99 <= N <= 99
?
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
?
Boundary Value Analysis -
examples
Input Boundary Cases
A number N such that: -100, -99, -98
-99 <= N <= 99 -10, -9
-1, 0, 1
9, 10
98, 99, 100
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
?
Boundary Value Analysis -
examples

Input Boundary Cases

A number N such that: -100, -99, -98
-99 <= N <= 99 -10, -9
-1, 0, 1
9, 10
98, 99, 100
Phone Number Area code: 199, 200, 201
Area code: [200, 999] Area code: 998, 999, 1000
Prefix: (200, 999] Prefix: 200, 199, 198
Suffix: Any 4 digits Prefix: 998, 999, 1000
Suffix: 3 digits, 5 digits