Software Testing & Quality Assurance

Prof. Dr. Tariq Javid

Graduate School of Engineering Sciences & Information Technology
Faculty of Engineering Sciences & Technology
Hamdard University
Spring 2024
Functional Testing Techniques

2
 Introduction

• William E. Howden developed the idea of functional testing of programs

while visiting the International Mathematics and Statistics Libraries (IMSL)
in Houston in 1977–1978
• IMSL libraries are a comprehensive set of mathematical and statistical
functions that programmers can embed into their software applications
• IMSL uses proven technology that has been thoroughly tested, well
documented, and continuously maintained

3
Function, 1/4

4
Function, 2/4

5
Function, 3/4

6
Function, 4/4

7
 Key Elements of a Function

• Three key elements of a function are its input, output, and expected
transformation of input to output
• Ignoring the details of the actual transformation of input to output, we
analyze the domains of the input and the output variables of programs to
generate test data

8
Key Concepts in Functional Testing

9
 Types of Variables

• One can identify the domain of an input or an output variable by analyzing

the requirements specification and the design documents
• Types of variables are commonly used as input to and output from a large
number of systems for numeric calculations, the case of Howden
• Numeric variables, arrays, substructures, and subroutine arguments; and
their important values

10
 Numeric Variable

• The domain of a numeric variable is specified in one of two ways

• A set of discrete values, for example, {23, 79}

• A few contiguous segments of values, e.g., {min_value … max_value}

11
12
 Input, Output

• Module accepts three input variables: MODE, CLOCK, ADDRESS

• Generates values of the output variable: INDEX

13
 Input

• MODE: {23, 79}

• CLOCK: 28-bit unsigned number, upper 27 bits
MIN = 0x0000000, MAX = 0xFFFFFFF
• ADDRESS: 48-bit unsigned number, lower 20 bits
MIN = 0xyyyyy0000000, MAX = 0xzzzzzFFFFFFF

14
 Output

• INDEX: Range of values

MIN = 0, MAX = 22 if MODE = 23
MIN = o, MAX = 78 if MODE = 79

15
 Selection Criteria, 1/2

• Having characterized the domain of a numeric variable as a discrete set of

values or as a set of contiguous segments, test data are chosen by applying
different selection criteria depending upon the use of those variables
(i) input, (ii) output, (iii) dualuse, and (iv) multipletype

16
 Selection Criteria, 2/2

• A dual-use variable is one that holds an input to the system under test at the
beginning of system execution and receives an output value from the
system at some point thereafter
• A multiple-type variable is one that can hold input (or even output) values of
different types, such as numeric and string, at different times

17
 Selection Criteria for Input Variables, 1/3

• Input domain is a discrete set of values, tests involve each value

• Domain of the input variable MODE consists of the set {23, 79}
• Module is tested at least once with MODE = 23 and at least once = 79

18
 Selection Criteria for Input Variables, 2/3

• If the domain of a variable consists of one or more segments of values, then

test data are selected as follows:
• Consider the minimum value of a segment
• Consider the maximum value of a segment
• Consider a typical representative value in a segment

19
 Selection Criteria for Input Variables, 3/3

• Consider certain values which have special mathematical properties. These

values include 0, 1, and real numbers with small absolute values
• Consider, if possible, values lying outside a segment. Here the idea is to
observe the behavior of the program in response to invalid input
• In case the conceptual minimum (maximum) of a variable is −∞ (+∞), then a
large negative (positive) value is chosen to represent −∞ (+∞)

20
 Selection Criteria for Output Variables, 1/3

• If the domain of an output variable consists of a small set of discrete values,

then the program is tested with input which results in the generation of each
of the output values

21
 Selection Criteria for Output Variables, 2/3

• The output variable of the frequency hopping box has a domain of two
discrete sets {0, . . . , 22} and {0, . . . , 78} for MODE = 23 and MODE = 79,
respectively
• The frequency hopping box must be adequately tested so that it produces
each of the output values as desired
• For a large set of discrete values, the program is tested with many different
inputs that cause the program to produce many different output values
22
 Selection Criteria for Output Variables, 3/3

• If an output variable has a domain consisting of one or more segments of

numbers, then the program is tested as follows:
• Test the program with different inputs so that it produces the minimum values of
the segments
• Test the program with different inputs so that it produces the maximum values of
the segments
• Test the program with different inputs so that it produces some interior values in
each segment
23
 Selection Criteria for Dual-Use Variables

• A dualuse variable often serves as an input to a program (or function) and

holds the output from the program (or function) at the end of the desired
computation
• Consider a test case such that the program produces an output value which
is different from the input value of the same dual-use variable
• Consider a test case such that the program produces an output value which
is identical to the input value of the same dual-use variable.
24
 Selection Criteria for Multiple-Type Variables

• Sometimes an input variable can take on values of different types

• For example, a variable may take on values of type integer in one program invocation and of
type string in another invocation
• If an input or output variable can take on values of different types, then the following criteria
are used:
• For an input variable of multipletype, the program is tested with input values of all the types
• For an output variable of multipletype, the program is tested with different inputs so that the
variable holds values of all different types

25
 Array

• An array holds values of the same type, such as integer and real
• Individual elements of an array are accessed by using one or more indices
• In some programming languages, such as MATLAB, an array can hold values
of both integer and real types

26
 Array Properties

• Array can have one or more dimensions … to control loops, max. min., etc.
• Individual array elements are considered as distinct numeric variables
• Portion of an array can be collectively interpreted as a distinct substructure
with specific application-dependent properties … diagonal, upper triangular,
lower triangular, etc.

27
 Substructure

• In general, a structure means a data type that can hold multiple data elements
• In the field of numerical analysis, a matrix structure is commonly used
• For example, a set of n linear equations in n unknown quantities is represented
by an n × (n + 1) matrix
• The n rows and the first n columns of the matrix represent the coefficients of
the n linear equations, whereas the (n + 1)th column represents the constants of
the n equations
28
 Subroutine Arguments, 1/2

• Some programs accept input variables whose values are the names of
functions
• Such programs are found in numerical analysis and statistical applications
• Functional testing requires that each value of such a variable be included in
a test case

29
 Subroutine Arguments, 2/2

• Consider a program P calling a function f (g, param_list), where f () accepts a

list of parameters denoted by param_list and another parameter g of
enumerated type
• When f () is invoked, it invokes the function held in g
• Let the values of g be represented by the set {g1, g2, g3}
• We design three test cases to execute f (g1, list1), f (g2, list2), and f (g3, list3)
such that, eventually, g1(), g2(), and g3() are executed
30
 Test Vector, 1/3

• A test vector, also called test data, is an instance of the input to a program
• It is a certain configuration of the values of all the input variables
• Values of individual input variables chosen must be combined to obtain a
test vector

31
 Test Vector, 2/3

• If a program has n input variables var1, var2, . . . , varn which can take on k1,
k2, . . ., kn special values, respectively, then there are k1 × k2 × · · · × kn
possible combinations of test data
• For module, if k1 = 2 for MODE, k2 = 3 for CLOCK, and k3 = 3 for ADDRESS;
then possible combinations of test data = 2 x 3 x 3 = 18

32
Test Vector, 3/3

33
 Need to Reduce Number of Test Vectors

• Example: Tax filing software system

• All possible combinations of few special values of large number of input variables is
challenging
• For n input variables, each of which can take on k special values, there are k^n test
vectors, for example, more than one million test vectors for k = 3 and n = 20
• Therefore, there is a need to identify a method for reducing the number of test
vectors obtained by considering all possible combinations of all the sets of special
values of input variables
34
 Reducing Number of Test Vectors

• Howden suggested that we produce all possible combinations of special

values of variables falling in the same functionally related subset
• In this way, the total number of combinations of special values of the input
variables is reduced

35
 Identifying Functionally Related Variables

• It is difficult to give a formal definition of the idea of functionally related

variables, but it is easy to identify them
• Variables are functionally related if appearing in the same:
• Assignment statement
• Branch predicate, for example, the condition part of an if statement

36
Reducing Number of Test Vectors through
Identifying Functionally Related Variables

37
38
39
Testing a Function in Context, 1/4

40
 Testing a Function in Context, 2/4

• Let us consider a program P and a function f in P as shown in Figure 9.4

• The variable x is an input to P and input to f as well
• Suppose that x can take on values in the range [−∞, +∞] and that f is called
only when the predicate x ≥ 20 holds

41
 Testing a Function in Context, 3/4

• If we are unaware of the predicate x ≥ 20, then we are likely to select the
following set of test data to test P: x = +k, x = −k, x = 0; k is a number with a
large magnitude
• Function f will be invoked just once for x = +k, assuming that k ≥ 20, and it
will not be invoked when P is run with the other two test data
• Testing function f in isolation vs. testing function f in context

42
 Testing a Function in Context, 4/4

• Two data points (x = −k and x = 0) are invalid data as they fall outside the range of x for f in P
• Valid range of x for f is [20, +∞], and functional testing in context requires us to select the
following values of x:
x = k where k ≫ 20
x = y where 20 < y ≪ k
x = 20
where the symbols ≪ and ≫ are read as “much larger” and “much smaller,” respectively.

43
 Functional Testing Main Points

• Identify the input and the output variables of the program and their data
domains
• Compute the expected outcomes as illustrated in Figure 9.5a for selected
input values
• Determine the input values that will cause the program to produce selected
outputs as illustrated in Figure 9.5b

44
45
 Generating Test Data by Analyzing Input
Domain

• The number of test cases obtained from an analysis of the input domains is
likely to be too many because of the need to design test vectors
representing different combinations of special values of the input variables.
• Generation of expected output for a certain test vector is relatively simple
• This is because a test designer computes an expected output from an
understanding and analysis of the specification of the system

46
 Generating Test Data by Analyzing Output
Domain
• The number of test cases obtained from an analysis of the output domains is
likely to be fewer compared to the same number of input variables because
there is no need to consider different combinations of special values of the
output variables.
• Generating an input vector required to produce a chosen output value will
require us to analyze the specification in the reverse direction
• Reverse analysis will be a more challenging task than computing an expected
value in the forward direction
47
 Functional Testing at the Computing Element
Level
• We have discussed the ways to apply the idea of functional testing to an entire
program
• However, the underlying concept, that is, analyzing the input and output
domains of a program, can as well be applied to individual modules, functions,
or even lines of code aka code block
• This is because every computing element can be described in terms of its
input and output domains, and hence the idea of functional testing can be
applied to such a computing element
48
Example

49
50
51
 Developing Functional Test Cases

• Methodology for developing functional test cases is an analytical process that decomposes
specification of a program into different classes of behaviors
• Functional test cases are designed for each class separately by identifying the input and
output domains of the class
• Identification of input and output domains help in classifying the specification into different
classes
• However, often, in practice, the total number of input and output combinations can be very
large
• Several well-known techniques are available to tackle this issue
52
 Functional Testing Techniques

• Pairwise Testing
• Equivalence Class Testing
• Boundary Value Analysis
• Decision Tables
• Random Testing
• Error Guessing
• Category Partition
53