Functional (Black Box) Testing

1
 Outline:

⚫ Specification / Black Box Testing

⚫ Specification-based test construction techniques

⚫ Black Box Testing [ Equivalence portioning, Boundary Value

Analysis] ,
⚫ White Box Testing and
⚫ Grey Box Testing
⚫ Its Advantages and Disadvantages

2
 How do you test a system?

3
Specification Based Testing
 Classification based on the source of information to derive test cases:

⚫ Black-box testing (functional, specification-based)

[ Equivalence portioning, Boundary Value
Analysis]
⚫ White-box testing (structural, program-based)
⚫ Grey box testing(Classes, Objects, Design)

Input
determined by... Result
Actual
output
… requirements* Black box compared
with
* from previous phase required

4
 Specification Based Testing
⚫ There are numerous forms of testing all with a slightly different purpose and focus.

Black Box Testing

• Black box testing verifies the functionality of an application—not necessarily
having specific knowledge of the application’s code/internal structure.

• Black box tests are normally based on requirements and the functionality
specified in them.
• Their purpose is to confirm that a given input reliably produces the anticipated
output condition.
• This testing process is performed by quality assurance (QA) teams.

5
 Specification Based
Testing
White Box Testing
• White box testing is based on the knowledge of the internal logic of an application’s
code and includes tests such as coverage of code statements, branches, paths, and
conditions.
• This testing process is generally performed by software developers by using a
targeted set of prearranged test cases. The purpose of white box testing is to confirm
internal integrity and logic of the artifact and code.

Gray Box Testing

Gray-box testing is a combination of white-box testing and black-box testing.

The aim of this testing is to search for the defects, if any, due to improper structure or
improper usage of applications. A gray-box tester partially knows the internal structure,
which includes access to the documentation of internal data structures as well as the
algorithms used. Gray-box testers require both high-level and detailed documents
6 describing the application, which they collect in order to define test cases.
 Black, Gray and White-box Testing

Input determined by... Actual output

compared
… requirements with
required output

Gray box As for black-

… requirements &
and white
key design elements box
testing

White box
Confirmation
…design of
expected
elements behavior

7
 Characteristics of Testable Software
 Operable: The better it works (i.e., better quality), the easier it is to test

 Observable: Incorrect output is easily identified; internal errors

are automatically detected

 Controllable: The states and variables of the software can be controlled

directly by the tester

 Decomposable: The software is built from independent modules that can be

tested independently

 Simple: The program should exhibit functional, structural, and code simplicity

 Stable: Changes to the software during testing are infrequent and do not

invalidate existing tests

 Understandable: The architectural design is well understood; documentation

8 is available and organized
 Black Box Testing
⚫ Black box testing is a technique of software testing which examines the
functionality of software without peering into its internal structure or coding.
The primary source of black box testing is a specification of requirements that
is stated by the customer.

⚫ In this method, tester selects a function and gives input value to examine its
functionality, and checks whether the function is giving the expected output
or not. If the function produces the correct output, then it is passed in testing,
otherwise failed. The test team reports the result to the development team
and then tests the next function. After completing testing of all functions if
there are severe problems, then it is given back to the development team for
correction.

9
 Generic Steps of Black Box Testing
• The black box test is based on the specification of requirements, so it is examined in
the beginning.

• In the second step, the tester creates a positive test scenario and an adverse test
scenario by selecting valid and invalid input values to check that the software is
processing them correctly or incorrectly.

• In the third step, the tester develops various test cases such as decision table, all
pairs test, equivalent division, error estimation, cause-effect graph, etc.
• The fourth phase includes the execution of all test cases.
• In the fifth step, the tester compares the expected output against the actual output.

• In the sixth and final step, if there is any flaw in the software, then it is cured and
tested again.

10
 Black Box Testing Steps
• Black box testing involves testing external interfaces to ensure
that the code meets functional and nonfunctional requirements.

The various steps involved in black box testing are the following:
1. Create test plans

2. Test the external interfaces

3. Perform load testing
4. Perform stress testing
5. Perform security testing
6. Perform globalization testing
11
11
 1. Create test plans

• The first step in the process of black box testing is to create

prioritized test plans.
• We can prepare the test cases for black box testing even before
you implement the application block.
• The test cases are based on the requirements and the functional
specification documents.
• The requirements and functional specification documents help
you extract various usage scenarios and the expected output in
each scenario.
The detailed test plan document includes test cases for:
• Testing the external interfaces with various types of input
• Load testing and stress testing
• Security testing
• Globalization testing
12
12
 2. Test the external interfaces
• Test the external interfaces for various type of inputs using
automated test suites, such as N Unit suites and custom prototype
applications.
• Ensure that the application block exposes interfaces that address
all functional specifications and requirements.
To perform this validation testing, do the following:
• Prepare a checklist of all requirements and features that are
expected from the application block.
• Create test harnesses (join) , such as N Unit, and small "hello
world"' applications to use all exposed APIs of the test
application block.
• Run the test harnesses.

13
13
 Cont.…
Testing for various types of inputs. After ensuring that the application block
exposes the interfaces that address all of the functional specifications, you
need to test the robustness of these interfaces.
You need to test for the following input types:
• Randomly generated input within a specified range
• Boundary cases for the specified range of input
• The number zero testing if the input is numeric
• The null input
• Invalid input or input that is out of the expected range

14
14
 Perform load testing Load test the application block to analyze the behavior at
various load levels.
• This ensures that it meets all performance objectives that are stated as
requirements.
Perform stress testing Stress test the application block to analyze various
bottlenecks and to identify any issues visible only under extreme load conditions,
such as race conditions and contentions.
Perform security testing Test for possible threats in deployment scenarios.
• Deploy the application block in a simulated target environment and try to
hack the application by exploiting any possible weakness of the application
block.
Perform globalization testing Execute test cases to ensure that the application
block can be integrated with applications targeted toward locales other than the
default locale used for development.

15
15
 Test
Procedure
⚫ The test procedure of black box testing is a kind of process in which the tester
has specific knowledge about the software's work, and it develops test cases
to check the accuracy of the software's functionality.
⚫ It does not require programming knowledge of the software. All test cases are

designed by considering the input and output of a particular function.

⚫ A tester knows about the definite output of a particular input, but not about

how the result is arising. There are various techniques used in black box
testing for testing like decision table technique, boundary value analysis
technique, state transition, All-pair testing, cause-effect graph technique,
equivalence partitioning technique, error guessing technique, use case
technique and user story technique.

16
 Test
Cases
⚫ Test cases are created considering the specification of the requirements.
⚫ These test cases are generally created from working descriptions of the

software including requirements, design parameters, and other specifications.

⚫ For the testing, the test designer selects both positive test scenarios by taking

valid input values and adverse test scenarios by taking invalid input values to
determine the correct output.
⚫ Test cases are mainly designed for functional testing but can also be used for

non-functional testing.
⚫ Test cases are designed by the testing team, there is no any involvement of

the development team of software.

17
 Test Case Design Techniques
⚫ The test case design technique or methods or approaches are need to be

followed by every test engineer while writing the test cases to achieve
the maximum test coverage. If we follow the test case design technique,
then it became process-oriented rather than person-oriented.
⚫ The test case design technique ensures that all the possible values that

are both positive and negative are required for the testing purposes. In
software testing, there are three major test case design techniques which
are as follows:
1. Error Guessing

2. Equivalence Partitioning

3. Boundary Value Analysis[BVA]

18 4. Decision Table
 Test Case Design Techniques

1: Error Guessing

19
 1. Error Guessing
• In this section, we will see the first test case design technique that is Error guessing
techniques.
• Error guessing is a technique in which there is no specific method for identifying the error.
It is based on the experience of the test analyst, where the tester uses the experience to
guess the problematic areas of the software.
• It is a type of black box testing technique which does not have any defined structure to
find the error.
• In this approach, every test engineer will derive the values or inputs based on their
understanding or assumption of the requirements, and we do not follow any kind of rules
to perform error guessing technique.
• The accomplishment of the error guessing technique is dependent on the ability and
product knowledge of the tester because a good test engineer knows where the bugs are

20
most likely to be, which helps to save lots of time.
 Examples of Error guessing method:

⚫ A function of the application requires a mobile number which must be of 10 characters.

Now, below are the techniques that can be applied to guess error in the mobile number
field:
• What will be the result, if the entered character is other than a number?
• What will be the result, if entered characters are less than 10 digits?
• What will be the result, if the mobile field is left blank?
⚫ After implementing these techniques, if the output is similar to the expected result, the
function is considered to be bug-free, but if the output is not similar to the expected
result, it is sent to the development team to fix the defects.
⚫ However, error guessing is the key technique among all testing techniques as it depends
on the experience of a tester, but it does not give surety of highest quality benchmark. It
does not provide full coverage to the software. This technique can yield a better result if
combined with other techniques of testing.
21
2. Equivalence Partitioning

⚫ Equivalence partitioning is a technique of software testing in which input data is

divided into partitions of valid and invalid values, and it is mandatory that all
partitions must exhibit the same behavior. If a condition of one partition is true, then
the condition of another equal partition must also be true, and if a condition of one
partition is false, then the condition of another equal partition must also be false.
The principle of equivalence partitioning is, test cases should be designed to cover
each partition at least once. Each value of every equal partition must exhibit the
same behavior as the other.

⚫ The equivalence partitions are derived from the requirements and specifications of
the software. The advantage of this approach is, it helps to reduce the time of testing
due to a smaller number of test cases from infinite to finite. It is applicable at all
levels of the testing process.
22
 Guidelines for Defining Equivalence Classes
• If an input condition specifies a range, one valid and two invalid
equivalence classes are defined
Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}

• If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}

• If an input condition specifies a member of a set, one valid and one

invalid equivalence class are defined
Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other
x}

• If an input condition is a Boolean value, one valid and one invalid class
are define
Input: {true condition} Eq classes: {true condition}, {false condition}
23
Examples of Equivalence Partitioning technique
Assume that there is a function of a software application that accepts a particular number of
digits, not greater and less than that particular number. For example, an OTP number which
contains only six digits, less or more than six digits will not be accepted, and the application
will redirect the user to the error page.

1. OTP(One Time Password) Number = 6 digits

24
Examples of Equivalence Partitioning technique
⚫ A function of the software application accepts a 10 digit mobile
number.

2. Mobile Number = 10 digits

25
Examples of Equivalence Partitioning technique
⚫ In both examples, we can see that there is a partition of two equally valid and invalid
partitions, on applying valid value such as OTP of six digits in the first example and
mobile number of 10 digits in the second example, both valid partitions behave same,
i.e. redirected to the next page.

⚫ Another two partitions contain invalid values such as 5 or less than 5 and 7 or more
than 7 digits in the first example and 9 or less than 9 and 11 or more than 11 digits in
the second example, and on applying these invalid values, both invalid partitions
behave same, i.e. redirected to the error page.

⚫ We can see in the example, there are only three test cases for each example and that is
also the principal of equivalence partitioning which states that this method intended
to reduce the number of test cases.

26
3. Boundary Value Analysis
⚫ Boundary value analysis is one of the widely used test case design technique for black
box testing. It is used to test boundary values because the input values near the
boundary have higher chances of error.

⚫ Whenever we do the testing by boundary value analysis, the tester focuses on, while
entering boundary value whether the software is producing correct output or not.

⚫ Boundary values are those that contain the upper and lower limit of a variable. Assume
that, age is a variable of any function, and its minimum value is 18 and the maximum
value is 30, both 18 and 30 will be considered as boundary values.

⚫ The basic assumption of boundary value analysis is, the test cases that are created using
boundary values are most likely to cause an error.

27
 3. Boundary Value Analysis
⚫ The 18 and 30 are the boundary values that's why tester pays more attention to these
values, but this doesn't mean that the middle values like 19, 20, 21, 27, 29 are ignored.
Test cases are developed for each and every value of the range.
Testing of boundary values is done by making valid and invalid
partitions. Invalid partitions are tested because testing of
output in adverse conditions is also essential.

Example:

• Imagine, there is a function that accepts a number

between 18 to 30, where 18 is the minimum and 30 is the
maximum value of a valid partition, the other values of
this partition are 19, 20, 21, 22, 23, 24, 25, 26, 27, 28 and
29.
• The invalid partition consists of numbers that are less than
18 such as 12, 14, 15, 16, and 17, and more than 30 such
as 31, 32, 34, 36, and 40. Tester develops test cases for
both valid and invalid partitions to capture the behavior of
the system on different input conditions.
2
8
 ⚫ The software system will be passed in the test if it accepts a valid number and
gives the desired output, if it is not, then it is unsuccessful. In another
scenario, the software system should not accept invalid numbers, and if the
entered number is invalid, then it should display error massage.

⚫ If the software which is under test, follows all the testing guidelines and
specifications then it is sent to the releasing team otherwise to the
development team to fix the defects.

29
 Boundary Value Analysis

Using BVA- Range

When the programming element is a range type, we can arrive at test cases using BVA

as follows:

For a range of values bounded by a and b, then test:

(Minimum Value - precision) , Minimum Value

(Minimum Value + precision), (Maximum Value - precision), Maximum Value

(Maximum Value + precision)

30
 Guidelines for Boundary Value Analysis
• If an input condition specifies a range bounded by values a and b, test cases
should be designed with values a and b as well as values just above and just
below a and b

• If an input condition specifies a number of values, test case should be

developed that exercise the minimum and maximum numbers. Values just
above and just below the minimum and maximum are also tested

• Apply guidelines 1 and 2 to output conditions; produce output that reflects the
minimum and the maximum values expected; also test the values just below
and just above

• If internal program data structures have prescribed boundaries (e.g., an array),

design a test case to exercise the data structure at its minimum and maximum
boundaries

31
 4. Decision Table Technique in Black Box Testing
⚫ Decision table technique is one of the widely used test case design techniques for
black box testing. This is a systematic approach where various input combinations
and their respective system behaviorare captured in a tabularform.

⚫ That's why it is also known as a cause-effect table. This technique is used to pick the
test cases in a systematic manner; it saves the testing time and gives good coverage
to the testing area of the software application.

⚫ Decision table technique is appropriate for the functions that have a logical
relationship between two and more than two inputs.

⚫ This technique is related to the correct combination of inputs and determines the
result of various combinations of input. To design the test cases by decision table
technique, we need to consider conditions as input and actions as output.

32
 …..Decision Table Technique in Black Box Testing
⚫ Most of us use an email account, and when you want to use an email
account, for this you need to enter the email and its associated password.
⚫ If both email and password are correctly matched, the user will be directed
to the email account's homepage; otherwise, it will come back to the login
page with an error message specified with "Incorrect Email" or "Incorrect
Password."
⚫ Now, let's see how a decision table is created for the login function in
which we can log in by using email and password. Both the email and the
password are the conditions, and the expected result is action.

33
 …..Decision Table Technique in Black Box
Testing
⚫ In the table, there are four conditions or test cases to test the login function. In the
first condition if both email and password are correct, then the user should be
directed to account's Homepage.
⚫ In the second condition if the email is correct, but the password is incorrect then the
function should display Incorrect Password. In the third condition if the email is
incorrect, but the password is correct, then it should display Incorrect Email.
⚫ Now, in fourth and last condition both email and password are incorrect then the
function should display Incorrect Email.
⚫ In this example, all possible conditions or test cases have been included, and in the
same way, the testing team also includes all possible test cases so that upcoming
bugs can be cured at testing level.
⚫ In order to find the number of all possible conditions, tester uses 2n formula where n
denotes the number of inputs; in the exampl the number of inputs is 2 (one is true
and second is false).
⚫ Number of possible conditions = 2^ Number of Values of the second condition
⚫ Number of possible conditions =2^2 = 4
⚫ While using the decision table technique, a tester determines the expected output, if
the function produces expected output, then it is passed in testing, and if not then
it is failed. Failed software is sent back to the developmentteam to fix the defect.
3
4
 Black Box Testing
Advantages Disadvantages
Well suited and efficient for large Limited coverage, since only a selected
code segments. number of test scenarios is actually
performed.

Code access is not required. Inefficient testing, due to the fact

that the tester only has limited
knowledge about an application.

Clearly separates user's perspective Blind coverage, since the tester

from the developer's perspective cannot target specific code segments
through visibly defined roles. or error prone areas.

Large numbers of moderately skilled The test cases are difficult to design.
testers can test the application with
no knowledge of implementation,
programming language, or operating
systems.

3
5
36