CHAPTER 2 : TYPES OF LEVELS OF TESTING

MARKS : 18
2.1 Levels of Testing
2.2 Unit testing : driver , stub
2.3 Integration testing :Top down integration, Bottom up integration, Bi- directional integration
2.4 Testing on web application : Performance testing: load testing, stress testing, security testing,
client server testing
2.5 Acceptance testing : alpha testing and beta testing , special tests :regression testing, GUI testing.

2.1 Levels of testing

There are different levels during the process of testing. In this chapter, a brief description is provided
about these levels.
Levels of testing include different methodologies that can be used while conducting software testing.
The main levels of software testing are −
 Functional Testing
 Non-functional Testing
Functional Testing
This is a type of black-box testing that is based on the specifications of the software that is to be
tested. The application is tested by providing input and then the results are examined that need to
conform to the functionality it was intended for. Functional testing of a software is conducted on a
complete, integrated system to evaluate the system's compliance with its specified requirements.
There are five steps that are involved while testing an application for functionality.
Steps Description

I The determination of the functionality that the intended application is

meant to perform.

II The creation of test data based on the specifications of the application.

III The output based on the test data and the specifications of the
application.

IV The writing of test scenarios and the execution of test cases.

V The comparison of actual and expected results based on the executed

test cases.
An effective testing practice will see the above steps applied to the testing policies of every
organization and hence it will make sure that the organization maintains the strictest of standards
when it comes to software quality.
Non functional testing refers to aspects of the software that may be related to a specific function or
user action, such as scalability or other performance , behavior under certain constraints or security.
The major levels of testing are as shown in the figure below:
Unit Testing
This type of testing is performed by the developers before the setup is handed over to the testing team
to formally execute the test cases. Unit testing is performed by the respective developers on the
individual units of source code assigned areas. The developers use test data that is separate from the
test data of the quality assurance team.
The goal of unit testing is to isolate each part of the program and show that individual parts are
correct in terms of requirements and functionality.
Limitations of Unit Testing
 Testing cannot catch each and every bug in an application.
 It is impossible to evaluate every execution path in every software application. The same is
the case with unit testing.
 There is a limit to the number of scenarios and test data that the developer can use to verify
the source code. So after he has exhausted all options there is no choice but to stop unit testing
and merge the code segment with other units.
Integration Testing
The testing of combined parts of an application to determine if they function correctly together is
Integration testing. There are two methods of doing Integration Testing Bottom-up Integration testing
and Top Down Integration testing and bidirectional testing.
Bottom-up integration testing begins with unit testing, followed by tests of progressively higher-
level combinations of units called modules or builds.
Top-Down integration testing, the highest-level modules are tested first and progressively lower-
level modules are tested after that. In a comprehensive software development environment, bottom-up
testing is usually done first, followed by top-down testing.
Bidirectional or Sandwich integration testing, the testing is done in both the ways top down and
bottom up approach. Generally applicable for larger types of software testing.

System Testing
This is the next level in the testing and tests the system as a whole. Once all the components are
integrated, the application as a whole is tested rigorously to see that it meets Quality Standards. This
type of testing is performed by a specialized testing team.
Why is System Testing so Important
System Testing is the first step in the Software Development Life Cycle, where the application is
tested as a whole.
The application is tested thoroughly to verify that it meets the functional and technical
specifications.
The application is tested in an environment which is very close to the production environment
where the application will be deployed.
System Testing enables us to test, verify and validate both the business requirements as well as the
Applications Architecture.

Acceptance Testing
This is arguably the most importance type of testing as it is conducted by the Quality Assurance
Team who will gauge whether the application meets the intended specifications and satisfies the
client’s requirements. The QA team will have a set of pre written scenarios and Test Cases that will
be used to test the application.
More ideas will be shared about the application and more tests can be performed on it to gauge its
accuracy and the reasons why the project was initiated. Acceptance tests are not only intended to
point out simple spelling mistakes, cosmetic errors or Interface gaps, but also to point out any bugs in
the application that will result in system crashers or major errors in the application.
By performing acceptance tests on an application the testing team will deduce how the application
will perform in production. There are also legal and contractual requirements for acceptance of the
system.
2.2 Unit testing : drivers and stubs

1. Unit is the smallest testable part of the software system.

2. Unit testing is done to verify that the lowest independent entities in any software are working fine.
3. The smallest testable part is isolated from the remainder code and tested to determine whether it
works correctly.
4. When developer is coding the software it may happen that the dependent modules are not
completed for testing, in such cases developers use stubs and drivers to simulate the called (stub) and
caller (driver) units.
5. Unit testing requires stubs and drivers, stubs simulates the called unit and driver simulates the
calling unit.
1. STUBS:
i. Assume you have 3 modules, Module A, Module B and module C.
ii. Module A is ready and we need to test it, but module A calls functions from Module B and C
which are not ready, so developer will write a dummy module which simulates B and C and returns
values to module A.
iii. This dummy module code is known as stub.
Figure 1: The Stub Flow Diagram
iv. The above diagrams clearly states that Modules 1, 2 and 3 are available for integration, whereas,
below modules are still under development that cannot be integrated at this point of time.
v. Hence, Stubs are used to test the modules.
2. DRIVERS:
i. Now suppose you have modules B and C ready but module A which calls functions from module B
and C is not ready so developer will write a dummy piece of code for module A which will return
values to module B and C.
ii. This dummy piece of code is known as driver.

Testing that occurs at the lowest level is called unit testing or module testing. As the units are
tested and the low-level bugs are found and fixed, they are integrated and integration testing is
performed against groups of modules. This process of incremental testing continues, putting
together more and more pieces of the software until the entire product—or at least a major portion
of it—is tested at once in a process called system testing.
With this testing strategy, it’s much easier to isolate bugs. When a problem is found at the unit
level, the problem must be in that unit. If a bug is found when multiple units are integrated, it
must be related to how the modules interact. Of course, there are exceptions to this, but by and
large, testing and debugging is much more efficient than testing everything at once.

Fig 1 : A test driver can replace a real software and more efficiently test a lower level module.
In bottom-up
testing (see Figure 1 above), you write your own modules, called test drivers,that exercise the
modules
you’re testing. They hook in exactly the same way that the future real modules will. These drivers
end test-case data to the modules under test, read back the results, and verify that they’re correct. You
can very thoroughly test the software this way, feeding it all types and quantities of data, even ones
that might be difficult to send if done at a higher level.
Fig .2 : A test stub sends data up to the module being tested.
Look at Figure 2. In this case, a low-level interface module is used to collect temperature data from
an electronic thermometer. A display module sits right above the interface, reads the data from the
interface, and displays it to the user. To test the top-level display module, you’d need blow torches,
water, ice, and a deep freeze to change the temperature of the sensor and have that data passed up the
line.
Rather than test the temperature display module by attempting to control the temperature of the
thermometer, you could write a small piece of code called a stub that acts just like the interface
module by feeding temperature values from a file directly to the display module. The display module
would read the data and show the temperature just as though it was reading directly from a real
thermometer interface module. It wouldn’t know the difference. With this test stub configuration, you
could quickly run through numerous test values and validate the operation of the display module.

Fig 3 : Unit test environment / concept of stubs and drivers

Stub and Driver are two different types of codes that are used in software development, specifically in
software testing. Stubs are used as functions in Top Down Integration, whereas drivers are used as
functions in Bottom Up Integration. Stubs and Drivers are commonly used in porting, distributed
computing, as well as general software development and testing.
Stub and Driver are two different types of codes that are used in software development, specifically in
software testing. There are times during software development when a part of the code must be tested.
This allows the developed to make sure that this part of the code is working fine before he moves on
to the next part of the code. However, this part of the code may not function properly without inputs
from the other parts of the code, which have yet to be written. This is where dummy codes come in
handy.
Dummy codes are pseudocode, which basically means that they are fake codes that help stimulate the
behavior of existing code. This means that the developer can used these dummy codes in place of the
parts of the code that have not yet been written in order to replicate the outputs the original code will
give. These outputs can then be used to test the part of the code that has been written.
Stubs and Drivers are two different types of dummy codes and hence are used differently. Stubs are
used as functions in Top Down Integration, whereas drivers are used as functions in Bottom Up
Integration. Stubs and Drivers are commonly used in porting, distributed computing, as well as
general software development and testing.
Suppose that there are three pieces of code: A, B and C, and they run in that direction, i.e. A-> B ->
C. Now code A has been developed and needs to be tested, but codes B and C still need to be
developed, and code A will not run without the input of B and C. Hence, stubs are used in place of B
and C. These stubs imitate the codes B and C and even give output that are similar to what B and C
would give. This is called Top Down Integration, i.e. testing from the top to the bottom.
Similarly, imagine now that codes B and C have been developed and need to be test, however code A
still needs to be developed, and likewise the other two codes won’t work without the inputs from
code A. Hence, a driver can be used in place of code A. This driver would then imitate code A and
give the input to B and C, similar to what the actual code A would. This is called Bottom Up
Integration, i.e. testing from the bottom up.
Stubs and Drivers are basically routines that don’t actually do anything except declare themselves
and the parameters they accept. The rest of the code can then take these parameters and use them as
inputs. The stubs and drivers are just commonly used as placeholders for the routines, i.e. codes that
still need to be developed. Hence, the stubs and drivers contain just enough code to allow it to be
compiled and linked with the rest of the program.
Differences between stubs and drivers :

Stubs Drivers
1. Used in Top-Down Integration Testing. 1.Used in Bottom-Up Integration Testing.
2. A piece of code that emulates the called 2. A piece of code that emulates a calling
function. function.
3. Stubs stimulates the activity of missing and 3. Drivers pass test cases to another code and
not developed modules. invoke modules under testing.
4. These are created when high level modules 4. These are created when lower level modules
are tested and lower level modules are not are tested and higher level modules are not yet
developed. developed.
2.3 Integration Testing :
INTEGRATION TESTING is a level of software testing where individual units are combined and
tested as a group. The purpose of this level of testing is to expose faults in the interaction between
integrated units. Test drivers and test stubs are used to assist in Integration Testing.
Definition by ISTQB
 integration testing: Testing performed to expose defects in the interfaces and in the
interactions between integrated components or system.
Approaches
 Big Bang is an approach to Integration Testing where all or most of the units are combined
together and tested at one go. This approach is taken when the testing team receives the entire
software in a bundle. So what is the difference between Big Bang Integration Testing and
System Testing? Well, the former tests only the interactions between the units while the latter
tests the entire system.
 Top Down is an approach to Integration Testing where top-level units are tested first and
lower level units are tested step by step after that. This approach is taken when top-down
development approach is followed. Test Stubs are needed to simulate lower level units which
may not be available during the initial phases.
 Bottom Up is an approach to Integration Testing where bottom level units are tested first and
upper-level units step by step after that. This approach is taken when bottom-up development
approach is followed. Test Drivers are needed to simulate higher level units which may not be
available during the initial phases.
 Sandwich/Hybrid is an approach to Integration Testing which is a combination of Top Down
and Bottom Up approaches.

Fig : Individual pieces of code are built up and tested separately, and then integrated and tested again.
As the units are tested and the low-level bugs are found and fixed, they are integrated and integration
testing is performed against groups of modules. This process of incremental testing continues, putting
together more and more pieces of the software until the entire product—or at least a major portion
of it—is tested at once in a process called system testing.
With this testing strategy, it’s much easier to isolate bugs. When a problem is found at the unit level,
the problem must be in that unit. If a bug is found when multiple units are integrated, it must be
related to how the modules interact. Of course, there are exceptions to this, but by and large, testing
and debugging is much more efficient than testing everything at once.
There are two approaches to this incremental testing: bottom-up and top-down. In bottom-up
testing , we write own modules, called test drivers, that exercise the modules we’re testing. They
hook in exactly the same way that the future real modules will. These drivers send test-case data to
the modules under test, read back the results, and verify that they’re correct.
One can very thoroughly test the software this way, feeding it all types and quantities of data, even
ones that might be difficult to send if done at a higher level.
1. Bottom-up Integration
In the bottom-up strategy, each module at lower levels is tested with higher modules until all modules
are tested. It takes help of Drivers for testing
Diagrammatic Representation:

Advantages:
 Fault localization is easier.
 No time is wasted waiting for all modules to be developed unlike Big-bang approach
 In this approach development and testing can be done together so that the product or
application will be efficient and as per the customer specifications.
Disadvantages:
  Critical modules (at the top level of software architecture) which control the flow of
application are tested last and may be prone to defects.
 An early prototype is not possible
 The key interface defects are caught at the end of the cycle.
 Test drivers are required to be created for modules at all levels except the top control.
 The critical modules, which control the flow of the application are tested last and may be
prone to defects.
2. Top-down Integration:

In Top to down approach, testing takes place from top to down following the control flow of the
software system.
Takes help of stubs for testing.
Diagrammatic Representation:

Advantages:
 Fault Localization is easier.
 Possibility to obtain an early prototype.
 Critical Modules are tested on priority; major design flaws could be found and fixed first.
 The tested product is extremely consistent in this approach as the integration testing is
performed in an environment that is similar to the real world environment.
 The stubs in top-down approach can be written in lesser time compared to drivers as they are
simpler to author.
 Here, fault localization is easier.
 In top-down approach the critical modules are tested on priority, which further helps testers in
finding major design flaws as early as possible.
Disadvantages:
 Needs many Stubs.
 Modules at a lower level are tested inadequately.
 The basic functionality of the software is tested at the end of the cycle.

3. Bidirectional /Hybrid/ Sandwich Integration

In the sandwich/hybrid strategy is a combination of Top Down and Bottom up approaches. Here, top
modules are tested with lower modules at the same time lower modules are integrated with top
modules and tested. This strategy makes use of stubs as well as drivers.
bidirectional integration testing is a culmination of both incremental as well as non incremental
integration testing, wherein Bottom-Up approach is focused on middle to top layer, Top-Down
approach is concerned about layers from middle to downwards and the Big Bang approach is
followed for the middle layer. This type of testing combines the advantages of all the three
approaches and is mainly used to test large projects.
Advantages:
o Sandwich approach is very useful for large enterprises and huge projects that further
have several subprojects.
o When development follows a spiral model and the module itself is as large as a
system, then one can use sandwich testing.
o Top-Down and Bottom-Up approach both start as per development schedule.
o Units are tested and brought together to make a system.
o Integration is done downwards.
o The resources that are required are immense and big team perform both top-down and
bottom-up method of testing at a time or one after the other.
Disadvantages:
o As both Top-Down and Bottom-Up approaches are executed on the software, the cost
of testing is very high.
o It cannot be used for smaller systems with huge interdependence between the modules.
o It only makes sense when the individual subsystem is as good as completed system.
o Different skill sets are required for testers at different levels.

How to do Integration Testing?

The Integration test procedure irrespective of the Software testing strategies (discussed above):
1. Prepare the Integration Tests Plan
2. Design the Test Scenarios, Cases, and Scripts.
3. Executing the test Cases followed by reporting the defects.
4. Tracking & re-testing the defects.
5. Steps 3 and 4 are repeated until the completion of Integration is successful.
Brief Description of Integration Test Plans:
It includes the following attributes:
  Methods/Approaches to testing (as discussed above).
 Scopes and Out of Scopes Items of Integration Testing.
 Roles and Responsibilities.
 Pre-requisites for Integration testing.
 Testing environment.
 Risk and Mitigation Plans.

BOTTOM-UP
BASIS FOR TOP-DOWN INTEGRATION
INTEGRATION
COMPARISON TESTING
TESTING

Basic Uses stubs as the momentary Use test drivers to initiate

replacements for the invoked and pass the required data
modules and simulates the to the lower-level of
behaviour of the separated lower- modules.
level modules.

Beneficial If the significant defect occurs If the crucial flaws

toward the top of the program. encounters towards the
bottom of the program.

Approach Main function is written at first then Modules are created first
the subroutines are called from it. then are integrated with the
main function.

Implemented on Structure/procedure-oriented Object-oriented

programming languages. programming languages.

Risk analysation Collaborating the impact of internal Models are used to analyze
operational failures. the individual process.

Complexity Simple Complex and highly data

intensive.

Works on Big to small components. Small to big components.