Software Testing

Unit-II: TESTING LEVELS AND TYPES

2.1Testing Levels
During development the software is developed in terms of components or units, and these units
are combined into modules, and modules are combined to build the system. It will be very
complex if we start the testing after the complete system is developed. Therefore we have to start
the testing at every level of the development. So first we need to test the components/units, and
then we have to test the modules, and then we have to test the system.

The below figure show each level of testing on specific entity. Unit testing is done to test the
source code of the software. Integration testing is done to test the design of the software. System
testing is done to test the SRS document. And finally, Acceptance testing is done to test the
client/user requirement. To check whether the software satisfies the user needs, the software has
to install in the actual environment and tested.

Testing in Actual Business Needs

Environment

Acceptance User Requirement

Testing

System
SRS
Testing

Integration Design
Testing

Unit Code
Testing

2.1.1 Unit/Component Testing

A unit/component is the smallest entity in the software. Every unit has to be tested separately to
check whether it is as per the specifications or not. Unit/component testing is done only by the
development engineers. As it is not possible to test a unit individually, additional piece of code
may be written to test the units.
For Example: Suppose you have implemented a stack. You have to write a main function, a push
function to push an element into the stack and a pop function to pop an element from the stack.
You can consider each of the three functions as a unit. Initially you may just write the code for
B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 1
 Software Testing

main function. How do you test it without push and pop functions? Instead of the statement that
calls the push function, you can just write a statement that prints “call push here”.
Similarly, you can write a print statement that prints “call pop here” in the place of the statement
that calls pop function. If the main program works, then you can develop the push function,
replace the above print statement with the call to push function, and then test the main program
along with the push function. Then you can develop the pop function and test the program for
entire stack implementation.
NOTE: It may not be possible to test a component/unit in isolation. You may need to write
additional code (stubs, drivers, simulators) to do the testing.
2.1.2 Module Testing
A module is an independent entity in the software. The tested units are integrated into a module
and each module is tested separately for the specifications.
For Example: if the software is a database management system, the modules can be database
(back-end) and the user interface (front-end). The database can be tested separately using the
SQL commands to check correct data retrieval. However, validation of the front-end (forms)
requires a database. So, after testing the back-end module, the front-end can be tested, after
integration.
After completion of testing a module, a module-level test report has to be prepared.
2.1.3 Integration Testing
After the modules are tested, the modules can be integrated together. It is a difficult task to
integrate all the modules together and start testing, as it would be very difficult to do the
debugging. The integration has to be done incrementally and testing is performed step by step.
Once all the modules are integrated together, the system is ready for testing.
Consider an e-learning portal. Some important modules in this portal are given in Figure 2.2.
Broadly, the portal consists of three modules: admin module, tutor module and student module.

E-Learning Portal

Admin Tutor Student

Module Module Module

User Tutor Content Online Test Chat Email

Management Management Management Allocation

Student Program Online Online

Management Monitoring content Test

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 2

 Software Testing

Figure 2.2 various modules in an e-learning portal

The student module in turn consists of a number of sub-modules, such as chat, email, on-line
content and on-line tests. The tutor module contains sub-modules for content management, on-
line tests allocation and progress monitoring. The admin module consists of sub-modules, such
as user management, student management and tutor management. During the construction /
implementation stage, all the sub-modules (or components) are developed.
You can start integrating the sub-modules of the student module, build the complete student
module and then test it. Similarly, you can integrate and test the tutor module and the admin
module. After each of the modules is tested to a great extent, integrate all the three modules and
build the complete portal.
2.1.4 System Testing
Once all the modules are integrated, the system is ready for a full-fledged testing. There are
various approaches used for system testing. There various types of testing the system.
To start with, the system has to be tested for its functionality. These is done by going through the
SRS document and for each requirement, carry out the test cases design, execute the test cases,
and compare the expected results with the actual results.
In addition to functional testing, the system has to be tested for its non-functional requirements,
such as:
• Performance requirements
 Accessibility to physically handicapped persons
• Portability
• Security
• Reliability
• Safety
• Conformance to national/international standards
• Interoperability with other hardware/software
• Conformance to any legal/statutory requirements
2.1.5 Acceptance Testing
After the system testing is completed in your premises, the software has to be approved by the
customer. The customer will carry out a testing which is called Acceptance testing, and if there
are no errors/defects found, then the customer will pay your money. If we encounter problems
during this testing, the customer will be unhappy, and he will carry out the testing more
rigorously next time. Acceptance testing can be done either at the developer's or client premises.
Depending on the client requirements and contractual obligations, the acceptance testing may be
of different types:
• User acceptance testing: The user acceptance testing is carried out at the developer's /
user's premises based on an acceptance test plan.
• Operational acceptance testing: The software is installed in the actual working
environment and the performance observed over a period of time to check how the
software meets the users' operational needs.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 3

 Software Testing

• Contractual fulfillment testing: If there are any special contractual obligations, such as
environmental testing for the hardware, testing the system for air-borne applications
(such as in aircraft), then testing is done to fulfill these obligations.
• Regulatory acceptance testing (or conformance testing): The software may be
required to conform to international standards or meet some government requirements. In
such a case, the necessary testing is done. This testing may be done at third party
laboratories or government laboratories.
The various stages of testing are shown in Figure 2.3. For each stage of testing, a test plan has to
be prepared and after the testing is done, a test report has to be prepared. Figure 2.4 describes
pictorially the complete testing process as discussed in this section along with various test plans
to be written at each level.
Unit
Unit
Testing
Testing

Module Module
Testing Testing

Sub-System
Testing Sub-System
Testing

System
Testing

Acceptance
Testing
Figure 2.3 Stages of testing

Client
Design SRS Requirement

Unit Test Module System Acceptance

Plan Test Plan Test Plan Test Plan

Unit Combine Unit Module Interconnect System Acceptance

Test into Modules Test Modules Test Test

Figure 2.4: Levels of Testing and corresponding test plans.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 4
 Software Testing

2.2 Testing Approaches

Testing approaches helps in easier debugging. There are following testing approaches:
2.2.1 Static Testing vs. Dynamic Testing
After the code is written, there are two ways of testing this code-static testing and dynamic
testing.
In static testing, the code is checked without executing it; hence it is also known as desk
checking. This can be done either formally in a meeting or informally by two or more
development/QA engineers.
In dynamic testing, the code is executed and tested. The code is compiled, executed with test
inputs, and if the actual results match with the expected results, then the software can be
considered as OK.
NOTE: In static testing, the code is checked without executing it through reviews/inspections. In
dynamic testing, code is executed and checked. Both types of testing are required to be done.
2.2.2 Positive Testing vs. Negative Testing
Positive testing is testing the software to ensure that it does what it is intended to do. Negative
testing is testing the software to ensure that it does not do anything; which is not intended to do.
We need to test the software to check whether it is doing what it is intended to do. But that is not
enough. We also need to check whether it is doing anything, which it is not intend to do. This is
differentiated by positive testing and negative testing.
2.2.3 Top-down Testing vs. Bottom-up Testing
In top-down approach, testing is done from the top of hierarchy. Dummy routines called stubs
are introduced that simulate a module.
In bottom-up approach, testing is done from the bottom of hierarchy. Dummy routines called
drivers are introduced that invoke a module.
Consider testing the software that implements a stack. The program contains a main function and
functions to push and pop. In top-down approach, first the main function is tested. For this, in the
place where a push function is called, a print statement can be introduced which prints call the
push function. Similarly at the point where a pop function is called, a print statement can be
introduced which prints pop function is called. In bottom-up approach, a driver function is
written, which just calls the push function first and then testing is done to ensure that the push
works well. Similarly, the pop function is tested. And subsequently, all the functions are
combined together and tested.
2.2.4 Functional Testing vs. Structural Testing
In functional testing, software is tested for its functionality without considering the
implementation details. This type of testing is also known as black box testing.
In structural testing, the implementation / structural details are taken into consideration. This
type of testing is also known as white box testing.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 5

 Software Testing

Structural Testing:
A structure of a program differs at different levels. At the component level, the structure is the
code. At integration level, the structure is the tree diagram that shows the interconnecting
modules. At system level, the structure is the menu structure, or web page structure or a business
process depending on the application.
NOTE: Structural testing is also called clear-box testing, glass box testing and white box testing.
Structural testing at component level is used to test the implementation of the program. Here,
the source code is looked into for testing the software. Structural testing involves.
a) statement coverage
b) condition coverage
c) path coverage
d) function coverage
a)Statement Coverage
It is advisable that every line in the code is executed at least once. Statement coverage ensures
that each and every statement is executed, that shows all the statements in the program are tested.
Checking it manually is difficult, but tools are available for this. Profilers used to carry out this
statement coverage testing. Profilers indicate the number of times each line in the code is
executed. When you execute a test case, the profiler indicates which lines of code have been
executed and which lines have not been executed.
b)Condition Coverage
It is advisable that every condition in the code is executed at least once. Condition coverage
ensures that each and every condition is executed, that shows all the conditions in the program
are tested.
In branch coverage, inputs are given in such a way that, each branch is executed at least once.
For example:
if (a > 6 && b < 5)
then <statements;
else <statements> ;
Here it has to be certainly ensured that both the 'then' statements and 'else' statements are
executed at least once. The test cases are chosen in such a way that the condition becomes true
and also false. In addition, it is necessary to ensure that each individual condition in the
conditional statement is made true and false, i.e. a > 6 is true and false; b <5 is true and false.
Condition coverage in percentage = (no. of conditions executed / total conditions) * 100
c)Path Coverage
It is advisable that every path in the code is executed at least once. Path coverage ensures that
each and every condition is executed, that shows all the paths in the program are tested.
You can convert a given program into a flow chart and you will know pictorially how many
paths are there. You need to design test cases to ensure that each path is tested at least once.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 6

 Software Testing

As an example, consider the flow chart shown in Figure 2.5(a) and the corresponding flow path
given in Figure 2.5 (b). Here, there are two paths to be tested. Hence, test cases need to be
formulated to ensure that both the paths are tested.

Find d

D>0 ?

Calculate Calculate
Real Roots IMG Roots

STOP

Figure 2.5: (a) Flow Chart; b) Corresponding Flow Diagram

Note that path coverage deals with the logical paths in the program. For small programs, the path
coverage can be easily ensured by designing the necessary test cases, however in practical
systems with millions of lines of code, 100% path coverage is not that easy.
Path coverage in percentage = (no. of paths executed / total paths) * 100
4)Function Coverage
It is advisable that every function in the code is executed at least once. Function coverage
ensures that each and every function is executed, that shows all the functions in the program are
tested.
Developing test cases for function coverage is rather easy as compared to developing test cases
for path coverage and condition coverage
Structural Testing at Integration Level
At integration level, the structure is the tree diagram that shows the interconnecting modules.
Consider the various modules of an e-learning portal given in Figure below figure. In this
software, when we carry out the testing at module level, we can consider the various sub-
modules in that module and then work out the testing strategy.
In the student module, the chat, e-mail and on-line content sub-modules can be tested
independently. However, to test the on-line tests sub-module, we need to also consider the on-
line tests allocation sub-module of the tutor module. So, at the testing level, we can consider the
structure of the modules for finalizing the testing strategy.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 7

 Software Testing

E-Learning Portal

Admin Tutor Student

Module Module Module

User Tutor Content Online Test Chat Email

Management Management Management Allocation

Student Program Online Online

Management Monitoring content Test

Structural Testing at System Level

At system level, the structure is the menu structure or a web page structure depending on the
application.
When you consider the structure at the system level, you look at the major modules in the total
system. For instance, in the e-learning portal, there are three modules admin module, tutor
module and student module. You can to start with test the student module, then the tutor module
and finally the admin module.
In the case of a web page, the home page of the web site gives the overall structure of the web
site. When you click on each link, you get the sub-links. In top-down structural testing, you can
begin at the home page, and go down to the leaves.
Black Box Testing vs. White Box Testing at Code Level
Both black box testing and white box testing have to be done for every software package.
Black-box testing is a system testing in which the functionality of the software is tested. The
software is considered as a black box, i.e. you will not be able to see the implementation details
(or the code).
White Box Testing is a testing in which structure of the program is tested. The software is
considered as a white box, i.e. you will be able to see the implementation details (or the code).
White box testing presents some advantages in detecting the defects as well as reducing the
testing time/effort.
White box testing helps in finding out whether the programmer made any mistake at the
boundary values of loops. For example, the loop statement
for (i =0; i <= 100; i++)
is executed 101 times. Does the programmer really want 101 times or only 100 times? Invariably
programmers make mistakes at the boundary values. So, testing has to be done at loop
boundaries. This type of testing cannot be done through black box testing and when the
functionality is not met, debugging is extremely difficult when only black box testing is done.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 8

 Software Testing

2.2.5 Mutation Testing

Mutation testing is required to ensure that the software does not fail. It is also a good debugging
mechanism. After the software works correctly, mutation testing can be done to simulate wrong
inputs and reduce the risk.
In mutation testing, program is modified slightly to obtain mutants of the program. Different
mutants are tested with the same test cases. If the mutant fails, and the actual program works
correctly, confidence is gained in the program, and test cases are considered as good, i.e.
different mutants should give different results. Generally each mutant will have only one change.
To produce mutants, mutation operators are defined. The mutation operators can be
• Constant replacement
• Variable replacement
• Arithmetic operator replacement
• Relational operator replacement
• Goto label replacement
2.2.6 Confirmation Testing
When a defect is reported in the software, the developer makes some change to the code to
remove that defect. When the developer indicates that the defect has been fixed, it has to be
confirmed by the test engineer that the defect has been indeed removed. This is called
confirmation testing.
If the developer comes to know that you are re-testing to check whether what he said is correct,
he may get angry. Don't worry, as a test engineer you need to do your job.
NOTE: Test engineers trust the developers, but nevertheless verify whether the software is really
defect-free.
2.2.7 Regression Testing
When a defect is reported by the test engineer, the developer will change the code and fix the
defect. It is likely that the change made in the code may lead to another defect that may not be
visible immediately. So, whenever a change is made to the source code, one has to ensure that
there are no bad effects of the change on the other parts of the software. Regression testing is
done precisely to ensure that change in one part of the software has no bad effect on other parts
of the software.
Whenever a change is made to the source code, a set of pre-defined test cases has to be run to
check whether any other portion of the software is affected.
2.3 Types of Testing
To release a quality product, the software has to be tested in such a way that it will meet all the
requirements of the user as well as all the possible environments, in which the user is likely to
use the software. The software installations may differ-number of users, type of users, operating
system, network environment, etc. Based on these varying environmental requirements, the test
plan has to be prepared in such a way that these issues are taken care of. We address these issues
and the various test methodologies in this section.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 9

 Software Testing

2.3.1 Smoke Testing

Smoke testing usually done on hardware components. When you develop the hardware (e.g.
Build Printed Circuit Board (PCB) by assemble the components and connect the power supply),
just before you start the testing, what will you do? You pray to God and then switch on the
power supply. If you don’t see any smoke then you can say that the hardware has passed the
smoke test, otherwise the smoke test failed (Figure 2.7).

Figure 2.7 Smoke Test failed

When you integrate all the modules and when your complete software product is ready, you can
test the basic functionality of the software without going into every detail of each and every
feature. You just test whether the software is working OK for its basic features. This is called
smoke testing in software. And, luckily for software engineers, even if the software does not pass
the smoke test, you do not see any smoke!
NOTE: Smoke testing is also referred to as Build Verification Testing (BVT).
2.3.2 Black Box Testing
Black-box testing is a system testing in which the functionality of the software is tested. The
software is considered as a black box, i.e. you will not able to see the structural / implementation
details (such as the code).
In Black box testing, defined inputs are given and from which defined outputs are obtained
(Figure 2.8).

Compilable Block

Test Input Test Output

Figure 2.8 Black Box testing

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 10

 Software Testing

Since we are not worried about the internet structure of the program, we are only carrying out a
functional testing. Test cases have to be designed to ensure that the each and every functionality
given in the user requirements is tested.
2.3.3 White Box Testing
In white box testing, the structure of the program is taken into consideration. Using this approach
the logic behind the code can be checked. The objective is to ensure that each and every line of
the code is tested. Consider the example:

1f ( i < 5 && j > 6)

do this;
else
do that's

If the test case i = 3 and j = 7 is given as input, only the true condition is satisfied and the
corresponding statements are executed. A test case that makes the condition false also needs to
be given, such as i = 6 and j = 3.

…………………………………………………
………………………………………………….
if(a>5 && b<b)
{
k=10
Test Input else Test Output
k=0;
}
Figure 2.9 White-box testing
Run time profilers are used to indicate that which statement is executed and how many times.
For developing highly reliable software, white box testing is a must.
2.3.4 Interface Testing
Interface testing is done to test whether the interface between two subsystems is OK. Interface
testing may be done by the development engineers while carrying out integration. When two
modules have to be integrated, they can be integrated only when the interfaces are correctly
defined. In addition to this type of interface testing by developers, test engineers also may need
to test the interfaces. These interfaces can be:
Hardware-hardware interfaces: These are used to integrate two hardware devices. For
example, suppose you have to interconnect two computers, you have to specify the following
parameters: (a) physical connection (b) electrical parameters and (c) protocol for communication.
If you are testing software, you have to ensure that all the interface specifications are correctly
given and then test whether the necessary communication can be established between the two
hardware devices.
B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 11
 Software Testing

Software-software interfaces. These interfaces are used in every application for module
integration. For example Java SDK gives you APIs so that you can develop your own
applications. APIs are the most widely used mechanism for providing software interfaces. If you
are testing an interface that has to be given to the customer, you need to write the test routines for
ensuring that the interface is as per the mechanism specified by the customer.
Hardware software interfaces: In PC/mainframe computers, generally the interface between
the application and the hardware is through the operating system and hence hardware-software
interface is not difficult. But Testing and debugging hardware-software interfaces is more
difficult in embedded systems because the hardware and software are tightly coupled in
embedded systems.
Human-Computer Interface (HCI): This testing differs from application to application. In
PC/mainframe/client-server/web applications, HCI testing is simply testing the GUI. But then,
testing the GUI is not easy-you need to test each and every aspect of the GUI and check the
functionality. In addition, you need to test non-functional aspects such as usability or ease-of-
use, esthetics, etc.. For example, the number of clicks required to carry out a particular action can
be one parameter. If the GUI requires too many clicks and if the screens are deeply nested, then
the GUI is not easy to use.
2.3.5 Use Case Testing
In use-case testing, various use-cases are worked out. The exact sequence of operations done by
a user is worked out and testing is done in the same sequence, to check whether the desired
results are obtained. For example, consider a database application. The GUI for entering the
employee details in a database is shown in Figure 2.10.
Testing is done in by following the use case procedure, i.e. enter the details of a new employees,
add an employee details and then take a print-out and check whether the correct data is printed or
not. If the employee makes a mistake, he will click on the modify button, which modifies the
data and the report is checked again. Use cases can be worked out for adding employee details,
modifying an existing record and deleting a record.

Figure 2.10
B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 12
 Software Testing

2.3.6 Gorilla Testing

Gorilla testing is also called monkey testing or chimpanzee testing. More commonly it is referred
as random testing (Below Figure.)

Imagine that the software developed by the team is given to a Gorilla for testing. The Gorilla will
randomly press some keys, which may be irrelevant to the software and, it may sometimes press
the keys that are acceptable inputs.
This type of testing would bring out the defects when wrong input is given. Ideally, the software
should not misbehave when wrong inputs are given. If the software fails, then the user will lose
the confidence. Gorilla testing is used to check whether defensive programming has been done or
not--through defensive programming, software is made to tolerate wrong inputs.
As it is very difficult to recruit a Gorilla for carrying out this type of testing, you can ask your
project manager to act as one! The testing process is simple, just keep pressing some keys
randomly and check whether the software fails. If a wrong input is given, necessary error
messages have to be displayed. If the system hangs, it is a major defect; if the error messages are
not displayed, the error handling routines need to be improved.
Gorilla testing is highly beneficial for testing games software. When a child gets lost while
playing a game, it randomly presses lots of keys. If the software hangs, then it is really bad
software.
2.3.7 Alpha Testing
When the software reached a mature stage of development, the ends users can test the software
in the presence of the development team so that the users can be guided how to use the software
and the developers can rectify any major problems encountered during the usage of the software.
This type of testing is called alpha testing.
It is always advisable to carry out alpha testing for every project because this will help in getting
an early feedback from the user. Alpha testing is carried out by the users in the presence of the
development team.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 13

 Software Testing

2.3.8 Beta Testing

Beta testing is carried out at the user's premises, in the absence of the development team. Beta
testing detects the problem associated with the usability as well as Performance. Beta testing is
important for generic software products, such as operating systems, database engines, compilers,
etc.
During the test plan formulation, the following aspects need to be decided:
 The number of beta test sites: The number depends on the type of the product. For
generic product which is to be released to the market, a large number of test sites are
required to test the software. Sometimes, a limited beta testing is carried out with a small
number of sites and then subsequently more sites are added.
 The environment required for the beta testing: The exact hardware and software
configurations required need to be specified.
 The support services to be provided: During the beta testing, if the users encounter
problems, they need to be provided with a help line. The support mechanism needs to be
worked out and specified.
 Defect reporting mechanism: When a user finds a defect, the mechanism for reporting
the defect needs to be specified.
 Beta testing period: The testing period has to be fixed depending on the complexity of
the product (can vary from 2 weeks to 2 months).
2.3.9 Field Trial/Operational Testing
Some software products have to be tested in real working environment before releasing them to
the client for operational use. In Field trial testing the software is tested in actual working
environment. Field trial is similar to beta testing—it is normally conducted on customized
software.
For example, suppose your organization developed messaging software that works over a
satellite network (Figure 2.12).

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 14

 Software Testing

During the development time, you would have tested the software in the lab by simulating a
satellite network. However, before making the software available for the users on the satellite
network, you would like to conduct a field trial by installing the software on the satellite network
and testing it through a group of selected users. Once you find out that the software is working
fine, it can be then released for operational use.
A log sheet that gives all the problems encountered during the field trial has to be maintained
with the following details:
• Date
• Preconditions (status of the software before the problem)
• Description of the problem encountered
• Post-condition (status of the software after the problem)
• Impact (whether the software has become unusable or system has to be rebooted,
etc.)
Based on this field trial report, all the defects have to be removed and then the software can be
released.
2.3.10 Performance Testing/Load Testing
Performance testing focuses on the performance parameters, such as the response time
throughput etc. This type of testing is compulsory for client/server applications and web
applications because these applications are accessed simultaneously by a large number of users.
For example, in database systems the response time is a time to obtain a report after clicking on a
specific button. It may be difficult to specify the response times for each and every report to be
generated, so it can be specified as 30 seconds is a reasonable period, but not two minutes.

Figure 2.13 Load testing of client / server application using automated tools
Suppose you need to test a database application or a web application when 100 users
simultaneously accesses the application. For doing the performance testing, you need 100
computers and 100 test engineers! This is not practical; Therefore Performance testing tools has
to be used to do the testing. On a single computer, these tools simulate multiple users (say, 50
users), Hence, with just two computers, you can do the performance testing, as shown in figure
2.13. The simulated users are called virtual users in the performance tool jargon.

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 15

 Software Testing

2.3.11 Stress Testing

Stress testing is done to test the software at its limits of performance. For example, if the
Software is a DBMS that is expected to give a response time within the specified limit for 16
simultaneous users, the software has to be tested while slightly more than 16 users are using the
database at the same time, Generally, it is not done and the software is tested for four or five
users and delivered to the client. When the client starts using the software in actual environment,
the response increases and the client rejects the software.
Another type of stress testing is done on the resource requirement. If the RAM requirement is
specified as 64 MB for a specific application, the software needs to be tested not just on the 128
MB machines where the software was initially developed, but also on machines with 64 MB, It is
useful to test the software on 32 MB machines and check the performance as well.
The test set up for the stress testing needs to be defined as a part of the acceptance test plan.
NOTE: Performance/load/stress testing is generally done using automated test tools, such as
Load Runner, JMeter, etc. These tools simulate the actions of multiple users on a single machine
(in other words, virtual users are created) and hence the testing can be done with minimal
infrastructure and manpower.
2.3.1 2 Accessibility Testing
Accessibility testing is done to check whether an application or web site can be accessed by
persons with disabilities. Many governments are now bringing out acts making it mandatory to
make the web sites accessible by persons with disabilities, such as vision impairment, physical
impairment, and hearing impairment and cognitive impairment. To achieve this objective, World
Wide Web Consortium (W3C) released Web Content Accessibility Guidelines (WCAG). When a
web site is constructed, these guidelines need to be followed.
Accessibility testing is yet to gain popularity, but specialists are required to carry out this testing
as governments make it compulsory to make the Internet resources available to the disabled
persons. A number of tools are now available to do the web accessibility testing. You can get a
wealth of information on accessibility testing including the problems faced by persons with
disabilities while accessing the Internet, at the web site www.webaim.org.
You can get an idea of accessibility properties from Windows Accessibility Properties (Start 
Settings  Control Panel Accessibility Options). A representative screen shot is shown in
Figure 2.14.

Figure 2.14 Windows Accessibility Properties

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 16
 Software Testing

2.3.13 Conformance Testing

In the non-function requirements, standards are given as a requirement. For different application
domains, different standards are available. Conformance testing is the testing carried to ensure
that the software meets these standards.
The standards are specified by national/international bodies. These standards are specified to
provide:
• Interoperability among different equipment vendors (e.g. to make the mobile phone of
one manufacturer communicate with another mobile phone of another manufacture)
• Safety to the users (e.g. to ensure that a mobile phone does not emit high radiation to
damage the human body)
• Compatibility with the government regulations (e.g. to ensure that the healthcare software
is as per HIPAA guidelines)
• Special features such as accessibility to the disabled (e.g. W3C guidelines for
accessibility of web sites)
• Privacy to the users
• Security of the users
There are a large number of international standards bodies, to formulate specifications /
guidelines for different aspects of conformance in different application domains. Some important
bodies are:
• American National Standards Institute (ANSI)
• International Organization for Standardization (IOS)
• International Telecommunications Union (ITU)
• Internet Engineering Task Force(IETF)
• World Wide Web Consortium(W3C)
Testing the software for conformance to standards is a highly specialized activity. For instance, if
you are developing automotive software (software for embedded system used in cars), MISRA
(Motor Industry Software Reliability Association) gives a number of guidelines for developing
embedded software for automotive applications. Similarly, a number of standards are available
for nuclear plants, chemical plants, railway signaling, pharmaceutical industry, air-borne
systems, defense electronics, Internet protocols, telecommunication networks and subscriber
equipment, etc.
2.3.14 Internationalization Testing
When software is developed for international market, it is necessary to provide the option to set
different regional preferences for numbers, currency, time, date, etc.
For example, in the Indian numbering system, the numbers are denoted by 1,00,000 (for one
Lakh) and the equivalent American numbering system is 100,000 (hundred thousand).
Similarly date format differs: in some countries it is date/month/year whereas in other countries
it is month/date/year. If the computer accepts only one format, it is very inconvenient.
You can get the various settings for internationalization from Windows regional options:
Start  Settings  Control Panel  Regional Options
B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 17
 Software Testing

A representative screen shot for date format is shown in Figure 2.15.

2.3.15 Security Testing

Nowadays, security of computing systems and networks has become very important as cyber-
crime is becoming a great threat. Organizations have to protect their intellectual properties,
service organizations have to protect their users' information, governments have to protect their
information assets and individuals have to protect their privacy. To ensure that information
systems and networks are not open to threats, security testing is done. Security testing is done to
find out the vulnerabilities of systems and networks.
To provide security a number of security products are installed, such as anti-virus software,
firewalls, access control systems, intrusion detection system, intrusion prevention systems,
packet analyzers, etc.
Hackers are very expert in breaking the so-called secure systems and spread virus, commit
crimes such as stealing credit card information, etc. So, security testing is a very challenging
task. A new type of person called ethical hackers is now coming up who specialize in security
testing.
Organizations like banks, insurance agencies, credit card agencies, e-commerce portal hosting
companies, web service offering organizations, governments, etc. are now hiring ethical hackers
by paying very high fee to carry out security testing of their applications.
2.3.16 Maintenance Testing
Sometimes the test engineers test the software which is not presently under development. A
software package would have been developed many years ago and it is being used all these
years. The software may needs to be changed due to a number of reasons:
 Enhance some features of the software (modification of the software)
 Make the software run on another platform (migration of the software)
 Fix a bug in the software (correction to the software)
In such a case, the source code written (perhaps years ago) needs to be studied and modifications
made. This is a very challenging task because it is extremely difficult to understand the code
written by someone else and then modify it. Maintenance testing involves regression testing of
the software when changes are made to the code, in the absence of the original developers. Test
engineers also need to study the changes that have been made to the code and design test cases.
Here, the test engineers need to get involved in white box testing.
B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 18
 Software Testing

2.3.17 Acceptance Testing

Acceptance testing is the most important testing as this decides whether the client approves the
product or not. The criteria for acceptance testing should be decided at an early stage of product
development. By the time the design is completed, a draft Acceptance Test Plan (ATP) has to be
prepared and given to the client for approval. After discussions, the acceptance test plan has to be
approved by both parties. The ATP should be very detailed, so detailed that if testing is done as
per the ATP step by step, all the functionality and performance parameters are tested. The
functionality is tested.
The format for the ATP is given in Form 2.1.
Acceptance Test Plan
Name of the project: ______________________________________________________
Reference of requirements document: ________________________________________
Reference of design document: ______________________________________________
Test set-up required for the acceptance testing: _________________________________
Types of testing to be carried out: ____________________________________________
Test for each functionality
To test what: _____________________________________________________________
Pre-conditions: ___________________________________________________________
Post-conditions: __________________________________________________________
Test cases (to be selected based on equivalence
class partitioning, boundary value: analysis, cause
cause-effect graphs, and use cases): ___________________________________________
Expected results: _________________________________________________________
Testing process deliverables
Test results: _____________________________________________________________
Deviations: ______________________________________________________________
In case of deviations/conflicts, the process of resolution: _________________________
The final approving authority in case of differences of opinion: _____________________
Client representative: _____________________________________________________
Development team representative: ___________________________________________
Acceptance testing time frame: ______________________________________________
Resources allocated for acceptance testing: _____________________________________
Testing personnel (developer side and client side): _______________________________
Documentation personnel: __________________________________________________
Form 2.1 Acceptance Test Plan Format

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 19

 Software Testing

Testing is done from the following angles:

 Usability / learnability of the software: whether the software is user-friendly or any
improvements still need to be done on the GUI.
 User management: whether the system administrator is able to carry out user
management efficiently, by adding/deleting users, password management, etc.
 Security: whether the necessary security controls are incorporated in the software
through access control mechanisms, such as roles of users, password management, etc. If
the application is web-based, then the security testing has to be very rigorous.
 Backup mechanisms
 Disaster recovery management
2.3.18 Documentation Testing
Test engineers need to test the documents that need to be supplied to the customer, the most
important document being the user manual. Depending on the type of software and the
requirements of the customer, other manual that may need to be given to the customer are:
 Administrator manual
 Developer's manual (if you are giving some APIs with your software)
 Installation manual
 Design manual (if you need to transfer the intellectual property rights to the customer)
Documentation testing involves checking the software for accuracy and readability. You also
need to check whether the manuals are a true representation of the actual software that is being
delivered.
The documents should be as per RICADO178-B standard. This standard, whose thrust is safety,
gives a long list of documents at need to be written and submitted. These documents are divided
into the following 10 categories:
 Planning
 Standards
 Development
 Software Requirements Verification
 Software Design Verification
 Software Code Verification
 Verification of Integration Process
 Verification of Verification Process
 Configuration Management
 Software Quality Assurance

B.L.D.E.A’S SSM POLYTECHNIC, CSE (S.R. INAMDAR) Page 20