IJCSI International Journal of Computer Science Issues, Vol.

7, Issue 3, No 1, May 2010 11

ISSN (Online): 1694-0784
ISSN (Print): 1694-0814

Different Forms of Software Testing Techniques for Finding

Errors
Mohd. Ehmer Khan

Department of Information Technology

Al Musanna College of Technology, Sultanate of Oman

Abstract
Software testing is an activity which is aimed for evaluating an Testing is a process to identify the correctness
attribute or capability of a program and ensures that it meets and completeness of the software.
the required result. There are many approaches to software
testing, but effective testing of complex product is essentially a The general objective of software testing is to
process of investigation, not merely a matter of creating and
following route procedure. It is often impossible to find all the
affirm the quality of software system by
errors in the program. This fundamental problem in testing systematically exercising the software in
thus throws open question, as to what would be the strategy carefully controlled circumstances.
that we should adopt for testing. Thus, the selection of right
strategy at the right time will make the software testing Classified by purpose software testing can be divided
efficient and effective. In this paper I have described software into [4]
testing techniques which are classified by purpose.
Keywords: Correctness Testing, Performance Testing, 1. Correctness Testing
Reliability Testing, Security Testing 2. Performance Testing
3. Reliability Testing
4. Security Testing
1. Introduction
Software testing is a set of activities conducted with the 2. Software Testing Techniques
intent of finding errors in software. It also verifies and
validate whether the program is working correctly with Software testing is a process which is used to measure
no bugs or not. It analyzes the software for finding bugs. the quality of software developed. It is also a process of
Software testing is not just used for finding and fixing of uncovering errors in a program and makes it a feasible
bugs but it also ensures that the system is working task. It is useful process of executing program with the
according to the specifications. Software testing is a intent of finding bugs. The diagram below represents
series of process which is designed to make sure that the some of the most prevalent techniques of software
computer code does what it was designed to do. testing which are classified by purpose. [4]
Software testing is a destructive process of trying to find
the errors. The main purpose of testing can be quality
assurance, reliability estimation, validation or SOFTWARE
TESTING
verification. The other objectives or software testing
includes. [6][7][8]

The better it works the more efficiently it can

be tested.
Correctness Security
Better the software can be controlled more the Testing Testing
testing can be automated and optimized.

The fewer the changes, the fewer the disruption

to testing.
Performance Reliability
A successful test is the one that uncovers an Testing Testing
undiscovered error.

Fig. 1 Represent different software testing techniques which are

classified by purpose
IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May 2010 12
ISSN (Online): 1694-0784
ISSN (Print): 1694-0814

2.1 Correctness Testing Design Based Testing

Open Box Testing
The most essential purpose of testing is correctness Transparent Box Testing
which is also the minimum requirement of software. Clear Box Testing
Correctness testing tells the right behavior of system Glass Box Testing
from the wrong one for which it will need some type of Structural Testing
Oracle. Either a white box point of view or black box
point of view can be taken in testing software as a tester Some important types of white box testing techniques
may or may not know the inside detail of the software are:
module under test. For e.g. Data flow, Control flow etc.
The ideas of white box, black box or grey box testing are 1. Control Flow Testing
not limited to correctness testing only. [4] 2. Branch Testing
3. Path Testing
CORRECTNESS
TETSING
4. Data flow Testing
5. Loop Testing

There are some pros & cons of white box testing-

Pros-

1. Side effects are beneficial.

White Box Black Box
2. Errors in hidden codes are revealed.
Testing Testing

3. Approximate the partitioning done by execution

equivalence.

4. Developer carefully gives reason about

implementation.
GREY
BOX
TESTING Cons-

1. It is very expensive.
Fig. 2 Represent various form of correctness testing
2. Missed out the cases omitted in the code.
2.1.1 White Box Testing 2.1.2 Black Box Testing
White box testing based on an analysis of internal Basically Black box testing is an integral part of
working and structure of a piece of software. White box Correctness testing but its ideas are not limited to
testing is the process of giving the input to the system correctness testing only. Correctness testing is a method
and checking how the system processes that input to which is classified by purpose in software testing.
generate the required output. It is necessary for a tester
to have the full knowledge of the source code. White Black box testing is based on the analysis of the
box testing is applicable at integration, unit and system specifications of a piece of software without reference to
levels of the software testing process. In white box its internal working. The goal is to test how well the
testing one can be sure that all parts through the test component conforms to the published requirement for
objects are properly executed. [2][10] the component. Black box testing have little or no regard
to the internal logical structure of the system, it only
Analyze
OUTPUT
examines the fundamental aspect of the system. It makes
INPUT PROCESS
System Internal sure that input is properly accepted and output is
Working correctly produced. In black box testing, the integrity of
external information is maintained. The black box
Fig. 3 Represent working process of White Box Testing testing methods in which user involvement is not
required are functional testing, stress testing, load
Some synonyms of white box testing are [5] testing, ad-hoc testing, exploratory testing, usability
testing, smoke testing, recovery testing and volume
Logic Driven Testing testing, and the black box testing techniques where user
involvement is required are user acceptance testing,
IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May 2010 13
ISSN (Online): 1694-0784
ISSN (Print): 1694-0814

alpha testing and beta testing. Other types of Black box execution, analysis and reporting. This testing is
testing methods includes graph based testing method, conducted to evaluate the compliance of a system or
equivalence partitioning, boundary value analysis, component with specified performance requirement. [2]
comparison testing, orthogonal array testing, specialized Evaluation of a performance of any software system
testing, fuzz testing, and traceability metrics. [2] includes resource usage, throughput and stimulus
response time.

Analyze only By performance testing we can measure the

INPUT PROCESS fundamental OUTPUT
System characteristics of performance of any applications. One
aspects
of the most important objectives of performance testing
is to maintain a low latency of a website, high
throughput and low utilization. [5]
Fig. 4 Represent working process of Black Box Testing
PERFORMANCE
There are various pros and cons of black box testing- [5] TESTING

Pros-

1. Black box tester has no bond with the code.

2. Tester perception is very simple.

Load Stress
3. Programmer and tester both are independent of Testing Testing
each other.

4. More effective on larger units of code than Fig. 5 Represent two types of performance testing
clear box testing.
Some of the main goals of performance testing are: [5]
Cons-
Measuring response time of end to end
1. Test cases are hard to design without clear
transactions.
specifications.
Measurement of the delay of network between
2. Only small numbers of possible input can
client and server.
actually be tested.
Monitoring of system resources which are
3. Some parts of the back end are not tested at all.
under various loads.
2.1.3 Grey Box Testing Some of the common mistakes which happen during
performance testing are: [5]
Grey box testing techniques combined the testing
methodology of white box and black box. Grey box Ignoring of errors in input.
testing technique is used for testing a piece of software Analysis is too complex.
against its specifications but using some knowledge of Erroneous analysis.
its internal working as well. [2] Level of details is inappropriate.
Ignore significant factors.
Grey box testing may also include reverse engineering to Incorrect Performance matrix.
determine, for instance, boundary values or error Important parameter is overlooked.
messages. Grey box testing is a process which involves Approach is not systematic.
testing software while already having some knowledge
of its underline code or logic. The understanding of There are seven different phases in performance testing
internals of the program in grey box testing is more than process: [5]
black box testing, but less than clear box testing. [11]
Phase 1 Requirement Study
2.2 Performance Testing Phase 2 Test plan
Phase 3 Test Design
'Performance Testing' involve all the phases as the Phase 4 Scripting
mainstream testing life cycle as an independent Phase 5 Test Execution
discipline which involve strategy such as plan, design, Phase 6 Test Analysis
IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May 2010 14
ISSN (Online): 1694-0784
ISSN (Print): 1694-0814

Phase 7 Preparation of Report handle the anticipated no. of users or not. This can be
Requirement done by making the virtual user to exhibit as real user so
Collection that it will be easy to perform load testing. It is carried
only to check whether the system is performing well or
not. The main objective of load testing is to check
whether the system can perform well for specified user
Preparation of or not. Load testing increases the up time for critical
Plan web applications by helping us to spot the bottle necks
in the system which is under large user stress.

Load testing is also used for checking an application

Designing against heavy load or inputs such as testing of website in
order to find out at what point the website or
applications fails or at what point its performance
degrades. [2][5]

Scripting Two ways for implementing load testing are

1. Manual Testing: It is not a very practical option as it

is very iterative in nature and it involves [5]
Measure response time
Execution Compare results

2. Automated Testing: As compared to manual load

testing the automated load testing tools provide
more efficient and cost effective solutions. Because
Analysis
with automated load testing, tools test can easily be
rerun any number of times and decreases the
chances of human error during testing. [5]
NO
2.2.2 Stress Testing
Is Goal
We can define stress testing as performing random
Achieved?
operational sequence, at larger than normal volume, at
faster than normal speed and for longer than normal
YES
periods of time, as a method to accelerate the rate of
finding defects and verify the robustness of our product,
or we can say stress testing is a testing, which is
Preparation of conducted to evaluate a system or component at or
Report Final Report is
Prepared beyond the limits of its specified requirements to
determine the load under which it fails and how. Stress
testing also determines the behaviour of the system as
user base increases. In stress testing the application is
Fig. 6 Represent Performance Testing Process tested against heavy loads such as large no. of inputs,
large no. of queries, etc. [2] [5]
Typically to debug applications, developers would
execute their applications using different execution There are some weak and strong points of stress testing.
stream. Which are completely exercised the applications
in an attempt to find errors. Performance testing is Weak Points
secondary issue when looking for errors in the
applications but, however, it is still an issue. 1. Not able to test the correctness of a system.
2. Defects are reproducible.
There are two kinds of performance testing: 3. Not representing real world situation.

Strong Points
2.2.1 Load Testing
1. No other type of test can find defect as stress
Load Testing is an industry term for the effort of testing.
performance testing. The main feature of the load testing 2. Robustness of application is tested.
is to determine whether the given system is able to
IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May 2010 15
ISSN (Online): 1694-0784
ISSN (Print): 1694-0814

3. Very helpful in finding deadlocks.

Confidentiality: By security testing, we will ensure
2.3 Reliability Testing the confidentiality of the system i.e. no disclosure of
the information to the unknown party other than
intended recipient.

Integrity: By security testing, we will maintain the

integrity of the system by allowing the receiver to
Reliability Robustness
Testing Testing
determine that the information which he is getting is
correct.

Authentication: Security testing maintains the

Fig. 7 Represent Reliability testing authentications of the system and WPA, WPA2,
WEP are several forms of authentication.
Reliability Testing is very important, as it discover all
the failures of a system and removes them before the Availability: Information is always kept available
system is deployed. Reliability testing is related to many for the authorized personnel whenever they needed
aspects of software in which testing process is included; and assures that information services will be ready
this testing process is an effective sampling method to for use whenever expected.
measure software reliability. Estimation model is
prepared in reliability testing which is used to analyze Authorization: Security testing ensures that only the
the data to estimate the present and predict future authorized user can access the information or
reliability of software. [4][2] particular service. Access control is an example of
authorization.
Depending on that estimation, the developers can decide
whether to release the software or not and the end user SECURITY
will decide whether to adopt that software or not. TESTING

Based on reliability information, the risk of using

software can also be assessed. Robustness testing and
stress testing are the variances of reliability testing. By Security Ethical
Robustness we mean how software component works Auditing Hacking
under stressful environmental conditions. Robustness
testing only watches the robustness problem such as
machine crashes, abnormal terminations etc. Robustness
testing is very portable and scalable. [4] Security Penetration
Scanning Testing
2.4 Security Testing
Security Testing: Security testing makes sure that only
the authorized personnel can access the program and
only the authorized personnel can access the functions Vulnerability Posture Risk
Scanning Assessment Assessment
available to their security level. Security testing of any & Security
developed system or (system under development) is all Testing
about finding the major loopholes and weaknesses of a
system which can cause major harm to the system by an Fig. 8 Represent various type of security testing
authorized user. [1][2]
Different types of security testing in any organization
Security testing is very helpful for the tester for finding are as follows: [3]
and fixing of problems. It ensures that the system will
run for a ling time without any major problem. It also 1. Security Auditing and Scanning: Security
ensures that the systems used by any organization are Auditing includes direct inspection of the
secured from any unauthorized attack. In this way, operating system and of the system on which
security testing is beneficial for the organization in all it is developed. In Security Scanning the
aspects. [1][2] auditor scan the operating system and then
tries to find out the weaknesses in the
Five major concepts which are covered by security operating and network.
testing are
IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May 2010 16
ISSN (Online): 1694-0784
ISSN (Print): 1694-0814

2. Vulnerability Scanning: Various vulnerability 3. Reliability testing, which discovers all the failure of
scanning software performs Vulnerability the system and removes them before the system
Scanning, which involves the scanning of the deployed.
program for all known vulnerability.
4. Security testing makes sure that only the authorized
personnel can access the system and is further
3. Risk Assessment: Risk Assessment is a divided into Security Auditing and Scanning,
method in which the auditors analyze the risk Vulnerability Scanning, Risk Assessment, Posture
involved with any system and all the Assessment and Security Testing, Penetration
probability of loss which occurs because of Testing and Ethical Hacking.
that risk. It is analyzed through interviews,
discussions, etc. The successful use of these techniques in industrial
software development will validate the results of the
4. Posture Assessment and Security Testing: research and drive future research. [8]
Posture Assessment and Security Testing help
the organization to know where it stands in References:
context of security by combining the features
of security scanning, risk assessment and [1] Software testing-Brief introduction to security
ethical hacking. testing by Nilesh Parekh published on 14-07-2006
available at http://www.buzzle.com/editorial/7-14-
5. Penetration Testing: Penetration Testing is an 2006-102344.asp
effective way to find out the potential [2] Software testing glossary available at
loopholes in system and it is done by a tester http://www.aptest.com/glossary.html#performance
testing
which forcibly enters into the application [3] Open source security testing methodology manual of
under test. A tester enters into the system with PETE HERZOG and the institute for security and
the help of combination of loopholes that the open methodology-ISECOM.
application has kept open unknowingly. [4] Software testing by Jiantao Pan available at
http://www.ece.cmu.edu/~roopman/des-
6. Ethical Hacking: Ethical Hacking involves 899/sw_testing/
large no. of penetration test on a system under [5] Software Testing by Cognizant Technology Solution.
test. To stop the forced entry of any external [6] Introduction to software testing available at
elements into a system which is under security http://www.onestoptetsing.com/introduction/
[7] Software testing techniques available at
testing. http://pesona.mmu.edu.my/~wruslan/SE3/Readings/
GB1/pdf/ch14-GB1
[8] Paper by Lu Luo available at
3. Conclusion http://www.cs.cmu.edu/~luluo/Courses/17939Report
.pdf
Software testing is an important technique for the [9] Security testing-wikipedia the free encyclopedia
improvement and measurement of a software system available at http://en.wikipedia.org/wiki/security-
quality. But it is really not possible to find out all the tetsing.
errors in the program. So, the fundamental question [10] White box testing from wikipedia, the free
arises, which strategy we would adopt to test. In my encyclopedia.
[11] Software testing for wikipedia available at
paper, I have described some of the most prevalent and
http://en.wikipedia.org/wiki/grey_box_testing#grey_
commonly used strategies of software testing which are box_tetsing
classified by purpose and they are classified into [5]

1. Correctness testing, which is used to test the Mohd. Ehmer Khan

right behavior of the system and it is further
I completed my B.Sc in 1997 and M.C.A. in 2001 from Aligarh
divided into black box, white box and grey box
Muslim University, Aligarh, India, and pursuing Ph.D (Computer
testing techniques (combines the features of Science) from Singhania University, Jhunjhunu, India. I have
black box and white box testing). worked as a lecturer at Aligarh College Engineering &
Management, Aligarh, India from 1999 to 2003. From 2003 to
2005 worked as a lecturer at Institute of Foreign Trade &
2. Performance testing, which is an independent
Management, Moradabad, India. From 2006 to present working
discipline and involves all the phases as the as a lecturer in the Department of Information Technology, Al
main stream testing life cycle i.e. strategy, plan, Musanna College of Technology, Ministry of Manpower,
design, execution, analysis and reporting. Sultanate of Oman. I am recipient of PG Merit Scholarship in
MCA. My research area is software engineering with special
Performance testing is further divided into load
interest in driving and monitoring program executions to find
testing and stress testing. bugs, using various software testing techniques.