International Journal of Computer Science and Information Security (IJCSIS),

Vol. 17, No. 7, July 2019

Emerging Trends in Software Testing Tools &

Methodologies: A Review
Jawwad Ibrahim1, Sundas Hanif2, Sobia Shafiq3, Saeed Faroom4
1,2,3
Department of CS & Software Engineering, APCOMS, Pakistan
1
[email protected]
2
[email protected]
3
[email protected]
4
University of Lahore, Gujrat Campus, Pakistan
4
[email protected]

Abstract— Generally Software Testing is an important stage in or automated software testing. In manual testing the tester
software quality assurance process. During the Whole software prepares specific test cases manually that Executed in a way
Development life cycle Errors & Bugs can be injected at any stage. to find the errors/bugs. In Automated testing technique, it is a
All the time software testing is a very difficult & expensive process speedy process as it is mainly done with machines with less
in order to ensure the quality of developed product. In manual
testing the tester prepares specific test cases manually that
human intervention and consideration [6].
Executed in a way to find the errors/bugs. In Automated testing In our research we are have acknowledged that how to
technique, it is a speedy process as it is mainly done with machines find or get a set of test cases to test a software system to
with less human intervention and consideration. The Objective of ensure the Quality Constrain in Developed product. In this
this Paper is to find how to find/get a set of different emerging paper software testing tools and techniques are described.
tools and methodologies to test a software system to ensure the This study mainly focuses on the comparison of different test
quality constrain in developed product. We will do the comparison case generation techniques & automation of software testing
of different testing tools on the basis of existing literature and also execution phase in testing process [7].
do the comparative study of different automated testing
methodologies which is helpful in the selection of testing tool and
methodology which is helpful in terms of Cost and Time. This II. LITERATURE REVIEW
paper addresses the emerging trends in software testing tools and Software Testing is very important procedure in almost
techniques feasible at different stages of SDLC.
every new developing software project. Testing is not only
concerned to test that your product that works perfectly to the
Keywords: Automation Testing, Automatic Testing, Testing given inputs but more to make sure that the developed product
Methodologies, Testing Tools is free of bugs [1]. Finding the bugs from the product also
called as debugging. Testing represent 40% of the total budget
I. INTRODUCTION for a software product under development. Test engineers
Software Testing is an important Part of Software Quality have to prepare test documents and test plan before starting
Assurance. Testing is not a one-person job it is a team work the test process. Test document is very important as if old test
may be smaller ore larger it completely depends on the size engineer leaves and a new person can start working without
and complexity of the software product under testing [4]. wasting the time. Test plan is dynamic document that is
Software testing is systematically and step by step process of prepared in beginning as the requirement engineering phase
detection of errors/bugs of different classes by reducing the completed [2]. To analyze various testing techniques which
time & effort. Recently, Software Testing appeared as a most already exists, available literature was studied extensively.
important & popular part in Software development industry. There are a number of Tools and Techniques available for
Software testing is a significantly time and cost consuming the automated testing. The right selection of Testing tool and
process in software development. Software testing focuses technique may vary with the product to product as it depends
mainly in two operations which are respectively Verification on the product we are going to test [8].
& Validation. Testing process must be incorporated in whole
SDLC. Effective Software testing is the key to development A. Software Testing Automation Tools
of a quality software product with more user satisfaction, very 1) Ranorex: It is economical tool which is used for
low maintenance cost, more accuracy and reliability. Mainly programmed testing. It is also effective tool as it tests the
there are Testing Strategies could be Manual software testing software product from users prospective. One can use

108 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 17, No. 7, July 2019

multiple repositories at a time in case if multiple b) Selenium RC

software testers are working on the one product. It is c) Selenium Web Driver
purely written in .net programming Language. Main d) Selenium Grid
functionalities of this tool are its image-based detection
and containing Ranorex Recorder for Replaying Record 6) Watir: It is an automated web testing tool which is
[13]. available open source. The main feature of this tool is that it is
easy to use and easy to maintain. It works only with three
2) Rational Functional Tester (RFT): RFT is an browsers which are as follows Opera, Internet Explorer and
automated testing tool from the rationale Software. RFT Firefox [8][18].
is developed by IBM in 1999, It is purely written in OOP
programming language. It is mainly done to find out that 7) Windmill: It is an automated web testing tool which
the newly implemented functionalities do not disabled provides the automation testing with debugging capabilities.
and interrupt the previous functionalities [8]. This tool is The main purpose of the windmill is to develop test case
used because it enables regression testing and it frees up writing with ease and portability [19].
quality assurance departments from maintaining and
executing basic test plans and cases [14][15]. 8) Sahi: Sahi is an automated testing tool. It is an open source
testing tool for the web applications only. The main users of
3) Janova: It is also the important automated testing tool. this tool are:
It is not any language based and it does not require any
written scripts. It is simply English based tool that makes a) Web application developers for reproducing and
it easy to use for everyone. It is cloud based system. fixation of bugs [8].
Main advantage of this tool is that it is cost is very low as b) Quality Analysts for Functional testing [21].
10$ for a month. You do not need to download and c) Business analysts who use this for the definition
install any software [15]. and verification of functionality [21].

4) TestComplete: It is an automated self-testing tool. B. Software Testing Automation Methodologies:

TestComplete provides tester the ability to create
automated test cases for MS windows, Web based 1)Smoke Testing: This methodology this is portrayed, as
applications, Android based applications and IOS based soon developer team handed alpha build to the testing
applications. It is called as one of primary automated team. Smoke testing also called as functional testing of a
self-testing tool in the world. Many of the world leading software product as it focuses on the main features of the
companies uses this tool in their systems. One also can application under development [3]. Upon the main
develop scripted tests. It contains simple Operation successful testing of main features of the application the
concepts which are a plus point for the user [16][20]. testing team further convey the results of testing with the
whole of team working on the project. The smoke testing
5) Selenium: It is an automated web testing tool which is is mainly done because we want to know that is there
available open source and it mainly used for the testing any main failure in the build as next step is to hand over
of web browsers across the different platforms. Selenium the build for exhaustive testing team [22][23]. It is
provides a test domain specific language for writing tests further divided in two types which are as follows:
for many famous programming languages including [20];
a) Manual Smoke Testing
a) C# b) Automated Smoke testing
b) Groovy
c) Java 2) Regression Testing: The Fixation of error/defects is
d) Perl the process to remove the problems which was earlier
e) PHP reported previously and after fixation make sure that
f) Python these are not occur and cause any ripple effect on same
g) Ruby area of the product [24].The main theme of Regression
testing is to get to know that any effect on functionality
Selenium can be narrow down into four components of application where the developer already change the
[17]. Which are as follows: code for bug fixation .In regression testing Test team has
to test whole of the scenario rather than particular
a) Selenium IDE scenario.
Regression testing may be further divided into as before these units are integrated and merged as a whole
follows: system unit. Unit testing is always made by system
a) Reduction of Test Suites programmers and system developers can perform this
b) Change based method task with help of tester [9][25]. Unit testing further
3) Unit Testing: Unit testing is made for the purpose of divided into the following:
testing each and every unit of the system individually

109 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 17, No. 7, July 2019

a) NUnit Test tool tools and computer system [26].The main objective of
b) xUnit Architecture automated testing is minimize the human testing effort
with a minimum set of test scripts by using automated
4) Automated testing: Automated testing is one of the testing tools. In automated testing the test manager is the
most popular and handy in field of software engineering one who decide which application functional area is
for the purpose of software testing where testers can considered for automated testing and which areas for the
Unhidden main flaws in the application can report these manual testing ore other testing strategies [9].
issues and bugs automatically using Automated testing

III. COMPARISON
The following table summarizes the comparison made between different software testing tools.

TABLE 1
COMPARISON OF DIFFERENT SOFTWARE TESTING TOOLS

Table 1 Table 2
Criteria Test Ranorex Criteria Ranorex RFT JANOVA
Complete Install Required Yes Yes No
Test Recording Yes Yes Cloud No No Yes
Desktop application Testing Yes Yes Knowledge of scripts No No No
Data driven Testing Yes Yes required
Test Result reports Yes Yes Access to code required No Yes No
Playback Yes Yes User Friendly interface Yes Yes Yes
Identifying the tested object Yes No Tutorial on How- to- use Easy to follow From Help Not easy to
Plug- ins and extensions Yes No find

TABLE 2
COMPARISON OF DIFFERENT AUTOMATED SOFTWARE TESTING METHODOLOGIES

FEATURES SELENIUM QUICK TEST TEST COMPLETE

PROFESSIONAL
Licensing Cost “It is open source. So, there’s no “Licensed and very Expensive, “$2K Enterprise Seat License”
licensing or renewal cost for this Ten user license costs approx.
tool. It’s free of cost” [19]. 60L”.

Application support “Web Applications only it “A client server application” “All of this included right out of
supports addition of plug-ins to Only. It also supports add-ons, the box there are no plug-ins or
achieve desired results that are but user needs to purchase license add-ons to buy. You can install
not provided by Selenium Core. for them”. Test Complete and immediately
Since, selenium is open source, create any test against any
plug-ins are also available free of application”.
cost” [21].
Object Oriented Language “Supports Java, .Net, Perl, PHP, “Scripts can be developed only in “Test Complete supports
support and Scalability Python and Ruby” [27]. VBScript or JavaScript”. scripting in VBScript, JSScript,
DelphiScript, C++Script and
C#Script, so you can create
scripts in the language”.
Support for operating “Supports Windows “QTP supports only Windows “Windows 7, Windows Vista,
system/platforms PC/MAC/UNIXPlatforms”. XP”. Windows Server 2008 or later
operating systems”.
Programming skills “For using Selenium, one needs “QTP is quite easy to use. It is “TC is good for both webs based
to have programming skills”. quite easy to edit the script, and desktop application”.
parameterize, navigate, playback
and validate the results” [27].
Usage “Selenium needs quite a bit of “QTP is quite easy to learn in a “Support for all 32-bit and 64-bit
expertise”. short time”. window application”.
Database applications “With Selenium one needs to “QTP works very well with “TC works very well database
exert hard to do the same job” database applications”. application”.
[28].

110 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 17, No. 7, July 2019

Platform dependency “With Selenium these tasks can “It is difficult to deploy smoke “It is difficult to deploy
be easily accomplished” [12]. tests for web applications using application using” [29].
QTP especially with Windows7”.
Report Generation “Selenium users do not enjoy “With QTP we can easily “Report generation is an easy-to-
such luxury as enjoyed”. generate most comprehensive use utility that is support along
reports due to the availability of with TC and lets you generate
an efficient online help” [29]. dump files”.

IV. MOTIVATION

The major concern of this research is to do a review of

different testing tools and methodologies to select a better set
of tools to test a product.

V. CONCLUSION
In this study we discussed the different Emerging Trends of [13] Pallavi Bhuarya, Anuradha Chatterjee ,Shruti Nupur “Mobile Application
Testing: Tools & Challenges” IJECS ,Vol 5 ,10 October 2016.
tools and methodologies of Software Testing available in
market. In any of engineering project the main focus of [14] V.Sathyavathy,”Evaluation of Software Testing Techniques Using
Artificial Neural Network ”IJECS Vol.6 Issue 3, March 2017,ISSN 2319-
engineers is that the project under development is a quality 7242.
project. Testing is always a tough and time Taking Task. [15] Rasneet Kaur ChauhanȦ* and Iqbal SinghḂ“Latest Research and
Automatic testing is a time saving and speedy process over Development on Software Testing Techniques and Tools” INPRESSCO
manual testing. We have done the comparison of different ,Vol 4(4) August2014.
testing tools on the basis of existing literature and also do the [16] Neha Dubey, Mrs. Savita Shiwani “Studying and Comparing Automated
comparative study of different automated testing methodologies Testing Tools; Ranorex and TestComplete”.Online publication in IJECS 5
May 2014.
which is helpful in the selection of testing tool and
[17] Shivkumar Hasmukhrai Trivedi “Software Testing Techniques ,ijarccsse
methodology which is helpful in terms of Cost and Time. This ,Vol.2,Issue10,October 2012.
research will help in the way to find a useful set of tools and
[18] Neha Bhateja ,” A Study on Various Software Automation Testing Tools“
methodologies according to their product. ,ijarcsse,Vol.5,issue6,june 2015
[19] Richa Rattan, “Comparative Study Of Automation Testing Tools: Quick
Test Professional & Selenium”, Vsrd International Journal Of Computer
Science & Information Technology, Vol. 3 No. 6 June 2013.
[20] Keith Stobie, Too Darned Big To Test, ACM Digital Library, Queue –
VI. REFERENCES Quality Assurance, February 2005, Vol 3, Issue 1.
[21] William White, Sifting through the Software Sandbox SCM meets QA,
ACM Digital Library, Queue – Quality Assurance, February 2005, Vol 3,
[1] Ingrid A. Buckley,Winstom S. Buckley “ Teaching Software Testing Issue 1.
using Data Structures,” in IJACSA, vol. 08,2017.
[22] Sneha Khoria and Pragati Upadhyay “Performance Evaluation and
[2] Arti Rana, Arvind Singh Rawat, AnchitBijalwan,”Process of Finfing Comparison of Software Testing Tools”, in VSRD International Journal
defects in software testing,” in ACSIS ,vol. 10,2017. of Computer Science Information Technology, Vol. 2 No. 10 & October
[3] Jyoti , Susheela Hooda, “A systematic review and comparative study of 2012.
existing testing techniaues for aspect-oriented software systems,” in IRJET, [23] Software Test Automation - http ://en.wikipedia.org/wiki/Test automation
vol. 5,Issue 5, May 2017.
[24] Mohd. Ehmer Khan, “Different Forms of Software Testing Techniques for
[4] GaganR. Jaya , A. Senthil Kumar, “Quality assurance for business needs Finding Errors”, in IJCSI International Journal of Computer Science
using software testing concepts,” in IJARCCE,Vol 6,Special issue 1, Jan 2017. Issues, Vol. 7, Issue 3, No 1, May 2010.
[5] Dr. P. Maragathavalli, A. Saranya,“ Prioritization on software reliability [25] Jovanovich and Irena, “Software Testing Methods and Techniques”, May
assesment using adaptive testing strategy” inIJSRCSEIT,Vol 2,Issue 3, 2017. 26, 2008.
[6] Olekasandr Dorenskyi,“The assesment method of test suites for testing of [26] Quick Test Professional entry in Wikipedia: Http: //en.wikipedia.org.
information security systems software,” inITSEC, vol.7, 2017. /wiki/HP_QuickTest_Professional.
[7] Saniyam galhotra,Yuriy burn,Alexandra meliou” Fairness testig:testing [27] Mercury Quick Test Professional tutorial, version 8.0.Mercury Interactive
software for discrimination,”in IJICSE, vol.4,Issue 3,May 2017. Corp oration, Documentation, 2004.
[8] Alka bamota, Amanjot kaur randhawa,” Software testing techniques,”in [28] Automation testing- www.guru99.com/automationtesting. html.
IJICSE, vol.4,Issue 3,May 2017.
[29] Richa Rattan, Department of Computer Science, Hindu Engineering
[9] Farrukh Latif Butt, Shahid Nazir Bhatti, Sohail Sarwar, Amr Mohsen Jadi, College, Sonipat, Haryana, INDIA, “Comparative study of automation
Abdul Saboor,” Optimized software resting techniques in agile process- a testing tools: Quick Test Professional & Selenium” in International
systematic approach,”in IJACSA, vol.8,Issue1,2017. Journal of Computer Science & Information Technology, Vol. 3 No. 6
[10] Rasneet kaur chohan, iqbal singh, “ Latest research and development on June 2013.
software testing techniques and tools,” in IJCET, Vol 4, No 4, Aug 2014.
[30] Manjit Kaur, Raj Kumar, “Comparative study of automated testing Tools:
[11] Rasneet kaur chohan, iqbal singh, “ Latest research and development on Test Complete and Quick TestPro, International Journal of Computer
software testing techniques and tools,” in IJCET, Vol 4, No 4, Aug 2014. Applications”, in Volume 24-No. 1, June 2011.
[12] Harpreet Kaur, Dr. Gagan Gupta “Comparative Study of automated [31] Vishawjyoti and Sachin Sharma,dec 2012,”Study and Analysis of
testing tools : Selenium, Quick Test Professional and Testcomplete,” in Automation Testing Techniques”, in International University, Faridabad,
IJCET, Vol 4, No 4, Aug 2014. Vol3, No 12,36-43.

111 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 17, No. 7, July 2019

[32] SanjeevDhawan,NirmalKumar, Divya Sethi, “Automated Testing of Web [33] NidhikaUppal, AP,may 2012, “International Journal of Computer
Enabled Systems through FSM and Quick Test Professional”, in Applications” ,Volume 46– No.
International journal of software and web science (IJSWS) .

112 https://sites.google.com/site/ijcsis/
ISSN 1947-5500