© 2019 JETIR December 2019, Volume 6, Issue 12 www.jetir.

org (ISSN-2349-5162)

Artificial Intelligence in Software Test

Automation: A Systematic Literature Review
Dhaya Sindhu Battina
Sr. Data Engineer & Department of Information Technology
CA, USA

Abstract— The main aim of this paper was to review how towards autonomous testing. In software testing tools, AI is
artificial intelligence works in software test automation. being utilized to make the software development lifecycle
When it comes to software engineering, artificial simpler for the team working on the product. In software
intelligence (AI) has had a significant influence, and development and testing, artificial intelligence (AI) may be
software testing is no exception. With artificial intelligence used to automate and minimize the number of dull and
(AI), the goal of software test automation may be closer laborious operations that must be performed manually
than ever before. To some extent, the paradigm has [2,3]. We must make certain that the test is always
changed during the previous two decades [1]. Everything performed with an empty cart before adding any products.
about the testing process has been a positive experience, This helps to avoid distorted findings and adheres to sound
starting with manual testing and progressing to automated automation principles. "Maintenance" is the largest issue
testing, where Selenium is acknowledged to be one of the with test automation. As software complexity rises, we
best test automation tools. As a result, in today's high-speed must write more tests to keep up. It's because of this that
IT landscape software testing must come up with fresh we're swamped with testing and maintenance. It takes a lot
testing approaches that are based on solid research. The of time and effort to debug and resolve tests that fail.
emergence of AI-based testing has been very beneficial for Recent research shows that maintaining tests takes roughly
this aim [1]. A computer's ability to learn without human 40% of the time spent by testers [4].
involvement may be fully simulated by AI algorithms and II. PROBLEM STATEMENT
machine learning (ML). While AI and ML entail the
construction of distinct and unique algorithms to access The main problem that this paper will address is to
data and learn from it by identifying patterns to make review artificial intelligence in software test automation.
conclusions, these predictions are intended to be employed Automated software testing has a major difficulty, which
in software testing to their full potential [1]. will be examined in detail in this paper. As a result of a lack
of intelligence and premature human involvement, today's
Keywords: Artificial intelligence, automation, Software technologies are forced to cope with ineffective test runs
test automation, software engineering, AI systems [5]. As a result, there will be no way to identify test
I. INTRODUCTION mistakes, code flaws, or other important obstacles in the
The significance of technology in our professional and testing environment. When it comes to testing, artificial
personal life constantly changes at a breakneck rate, and we intelligence makes it possible for users to move over their
must keep up with it. The digital revolution is now affecting current challenges and improve their productivity. Instead
every part of life, from household appliances to virtual of prioritizing the feature's testing, most teams just assign
reality headsets. To an international audience, companies that task to whoever happens to be available at the moment
create applications utilized by hundreds of thousands or [5]. To test software effectively, we need testers that are
perhaps million worldwide [2]. The agile quick delivery inquisitive about the product and have a critical
methodology is used by the vast majority of those perspective. These testers should question the product and
companies, resulting in fresh launches every two weeks on assess it objectively.
average. These programs must be thoroughly tested before III. LITERATURE REVIEW
each launch to provide the best possible experience for the A. An Overview of Artificial Intelligence for
end-user. Manual testing cannot keep up at that rate. Every Software Testing
company, no matter how big or little, views software and
application testing as a critical phase in the development Software testing is a critical step in ensuring the
cycle. There are several important components of a application's customers are satisfied. Test automation is
program that are validated by this process [2]. To be sure, carried out in a controlled manner in which an application
manual testing gets more inefficient, time-consuming, and is monitored under specified situations, allowing testers to
expensive as software expands and new functionality is gauge the threshold and potential dangers associated with
introduced. Test automation, which automates critical the software's deployment [6]. In software testing, artificial
processes & operations in detail to boost the quality & intelligence (AI) aids in the prevention of application fail-
effectiveness of human testers, is increasingly being added overs that might be costly to both the program and the
to prevent such concerns. company in the long run. Tests using Artificial Intelligence
In software development, the usage of artificial are becoming more important as AI becomes increasingly
intelligence (AI) is still in its development, and the amount prevalent in our daily lives. Even with autonomous driving
of autonomy is still considerably lower than observed in automobiles, there is still the risk of human lives being
more mature fields of work including such self-driving endangered if the car's intelligence isn't working correctly
systems or voice-assisted control, but it is still moving and it makes the incorrect judgment or responds too slowly
[6]. An introduction to the advantages and requirements of
JETIR1912176 Electronic copy available at: https://ssrn.com/abstract=4004324
Journal of Emerging Technologies and Innovative Research (JETIR) www.jetir.org 1329
 © 2019 JETIR December 2019, Volume 6, Issue 12 www.jetir.org (ISSN-2349-5162)
artificial intelligence in software testing will be provided in testing easy for the users by using computer vision. Sikuli
this article. Robotic testing is in charge of carrying out Test was created to run on any operating system. Because
routine activities more accurately and quickly. of this, it may be used to test personal computers, online,
and mobile (Android) apps [8]. The tool seems to be
actively being developed as SikuliX at the moment.
Testilizer can generate test scenarios for software
applications leveraging SVM from Selenium scripts that
are already in existence. It starts with Selenium tests and
may create additional test cases for application states that
haven't been reached yet. It's necessary to have Crawljax
installed before running any tests on that system [9].
Automating Android GUI tests is made easier using
SwiftHand's GUI test automation features. This approach is
used to explore the model of the graphical user interface
(GUI) of the program under test [10]. SwiftHand then
makes use of it to produce the necessary inputs for
inspecting the software's previously unvisited levels.
SwiftHand is compatible with Linux and Mac. The tool's
GitHub source has comprehensive installation and uses
instructions [11].
Figure I: Framework for software test automation D. The Importance of Artificial Intelligence in
B. How is Artificial Intelligence Shaping the the Software Testing Process
Dynamics of Software Testing?
In the field of software development, software testing
We're leaning more and more on AI to make the is a critical step in the process. Nevertheless, due to the lack
application more secure (AI). We may be handing over of resources and time, programmers are often unable to
much of the testing to AI as it becomes more automated. perform comprehensive testing (a test technique in which
So, instead of human-driven testing, we're heading towards all conceivable data configurations are tested) on
a situation where robots execute test scripts in place of applications. To automate recurring patterns, we want a
people [6]. Machine learning and self-improvement will system capable of intelligently recognizing areas that will
need some human input, but it will be minimal. Hence, the be developed and more concentrated. The greatest time,
creation of a group focused on the Grand Dream of Testing money, and resources go into software testing. In addition,
has become critical, where everything is automated without developers are looking for speedier releases, and Artificial
human interaction and technologies provide superior Intelligence is a good solution [12]. A human tester adds
testing than existing application test teams [6,7]. Consider unnecessary expenses and energy since 80 percent of
going one step further and imagining an environment where testing consists of repeating tests that the software already
software can test, diagnose, and cure itself on its own. has. Artificial Intelligence may assist automate these
C. Artificial Intelligence-enhanced Software tools procedures more effectively. To discover application
difficulties, it would be a good practice to use both
Model-Inference driven testing (MINTest) is used for cognitive abilities and AI automation to create unique and
software test automation and may be used to generate test novel software systems. Repetitive labor should be
cases using the C4.5 method. It describes itself as a automated using Artificial Intelligence, such that just 80%
framework for unit and integration testing on its website of testing processes need human creativity and thinking
[7]. It is implemented for the Linux operating system (OS). [12].
Using AutoBlackTest (Automated Black-Box Testing), the When it comes to creating smarter and more efficient
supervised learning method known as QLearning is applications for end-users, artificial intelligence algorithms
implemented. The tool's primary purpose is to generate may be a huge assist in the testing sector. However,
GUI test cases automatically. GitHub says it's only understanding how Artificial Intelligence (AI) may be used
compatible with IBM Rational Functional Tester running wonderfully is critical. Artificial intelligence algorithms
on Windows [7,8]. It's impossible to determine if the that access automation in the same way as an actual human
program works with Windows OS versions higher than 8.1 would. The next step is to identify the parts of the process
and JRE based on the evidence currently accessible. where Artificial Intelligence may improve performance,
AimDroid is a GUI testing platform for Android apps and then use a machine learning or learning techniques
created by Google. Exploration of the app's activity is used algorithm to do so [13]. An algorithm that encourages the
for automated testing. Using the program, tests may be run process, aids testers in finding the most problems in the
and results reported to the user. Fusing was employed as an shortest time, and increases the reliability and accuracy of
AI enhancement [8]. One of AimDroid's drawbacks and the application is a plus [13]. Developers may then utilize
concerns is that the smartphone must be rooted: the user of the results to iterate on the product and learn from their
the device is given root access. mistakes.
To fix GUI test breakage, Vista makes use of computer
vision technology from the past. A successful test is
recorded in the web-based GUI. It is possible for Vista to
restore test scripts that fail on a subsequent version of an
application by comparing their current status with what was
documented before. In specific Selenium scripts, the
program presently supports repairing Java scripts. To write
GUI tests, you may use the Sikuli Test, an algorithm that is
efficient that lets you use the visual notation (such as an
image of an element to help identify it on the screen) while
utilizing visual notation. The program makes automated
JETIR1912176 Journal Electronic copy Technologies
of Emerging available at: https://ssrn.com/abstract=4004324
and Innovative Research (JETIR) www.jetir.org 1330
 © 2019 JETIR December 2019, Volume 6, Issue 12 www.jetir.org (ISSN-2349-5162)
E. The advantages of using artificial intelligence be a part of test automation. According to an InfoWorld
in software testing study, 88% of firms automate 50% or more of their tests,
which results in quicker testing cycles, 71% higher test
coverage, and 68% better problem detection. Many
American organizations are aiming to expand their
automated testing portfolios as Agile and DevOps adoption
increases [15,16]. Test automation has grown by 85% in the
last two years, according to app developer Magazine. Open
Source software technologies freely accessible on the
market today have contributed to this increase. As a
decentralized architecture for resilient and flexible
cybersecurity management, cybersecurity mesh is a term to
remember. As a result of cybersecurity mesh, the perimeter
may be refocused on securing people or things rather than
just their physical location. System scalability and
adaptability will be determined via this kind of testing. The
goal of artificial intelligence (AI) in software testing is to
make testing more intelligent and efficient. The use of
Fig i: The benefits of integrating Artificial Intelligence in
artificial intelligence (AI) and machine learning (ML) helps
Software Testing
automate and enhance testing [16]. Software testing using
i. Enhanced Precision
artificial intelligence (AI) saves time and allows teams to
When doing frequent manual software testing, even work on more difficult tasks, such as developing inventive
the most seasoned tester is prone to make errors. To aid new features. The phrase "Mobile First" and providing the
with this, automated software tests conduct the same or user with a mobile platform via mobile internet, hybrid
repeated tasks properly every time, ensuring that accurate mobile apps were popular throughout the world around 5
findings are recorded every time. The time saved by years ago. Artificial intelligence (AI) is the newest fad (AI).
automating manual testing allows the testers to work on In self-driving vehicles, voice recognition, machine vision,
more complex features and new automated tests. healthcare, finance, and now test automation, hardly a day
ii. Beyond Manual Testing's Restrictions goes by without someone or an article announcing some
It is almost hard for even the largest software form of AI progress [17].
development/quality assurance organizations to conduct V. ECONOMIC BENEFITS
controlled web application testing involving 1000+ users. As AI advance at a fast rate in the US, the advantages
One may mimic tens, hundreds, or thousands of virtual to the economy will only grow. To address many technical
users using automated testing, and these people can then be issues in the fields of healthcare, driverless vehicles, search
combined with a web-based application, program, or engines, predictive modeling, and a lot more testing,
network [13]. companies like Apple have begun spending more on AI. It
iii. Benefits for developers and testers. has an impact on all businesses, large and small. By 2030,
The developers can detect issues faster by using shared AI is anticipated to boost the world economy by a
automated tests before submitting them to the QA team. whopping $15.7 trillion. The application of artificial
Tests may be performed periodically anytime the source intelligence (AI) in software development will be one of the
code is modified, checked in, and failures can be reported fastest-rising technologies in the industry in the next years.
to the team or the developer. These kinds of features Quality assurance will benefit from AI-based technologies
provide developers more confidence and save them time at by automating manual work and speeding up sprints within
the same time. the SDLC [17,18]. Software quality assurance. One of the
iv. Increasing the total number of tests main goals of the US software business is to reduce
conducted software development costs while also enhancing the
quality of the software. Though computerized economies
It's possible to do more tests with more depth and
are becoming more complicated, so is the underlying
breadth thanks to automated software testing. Automated
software required to sustain them. Software applications
software testing may examine the contents of memory and
are now measured in terms of millions of lines of code
files, as well as internal program modes and datasets, to
rather than tens of thousands of lines of code, as was
evaluate whether or not the software is working as it should
formerly the case. Concerns about software quality have
under certain conditions. Overall, software test automation
grown as software complexity has increased and the typical
allows for the execution of over 1000 distinct test cases in
market living standards of many software products have
a single test run, providing coverage that is not feasible
decreased. To conduct a thorough investigation, two
with human software testing [14].
industrial groups were chosen: automobile and aerospace
v. Time and money saved equates to a quicker
equipment makers, as well as banking and finance players
time to market and associated digital communications equipment
Manually performing such tests may be time- and producers.
money-consuming when software tests are redone after VI. CONCLUSION
every modification in the source code [15]. However, once
This study looked at how artificial intelligence may be
they've been written, automated tests may be reused
incorporated into the software testing process using test
indefinitely at no extra charge and a significantly faster
automation. Overall, the purpose of this study was to
rate. Software testing may be done in a matter of hours
generate interest in artificial intelligence (AI) as a viable
rather than days, saving time and money.
tool for use in the software testing automation industry. A
IV. FUTURE IN THE U.S Systematic Literature Review (SLR) was carried out to
The automation of software testing is advancing achieve the objective. In the near future, even "Continuous
rapidly in the United States. The software industry's long- Testing" would be unable to keep up with the smaller
term prospects. The usage of tools to assist with testing will delivery cycle durations, increased technological
JETIR1912176 Electronic
Journal copy Technologies
of Emerging available at: https://ssrn.com/abstract=4004324
and Innovative Research (JETIR) www.jetir.org 1331
 © 2019 JETIR December 2019, Volume 6, Issue 12 www.jetir.org (ISSN-2349-5162)
complexity, and growing rates of change. We are rapidly
nearing this point. For robots, the Internet of Things, and
other cutting-edge technologies, the testing evolution must
be carried out to ensure the efficiency necessary. While
working on the IoT and practically driving "self-driving"
automobiles, we need to learn how to work smarter, not
harder, to ensure quality in an age where software processes
an unthinkable number of data points in real-time.
Although artificial intelligence (AI) continues to advance,
it is clear that simulating the human brain is a difficult
undertaking. It's important to remember that applications
are used by people, and the technical improvements being
developed consider that. This helps to guarantee a high-
quality product.

REFERENCES
1. S. Amaricai and R. Constantinescu, "Designing a Software
Test Automation Framework", Informatica Economica, vol.
18, no. 12014, pp. 152-161, 2014.
2. D. Banerjee and K. Yu, "3D Face Authentication Software
Test Automation", IEEE Access, vol. 8, pp. 46546-46558,
2020.
3. L. Damm and L. Lundberg, "Results from introducing
component-level test automation and Test-Driven
Development", Journal of Systems and Software, vol. 79, no.
7, pp. 1001-1014, 2006.
4. C. Jordan, F. Maurer, S. Lowenberg and J. Provost,
"Framework for Flexible, Adaptive Support of Test
Management by Means of Software Agents", IEEE Robotics
and Automation Letters, vol. 4, no. 3, pp. 2754-2761, 2019.
5. J. Kasurinen, O. Taipale and K. Smolander, "Software Test
Automation in Practice: Empirical Observations", Advances in
Software Engineering, vol. 2010, pp. 1-18, 2010.
6. D. Kumar and K. Mishra, "The Impacts of Test Automation on
Software's Cost, Quality and Time to Market", Procedia
Computer Science, vol. 79, pp. 8-15, 2016.
7. C. Rankin, "The Software Testing Automation
Framework", IBM Systems Journal, vol. 41, no. 1, pp. 126-
139, 2002.
8. A. Bertolino, H. Foster, J. Jenny Li and H. Zhu, "Special
section on automation of software test", Journal of Systems
and Software, vol. 86, no. 8, p. 1977, 2013.
9. V. Garousi and F. Elberzhager, "Test Automation: Not Just for
Test Execution", IEEE Software, vol. 34, no. 2, pp. 90-96,
2017.
10. B. Green, "Software test automation", ACM SIGSOFT
Software Engineering Notes, vol. 25, no. 3, pp. 66-66, 2000.
11. J. Hollingum, "Reflex puts its software capabilities to the
test", Assembly Automation, vol. 7, no. 1, pp. 21-23, 1987.
12. D. Hutton, "Software Test Automation: Effective Use of Test
Execution Tools20005Mark Fewster and Dorothy Graham.
Software Test Automation: Effective Use of Test Execution
Tools. Reading, MA: Addison Wesley Longman 1999. 592
pp., ISBN: ISBN 0‐201‐33140‐3 $39.95 (£25
approx.)", Kybernetes, vol. 29, no. 3, pp. 392-398, 2000.
13. D. O'Shea, F. Ortin and K. Geary, "A virtualized test
automation framework: A DellEMC case study of test
automation practice", Software: Practice and Experience, vol.
49, no. 2, pp. 329-337, 2018.
14. J. Park and J. Choi, "Test Framework Development for
Software Reliability Test using Formal Method", International
Journal of Software Engineering and Its Applications, vol. 10,
no. 8, pp. 151-158, 2016.
15. M. Polo, P. Reales, M. Piattini and C. Ebert, "Test
Automation", IEEE Software, vol. 30, no. 1, pp. 84-89, 2013.
16. K. Wiklund, S. Eldh, D. Sundmark and K. Lundqvist,
"Impediments for software test automation: A systematic
literature review", Software Testing, Verification and
Reliability, vol. 27, no. 8, p. e1639, 2017.
17. S. Jan, A. Javed and M. Majeed, "Quality Category Matrix to
Ensure the Quality of Software Product", International
Journal of Computer and Communication Engineering, pp.
175-178, 2012.
18. B. Kitchenham, "Towards a constructive quality model. Part 1:
Software quality modelling, measurement and
prediction", Software Engineering Journal, vol. 2, no. 4, p.
105, 1987.
JETIR1912176 Electronic
Journal copy Technologies
of Emerging available at: https://ssrn.com/abstract=4004324
and Innovative Research (JETIR) www.jetir.org 1332