Three-module framework for automated software

testing
Ilona Kozak Andrii Berko
2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT) | 979-8-3503-3431-9/22/$31.00 ©2022 IEEE | DOI: 10.1109/CSIT56902.2022.10000806

Information systems and neworks department Information systems and neworks department
Lviv Polytechnic National University Lviv Polytechnic National University
Lviv, Ukraine Lviv, Ukraine
[email protected] [email protected], 0000-0003-2892-9519

Abstract–Different types of automated test frameworks are or action words for each function or method that will
used for different purposes, i.e., to investigate features of the be executed;
interface, functionality, etc. Each of these frameworks has
different approaches to the testing process, and therefore, own • hybrid Driven Testing Framework is a combination
advantages and disadvantages. Repositories seems to be helpful of two or more frameworks, mentioned above. It takes
tool while developing various types of software testing systems. advantage of other frameworks to create the specific
Such software testing system should meet business test environment;
requirements, as well as user, functional, and non-functional
requirements. Such requirements can be grouped into • behavior Driven Development Testing Framework,
technological, financial, ergonomic, time, etc. The designed and as a goal has a creation of a framework that allows an
developed three-module framework for automated software active participation of business analysts, developers,
testing was uploaded to the repository and a sub-project was testers, etc. This requires increased cooperation efforts
prepared. between developers and test teams.
1.2. Functional and non-functional testing types
Keywords—automated test framework, test approaches, testing
The main goal of functional testing is to ensure that
tools, remote storage. software performs according to specifications and meets user
expectations.
INTRODUCTION
A quality assurance strategy helps a company create 1.2.1. API testing
products and services that meet customer needs, expectations, Application Programming Interfaces (API) testing is a
and requirements. This ensures high-quality product type of software testing when testing cannot be done on an
offerings that build trust and loyalty among customers [1, 2]. interface because there is no one. The API accepts requests
The standards and procedures defined by the quality from the device and receives a response from the server. Due
assurance program help prevent product defects before they to the nature of API testing, it cannot be tested manually, and
occur. specific API testing tools must be selected [6].
To prevent business losses, such as finances and time, in 1.2.2. UI testing
software design is important to guarantee the quality of the The user interface (UI, also known as the graphical user
software. The thoughtful design of such tools is a relevant interface) is the most important component of any mobile or
task, both theoretical and practical. Automated frameworks web application. Interface interaction is the way users access
were developed and built to fulfill the needs of the software the core functions of any application. The UI testing covers
developers in electronic circuits assembling, cloud wide range of verifications, from the smallest piece of logic,
computing, IoT applications, etc. UI workflows, to navigation, transitions, calculations,
checking all buttons, and more. The process involves testing
1. STATE OF ART all visual indicators and graphical icons, including menus,
1.1 Types of automated test frameworks switches, text boxes, flags, toolbars, colors, fonts, etc. [7, 8].
Universal design demands should also be evaluated [9].
There are many test frameworks with a different
combination of modules, test layers, automated test tools, and User interface testing is performed manually or with an
libraries [3]. There are also several types of automated test automated testing tool. Regardless of the method used, the
frameworks [4, 5]: goal is for all interface elements to conform to the requested
specifications.
• modular Testing Framework, when testers create
modules of test scenarios, dividing the complete tested Key aspects examined in user interface testing include:
application into smaller independent tests; • visual design;
• data-driven Framework is a data-driven test • functionality;
automation structure, focused on separating the logic
of test scripts and test data. The test data set is stored • usability;
in external files or resources such as MS Excel tables,
MS Access tables, SQL database, XML files, etc.; • productivity;

• keyword Driven Testing Framework is also known • compliance.

as tabular testing, or action-based testing. In keyword- The scope of UI testing can be very broad.
based testing, we use a table format to define keywords
454

979-8-3503-3431-9/22/$31.00 ©2022 IEEE

Authorized licensed use limited to: Universitas Indonesia. Downloaded on October 25,2023 at 17:09:19 UTC from IEEE Xplore. Restrictions apply.
 To achieve the goal of the research and develop the
automated test framework, the browser automation library
1.2.3. Performance testing Selenium, testing service RestAssured, and Gatling testing
Performance testing is the evaluation of the system tools should be used.
performance in terms of sensitivity and stability under a
certain load. Performance tests are usually performed to test As the goal is to developed software continuous detection
the speed, reliability, and size of an application. The process of software defects, a multi-module system will be developed.
includes such performance indicators as: The first module aims to solve the problem of testing the user
interface. These classes are subprocesses of three main layers,
• browser, page, and network response time; i.e., Core Layer, Business Layer, and Test Layer [6]. The
• server request processing time; second module includes main classes for the correct
implementation of the API testing framework. The third
• acceptable simultaneous volumes of users; module includes main classes for the correct implementation
of the performance test framework.
• processor memory consumption;
While developing a system, it is necessary to choose
• the number and type of errors that may occur in the resources properly. The necessary resources, their volume,
program. and quantitative indicators for working with the product are
Performance testing ensures that the software meets listed at Fig. 1.
expected service levels and provides positive user experience.
The goal of the research is to ensure software quality by
development of an automated test framework to be a core
layer repository, available for extension, and be a library for
the test framework. The developed framework will use a set
of tools and libraries for automated testing, and implementing
such a framework defines a single format for testing. Also, it
should include a report of results generation, conveniently
defined test data (constants, datasets, etc.), and combine
libraries specific for software testing.
2. MAVEN REPOSITORY
As a tool to build and manage Java-based project, the
Maven was chosen. Its developers state that the Maven tool
helps with the comprehension of any Java-based project [10].
Maven allows to make the build process easy, it provides a
uniform build system and a quality project information,
encouraging better development practices [10].
In Maven terminology, a repository is a “directory that Fig. 1. Components characteristics.
stores all project and library jars, plugins, or any other project-
specific artifacts that can easily be used by Maven” [11]. The developed system should meet functional and non-
There are three types of Maven repositories: local, central, functional requirements FURPS+, such as functionality,
and remote. usability, reliability, performance, and operational suitability
[12, 13]. Some additional characteristics are relevant, too:
A local Maven repository is a folder located on a transaction load, processing complexity, reuse, ease of
computer. It is created when any Maven command runs for the installation, ease of administration, distributed data
first time. The local Maven repository stores all project processing.
dependencies (library jars, plugin jars, etc.).
IntelliJ IDEA was chosen as the environment for
The central Maven repository is a repository provided by developing the automated test framework. Every component
the Maven community. It contains many commonly used of IntelliJ IDEA is designed to maximize development
libraries. To browse the contents of the Maven central productivity. A smart code editor combined with an
repository, the Maven community has provided a URL ergonomic design makes development efficient. After
(https://search.maven.org/#browse). Using this library, the indexing the source code, IntelliJ IDEA provides many
developer can search for all available libraries in the central opportunities for fast and efficient development, such as smart
repository. autocompletion, real-time code analysis, and reliable
Sometimes Maven also doesn't find the mentioned refactoring [14-19].
dependency in the central repository, and an error message The UI-tier and API-tier code is written in Java 11. Java
outputs to the console. To prevent this situation, Maven offers is well suited for writing automated tests. It is not necessary
the concept of a remote repository, which is a developer's to write new functions for it, as they are already designed as
repository containing the required libraries or other project libraries for testing. And for the performance tier, the code is
jars. written in Scala. Based on Scala, Gatling is used here, which
can implement logic using functions.
3. ANALYSIS OF PROJECT REQUIREMENTS

455

Authorized licensed use limited to: Universitas Indonesia. Downloaded on October 25,2023 at 17:09:19 UTC from IEEE Xplore. Restrictions apply.
 Selenium is a comprehensive open-source project for development. It clearly emphasizes quality control of the
several tools and libraries, aimed at supporting web browser development process [22]. The type of strategy for developing
automation [20]. Selenium provides a playback tool to create this product is customer focus and speed of execution.
functional tests without having to learn the test scripting
language (Selenium IDE). It also provides a test language 5. THREE-MODULE TESTING SYSTEM
(Selenese) for writing tests in several popular programming 5.1. Classes of objects to perform main system functions
languages. The tests can then be run in most modern web
browsers. At the beginning of the automated testing software
development, the common classes and often used functions
RestAssured is a Java library for testing RESTful APIs. that were analyzed. Developing the first module, which solves
The code written using this library has a simple and clear the problem of testing the user interface, it is necessary to
syntax. In the test, you can request to the API in one line of define and specify the classes of objects that implement the
code and use the DSL syntax to check the result. main processes of the system functioning. As mentioned
above, the classes are subprocesses of three main layers —
4. EXPECTED EFFECTS OF SYSTEM IMPLEMENTATION Core Layer, Business Layer, and Test Layer.
To evaluate the developed framework, we used some
The API testing scheme is in Fig. 2, the scheme of
criteria, and the framework was evaluated by effect (namely
Performance module is at Fig. 3.
technological, time, financial, economic, ergonomic), and by
goals. The unit of measurement is a quantitative assessment
(QA, in the range from 0 to 5). It is convenient to evaluate the
effects on a scale from 0 to 5 points, for example, the
flexibility increasing, when 0 means no effect on operation of
the system, and 5 means the flexibility of the query system and
the construction of arbitrary reports affects the data model,
which is changed by the user interactively (Table 1).

TABLE I. THE EFFECTS OF SYSTEM IMPLEMENTATION

Goal Effect QA
Main goal
Designing an automated test framework
that will test the software using Selenium Technological 5
API, RestAssured, and Gatling tools
First-level goals (Operational)
Data collecting Technological 5 Fig. 2. UML-diagram of classes of objects of the API module
System design Technological 5
Implementation of functions Technological 5
Second-level goals (Tactical)
Unification of information protocols for
data exchange between parts of the Technological 5
program
Ensuring interoperability between
Time 5
database and product
Ensuring information transfer and Technological
3
management
Ensuring information security Technological 2
Reducing capital costs for technical
Financial 4
support
Providing free content Financial 4
Improving the reliability of work Technological 2
Reduction of technical staff Financial 3 Fig. 3. UML-diagram of classes of objects of the performance module
Increase efficiency Technological 5
Increasing the performance of end users Ergonomic 2 5.2. Uploading a project to a storage
Increased usability and flexibility Technological 3
After launching the project using the Maven mvn install
Ability to cope with the growing amount
of work
Time 4 command, it is necessary to go through the procedure of
downloading the finished artifact to the remote Maven
Central Repository [11].
The greatest effect of the implementation of the developed
three-module framework is technological. The financial and For hosting, Sonatype OSSRH (OSS Repository Hosting)
time effects are also significant, the ergonomic effect gives an [23] was chosen, which uses the Sonatype Nexus Repository
increase in the performance of end users only. Manager to provide a repository hosting service for open-
source project binaries.
The software product was developed using the Scrum
development methodology, which has already proven itself in To create a repository on OSSRH (OSS Repository
the IT industry as a flexible methodology that provides the Hosting), it is necessary to create a request. Next, we wait for
ability to quickly adapt to changing requirements [21]. Scrum the result of the project verification, obtain permission to use
is a project management approach for agile software Group, ID and check the artifacts available at the link in the
comments.
456

Authorized licensed use limited to: Universitas Indonesia. Downloaded on October 25,2023 at 17:09:19 UTC from IEEE Xplore. Restrictions apply.
 After that, you need to work on the file pom.xml, after Access, 7, art. no. 8572685, pp. 1888 - 1896. 2019. DOI:
completing the necessary steps, presented on Central 10.1109/ACCESS.2018.2886272.
Sonatype [22]. [5] C. Merina, N. Anggraini, N. Hakiem, “A comparative analysis of test
automation frameworks performance for functional testing in android-
Now we check the performance of the project by running based applications using the distance to the ideal alternative method,”
mvn install. In case of a successful result, run mvn clean Proceedings of the 3rd International Conference on Informatics and
Computing, ICIC 2018, art. no. 8780548. 2018. DOI:
deploy. If this command is successfully executed, an 10.1109/IAC.2018.8780548
appropriate message will be received (see Fig. 4). [6] A. Ehsan, M.A.M.E. Abuhaliqa, C. Catal, D. Mishra, “RESTful API
Testing Methodologies: Rationale, Challenges, and Solution
Directions,” Applied Sciences (Switzerland), 12 (9), art. no. 4369.
2022. DOI: 10.3390/app12094369.
[7] G.D. Everatt, R. McLeod Jr., Software Testing: Testing Across the
Entire Software Development Life Cycle. John Wiley & Sons., 2007,
p. 99- 121.
[8] V. Mukhin, Y. Kornaga, Y. Bazaka, I. Krylov, A. Barabash,
A. Yakovleva, O. Mukhin, “The Testing Mechanism for Software and
Services Based on Mike Cohn's Testing Pyramid Modification,”
Proceedings of the 11th IEEE International Conference on Intelligent
Fig. 4. Successful unloading of the project. Data Acquisition and Advanced Computing Systems: Technology and
Applications, IDAACS 2021, 1, pp. 589 - 595. 2021. DOI:
After unloading, the project becomes available for use on 10.1109/IDAACS53288.2021.9660999
the Sonatype host. The finished artifact can be used already [9] V. Pasichnyk, T. Shestakevych, N. Kunanets, V. Andrunyk, “Analysis
in the project. of completeness, diversity and ergonomics of information online
resources of diagnostic and correction facilities in Ukraine,” CEUR
Since this is only a snapshot that is not versioned, one Workshop Proceedings, 2105, pp. 193 – 208. 2018.
must also run the command mvn versions:set - [10] Apache Maven Project. Available at:
DnewVersion=1.0.0, thereby installing the release version. https://maven.apache.org/index.html
Next, the mvn deploy -P release command, which will start [11] Maven - Repositories. Available at:
https://www.tutorialspoint.com/maven/maven_repositories.htm.
the process of uploading the project to Maven Central
[12] A. Berko, V. Alieksieiev, V. Lytvyn, “Knowledge-based big data
Repository. cleanup method,” CEUR Workshop Proceedings, 2386,
As a result, the developed software was successfully pp. 96 – 106. 2019.
tested, which proves the that a three-module unified test [13] I. Pelekh, A. Berko, V. Andrunyk, L. Chyrun, I. Dyyak, “Design of a
system for dynamic integration of weakly structured data based on
framework can be used as a library from the Maven Central mash-up technology,” Proceedings of the 2020 IEEE 3rd International
Repository. The developed software uses OSS Repository Conference on Data Stream Mining and Processing, DSMP 2020, art.
Hosting. The import of the project is possible in the network no. 9204160, pp. 420 - 425. 2020. DOI:
only, developed dependencies reduce repetition and unify 10.1109/DSMP47368.2020.9204160.
frameworks, dependencies use gpg encryption key. [14] N. Melnykova, N. Shakhovska, V. Melnykov, M. Zakharchuk,
Versioning ensures reliability, and with proper versioning, the M. Logoyda, V. Mahlovanyi, “The Applying Processing Intelligence
Methods for Classify Persons in Identify Personalized Medication
continuous integration is possible. Decisions,” 2020 10th International Conference on Advanced
Computer Information Technologies, ACIT 2020 - Proceedings, art.
CONCLUSIONS no. 9208822, pp. 422 - 425. 2020. DOI:
The variety of automated test frameworks serves to 10.1109/ACIT49673.2020.9208822.
improving the quality of the software on different levels, such [15] P. Napieralski, M. Kowalczyk, "Detection of vertical disparity in three-
dimensional visualizations", Open Physics, 2017, 15.1, pp. 1028-1033.
as API, UI, and performance level. To create a complex
software for such automated testing, Maven repository was [16] A. Wojciechowski, K. Fornalczyk, “Exponentially smoothed
interactive gaze tracking method”, Lecture Notes in Computer Science
used as a tool of Java-based project comprehension. The (including subseries Lecture Notes in Artificial Intelligence and
resources for the OS, RAM, programming languages etc. were Lecture Notes in Bioinformatics), 2014, 8671, pp. 645-652.
defined. The designed software will work using the Selenium [17] I. Tsmots, V. Teslyuk, A. Batyuk, V. Khavalko, A. Mladenow,
API, RestAssured and Gatling tools. “Information-analytical support to medical industry,” CEUR
Workshop Proceedings, 2488, pp. 246 – 257. 2019.
The target audience and main users of the developed three- [18] O. Matviykiv, N. Bokla, T. Klymkovych, U. Marikutsa, I. Farmaha,
module framework for automated software testing are the M. Lobur, M. Banas, “Lab-chip diagnostic device for the rainwater
engineers who work on a software product and include testing monitoring system using wireless sensors network,” Proceedings of the
of this product in the development process. 26th International Conference "Mixed Design of Integrated Circuits
and Systems", MIXDES 2019, art. no. 8787185, pp. 241 - 245. 2019.
REFERENCES [19] I. Zheliznyak, Z. Rybchak, I. Zavuschak, “Analysis of clustering
algorithms,” Advances in Intelligent Systems and Computing, 512, pp.
[1] T. Borovska, D. Grishin, I. Kolesnik, V. Severilov, I. Stanislavsky, 305 – 314. 2017. DOI: 10.1007/978-3-319-45991-2_21.
T. Shestakevych, “Research and development of models and program
for optimal product line control “Advances in Intelligent Systems and [20] Automation in Selenium: Page Object Model and Page Factory.
Computing, 1080 AISC, pp. 186 - 201, 2020. Available at: https://www.toptal.com/selenium/test-automation-in-
[2] S. Korobka, M. Babych, R. Krygul, A. Zdobytskyj, “Results of research selenium-using-page-object-model-and-page-factory.
into technological process of fruit drying in the solar dryer”, Eastern- [21] K. Schwaber, Agile Project Management with Scrum, Microsoft Press,
European Journal of Enterprise Technologies, 2018, 1 (8-91), pp. 64 - January 2004, 163 p. ISBN 0-7356-1993-X11.
73. DOI: 10.15587/1729-4061.2018.122816. [22] OSS Repository Hosting. Available at:
[3] F. Xue, J. Wu, T. Zhang,” Learning-Replay Based Automated Robotic https://central.sonatype.org/publish/publish-guide/.
Testing for Mobile App,” Mobile Information Systems, 2022, art. no.
1084602. 2022. DOI: 10.1155/2022/1084602.
[4] D. Banerjee, K. Yu, “Integrated Test Automation for Evaluating a
Motion-Based Image Capture System Using a Robotic Arm,” IEEE

457

Authorized licensed use limited to: Universitas Indonesia. Downloaded on October 25,2023 at 17:09:19 UTC from IEEE Xplore. Restrictions apply.