Case study: Transition from Manual to Automated Testing

Introduction: ​As applications and software grow larger and more complex, software testing has
been put in a core position to make sure that the defects or bugs will not break the software.
However, it has been generally regarded that the traditional manual software testing simply
cannot satisfy the requirement of testing more often and earlier also with better code
coverage. To support testers in testing the software, different approaches have been proposed
for producing good test coverage. As a result, a wide range of techniques for automated tests
has been explored to complement manual testing and to have extensive test coverage.

The Challenge: ​With over 200 manual test cases to execute for each regression cycle, the
quality assurance process for the client’s application was quickly becoming very challenging to
maintain and execution time was too long. The application was constantly growing with more
added features besides a convoluted business layer. Due to repeated tests performed on the
application with confined test coverage, a lot of bugs were missed out compromising the
overall quality of the product. Thus, there was a need to discover a framework which can
address this situation. The framework should be able to reduce testing times, increase the
number of test cases broaden the test coverage and improve the overall quality of the system,
without presenting extra testing time.

The Approach: In light of this experience, a comprehensive and smart test automation
framework is designed from the scratch which will suffice all the testing requirements, increase
the testing coverage and significantly improve the quality of the software. Due to the complex
nature of the application, a detailed analysis and market research was conducted that identified
the tools and technology needed to develop a sustainable test automation engine to mitigate
the problem caused by the manual testing process.

● The Automated scripts are written in Java using selenium as a testing suite.
● TestNG is used as a test automation framework.
● Extent 3 to generate dynamic reports.
● Apache Ant is used to build code.
● Azure Repos for version control.

The Implementation: ​The chosen tools and technology are put together to form an extensible,
robust and easy to maintain test automation framework to drive the entire testing process.
Below postulates describe the step by step implementation of the automation engine and the
details of the solution.

1
 Process Solution Provided

Inspecting the test cases to Test cases were old and many new changes occurred in the application, due to
evaluate the test coverage. which the tests were less effective. The test cases were rewritten with the
latest logic and the test scenarios which can be automated are identified.

Converting the test cases The underlying logic of the test cases was translated to test scripts using Java
into automation scripts. as a scripting language to perform the automation​. ​The scripts were designed
to be easily manageable, highly extensible and having maximum test
coverage.

Using Selenium, each page is divided into chunks of methods called as Page
Object. These methods are called into another larger methods to drive the
Developing the test scripts. test logic. The test scripts receive the test data from an excel sheet which acts
as a test data reservoir. Again, each component of the page receives the test
data from the same excel. In this way, the same test scripts can be executed
with a different set of data.

The test scripts are smart enough to get executed in any environment with
minimal to none change. For instance, the test scripts are capable of executing
in any browser version. This is achieved by having a robust framework to drive
Designing the Framework. the entire test suites. Technically, the test scripts are overlaid by the page
object framework. Alongside a mechanism to generate a dynamic test
summary report. The whole process is built and executed using Apache ant.

Once the test scripts are run, the tests are either classified as Passed or Failed
Determining the fault. depending on the checks on the functionality. TestNg regulates the entire
process and provides powerful assertions to validate a scenario and Java to
bind the test script with Selenium.

After executing the test suite, it is important to have an interactive and

summarized reporting feature in the framework. This is achieved by using
The test summary report. Extent reporting which dynamically generates reports in HTML format. In
particular, the status of each test step is portrayed along with corresponding
snapshots.

Building automated testing software is required to manage and organize the

execution flow. The building incorporates compiling, linking and packaging the
Building the automated code into a usable or executable form. During the build process, instructions
tests software code. are provided to compile the code, execute the test, generate the test
summary report and send email to the stakeholders about the test status.

Finally, after executing the test suite, the reports are published as email
communication to the stakeholder which emphasizes the test status either
Publishing the results. Passed or Failed and the bugs identified during execution. Using this report
the bugs are fixed and the test cases are updated to ensure these bugs will
not come across in future releases.

2
The Results: ​Importantly, with the automated tests in use, there is a significant decrease in the
testing times with larger test coverage and great improvement in overall system performance.
Furthermore, the automated test scripts are easy to maintain and very versatile to be
extendable for any test data. Also, various test suites such as smoke, sanity and UAT are derived
from the master suite to maximum test coverage. Moreover, Parallel testing has been achieved
which has further reduced test execution times.

Below are some graphs to illustrate the achievements with the automated test framework.

In conclusion, this test automation architecture not only the testing process evolved to a
greater extent, but also the quality of the product has increased immensely. The scripts
generated out of this framework are extended to perform other forms of testing such as
Security testing, Compatibility testing, Responsive testing and Load testing for detailed tests. A
well-structured test summary reports are emailed to the stockholders, which will describe the
test status along with snapshots.