Lecture 8: Manual and Automation Testing

Automation Testing

Here, we are going to learn about the following related topics of automation testing:

o Introduction to Automation testing

o Why do we need to perform automation testing?
o Different approaches used in automation testing
o Automation testing process
o What are different challenges faced during the automation testing process?
o Automation testing tools
o Benefits and Drawbacks of automation testing.

Introduction to Automation Testing

Another software testing method is automation testing, which is used some specific tools to execute the test scripts without any human
interference. It is the most acceptable way to enhance the efficiency, productivity, and test coverage of Software testing.

With the help of an automation testing tool, we can easily approach the test data, handle the test implementation, and compares the
actual output against the expected outcome.

65.2Mion Handling in Java - Javatpoint

In automation testing, the test automation engineer will write the test script or use the automation testing tools to execute the
application. On the other hand, in manual testing, the test engineer will write the test cases and implement the software on the basis of
written test cases.

In test automation, the test engineer can execute repetitive tasks and other related tasks. In manual testing, it is a tedious process to
implement the repetitive take again and again.

In other words, we can say that the main concentration of test Automation is to change the manual human activity with systems or
devices.

The automation testing process is a time-saving process as it spends less time in exploratory testing and more time in keeping the test
scripts whereas enhancing the complete test coverage.

Why do we need to perform automation testing?

o In software testing, automation testing is required to test the application because it offers us a better application with less effort
and time.
o Some organizations still perform only manual testing to test the application as those companies are not fully aware of the
automation testing process.
o But now, they are aware of automated testing and executing the test automation procedure in their application development
process.
o To implement the automation testing, we required pretty a considerable investment of resources and money.

The execution of automation testing provides us various advantages, which are as discussed below:

o Reusability
o Consistency
o Running tests anytime (24/7)
o Early Bug detection
o Less Human Resources
1. Reusability

We can re-use the test scripts in automation testing, and we don't need to write the new test scripts again and again. And, we can also
re-create the steps which are detailed as the earlier ones.

2. Consistency

As compared to manual testing, automation testing is more consistent and way faster than executing the regular monotonous tests that
cannot be missed but may cause faults when tested manually.

3. Running Tests 24/7

In automation testing, we can start the testing process from anywhere in the world and anytime we want to. And even we can do that
remotely if we don't have many approaches or the option to purchase them.

4. Early Bug Detection

We can easily detect the critical bugs in the software development process's initial phases by executing automation testing. It also helps
us spend fewer working hours to fix these problems and reduce costs

5. Less Human Resources

To implement the automation test script, we need a test automation engineer who can write the test scripts to automate our tests, rather
than having several people who are repeatedly performing the tedious manual tests.

Automation Testing Methodologies

Automation testing contains the following three different methodologies and approaches, which will help the test engineer to enhance
the software product's quality.

o GUI Testing
o Code-Driven
o Test Automation Framework

Now, let's understand the different approaches of automation testing one by one:
1. GUI (Graphical user interface) Testing

In this approach, we can implement that software or an application, which contains GUIs. So, that the automation test engineers can
record user actions and evaluate them many times.

We know that the Test cases can be written in several programming languages like JAVA, C#, Python, Perl, etc.

2. Code-Driven

The code-driven technique is the subsequent methodology used in automation testing. In this method, the test engineer will mainly
concentrate on test case execution in order to identify whether the several parts of code are performing according to the given
requirement or not.

Hence, it is very a commonly used method in agile software development.

3. Test Automation Framework

Another approach in automation testing is test automation framework. The test automation framework is a set of rules used to
generate valuable results of the automated testing activity.

Similarly, it brings together test data sources, function libraries, object details, and other reusable modules.

Automation Testing Process

The automation testing process is a systematic approach to organize and execute testing activities in a manner that provides maximum
test coverage with limited resources. The structure of the test involves a multi-step process that supports the required, detailed and
inter-related activities to perform the task.
The automation testing process completed in the following steps:

Step1: Decision to Automation Testing

It is the first phase of Automation Test Life-cycle Methodology (ATLM). At this phase, the main focus of the testing team is to
manage expectations from the test and find out the potential benefits if applying the automated testing correctly.

On adopting an automated testing suit, organizations have to face many issues, some are listed below:

o Testing tool experts are required for automation testing, so the first issue, to appoint a testing equipment specialist.
o The second issue is, choose the exact tool for the testing of a particular function.
o The issue of design and development standards in the implementation of an automated testing process.
o Analysis of various automated testing tools to choose the best tool for automation testing.
o The issue of money and time occurs as the consumption of money and time is high in the beginning of the testing.

Step2: Test Tool Selection

Test Tool Selection represents the second phase of the Automation Test Life-cycle Methodology (ATLM). This phase guides the
tester in the evaluation and selection of the testing tool.
Since the testing tool supports almost all testing requirements, the tester still needs to review the system engineering environment and
other organizational needs and then make a list of evaluation parameters of the tools. Test engineers evaluate the equipment based on
the provided sample criteria.

Step3: Scope Introduction

This phase represents the third phase of Automation Test Life-cycle Methodology (ATLM). The scope of automation includes the
testing area of the application. The determination of scope is based on the following points:

o Common functionalities of the software application that are held by every software application.
o Automation test sets the reusable range of business components.
o Automation Testing decides the extent of reusability of the business components.
o An application should have business-specific features and must be technically feasible.
o Automation testing provides the repetition of test cases in the case of cross-browser testing.

This phase ensures the overall testing strategy that should be well managed and modified if required. In order to ensure the availability
of skills, testing skills of a particular member and whole team are analyzed against the required specific skills for a particular software
application.

Step4: Test Planning and Development

Test planning and development is the fourth and most important phase of Automation Test Life -cycle Methodology (ATLM) because
all the testing strategies are defined here. Planning of long -lead test activities, the creation of standards and guidelines, an arrangement
of the required combination of hardware, software and network to create a test environment, defect tracking procedure, guidelines to
control test configuration and environment all are identified in this phase. Tester determines estimated effort and cost for the entire
project. Test strategy and effort estimation documents are the deliverables provided by this phase. Test case execution can be started
after the successful completion of test planning.

Step5: Test Case Execution

Test case Execution is the sixth phase of Automation Test Life -cycle Methodology (ATLM). It takes place after the successful
completion of test planning. At this stage, the testing team defines test design and development. Now, test cases can be executed under
product testing. In this phase, the testing team starts case development and execution activity by using automated tools. The prepared
test cases are reviewed by peer members of the testing team or quality assurance leaders.
During the execution of test procedures, the testing team directed to comply with the execution schedule. Execution phase implements
the strategies such as integration, acceptance and unit testing that have defined in the test plan previously.

Step6: Review and Assessment

Review and assessment is the sixth and final stage of the automated testing life cycle but the activities of this phase are conducted
throughout the whole life cycle to maintain continuous quality improvement. The improvement process is done via the evaluation of
matrices, review and assessment of the activities.

During the review, the examiner concentrates whether the particular metric satisfies the acceptance criteria or not, if yes, then it is ready
to use in software production. It is comprehensive as test cases cover each feature of the application.

The test team performs its own survey to inquire about the potential value of the process; if the potential benefit is less than sufficient,
the testing team can change the testing tool. The team also provides a sample survey form to ask for feedback from the end user about
the attributes and management of the software product.

What are different challenges faced during the Automation testing process, and how to overcome them?

The automation testing process carries about large numbers of advantages for an organization. Usually, with automated testing, the
testing team of the specified method of software validation can accomplish the amplified test coverage.

Though we may face various challenges during the automation testing process; therefore, we need a thorough follow-up process to
reach successful test automation execution.

Some of the common challenges are as discuss below, which we might encounter throughout the test automation process of an
application:
1. Lack of Highly Skilled Test Automation Experts

o The most fundamental reason behind lacking an automation testing process is the absence of highly skilled automation test
engineers.
o To design and maintain the test automation framework and test scripts, the test automation engineer needs to be skilled and
have details knowledge about automation technical skills.
o To create a test script correctly and sustain them, the team should implement the test scripts successfully to check the application
performance and fix the technical issues.
o To achieve the operative automation aspects, the automation test engineers should have strong knowledge about test automation
frameworks because it would be easy for them to design and execute the test scripts.

2. Appropriate Selection of Right Test Automation Tool

 o The next challenge, we mostly face while performing the automation testing is selecting thecorrect automation testing tool.
o A suitable detailed assessment of the application under test (AUT) is mandatory.
o With the accessibility of various automation testing tools for both open-source and paid tools. And it is essential to check
appropriately before accepting a tool.

3. Challenges with Scaling Test environments

o It is a crucial challenge in present day as testing teams that don't providing test environments in the cloud.
o It is essential to quickly deliver the changed test environments that automated testing requires, measure them up, execute the
tests, to make sure that the success while testing in the cloud.
o Generally, the internal teams have a partial number of test environments which they can use to execute the fewer tests and
deployed them at any given time.
o As a result, testing takes a much longer time. To overcome this, it is essential to move test environments to the cloud to scale
the test automation, and it will also reduce testing teams operating costs.

4. Including High Initial Investment Costs

o Generally, the execution of the automation testing process in the initial phase is a bit expensive because it is necessary to
evaluate, design, and build an automation testing framework, libraries, or reusable functions over a detailed AUT analysis.
o Additionally, if an automation test engineer picks the licensed test automation tool, the operating cost should be appraised.
o If the automation test engineer selects the open-source tools, then it should also be significant efforts spent on learning, training,
and maintaining them.
o Later, the selection of automation tools, either open-source or licensed, depends on the AUT priority and availability of cost
facility.

5. Miscommunication and lack of collaboration between Development and testing teams

 o Miscommunication or lack of proper collaboration between the developer and the test engineers' team and operations teams
does not lead us to successful test automation.
o Therefore, it is significant to place all the team (developer, testing, operation) members to categorize the test automation
objectives and set targets.
o To ensure the test automation success, these teams need to spend some time to get an effective communication and to get a
clear understanding of the business needs and project specifications.

Automation Testing Tools

Automation testing tools can describe in two categories, which are as follows:

o Functional Testing Tools

o Non-Functional Testing Tools
Let's understand them one by one in-details:

Functional Automation Testing Tools

The automation test engineer uses the functional automation testing tool to implement the functional test cases. For example, the
Repetitive Regression tests are automated under the function automation testing tools.

This type of tools can be further divided into two different parts, which are as below:

o Commercial Tool
o Open-source Tool
Commercial Functional Testing Tools

Commercial functional testing tools are those testing tools, which are not available freely in the market. These tools are also known
as licensed tools. The licensed tools include various features and approaches as compared to the open-source tools.

Some of the most important commercial tools are as follows:

o QTP[Quick Test Professional]

o Rational Functional Tester [RFT]
o TestComplete
o SoapUI
Let's take a look on the above listed commercial tools:

QTP

o QTP signify as Quick Test Professional, but now it is known as Micro Focus UFT (Unified Functional Testing).
o Mainly, it is used to automate the functional regression test cases of the web-based application or the software.
o In order to test the application, deploy the objects, and for analysis purposes, it is designed on the scripting languages such
as VBScript.
o It follows the concept of keyword-driven testing for defining the test creation and maintenance.
o The new test engineers can easily use the QTP tool because it allows them to develop the test cases right from the application.
Characteristics of QTP

Some of the most standard attributes of QTP are as follows:

o The record and playback feature enhance the credibility of the QTP tool.
o Both technical and non-technical test engineers can operate the QTP tool.
o It supports several software development environments such as SAP, JAVA, Oracle, and so on.
o We can check both web-based and desktop-based applications by using the QTP.

RFT

o RFT stands for Rational Functional Tester, which is used to execute the functional regression test cases.
o It follows the concept of Data-driven testing and GUI testing.
o It builds the automated functional tests by recording an end-user's activities on the system under test and repeating the actions
on request to perform a test.
Features of RFT

The RFT tool contains the following characteristics:

o RFT supports an extensive range of applications and protocols, like HTML, Java, .NET, Windows, Eclipse, SAP, Visual
Basic, PowerBuilder
o It sustains custom controls by proxy SDK like Java or.Net.
o It allows developers to develop the keyword-associated script; hence, it can be re-used again, which enhances the productivity
of the project.

TestComplete

o Another functional automated testing tool is TestComplete, which acquires by SmartBear Software.
o TestComplete has some inbuild capability, which helps the test engineers to develop the automated tests for Microsoft Windows,
Web, Android, and iOS applications.
o In this tool, the tests can be recorded, scripted, or manually created with keyword-driven processes and used for automatic
playback and error logging.
o It includes the three different modules, like Web, Desktop, and Mobile, where all the modules involve the functionality to
develop the automated tests on the particular platform.
Characteristics of TestComplete Tool

Below is the essential characteristic of the TestComplete Tool:

o It is used to test several types of specific applications such as Web, Windows, Android, iOS, .NET, VCL, and Java.
o It involves the issue-tracking templates, which can be used to develop or change items stored in issue-tracking systems.
o It records the key actions essentially to repeat the test and rejects all unnecessary efforts.

SoapUI

o SoapUI is the most extensively used automation tool. It is mainly used to test the web servicesand web APIs of
SOAP and REST interfaces.
o It allows the test engineers to test functional, regression testing, and other testing types on various Web services and APIs.
o SoapUI tool develops the mocks where test engineers can test real applications.
o It is entirely written on JAVA and Groovy programming languages.
Features of SoapUI

Some of the most common characteristics of the SoapUI tool are as follows:

o The core features of SoapUI contains various web services, such as development, mocking, Inspection, simulation, and
invoking.
o It supports all standard protocols and technologies like HTTP, HTTPS, AMF, JDBC, SOAP, WSDL, etc.
o SoapUI tool offers a fast and efficient framework which creates many web service tests.
o It provides the services to obtain data from several sources of web service without developing any code.

To get more information about the SoapUI tool, refers to the following link: https://www.javatpoint.com/soapui.

Open-source Functional Testing Tools

Open-source functional testing tools are those tools, which are available freely in the market. These tools have less functionality and
features than the commercial/licensed tools, but sometimes working on the commercial tool became an expensive process.

That's why some well-known organizations preferred to use open-source tools.

Following are the most commonly used open-source functional automation testing tools:
 o Selenium
o Sikuli
o Autoit

Let's understand the open-source functional automation testing tools one by one:

Selenium

Whenever we are talking about the open-source tools of automation testing, one name came into every automation test engineer's mind:
Selenium.

o Selenium is the highly recommended and widely used functional testing tool, which is well-suited for non-functional testing
tools.
o Selenium is an open-source tool which means, it does not require any licensing.
o We can only test the web application using the selenium tool, and the Stand-alone application cannot automate in Selenium.
o It is most commonly used to implement Functional test scripts.
o It can be combined with several automation testing tools to achieve continuous testing, for example, Maven and Jenkins.
o It can be associated with many devices such as TestNG and JUnit to manage the test cases and generate the test reports.
Characteristics of Selenium

Following are the basic features of selenium:

o Selenium can be easily deployed on several platforms like Windows, Linux, Solaris, and MacOS.
o Furthermore, it also supports operating systems for mobile applications, for example, iOS, Windows mobile, and android.
o Selenium supports several programming languages such as C#, Java, Perl, PHP, Python, and Ruby.
o Selenium become extremely resourceful while we were implementing the test script and analyzing it across many browsers
concurrently.
o It provides a user-friendly interface that helps the test engineer develop and perform the test scripts resourcefully and efficiently.

To get the complete information about Selenium, refers to the following link: https://www.javatpoint.com/selenium-tutorial.

Sikuli

o Another open-source functional automation testing tool is Sikuli.

o It is a GUI based test automation tool, which can easily automate the Flash objects as most of the automation testing tool like
selenium does not support the flash-object automation.
 o Most commonly Sikuli is used to interact with fundamentals of web pages and control the windows-based popups.
o With the help of Sikuli tool, we can easily test the windows application.

Features of Sikuli

The most standard features of the Sikuli tool are as discussed below:

o The Sikuli tool can be easily combined with Selenium WebDriver and all other automation testing tools by using the Sikuli Jar
files.
o We can also automate desktop applications or stand-alone applications with the help of the Sikuli tool.
o It provides a simple API, which implies that all methods can be retrieved using screen class objects.

Note: As we know that Selenium WebDriver can only test the web application, but on the other hand, the Sikuli tool can test the web-
based as well as window-based applications.

AutoIt

o AutoItis another open-source tool used for functional automation testing.

o It is a freeware scripting language designed to test the Windows GUI and general scripting.
o The AutoIt script is written in a primary language.
o It can replicate any grouping of keystrokes, mouse movement, and window or control operation.
Features of AutoIT

AutoIT tools include the following features:

o It uses a straightforward syntax for its scripting language that can be easily understood and executed to automate any process.
o It has an inbuilt RunAs function that helps in executing any external program using some outside user.
o It can even record and create a script on its own for detailed processes, which need to be automated.
o All types of standard Windows controls and other GUIs without any object identification issue can easily relate to the AutoIt
tool.

Non-functional Automation Testing Tools

The automation test engineer uses the non-functional automation testing tool to execute the non-functional performance test cases.

For example, testing the application's response time under significant load, let say, 100 users.

Just like functional automation testing tools, the non-functional automation testing tools divided into two different categories, which
are as below:

1. Commercial Tools
2. Open-source Tools
Commercial Non-Functional Automation Testing Tools

These are those tools, which cannot be used freely, as they required a proper license. The commercial tools have additional features
and functionality as compared to the other open-source testing tools.

The usage of these types of tools helps us to enhance the efficiency of the software product.

Let's see some of the most commonly used commercial non-functional automation testing tools.

o LoadRunner
o Silk Performer

LoadRunner [HP Performance Tester]

LoadRunner is one of the most popular non-functional tools. It is mainly used to support performance testing for a wide range of
protocols, several technologies, and application environments. LoadRunner is the licensed tool.

It rapidly classifies the maximum number of performance issues. And precisely predict the application scalability and size.
Feature of LoadRunner

Some of the most significant features of the LoadRunner tool are as follows:

o LoadRunner tool will help us to reduce the cost of distributed load testing and hardware and software costs.
o We can easily view and handle the XML data within the test scripts as it supports the XML scripting language.
o We can get detailed performance test reports by using the LoadRunner tool.
o It offers the operational tool for deployment tracking.

Silk Performer

Another non-functional automation testing tool is Silk Performer. It can test the various application environments with thousands of
simultaneous users.

It makes sure that applications and server up times are sustained when faced with the highest customer usage.

It is one of the most commonly used enterprise-class load and stress testing tools, supporting an extensive range of protocols.
Characteristic of Silk Performer

Following are the standard features of the Silk performer tool:

o It is used to pretends changeable virtual users.

o In this tool, the correlation and parameterization are user-friendly.
o It supports integrated server monitoring.
o It provides Version Control Management.
o There is no license condition required for controllers or Individual Protocols.
o It can quickly create reports with tables and graphs and allows customization.

Open-Source Non-functional Automation Testing Tools

The open-source non-functional automation testing tools can be used easily as they are freely available in the market. These tools have
less functionality as compared to the commercial testing tools.

But because they are open-source and used quickly, some organizations prefer to use these tools as they do not require any cost.

Some of the most commonly used open-source non-functional automation testing tools are as follows:

o JMeter
 o NeoLoad

JMeter

JMeter is one of the most significantly used open-source non-functional automation testing tools. JMeter is entirely designed on the
JAVA application to load the efficient test performance and measure an application's productivity.

It is mainly simplifying users or developers to use the source code for the enhancement of other applications. It is used to test the
implementation of both static and dynamic resources and dynamic web applications.

It is used to reproduce the massive load on a server, object, or network, group of servers to test its power or explore the complete
performance in multiple load types.

It is highly extensible to load the performance test in several types of servers like:

o Database Server: LDAP, JDBC

o Web Server: SOAP, HTTPS, HTTP
o Mail Server: POP3
Characteristics of JMeter

Some of the significant attributes of the JMeter tool are as follows:

o It sustains several testing approaches, for example, functional, distributed, and load testing.
o It supports a user-friendly GUI that is interactive and easy.
o It is a platform-independent tool as it is developed with the help of JAVA; therefore, it can execute on any platform that accepts
a JVM like Windows, Mac, and Linux, etc.

For more information about JMeter, refers to the below link: https://www.javatpoint.com/jmeter-tutorial.

NeoLoad

Another most commonly used open-source tool in automation testing is NeoLoad and Neotys develop it.

It is used to test the performance test scenarios and also helps us to identify the bottleneck areas in the web and the mobile application
development process.

It is faster as compared to other traditional tools. NeoLoad will support a wide range of web, mobile, and packaged applications, such
as SAP, Oracle, Salesforce, etc., that cover all our testing needs.
Features of NeoLoad

Below are some of the vital features of NeoLoad:

o It is also used to share and manage the test resources.

o It will support various frameworks and protocols such as HTTP/2, HTML5, API, AngularJS, Web Socket, SOAP, etc.
o It provides a robust code-less design.
o NeoLoad can modify the functional test script into the performance test scripts.
o NeoLoad can automatically update the test scripts and developed the real-time test results.

Advantages of Automation Testing

o Automation testing takes less time than manual testing.

o A tester can test the response of the software if the execution of the same operation is repeated several times.
o Automation Testing provides re-usability of test cases on testing of different versions of the same software.
o Automation testing is reliable as it eliminates hidden errors by executing test cases again in the same way.
o Automation Testing is comprehensive as test cases cover each and every feature of the application.
o It does not require many human resources, instead of writing test cases and testing them manually, they need an automation
testing engineer to run them.
o The cost of automation testing is less than manual testing because it requires a few human resources.

Disadvantages of Automation Testing

o Automation Testing requires high-level skilled testers.

o It requires high-quality testing tools.
o When it encounters an unsuccessful test case, the analysis of the whole event is complicated.
o Test maintenance is expensive because high fee license testing equipment is necessary.
 o Debugging is mandatory if a less effective error has not been solved, it can lead to fatal results.

Overview

In this tutorial, we have understood the automation testing, automation testing approaches, automation testing process, automation
testing tools, different challenges during the automation testing process, advantages and drawbacks of automation testing.

Finally, we can conclude that the Automation testing is a software testing technique, which implemented with the help of special
automated testing software tools.

It is the best approach to execute a test case suite, which helps us enhance test coverage, efficiency, and performance speed in software
testing.

The selection of automation testing tool, testing process, and team, are essential aspects in order to successful automation.

Automation testing is highly dependent on the technology the Application Under Test is built on.

Test Automation Maintenance Approach is an automation testing phase execute to test whether the new functionalities added to the
software are working fine or not.

For the successful testing process of the software, the manual and automation techniques go hand-in-hand. We should clear about the
automation process that it is used to decrease the testing time for certain types of tests.