Introduction to Automation Tools:

Automation :

Test automation is the use of software to control the execution of tests, the comparison
of actual outcomes to predicted outcomes, the setting up of test preconditions, and other
test control and test reporting functions. Commonly, test automation involves automating
a manual process already in place that uses a formalized testing process.

Automation Tools :

Many test automation tools provide record and playback features that allow users to
record interactively user actions and replay it back any number of times, comparing
actual results to those expected.

A growing trend in software development is to use testing frameworks. A framework is

an integrated system that sets the rules of Automation of a specific product.

Another important aspect of test automation is the idea of partial test automation, or
automating parts but not all of the software testing process.

Over the past few years, tools with graphical user interfaces (GUI) that help programmers
quickly create applications have dramatically improved programmer productivity. This
has increased the pressure on testers, who are often perceived as bottlenecks to the
delivery of software products. Testers are being asked to test more and more code in less
and less time. Test automation is one way to do this, as manual testing is time consuming.
As different versions of software are released, the new features will have to be tested
manually time and again. But, now there are tools available that help the testers in the
automation of the GUI which reduce the test time as well as the cost; other test
automation tools support execution of performance tests.

Many test automation tools provide record and playback features that allow users to
record interactively user actions and replay it back any number of times, comparing
actual results to those expected. However, reliance on these features poses major
reliability and maintainability problems. Most successful automators use a software
engineering approach, and as such most serious test automation is undertaken by people
with development experience.

A growing trend in software development is to use testing frameworks such as the xUnit
frameworks (for example, JUnit and NUnit) which allow the code to conduct unit tests to
determine whether various sections of the code are acting as expected under various
circumstances. Test cases describe tests that need to be run on the program to verify that
the program runs as expected. All three aspects of testing can be automated.
Another important aspect of test automation is the idea of partial test automation, or
automating parts but not all of the software testing process. If, for example, an oracle
cannot reasonably be created, or if fully automated tests would be too difficult to
maintain, then a software tools engineer can instead create testing tools to help human
testers perform their jobs more efficiently. Testing tools can help automate tasks such as
product installation, test data creation, GUI interaction, problem detection (consider
parsing or polling agents equipped with oracles), defect logging, etc., without necessarily
automating tests in an end-to-end fashion.

Test automation is expensive and it is an addition, not a replacement, to manual testing. It

can be made cost-effective in the longer term though, especially in regression testing.
One way to generate test cases automatically is model-based testing where a model of the
system is used for test case generation, but research continues into a variety of
methodologies for doing so.

Along with test automation, automation of the entire testing process is also very
important. Therefore, one must keep following points when thinking of test automation:

 Platform and OS independence

 Data driven capability (Input Data, Output Data, Meta Data)
 Customizable Reporting (DB Access, crystal reports)
 Email Notifications (Automated notification on failure or threshold levels)
 Easy debugging and logging
 Version control friendly – minimum or zero binary files
 Extensible & Customizable (Open APIs to be able to integrate with other tools)
 Common Driver (Ant or Maven)
 Headless execution for unattended runs (For integration with build process or
batch runs)
 Support distributed execution environment (distributed test bed)
 Distributed application support (distributed SUT)

Some of the automation tools include Load runner, winrunner and Quick Test
Professional

Functional and regression test performance

Adv n disadv of automation tools

automated tests are the reusability and therefore ease of maintenance of tests that have
been created at a high level of abstraction.

Selecting an automation tool framework

Framework approach in automation
A framework is an integrated system that sets the rules of Automation of a specific
product. This system integrates the function libraries, test data sources, object details and
various reusable modules. These components act as small building blocks which need to
be assembled in a regular fashion to represent a business process. Thus, framework
provides the basis of automation and hence simplifing the automation effort.

There are various types of frameworks. They are categoried on the basis of the
automation component they leverage. These are:

1. Data-driven testing

Test scripts are executed and verified based on the data values stored in one or more
central data sources or databases. These databases can range from datapools, ODBC
sources, csv files, Excel files, DAO objects, ADO objects, etc. Data-driven testing is the
establishment of several interacting test scripts together with their related data results in a
framework used for the methodology. In this framework, variables are used for both input
values and output verification values: navigation through the program, reading of the data
sources, and logging of test status and information are all coded in the test script. Thus,
the logic executed in the script is also dependent on the data values.

2. Modularity-driven testing

requires the creation of small, independent scripts that represent modules, sections, and
functions of the application-under-test. These small scripts are then used in a hierarchical
fashion to construct larger tests, realizing a particular test case.

It is a well-known programming strategy to build an abstraction layer in front of a

component to hide the component from the rest of the application. This insulates the
application from modifications in the component and provides modularity in the
application design. The test script modularity framework applies this principle of
abstraction or encapsulation in order to improve the maintainability and scalability of
automated test suites.

3. Keyword-driven testing

4. Hybrid testing
5. Model-based testing
Test management
Test managers must take a potentially infinite job—testing new applications—and
accomplish it within tight time and resource restraints. It’s a tall order, but successful test
managers have found proven ways to handle the challenges. Planning for testing on a
software project is often challenging for program managers. Test progress is frequently
unpredictable, and during software testing painful schedule and feature "surprises"
typically occur. Software testing is often viewed as an obstacle—more as a problem and
less as a vital step in the process. For this reason, testing is treated as a "black box" and
addressed at the end of the schedule. While budget and time may be allocated for it,
testing is not really managed in the same way as development. Typically, software
development is measured in terms of overall progress in meeting functional and business
goals. Software testing needs to be measured in similar terms to understand its true
progress and make informed decisions. By considering testing dimensions other than cost
and schedule, managers and other team members can better understand and optimize the
testing process, in effect opening the black box and managing testing more effectively. In
this way they can avoid costly and painful "surprises" late in the project.

Tasks involved in Managing Testing

 Defining and implementing the role testing plays within the organization.
 Defining the scope of testing within the context of each release/delivery.
 Deploying and managing the appropriate testing framework to meet the testing
mandate.
 Implementing and evolving appropriate measurements and metrics.
 To be applied against the product under test.
 To be applied against the testing team.

 Planning, deploying and managing the testing effort for any given
engagement/release.
 Managing and growing testing assets required for meeting the testing mandate:
 Team members
 Testing tools
 Testing processes

* Employing the appropriate testing framework or test architecture to meet the

organization's testing needs

Advantages of Automation
Manual testing is expensive. It requires people to click buttons and observe results. This
isn’t terribly expensive the first time through but the costs stay fixed as the product
progresses. If you have a daily build (as you should), you pay the cost daily. This
quickly adds up. Automation is expensive to create but the incremental cost is very
cheap. Automation is also consistent. No one will ever forget to run a test case with
automation.
 Because it is cheap, automation can be run in places where manual testing
cannot. Extensive tests can be run on daily builds. These can even be run in the wee
hours of the morning before everyone shows up for the day. The inexpensive nature of
automated testing is what allows unit tests and test driven development to be possible.
Some test cases cannot easily be done manually. When testing an API such as the
DirectShow API I often work with, there is no UI to drive it. At least some minimal
coding must be done to expose the API to a user. In these cases, automation is the
obvious choice.

Using Testing Effectively

By definition, testing is a repetitive activity. The methods that are employed to carry out
testing (manual or automated) remain repetitious throughout the development life cycle.
Automation of testing processes allows machines to complete the tedious, repetitive work
while human personnel perform other tasks. Automation eliminates the required “think
time” or “read time” necessary for the manual interpretation of when or where to click
the mouse. An automated test executes the next operation in the test hierarchy at machine
speed, allowing test to be completed many times faster than the fastest individual.
Automated test also perform load/stress testing very effectively.

Reducing Testing Costs

The cost of performing manual testing is prohibitive when compared to automated

methods. The reason is that computers can execute instructions many times faster and
with fewer errors than individuals. Many automated testing tools can replicate the activity
of a large number of users (and their associated transactions) using a single computer.
Therefore, load/stress testing using automated methods require only a fraction of the
computer hardware that would be necessary to complete a manual test.

Replicating testing across different platforms

Automation allows the testing organization to perform consistent and repeatable test.
When applications need to be deployed across different hardware or software platforms,
standard or benchmark tests can be created and repeated on target platforms to ensure that
new platforms operate consistently.

Greater Application Coverage

The productivity gains delivered by automated testing allow and encourage organization
to test more often and more completely. Greater application test coverage also reduces
the risk if exposing users to malfunctioning or non-compliant software.

Results Reporting
Full-featured automated testing systems also produce convenient test reporting and
analysis. These reports provide a standardized measure of test status and results, thus
allowing more accurate interpretation of testing outcomes. Manual methods require the
user to self-document test procedures and test results.

Disadvantages of Automation testing

It is not free. Automation is expensive to create. If you are able to amortize that cost
over a lot of runs, the incidental cost becomes low. On the other hand, if the test is
something that will only be run a few times, automation may be more expensive than
manual testing. Decision makers must consider the high initial cost before committing
their organization to automated tests.
Automation is limited in scope. After you have run your automated tests for the first
time, you are done finding new bugs. Never again will you find a new issue. You might
catch a regression, but if you missed the fact that clicking a particular combination of
buttons causes the app to crash, you’ll never find that issue. This is true no matter how
many times you run the automation. On the other hand, high quality manual testers will
take the opportunity to explore corner cases. In doing this, they will find many issues
that would otherwise go unnoticed until real users find them after the product is shipped.

Bugs in an automated test may mask a bug in the product and go unnoticed for long
periods of time. Automated testing does not emulate real users using a product nearly as
well as a real person.