EXPLORE HOW TO GET THE MOST

OUT OF YOUR TESTCOMPLETE TESTS

Scale your UI Test Automation
Strategy with TestLeft

LEARN MORE ABOUT TESTCOMPLETE

Contents
Building Your Own Test Environment? Think about Scalability, Cost,
and Maintenance 5

The Challenge with In-house Infrastructure and Cloud Virtualization Services 5

How Can TestComplete’s NEW Environment Manager Help You? 7
TestComplete Now Gives You Access to Over 500 Enviroments 7
Combine Manual, Automated, and Visual Testing Together 8
Access Real Machines for In-depth Debugging 9
Quickly Identify Issues 9
Built-in Reporting 10
Get More Done - Multitask While Your Test Runs 11
Run Tests in iOS, Mac, and Android Environments 11
TestComplete can help you speed As you will see later, the key features available in En-
vironment Manager can be difficult to replicate using
up your time to delivery as it allows
on-premise test environments or virtual machines.
you to easily create accurate and TestComplete’s Environment Manager also comes
repeatable automated tests that with visual and manual testing capabilities. This can
can be run across multiple devices, help you uncover layout and usability issues, which

platforms, and environments. often are difficult to find through automated test
scripts.
Once tests have been created however, testing teams are
By the end of this eBook, you will have a thorough
often faced with a troubling question:
understanding of how TestComplete 12.2 can not only
Do they build their own on-premise test help you drive greater efficiency and test coverage
environments, use Virtual Machines, or across devices, but also allow you to overcome the
subscribe to a cloud testing service such as hassle of managing and upgrading devices, virtual
CrossBrowserTesting? machines, and in-house labs.
The goal of this eBook is to help you get more out of Before we venture in that direction, let’s dive deep
your existing TestComplete tests. The next few pages into understanding the challenge of building and
are focused on how Environment Manager, available in maintaining your own test infrastructure instead of
TestComplete’s latest release, can help you overcome the using an out-of-the-box cloud environment like Envi-
challenges faced while maintaining and setting up your ronment Manager.
own test environments.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 4

Building Your Own Test The Challenge with In-house Infrastructure and
Cloud Virtualization Services
Environment? Think about Regardless of whether you are setting up an on-premise
Scalability, Cost, and infrastructure or using cloud virtualization services such

Maintenance as Amazon EC2 or Rackspace, you’ll still be required to

put in some manual effort while building, maintaining,
If you have decided to set up and build your own test and upgrading the test infrastructure.
infrastructure, you’re probably doing this in one of two
While the cost and time associated with setting up such a
ways:
test infrastructure can look minuscule at first, it can easily
1. Using an in-house infrastructure: In this case, your grow out of hand as more devices, browsers, and resolu-
operations or test team takes on the responsibility of tions are added to the mix to improve test coverage.
building an in-house infrastructure themselves, in-
There are five challenges teams frequently face with in-
cluding the servers, mobile and desktop devices, and
house infrastructure and cloud virtualization services.
integrations to CI tools.
1. Environment Cost: Regardless of which path you take
2. Using cloud virtualization services: The other way you
to set up a test environment, there are a few key factors
can go about setting up a test infrastructure is by spin-
that will define its costs.
ning up virtual machines using services such as Ama-
zon EC2 and Rackspace. First and foremost is the time your operations team will
spend on maintaining and upgrading environments,
database servers, and labs, among others. While cloud
virtualization services can help you overcome hardware
up front, there is still a manual component involved with
setting up servers to ensure the right configurations are
available when tests are being kicked off.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 5

Provisioning clean test environments often means spinning 3. Maintenance Cost: Other than the responsibility of
up new VMs or adding more on-premise servers, which maintaining the desktop hardware, virtual machines,
can be time consuming and adds to the environment cost. and other real devices, the team might also have to
maintain a whole host of other software licenses.
2. Device Cost: This represents the cost for maintain-
These licenses will add to the overall maintenance
ing and upgrading different desktop and mobile devices.
cost.
Keeping systems and devices up-to-date can become chal-
lenging as teams grow and new environments are added 4. Parallel Execution: The time it takes for your testing
to the mix to improve coverage. suite to complete a run can be drastically reduced by
running UI tests in parallel. Ensuring your in-house
As the team grows, ensuring particular combinations of
test infrastructure or cloud virtualization service can
operating systems, resolutions, and browser versions are
handle parallel execution often requires extra steps to
available for testing means adding new devices. This cost
configure test scripts.
can grow quickly when every operating system and brows-
er needs to be kept up-to-date based on the latest release. To add to this, running tests in parallel can get
If the testing team is not co-located, this can become more expensive, especially when you have to add devices
difficult as there might be some degree of lag, which in to scale tests across hundreds of browsers, operating
turn impacts the productivity of the testing team. systems, and resolution combinations.

5. Debugging &Reporting: Features such as videos

and screen shots can be helpful once tests have been
run to ensure they are debugged in real-time and so
that you can see what’s going on with a build.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 6

However, building an in-depth debugging mechanism
in-house can be challenging. The additional time and
effort you would spend while building in-house, real-time
reports and dashboards for common visibility across team
members could be costly.

How Can TestComplete’s NEW Environment

Manager Help You?
Environment Manager, available in TestComplete 12.2,
allows QA and dev teams to easily and quickly execute
and report on TestComplete tests across more than 500
test environments. This can help overcome the challenges
faced while setting up, maintaining, and upgrading an in-
house testing infrastructure or cloud virtualization service.

TestComplete Now Gives You Access to Over 500

Enviroments
TestComplete has always done a great job of helping you
create automated UI tests using script-free record and
replay functionality or by giving you the ability to choose
from multiple programming languages such as JavaScript
and Python, among others. Now, with Environment Manager
you have access to a real device lab for executing these
tests without setup or configuration. You also get access to
a few additional features with Environment Manager.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 7

Combine Manual, Automated, and Visual Testing Together
Through TestComplete’s integration with CrossBrowserTesting you have access to live environments for manual
testing and debugging, as well as visual testing so you can highlight differences between website displays on different
browsers. This is beneficial to your team because you can now diagnose responsive web and layout issues across
more than 1,500 mobile and desktop browsers. The image below shows you what visual testing looks like. In this case,
you can see there are 22 layout issues.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 8

Access Real Machines for In-depth Debugging
Through Environment Manager you can also access live desktop web browsers remotely to further drill down into
problem areas. Once you have access to the real machine, all you have to do is use native developer tools or set up
breakpoints to debug HTML, JavaScript, and CSS issues in greater detail. You can even grant your developers access
in the event they want to use real machines to debug defects further.

Quickly Identify Issues

Environment Manager has also expanded TestComplete’s logging capabilities. You can now record your entire UI test
from start to finish so you can pinpoint problem areas to find and fix errors quickly.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 9

Built-in Reporting
With Environment Manager’s in-depth reporting capabilities, you can check detailed metrics and reports to ensure
that proper test coverage exists across operating systems, browser versions, and resolution configurations. You can
also export reports into CSV and XLS files to share with other stakeholders. Often, getting in-depth reports like these
requires writing code or developing custom integrations.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 10

Get More Done - Multitask While Your Test Runs Run Tests in iOS, Mac, and Android
Environments
Wouldn’t it be great if a test run didn’t block your computer?
What if you could get work done at the same time your tests With Environment Manager you can also create
are running? and run tests in iOS, Mac, and Android Environ-
ments.
With Environment Manager, you can deploy your tests direct-
ly in environments in the cloud and in the meantime, you can Through the tool’s integration with
continue to use your computer. Whether it’s answering work crossbrowsertesting, you can perform manual
emails, talking to friends on skype, playing games, or watching as well visual testing across 1500+ environments,
cat videos, you’ll have the freedom to do what you want be- including iOS, Mac, and Android devices.
cause the test run won’t lock down your machine.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 11

In conclusion, if you’re on an older version of TestComplete, you should consider upgrading to the tool’s latest version
to take advantage of the key features of Environment Manager. Take a look at the comparison chart below to see what
you’re missing.

So what are you waiting for? Give TestComplete’s Environment Manager a try by upgrading to the newest version of
TestComplete here.

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 12

 With the small amount of work of setting up an automated data-driven
test in TestComplete, you will now have a reusable test script that allows
for easy configuration depending on your variables and database needs.
Building this data-driven test, is not only a reusable automated test for
data, but is a great way to learn some of the many valuable features
TestComplete has to offer in making a tester’s life a lot easier.

TRY IT FOR FREE TODAY

 Over 4 million software professionals and
25,000 organizations across 194 countries
use SmartBear tool

4M+ 25K+ 194

users organizations countries

See Some Succesful Customers >>

API TESTING PERFORMANCE CODE

RE AD I N E S S MONITOR I N G COL L ABORAT I ON

Functional testing through Functional testing, Synthetic monitoring for API, Peer code and documentation
performance monitoring performance testing and test web, mobile, SaaS, and review
management Infrastructure

SEE API READINESS SEE TESTING SEE MONITORING SEE COLLABORATION

PRODUCTS PRODUCTS PRODUCTS PRODUCTS

EXPLORE HOW TO GET THE MOST OUT OF YOUR TESTCOMPLETE TESTS 14