White Paper

Testing the Cloud

Last Updated: 2nd February, 2009
Introduction step in the SaaS model, where the on demand delivery is
not simply the specific item of software required, but the
users’ platform, thus allowing centralised control of the
Cloud Computing presents tangible benefits to businesses.
usage of each machine on the PaaS network. Again the
These can be both immediate and vast ranging from
delivery route in this model is the ‘Cloud.’
reductions in cost of ownership to location independence.
However along with these potential benefits comes a new
set of concerns; security, privacy, availability, performance
and integrity. Suitable testing must be at the core of any
Cloud solution to ensure the delivery of a safe, integrated
solution which meets the needs of the business it is to
serve.

The purpose of this paper is to understand the benefits

and concerns of a Cloud Computing solution and how
suitable testing can aid in realising the full potential of your
investment.

Cloud computing is the single largest trend in computing

infrastructure today, although commonly misunderstood or
incorrectly referred to as Grid Computing, Utility Computing
or Autonomic Computing. Although a Cloud solution may
make use of these types of backbone, Cloud Computing
What does Cloud Computing
is a much more generalised term in which little or no mean to business?
centralised infrastructure exists. Sets of loosely coupled
data centres work together achieving high utilisation levels It’s easy to see why Cloud Computing solutions appeal
to perform a common task, presenting GUI interfaces to more and more to business’ globally.
users through virtualisation or thin clients via the ‘Cloud’.
Indeed Cloud  Scalability – The distributed nature of Cloud solutions
allow utilisation to be spread evenly amongst available
Computing can be considered to be the next natural step servers meaning applications can scale on demand.
from Grid and Utility computing.
 Location independent access – Using thin clients or

SaaS as a type of Cloud virtualisation. The only limitation being internet access.

Computing  Reduced cost of ownership – Using service providers

with currently existing Cloud deployments means the
Typically Software as a Service (SaaS) would be considered need to own hardware to support the Cloud solution
to be a type of cloud computing. Software is held centrally, can be almost entirely removed.
not by local machines. This is presented to the user on an
 Server efficiency/utilisation – The spread of server
‘on demand’ basis usually by means of virtualisation. Central
workload improves server efficiency and increases
control of the application is retained allowing for reduction
utilisation of otherwise lesser used servers producing
in licensing, implementation and ongoing maintenance
greater return on investment.
costs. The delivery route in this instance is the ‘Cloud’ this
being the general term for the Internet. The term ‘Cloud’ is  Green IT – The reduced requirement for hardware,
used to describe networks and infrastructure which are not implementations and location dependence means
visible to the user, a potentially huge network black box. business’ can reduce the environmental impacts of its
IT infrastructure.

PaaS as a type of cloud  Instantly deployable environments – Environments

can be built and deployed for specific needs such as
computing test or deployment. The use of virtualisation using the
Cloud means that these environments can be scalable
Another common example of Cloud Computing is Platform to production size and deployed within far reduced
as a Service (PaaS). PaaS can be considered as the next timescales.

AppLabs.com
App_WhitePaper_Testing_the_Cloud_1v00 Page  © 2007 AppLabs
 Reduced maintenance cost – The centralisation of all workshops to understand what is being asked for in
IT components means that maintenance can be carried business requirements will save time and money later in
out centrally on a one time only basis. The maintenance the development lifecycle by removing potential software
is then mirrored in all end user instances of that Cloud defects before they are built.
solution.
Cloud Computing solutions claim to be scalable on
However, alongside all of these benefits Cloud Computing demand. How do business’ verify that the solution delivered
presents risks which must be mitigated effectively if Cloud is capable of coping with the workload which it is required
Computing is to be a viable option for business. to undertake? Load or Stress testing can be used to prove
that the developed solution can scale as required. By using
 Lack of control – When IT infrastructure is outsourced
test techniques and tools which are capable of applying
to an external third party how does the business
huge amounts of load on the solution the Cloud can be
maintain control of their data, impact down time, drive
accurately measured and its capacity verified.
technology change or influence decisions which may
impact their solution? Using a distributed Cloud Computing solution should
mean that the delivery will perform to a high standard.
 Security – How can business ensure the potentially
Performance testing techniques allow the systems
sensitive information which traverses the cloud is safe
performance to be measured and verified accurately. Using
and secure?
performance testing and load testing techniques in tandem
 Privacy concerns – What can business’ do to maintain allows an accurate image of the solutions ability over the
the privacy of their users and information when using cloud to be created. This in turn provides the comfort that
the cloud? the system is

 Data Integrity – When using third party solutions for constructed so as to be capable of acting as delivery
Cloud Computing what assurances do businesses mechanism for the business requirement.
have their valuable data remains intact?
Security testing can provide assurance that business
 Availability – Cloud Computing solutions rely on the critical data is stored and transported safely. Techniques
availability of their infrastructure to be able to function. such as Penetration Testing can prove that the mechanisms
Should a business critical Cloud Solution be unavailable which have been developed to maintain security will remain
for any time, what is the business impact? intact during potential attempts to compromise the Cloud
solution.
 Business Acceptability – How sure can a business be
that their third party solution is suitable for its intended Functional Testing
use?
How do business’ validate that the system will behave as
has been specified within requirements? System testing
Quashing Concerns techniques allow the proving of the systems behaviour within
its own entity. Before consideration of any deployment it is
Non-Functional Testing critical to prove that the system functions as it has been
designed, that the system components work together,
In the earliest stages of assessing the feasibility of a Cloud inputs and outputs are as expected and the overall resulting
Computing solution it is key for the business and Cloud system is of a suitable quality to release.
delivery provider to understand the requirement for a Cloud
solution within that business’ context. The business must Before any deployment, how do the business verify that
carefully document what they require from such a solution the integrated solution will behave as intended to facilitate
clearly and unambiguously. As with the development of business continuity? Integration testing allows the business
any solution, business requirements are the key driver to verify that the Cloud solution will work within the
to the success of the solution. As business’ look more current infrastructure and environments, proving that the
and more often to third parties to deliver these solutions, implementation of a Cloud solution does not detrimentally
the foundations for the delivery must be solid. Business impact any existing systems.
requirements are these foundations. Static Testing on Finally, the business requirements must be verified and
business requirements will ensure that these are concise validated to prove that the end result of the Cloud solution
and complete. Undergoing the process of reviews and will meet the documented needs of the business. User

AppLabs.com
App_WhitePaper_Testing_the_Cloud_1v00 Page  © 2007 AppLabs
Acceptance Testing will use business requirements to prove full effect within a cloud environment to perform automated
that the delivered Cloud solution meets those needs. testing tasks such as regression tests.

The location independence element of Cloud Computing Well known tools such as HP’s Load Runner or IBM’s
solutions makes these types of applications the most Rational Performance Tester are familiar to testers the world
versatile in terms of test. There is little need to develop over and are well suited to testing Cloud type solutions.
infrastructure which can be utilised offsite and thin client or Already performance test tools which are specifically
virtualisation are the most common types of interface. designed to test cloud applications are appearing on the
market such as LoadStorm’s or SOASTA’s Cloud Test both
The benefits of offshore testing are well documented and
suites of on demand test tools for cloud applications.
present as an instant option for testing Cloud Computing
solutions. Likewise, on-site testing allows immediate Taking into account the quickly evolving nature of Cloud
control and monitoring of test progress. Computing in today’s technology sector it is critical that
any tester who is expected to test cloud solutions has a
Service Enablement via Tools good understanding of what makes a Cloud Computing
application and distributed architecture, as well as a good
Major technology vendors including such household
understanding of the tools available and their strengths and
names as HP, Intel and Yahoo are presently collaborating
weakness for testing different types of cloud applications.
to create huge cloud ‘test beds’ consisting of many
thousands of processors working together as centres of
excellence in Cloud Computing. These test beds will allow Summary
users to test their cloud deployments at internet scale and
also understand how their systems and software actually Whilst Cloud Computing applications for business use are
behave within the cloud. With such huge investment by still in their infancy, analysts predict this type of development
some of the largest technology and Internet Service will form a major new wave in technology infrastructure.
Providers globally today, indicators are clearly pointing at The benefits of a Cloud Computing solution appeal to
cloud computing to be a major focus point for the industry IT managers the world over and cannot be ignored by
in the coming years. businesses striving to overcome the constraints of their
current IT hardware whilst struggling to justify the cost of
Current test tool offerings by the likes of HP and IBM are
investing in major upgrades. All the while understanding
ideal for non functional and automated testing in a cloud
the requirement and suitably testing that requirement lies at
environment. Already well established software such as
the core of solid Cloud Computing deliveries.
HP’s Quick Test Pro or IBM’s Rational Robot can be used to

AppLabs.com
App_WhitePaper_Testing_the_Cloud_1v00 Page  © 2007 AppLabs