Item Description LoadRunner OpenSTA

Protocols The Many supported. HTTP 1.0 / 1.1 / HTTPS

communication Protocols are (SSL) only.
protocols that can charged per item.
be captured, Has a multi-protocol
manipulated and recording feature.
replayed by the
tool.
Playback functions Replaying of the Extended logging Similar playback facilities,
script and script supports view of but no integrated
debugging parameter values comparison function. The
facilities. and Server debugging functions are in
messages. Also the controller, including set
view and break points and single
comparison with stepping.
‘recorded’ version
of web page view
and client response
messages.
Debugging facilities
in script generator,
step and
breakpoints.
Scripting language The medium used Called TSL, it uses Called SCL, it uses a
to represent the standard syntax for proprietary, “BASIC” like
captured protocol “C” & allows C language that has special
data and function libraries to automation
manipulate the be added. Has scriptingfacilities. Is
data for play-back. extensive limited in available
customised functions, such as string
functions for the manipulation and supports
different protocols direct DOM addressing.
supported by the
tool.
Extensibility The ability to Additional TSL or SCL script modules can be
increase the “C” function defined in â??Includeâ??
functionality of the libraries, limited to files. Open Source
tool. functional therefore new tool
capabilities of the functionality can be added
tool. using C++.
Scripting Interface The interfaces Captures in several Has low-level HTTP
supplied by the modes, high level protocol view and provides
tool application for context based and graphical tree
the purpose of low level HTTP representation of the â??
script editing. view. Has both a Document Object
graphical tree Modelâ?? (DOM)
 structure & a script structure. Viewable
view. Script view captured HTML rendering
has function and addressable server-
sensitive entry. header table. Some
language sensitive, syntax
colour coding
functionality.
Correlation The task of Automated Manual correlation using
substituting values correlation faculties. graphical interactive â??
in dynamic data to Including during DOMâ?? structure. Facility
enable successful recording, after to automatically generate
playback. recording and script code to aid variable
comparing substitution.
recordings with
playback results.
Not available for all
modes of capture.
Cookie Detection, HTTP header HTTP header cookies are
Management recording and cookies are managed automatically and
playback of HTTP managed can be manipulated
cookies. Both tools automatically and manually if required.
need additional can be manipulated
code to manage manually if
JavaScript required.
generated cookies.
Parameterisation Automatically Extensive facilities Extensive facilities for data
changing dynamic for data entry entry including wizard
data values for including wizard interface to automatically
more accurate interface to DB generate test data. Standard
emulation of real interrogation. No functions for sequential,
users. Often standard function to random and pseudo-
essential for lock data sources random data-file access.
session and maintain Has standard common
management. uniqueness of locking facilities for
concurrently maintaining uniqueness of
accessed data across parameters for an
distributed tests. individual load injector or
across all injectors on a
distributed test.
Controller Application that Facilities for real- Facilities for real-time
manages and time monitoring. monitoring. Simple drag &
conducts a test. Automatic scenario drop multi-scenario test
generation. configuration supporting
Individual control of modular scripting and
vusers, scripts and addition of new
groups of scripts. scenarios/Vus at run-time.
 Scheduling, No automatic scenario
percentage runs & generation. Allows test-
iterations. wide and user-specific
HTTP monitoring and
debugging while multi-user
load in progress.
Monitoring Resource usage Many supported. Supports Integrated real-
information is On-line graphics time, graph view of
captured during during execution. Windows NT/2000
execution. It can Supplied with Performance (Perfmon) &
be shown during Apache, Netscape & SNMP collectors. Various
execution and used IIS, other monitors measurements of test
to build are charged per progress including VU-
performance item. Results used specific, custom status and
reports. for reporting. New activity information. Web-
facility to allow relay allows monitors to
remote users to run on remote machines
monitor real-time beyond firewall. On-line
results via a browser graphics during execution,
based interface. & monitored results used
Note: To monitor for reporting.
through a firewall
requires TCPIP
access through a
specific port. Future
versions of
LoadRunner should
use HTTP
messaging & avoid
this issue.
Distributed tests The ability to Supports multiple Supports multiple load-
distribute the load-injectors injectors managed by a
generation of load managed by a single single controller. Uses
across multiple controller. TCP/IP if on the same
load-injector network or the Web-relay
machines. feature uses HTTP to
control load-injectors
located within remote
DMZs.
IP Spoofing The ability to Supports IP No inbuilt features to
emulate the spoofing including enable IP spoofing.
behaviour of automated router
different IP updates for IP
addresses forwarding.
accessing a
system.
 Particularly useful
with Load
balancing systems.
WAN/LAN The ability to New feature added No inbuilt features to
emulation emulate the to version 7.6. enable WAN/LAN
behaviour of Allows the emulation.
different network emulation of
infrastructures latency, packet loss,
during a test. link faults &
dynamic routing
effects over the
LAN used in a test.
Requires a special
license.
Caching The ability to Can control browser No specific faculties,
emulate the cache emulation although can be emulated
caching of pages during playback and with simple script code.
as performed by a control setting for
web browser. each individual
vuser.
User connection The ability to Can emulate No inbuilt features to
speed emulation emulate the different network emulate user-connection
different network speeds during speed emulation.
speeds that can be playback
used by real users.
Distributed/Remote To generate large Can control multiple Can control multiple load
load generation loads additional load generators and generators and collect
load generator collect results. results.
machines are
required. These
machines must be
centrally
controlled.
Can control load Can control load generators
generators located at located at remote networks
remote networks (through a firewall) if
(through a firewall) required using HTTP ports.
if required using
HTTP ports.
Reporting & The facilities to Sophisticated & Simple charts and graphs
analysis examine and large range of charts sufficient for analysing key
investigate the and graphs with load related statistics and
results of a test overlay facilities. resource usage monitors.
including timers -Automatic report Resource usage monitors
and monitored generation into MS- supports graph overlays.
resources and Word. Analyser is a Can be exported to
 display the results separate application Microsoft Excel. No
in graphical that can be license restrictions on
format. distributed to users. OpenSTA distribution thus
stats can be viewed by any
user with access. -Free
tools and excel macros
available through public
user-forums.
Scalability The ability of the Resource limitations Main resource used is
tool to generate are number of RAM. Tested for simple
numbers of virtual threads and RAM. ASP pages, reaching up to
users and the Approx 1 vuser per 3,000 users on load
corresponding ½ Mb RAM for generator of 1Gb RAM on
resource usage. NT/W2K. Windows a single P4 processor &
Actual resource 95, 98 & Unix are W2K. Unconfirmed report
use depends on the less efficient. Max. of a limit of 1664 Vus per
number, size and approx 1,500 VUs Win2K machine for
complexity of the per PC. complex scripts. Suspected
scripts. thread limitation. No
licensing limitations.
Initial Costs Purchase costs of From £16,000 for FREE� download from
software and basic package with SourceForge via
licenses, excluding no virtual users. www.OpenSTA.org.
upgrades or Additional charge Downloads available:
support. for each protocol Previous versions;
and monitored Automatic installer or
resource and for current source code (with
virtual users. simple build instructions
for MS C++ Visual Studio
6).
Costs of Virtual Most commercial Prices vary but for FREE. No licensing
users tools charge on the the purpose of limitations.
basis of the expectation:
number of virtual additional vusers
user available. cost from £10K
Extra hardware is for 25 VUs to
an additional cost. £66K for 1000
VUs. Also
temporary VU days
at £3.50 each per
day (min 1000).
This is not a quoted
price.
Support & The support By M.I., approx 1/5 Various independent
consultancy services available of initial cost per resources. etest associates
for the tool and annum. Includes charged from £50 per
 some costs. upgrades. MI & incident for remote tech
their partner support. Consultancy rates
companies also available on
provide consultancy
request.Numerous online
(including etest resources including web
associates). and e-mail forums.
Upgrades are free (approx
every 3-6 months)
Training The training MI has a range of Specialist companies
services available courses from around provide tailored training,
for the tool. £400 per day per prices vary.
person. Many
partners also
provide training.
System The operating MS windows 2000, MS windows 2000, NT4
requirements systems required NT4 (sp6a),XP-Pro (sp5+), XP-Pro.
to host the tool. (also 95 & 98
(Not the OS of the cannot run multi-
system under test). threaded) - Load
generators also
support limited
generator
functionality on:
Unix: HP, Solaris,
Linux

Hardware The hardware Min: Pentium 350 Min : Pentium 200 &
requirements. requirements to & 128M - Load 80MB RAM. Pref:
host the tool. (Not generators: Pentium Pentium 500MHz+ &
the hardware of 1GHz & 1 MB per 128MB+ RAM.
the system under VU.
test).
Access to source Availability of the Unavailable. Open Source GNU public
code source code of the license - ‘C++’ Language.
tool itself.
A users opinion. Opinion of Has a very user Easy to use interface and
Internet friendly user excellent scalability.
performance interface and Inbuilt results analysis is
engineers who fantastic monitoring slightly limited in
have experience & results analysis. comparison to
with both tools. Automatic LoadRunner. The captured
correlation and data is open and can also
improved script be exported to Excel in two
recording facilities clicks. Drag & drop
can aid productivity. scenario set-up and control
Very flexible is very intuitive and easy to
scripting functions interact with, simplifying
& great help creation of scenarios with
documents. modular scripts. Manual
Complexity of correlation can be a
options and layout headache but is eased
of controller are slightly by using third-
drawbacks. The party ‘Diff’ tools and the
availability (at a inbuilt GUI DOM
cost) of numerous addressing feature. A
protocols for relative shortage of
recording is a standard script language
significant strength. functions but enough to get
most HTTP load testing
jobs done. If not, with
“Includes” support and the
availability of the source
code, it is very extensible