Performance Testing

Checklist Packet
Everything you need to trigger thoughts, discussions
and actions in the projects you are working on
Performance Testing Checklist:
People & Processes within Performance Testing

These checklists are designed to trigger thoughts, discussions and actions. They are not a learning tool, more they are designed to help you ask questions about
the projects you are working on. We hope you can take this checklist concept and create adapted versions which will be more relevant to the projects you work
on.

Considerations Analysis Resources

❑❑ Why do you want to do performance testing? ❑❑ Does the software and performance testing ❑❑ What budget is available?
data recover from after long periods, crashes,
❑❑ What needs to be measured? ❑❑ Who is available to support the performance
(un)scheduled downtime?
testing (including issues that may arise)?
❑❑ Users
❑❑ What failures do you anticipate?
❑❑ Volume ❑❑ How will issues be reported?
❑❑ Response time ❑❑ How often will performance testing be per-
❑❑ Who is responsible for replicating issues?
❑❑ Throughput formed? [sporadic, weekly, nightly]
❑❑ Physical ❑❑ Who will maintain the performance tests?
❑❑ Will you be benchmarking your performance
❑❑ Resource consumption: CPU, Memory, against competitors, SLA’s, previous versions ❑❑ How flexible are the tools being used?
Disk I/O or other standards?
❑❑ What exactly are you performance testing? ❑❑ How costly are the tools and load traffic?
❑❑ What is the pass / fail / goals criteria for frame
❑❑ What type of performance testing are you rate, load time, etc? ❑❑ How can costs be reduced?
doing?
❑❑ How does the software plan to change in the
❑❑ Performance future and what effect will this have?
❑❑ Load
❑❑ Stress
❑❑ Is there room for exploratory performance
testing?

❑❑ How will tests be made to be realistic?

www.smartbear.com
EBOK_20131010_V1_PERFCHECKALL
Performance Testing Checklist:
Questions to Ask About Your Performance Test Data

Considering your performance testing data, have you thought about the following:

❑❑ Do you have a baseline of current performance to measure against?

❑❑ How much data can the test system hold?

❑❑ What are the accepted figures for peak and maximum users?

❑❑ How do user roles and hierarchies come into play? 

❑❑ Does data need to be unique? (e.g. user ids)

❑❑ Does additional data need to be setup so a user can complete their user journey?

❑❑ Does additional data need to be generated to test future volumes and scalability?

❑❑ Does the application require interaction between users?  Do they share data?

❑❑ How many times and how much data do you need to run the tests?

❑❑ Where will you get the data from? [Production / legacy / off the shelf data tool / bespoke]

❑❑ What’s the acceptable time between transactions / clicks / actions?

❑❑ How and to what should future tests be compared against?

❑❑ Do you have access to (Google) analytics to identify high volume user journeys and load conditions?

❑❑ Check your ability to reset the database back to baseline state between each run?

❑❑ Do you have permissions in the test database to perform backups/restores?

❑❑ Do you have enough spare storage in the test database to have backups?

www.smartbear.com
EBOK_20131010_V1_PERFCHECKALL
Performance Testing Checklists:
Tips for setting up a performance testing environment

Setup the performance test What are your resources going to Configure monitoring for the Schedule a cross-functional
environment configuration as cost? system under test: team to support the process:
similar to production as possible:

❑❑ What tools and resources can be ❑❑ Are the tools expensive? Consider ❑❑ Physical Resources: CPU, ❑❑ Business Analysts
made available for performance the mix of open-source and com- DISK, MEMORY, NETWORK
❑❑ System Administrators
testing? mercial tools
❑❑ Application: .NET, JVM, Web
❑❑ Database Administrators
❑❑ Load generation and coordination ❑❑ Are the tools supported? You Server, MQ/JMS, etc.
tools need help when the tool isn’t ❑❑ Functional Testers
❑❑ Database: Connections, Block-
working right
❑❑ System and application monitoring ing, Locking, Scans, Cache, ❑❑ Database Developers
tools ❑❑ Hardware costs - servers, net- Storage, I/O
work, storage, devices, etc. ❑❑ Application Developers
❑❑ Application and database profiling/
diagnostic tools ❑❑ System setup and configuration ❑❑ Network Engineers
- who can help you setup and
❑❑ Service and network virtualization ❑❑ Managers
configure the system
tools

❑❑ Results graphing and reporting

❑❑ Collaboration tools for chatting, file

❑❑ Consider using the same tools in

www.smartbear.com
EBOK_20131010_V1_PERFCHECKALL
Performance Testing Checklists:
Tips for setting up a performance testing environment

Do all network components in Is the architecture of Check the size of the Are your test environment Do you collect
production exist in test?  the test site the same as test environment resources being used performance stats
production? when compared to or shared by other automatically?
production: processes?

❑❑ If yes, then testing more accu- ❑❑ Are the tools expensive? ❑❑ If the test systems are ❑❑ Coordinate schedules ❑❑ Hardware, SO, dbms,
rately with load balancing and Consider the mix of smaller, it is accept- to give you an exclusive virtual machines, run-
latency will be possible open-source and com- able for tuning specif- time to run tests time (java, ,net, etc),
mercial tools ic components application server
❑❑ If no, then your testing scope ❑❑ Turn off any auto-sched- and application itself
will be limited to only testing ❑❑ Are the tools supported? ❑❑ If the test systems uled jobs that might dis- (caches, querys, ser-
systems You need help when the are the same, it is rupt your testing (cron, vices stats, etc))
tool isn’t working right acceptable for testing batch, etc.)
❑❑ You may discover network bot- realistic loads
tlenecks that limit your testing ❑❑ Hardware costs - serv-
ers, network, storage, ❑❑ If the test systems are
devices, etc. bigger, it is accept-
able for testing future
❑❑ System setup and con- capacity and scale
figuration - who can help
you setup and configure
the system

These checklists have been created as part of a crowdsourced community effort in collaboration with Software Testing Club & SmartBear. We’d like to say a spe-
cial thank you to everyone who contributed David Thompson, Thomas Lattner, Matias Reina, Dan Caseley, Simon Knight, Shane Dennis, Daniel Wiersma, Lindsey
Thomas, Maciej Szatanik, Chris Gallant, Skip Marler, Sandeep Maher and Mark Tomlinson.

www.smartbear.com
EBOK_20131114_V2_PERFCHECKALL