Scribd
Upload
63 views4 pages

Demandware Load Test Plan - Template

Uploaded by

Ashok Jagtap
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
63 views4 pages

Demandware Load Test Plan - Template

Uploaded by

Ashok Jagtap
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

<CUSTOMER_NAME> Load Test

______________________________________________________________________________

Template Overview + General Load Test Information


This is the Template that should be used to develop a Load Test Plan for a Storefront(s)
running on the Demandware platform.

The Demandware Support Team must be notified by a Support Case of Load Testing. This
will prevent the load generated from being mistaken by the Demandware Operations Team
as malicious. This notification process is outlined in the following Knowledgebase Article:
Load Test Notification. This completed Load Test Plan should be attached to this load test
support case.

It is recommended that any Partner or Engineer that will be load testing against the
Demandware platform go through Demandware’s Load Testing Best Practices course
online. This will help to insure optimal load testing project outcome. The course can be
completed in approximately ½ hour.
• Go to Demandware XChange > Education > Training > Demandware Load Testing
Best Practices

This first section can be removed from the Load Test Plan. It is intended as informational for
the Load Test Partner and/or Engineer.

Load Test Goal


The goal of this load testing should be to produce a load with similar characteristics to
production traffic patterns experienced during the holiday for the purposes of optimization
and tuning custom code, services timeouts, and site configuration. The custom code tested
during this project should be the code that will be in production.

Load Test Plan


This Load Test Plan will be used by Demandware Support Team to understand the target
load, load profile, and execution schedule..
This section will outline best practices for a load test plan including scripting scenarios,
execution guidance, target load, load distribution, and site config.

Recommended Load Test Scenarios


The load test scenarios represent the set of detailed actions that are scripted. These
scripted scenarios make up the set of actions that are available to help model traffic under
load. The standard set of commerce actions that are recommended to be defined as
scenarios for the load test should include:
• Search Products
• Browse Products
• Register User
• Add To Cart
• Checkout
• Place Order

For any action that can be conducted as a registered user where the user is logged in, it is
recommended that a version of that scenario be created for both anonymous and logged in.

Recommended Target Load


The target load will be based on analytics data from the <current_year> holiday season with
a projection factor added to account for future holiday growth. Demandware measures load
test volume using transactional units:
1. Visits Per Hour – The visit rate measurement is a one unique user transaction from
start to finish of a single load test scenario that does not place an order. The
number of times a virtual user can run through this transaction in 1 hour
represents the Per Hour rate.
2. Orders Per Hour – The order rate measurement is a one unique user transaction
from start to finish of a single load test scenario that results in a valid order
placement. The number of times a virtual user can run through this transaction in 1
hour represents the Per Hour rate.

<current_year> Holiday Data


There is only 1 storefront site on the Customer production realm instance: Customer_Brand.
This site was used to pull peak hourly data from the Traffic and Purchase Reports available
in Business Manager Analytics. The time period used was Black Friday, November 27,
<current_year> due to its having the highest volume.
• Peak Hourly Visit Rate: 151,418
• Peak Hourly Order Volume: 5,954

Target Load Projection Factor


The load test should be gauged to test for the next up and coming Holiday season. In order
to simulate this, the target volume should be the most recent peak holiday volume with a
growth factor. This factor is based on data from the previous 2 holiday seasons.
Measurement Visits Per Hour Orders Per Hour VPH % Increase Y0Y OPH % Increase YoY
Customer <currentYear-2> 63,860 2,740 40% 42%
Customer <currentYear-1> 106,885 4,687 30% 21%
Customer <currentYear> 151,418 5,954 NA NA

The average increase in both visit and order volume over the past few years is around 35%
growth. It is important to leave a conservative amount of growth in a projection factor, so a
factor of 2x the observed yearly growth over the past 2 years will be used: 75% or 1.75x

Load Test Target Load


The target load for the load test should be (<current_year> peak hourly) x (Calculated
Factor).
• Peak Hourly Visit Rate: (150,000 x 1.75) = 262,500 visits per hour
• Peak Hourly Order Volume: (6000 x 1.75) = 10,500 orders per hour
Recommended Load Profile
In order for a Load Test to have a more realistic load, the target load should be broken out
into different actions. This Load Distribution will help to exercise custom code in similar
ways to production usage patterns, which is important to identify performance hot spots
that are most likely to impact the Storefront during high traffic events.

The break out of the load should be based on Analytics data available through Demandware
Business Manager. It may also come from Business Analysts and vary based widely on the
event being simulated or the goal of the load test.

The below is an example load distribution for the set of suggested commerce scenarios. The
load distribution is specified in transactional units – Visits Per Hour and Orders Per Hour.

Total Visits Per Hour (VPH): 262,500


Total Orders Per Hour (OPH): 10,500

Suggested Scenarios
TBrowse 30% x VPH = 78,750
TSearch 30% x VPH = 78,750
TAddToCart 15% x VPH = 39,375
TRegisterUser 5% x VPH = 13,125
TCheckout 20% x VPH = 52,500
TOrder 100% x OPH = 10,500

Test Realm Instance and Configuration


The test realm instance will be provisioned to match production realm instance. This test
realm resource footprint should match the current production realm instance for the Web,
App and Database tiers. [Include your Production realm id or Production Business Manager
URL]

3rd Party Integrations


All 3rd Party integrations that will be in place when the custom code under test is in
production should be in place and available to the test realm instance during the load test.
Steps should be taken to ensure that the service providers can handle the load.
The configuration for all services timeouts for these integrations should match production.

Job Execution
The job execution schedule that will be in place when the custom code under test is in
production should be running in the test realm instance during the load test.

Load Test Project Schedule


This load testing is scheduled for the following dates:
• Environment Setup and Data Preparation: [from_date – to_date]
• Load Test Execution and Monitoring: [from_date – to_date]
• Resolve Issues / Rerun Load Test: [from_date – to_date]
• Delivery of Load Test Analysis Document: [date]

Load Test Execution Schedule


A load testing is designed to evaluate the custom code under volume conditions. Therefore,
any time there is a major revision planned for custom code – a load test should be executed
with the recommended load test scenarios, target volume and load distribution.
For Holiday Readiness, the load test should be executed no later than 2 months prior to the
holiday. This will allow for time to optimize and retest if necessary.

Legal Disclaimers:
THIS DOCUMENT IS CONFIDENTIAL INFORMATION OF DEMANDWARE, INC.
COPYRIGHT (c) 2016 Demandware, Inc.
The information contained in this document is provided by Demandware for general information purposes only and is based on information as of the
date of distribution and is subject to change. References to other entities and/or their services and products are not intended as an endorsement of
such other entities and represents solely Demandware's opinion with respect to such other entities and the matters described herein as of the date
hereof. This document and the information contained herein is for the benefit of the intended recipients only and may not be reproduced,
disseminated further or disclosed to any third party except as specifically permitted by the intended recipients' agreement(s) with Demandware. This
document and the information contained herein shall not be conclusive with respect to the measurement of, and may not be relied upon by the
recipients with respect to, the availability of the Demandware eCommerce Platform Services.

You might also like