Why performance testing?

 Do want to use an Movie Booking web-site to book a ticket

and wait for 15 minutes to know that there are no seats available?
Why performance testing?

How many of us like to click on a button and see the result –

“Error : Page not found”?
 Bad performance is bad for business
 Lack of proper performance testing results in loss of revenue,
loss of credibility, loss of customers
Why Applications become slow?

Multiple users using the application at the same point of time

Load on the application increases

Applications become slow

What is Performance Testing Check
Performance of
application

Application Under
Creating the test
load on the
application (Lets assume Gmail
application)
What are Vusers Can we have
Image real users
there were
1000 users

These
were real
users

Virtual Users
OR
Vusers
Types of users

 User Base (100000)  registered

 Application Users / Online users (100) (90 +10)
 Concurrent users (10)

Key Facts
 Number of concurrent users is not same as the number of
application users
Performance testing tools and protocols

8
 What is
Architecture?
What is architecture?
 An architecture is a

 combination of software and system components and connections

10
Importance of architecture

 Helps in
 Designing smaller and more manageable pieces of application
 Understanding dependencies between components
 Enables interaction
 Between different parties (stakeholders) who intend to develop the computer-
based system
 Between different systems and system components

11
What is a server

Software (Computer Program) or Hardware which

 Understands your request
 Process your request
 Send the response back
Application Server (APP Server)

An application server is a server program

in a computer in a distributed network that
provides the business logic
for an application program
 Web Server

An web server is full implementation of HTTP Protocol

A Browser is partial implementation of Http Protocol

 Application layers
Presentation layer
Accepts user inputs, displays data and any results computation on the user
interface

Application processing layer

Provides application specific functionality
e.g., in a banking system, banking functions such as open account, close
account, view account details etc.

Data management layer

It manages the system database

15
Client/Server - 2 Tier architecture

 Improves multi-user updating.

 These days again this model is picking up because of mobile apps

There are 2 types of Models

16
3-Tier Architecture
Presentation
Layer
Business logic tier between presentation logic and data tier
Business Logic
Layer

Data Access
Layer

Data

PC Database
(Thin Client) Server
Application
Server

17
Three tier architecture

 Advantages
 Separate processor might be present for each application layer

 This architecture is simpler to

 manage than fat-client model and also gives better performance
than a thin-client model

 Can be scaled easily

 more servers can be added as demands increase

18
N-Tier (Multi-Tier Architecture)
Business Logic
Layer

Presentation Layer Data Access Layer

Enterprise Data

Database
PC (Thin Client) Web Server Application Server
Server
 Specialized web servers were introduced which can generate presentation content
which is transferred to the browser on the client tier, which takes care of rendering
the user interfaces

19
What is performance testing?

 PT is testing an application for speed, stability and scalability in “Production

like Environment” under virtual user load to meet Non-Functional
requirements. (NFR’s)
 Speed: How fast the application responds to your request. In other words,
Response time should be less
 Stability: Performance should not degrade even after using the
Application for very long time.
 Scalability: Application’s ability to handle increasing user without
performance degradation
Purpose of Performance Testing

 To identify Performance Bottlenecks

 Functionality of the application should not be changed under real
world conditions

Key Notes:

Performance Tester are required to identify Performance Bottlenecks but not Defects
(Bugs)
Important Terms in PT

 PT: Performance testing

 NFR : Non-Functional testing.
 Response Time - Time taken by the system to respond to a specific
transaction request
 Think Time –Time taken for selecting new transaction after receiving
the response for previous transaction.
 Throughput - Transactions per second, Bytes per second
90 percentile response time

 When we execute a Performance Test, same transaction will be executed

multiple times.
 Since, the same transaction is executed multiple times, multiple response
times are captured
 Refer
Client willtobe
inserted spreadsheet
provided formin
with Max, 90% ,RTAvg,
calculations.
and 90% response times.
 LoadRunner automatically calculate the 90% response times
Performance Testing VS Performance
Engineering
Performance testing is a subset of performance engineering
Performance Testing Performance Engineering

Performance Testing usually deals PE not only deals with identifying the
with identifying issues in the issues but also eliminating the same.
Application Under Test
Types of Performance Testing

Load Testing

 This is a mandatory test which will be done prior to any test.

 This test usually simulate current user load on the AUT.
 Usually Peak load for the application is considered for this test.
 This test ensures that the AUT is stable and handle expected peak
load once the application is deployed.
Scenario Load Test – MERCURY SAMPLE PROJECT
Name
Scenario Load Test – Duration 1 hour.
Type
Scenario To simulate the peak Load and to monitor the performance of
Objective the MERCURY SAMPLE PROJECT online system

Steps The online load will be maintained at steady state for 1 hour
with only critical transactions
Entry All the Monitors are in place
Criteria Test Data is set-up
Shakedown completed successfully

Exit Criteria Response times meet the SLA

Test completion report is agreed upon by stakeholders
Stress Testing
 Stress Test is conducted by increasing the user load gradually until the application
breaks.
 Objective of this test is to obtain the breaking point or saturation point.
 Usually once the breaking point is attained there is a noticeable increase in errors
and also there is a big increase in response time.
 This test is performed to check if application can accept spikes.

Spike Testing
 Spike Testing is considered to be subset of Stress Testing.
 It is done by increasing the user load beyond anticipated load for short periods of
time.
Scenario Stress Test
Name
Scenario Type Stress Test – Duration N/A

Scenario To objective is verify that the application can handle the projected growth
Objective and to discover the breaking point of MERCURY SAMPLE PROJECT
Online

Steps 1. Ramp up to 150% of peak load volume and thereafter continuously

increase load until breaking point is found
Break point – When the error rate is more than 20% or response times
are really high

Entry Criteria All the Monitors are in place

Test Data is set-up
Peak Load test completed successfully

Exit Criteria Test completion report is agreed upon by stakeholders

Endurance Testing

 It is also called as Soak Test.

 This test is performed for long periods of time (8 hrs, 16 hrs, 1 day, 3
day) with expected user load.
 Purpose of this test is to identify performance bottlenecks like
Memory Leaks, connection leaks etc.
Scenario Soak Test – MERCURY SAMPLE PROJECT
Name

Scenario Type Endurance – Duration 8 hour.

Scenario To discover memory issues and bottlenecks that might occur under daily
Objective usage of the application

Steps Steady state Test is maintained for 8 hour with half the Peak Load.

Entry Criteria All the Monitors are in place

Test Data is set-up
Peak Load test completed successfully

Exit Criteria Test completion report is agreed upon by stakeholders

Scalability Testing

 Test is performed using the user load considering the growth of the
application under test down the years.
 This test is performed to check the capability to scale up or scale out
in terms of User Load.
Volume Testing

 It is a load test except that huge data populated in the database.

 The data populated in the DB is expected down the years.
 Purpose of this test is to check if there is any change in the response
time with increase DB volumes.
Performance Testing Lifecycle Stages
Non-functional Requirement Gathering and
Analysis

Test Requirement Test Strategy and

Planning

Test Plan Test Design and

Development

Iterations

Test Scripts Test Execution

Bottleneck
Analysis

Test Results Test Result

Analysis

Final Test Reports

Test Reports
Workload Load Modeling
Work load

 Includes the number of virtual users and the volume of transactions per user

 Estimation for number of users required for a given volume or vice-versa

 Collect information on
 various combination of business transactions

 For each business transaction

 information collected is probable number of users performing same transactions simultaneously
 during normal operation
 peak time operations.

 Refer to below work load modeling example

Little’s LAW

N = X * (Rt)

 N = No. of Concurrent Users

 X = Throughput (TPS)
 Rt = Response time (Secs)

N = X * (Rt + Zt) (For system with think time)

 Zt = Think Time
 Industry standard for Zt is 10 secs

Rt + Zt = Script Execution Time

Little’s LAW

 For Work load Modeling, Pacing is required to be calculated. So the formula

would be

N = X * (Rt + Zt + Pacing)

Rt + Zt = SET
NFR Gathering
Infrastructure Details

 As part of NFR gathering, PT team is supposed to capture the configuration details of

both production environment and performance testing environment
 Data collected
 Operating System of each server– OS version
 Hardware configuration of each server – Number of CPU’s, speed, memory etc..
 Details of server clusters.
 Support software on each server – Web server, application server, database

 Example
 Apache Tomcat Server 8
 1.6GHz processors with 32MB Cache, 16GB Memory, 100GB Hard Disks, HP-UX and
Java ES pre-installed
Transaction Details

 As part of NFR gathering, PT team is supposed to captures list of

performance critical transactions.

 What are Performance Critical Transactions:

 Transactions which are executed Frequently.
 Transactions which are critical for Business.
 Transactions that are suspected to have high resource requirements
Transaction Details (Contd..)

Example
For a banking application, following transactions are critical
 Account Summary Details
 Checking Transaction History
 Balance Transfer
 Login
 Logout

Following transactions are not critical

 Change Password
 Change Username
 Change Theme
 Order Checkbook
 Open a new account
Scalability Related data

 Few years down the line, there is chance that user load on the application
might increase because of the business growth.

 Because of this increased load, the performance of the application might get
affected.

 So, performance Testing team is supposed to capture the future user volume
growth.

Example
 There is a 100% increase in user load annually.
Workload Related data

 Application usage patterns and volumes for each performance Critical Transaction is
captured as part of this model.
 This data is identified by
 Interviewing Clients
 Analyzing existing logs
 Example
Data related to response times and other
metrics

 As part of NFR Collection, Performance Testing Team is suppose to capture

Response Time (RT) related data.

 After the Performance test execution, the test results are compared with the
SLA to determine if the application meets performance expectations
 Examples -
 When application is subjected to 1000 concurrent user load, “Login”
transaction should not take more then 2 seconds to complete
 When application is subjected to 1000 concurrent user load, CPU
utilization of the WebApp Server should never cross 60%.
DB Data

 Database volumes affect round times for operations that access the database

 To simulate realistic load on the application the DB tables should be loaded

with sufficient number of records

 The data retention model is used to capture the key DB tables and estimated
number of records in each of these tables

 All this data will help in simulating realistic database volumes during test
execution

45