Jmeter - My Notes
Jmeter - My Notes
Jmeter - My Notes
Please refer to the below flow diagram to understand each component and their relation
to specific modules of JMeter.
Now we would start touching each component of Jmeter along with use cases just to
know how it works and how can testers implement these in their testing. Please note that
we won’t be covering all Samplers, listeners in this article. We will work on the most
used ones and will take up rest in the next article when we create real time Test Plans.
Test Plan
Just as a simple test plan in Software Testing consists of all steps which execute the
script, JMeter's Test plan has the same purpose. Everything which is included in a test
plan is executed in a sequence which is top to bottom or as per the defined sequence in
the test plan.
Test Plan can be as simple as it could be, with Just ThreadGroup, Sampler, and Listener
and it starts getting more complex as soon as you start adding more elements like config
elements, preprocessors or controllers.
You can rename Thread Group name to your own. Just change the name and click
anywhere outside the Test Plan window, you would see the name getting changed.
It is very important to configure your thread group as per the test conditions.
For example, if you want to test how a web server behaves when 100 user hits it
concurrently, you can set Thread Group as below:
Basically, there are three main parameters which must be configured to generate
actual load or virtual users:
Number of Thread(Users) – It defines the number of virtual users. For testing
purpose, we should generate only a limited amount of load as generating huge
volume at once would mean consuming lot many threads which can ultimately
lead to high CPU utilization.
Ramp Up Period – This field is very important in controlling the load generation.
Ramp up period defined the amount of time in which the total load will be
generated.
Example 1:
It means all 10 users will be hitting servers concurrently as soon as a test is run
Example 2:
You all must have noticed the “Scheduler” checkbox in above screenshot. In case
you want your test to run at a specific time later then you can set the timings as
you can also see in the below screenshot. It means that every 1 sec, a new user
will be hitting server. The load will not be concurrent but will be in increments. By
the 10th second, all user would have hit the request.
Loop Count – It defines the number of times Thread Group will execute. If you
check the Forever check box your test will run forever unless you manually stop
it. This can be used to test something like “If your server doesn’t crash on
continuous load for few minutes”.
Samplers
So, how does a Jmeter knows what type of request has been sent to server???
It is through Samplers. Samplers are a must to add to a Test Plan as only it can
let Jmeter know what type of request need to go to which server and with any
predefined parameters or not. Requests could be HTTP, HTTP(s), FTP, TCP,
SMTP, SOAP etc.
Samplers can only be added to Thread Group not directly under Test Plan as Thread
Groups need to use a sampler to send a request to server URL under test. The sampler
can be added by path Thread Group -> Sampler -> HTTP Request.
HTTP Requests
These are the most common requests sent to the servers. Say, for example, we want
100 users to hit https://www.google.com concurrently, this can be done as described in
below screenshot:
The path is the navigation inside the main website. For example, if we want to hit
http://www.google.com/gmail then we can set “/Gmail” in path and rest remains
the same
Need not type “www” in the server name
Port Number is used if you are using any proxy server
Timeout Connect and Response can be set if you want to have a benchmark on
your server connection time and response time. Your request will fail if your
server takes more time to send response than configured one
You can also configure parameters to send with your request. For example: In
some cases, you may need to send Authorization Token with your request, so
you made add them in HTTP Request as below:
FTP Requests
Path-> Test Plan-> Thread Group-> Sampler-> FTP Request
FTP stands for File Transfer Protocol and it is used to upload or download a file from the
server. JMeter’s threads send requests to FTP servers to upload or download files from
there and measures the performance.
Local File is the location where you need to save the downloaded file
Use GET option if you are downloading from FTP server
User POST option if you are uploading any file on FTP server
We have lot many listeners which will be covered when we go through some real Test
Plans consisting Samplers, listeners, config elements etc.
Listeners
So, till now we have seen few samplers sending requests to the server but haven’t
analyzed the response yet. Performance testing is all about analyzing server responses
in various form and then presenting the same to the client.
Listeners are used to display the results of test execution so that testers get to know the
stats. We have around 15 listeners in Jmeter but mostly used ones are table, tree, and
Graph.
Listeners can be added directly under Test plan or under a sampler. The difference is,
when you add listener under a sampler, it will show the results of that sampler only. If
we add sampler directly under the test plan, it displays the result for all the Sampler up in
the hierarchy.
Latency: It is the time when the first piece of information is received i.e. the first
byte of data is received
Connect Time: It is the time taken to establish connection with the server
Sample Time: It is the time taken to receive complete data
Sample – Sequence of sample number
Bytes – Size of data received.
View Results in Tree:
This is another most commonly used listeners and provides detailed information with
request and response. One can also view the HTML page rendered in response apart
from viewing Json, XML, Text, RegEx.
It is very useful as testers can put assertions on the response received to be ensured
that the test passed. Jmeter results still show “Pass” even if the response is not desired.
For Example: Say, we hit HTTP request on any website www.xyz.com and in response
we expect <title> XYZ </title> or in simple words, when we hit this page company’s
home page opens with its name. If we haven’t put assertion, Jmeter will still display
results since the hit has gone to the server.
Please see below to know the format of results:
For viewing HTML page in response, click on the drop-down in the left pane and then
select “HTML”, Navigate to the response tab and check the page which is returned as
server’s response.
Work Bench
A workbench is a place where you can store those elements which are not in use in your
current test plan but which can be later copy pasted in it. When you save JMeter file, the
components which are present in workbench are not automatically saved. You must
save them separately by right clicking and choose “Save as “option.
You all might be thinking then what is the use of workbench, anyways it is easy to add
any component directly to a Jmeter’s Test Plan.
The reason of having workbench was that user could do some experiments and try out
new scenarios. As we already know elements in workbench are not saved so a user can
literally use anything and then throw away. But, there are some “Non-Test Components”
which are only available in WorkBench.
Jmeter only records the request sent to the server. Don’t get confused with “Record and
Play” functionality of QTP/Selenium. All the requests are recorded and testers can apply
the desired load on them to see the behavior.
This element is very important for scenarios where testers don’t know what all requests
are going on from their application. They can use Http(s) script recorder to record the
application under testing.
Performance Testing of mobile applications can also be done in this way by setting up
JMeter proxy and then recording the requests which our mobile app sends to the server.
Step by Step procedure for mobile performance testing will be explained in next article.
Assertions
Till now, we have covered how JMeter hits the server and how the responses are
displayed via listeners. To ensure that the response received is correct and as per
expectation, we need to add assertions. Assertions are simply validations which we
need to put on responses to compare the results.
Case: 10 users are concurrently hitting “google.com” server and Duration Assertion is
set to 1000ms.Please see below screenshots:
XML Assertion validates if response data has correct XML document in it and HTML
Assertion verifies the HTML syntax of the response received from a server.
Config Elements
Requests sent to the server can be further parameterize using some config elements
which are executed before the actual request. A simple example of it could be reading
values of a variable from a CSV file for which CSV Data Set Config is used.
Below are some of the important config elements used in the performance testing
of the web and mobile applications
CSV Data Set Config.
User Defined Variables
HTTPS Requests Default
HTTPS Cache Manager
HTTPS Cookie Manager
CSV Data Set Config
CSV data set config helps Jmeter in picking values of some parameters from a CSV file
rather than passing different parameter in each separate request. For example, if we
need to test login functionality with a different set of users and passwords, then we can
create two columns in a CSV file and enter the values there so that JMeter can pick one
for each request sent to the server.
Below is the flow of using CSV data set the config to test weather API for different cities
in India.
In this scenario, if we add an HTTP Request Default then we need not specify a server
name, path or any other properties like port number, connection time out properties.
Whatever is specified in the HTTP Request Default config element is inherited by all
HTTP requests.
Please see below how to add HTTP Request Default and specify server and path.
HTTP Cache Manager and HTTP Cookie Manager are used in making JMeter behave
as a real-time browser. HTTP Cache manager can clear cache after each request
whereas the other one can manage the cookies settings.
Logic Controllers And Timers
Logic controllers and timers help Jmeter control the flow of transactions. Timers ensure
the delay in each thread if need to test any server. For Example, if we need FTP request
to wait for 5 secs after HTTP request is completed, we can add timer there.
Logic Controllers are used to defining the flow of requests which are sent to the server. It
can also let you store requests for each module separately such as login and logout.
Pre-Processors
These are the elements which are executed before a sampler is executed. You can
attach Pre-Processor with the sampler for which you want to make some changes in the
request.
The simplest of a case could be adding “Sample Timeout” preprocessor with an HTTP
Request so that this request would run for only defined amount of time. It also makes
sure that this modification is done only for the parent Sampler.
The simplest of a case could be adding “Sample Timeout” preprocessor with an HTTP
Request so that this request would run for only defined amount of time. It also makes
sure that this modification is done only for the parent Sampler.
User Parameters
User Parameters are used to define values for the variables before they are used in the
Samplers. When JMeter executes this Pre-Processor element, it stores the values in the
variables which can be referenced by any Samplers within the same thread group.
If you have more threads than the number of users in “User Parameters” than extra
threads will re-iterate through the values again.
For Example, you have 5 users in your thread group but only 3 in pre-processor, then
your 4th and a 5th thread will use param1 and param2.
For example, if you have put a sample time out of 400 milliseconds, then all requests
which are taking time>400 will have a failed response.Please see below screenshots.
Controllers
Controllers are very important in building a real time JMeter test plan. It defines the
sequence in which the requests are sent to the server.
For example, If you want to test a web application in which you want login to happen
once and search, item selection requests to go one by one for all iterations. Controllers
make it possible by managing the flow of request going on the server under test.
Below are the most commonly used controllers in JMeter
Simple Controller
Module Controller
Once Only Controller
Interleave Controller
Loop Controller
If Controller
For Each Controller
Recording Controller and etc.,
Simple Controller
The simple controller does not perform any specific function. It is just a kind of container
in which you can keep your similar requests to make your test plan easily
understandable.
Loop Controller
If you want some specific request to run more iterations than what is specified in Thread
Group, you can put them under Loop Controller and enter the loop count in controller
settings.
Example: If you have a thread group with 1 user and 3 iterations, then all your requests
under this group will run for 3 times. Now, if you have two HTTP Sampler under a loop
controller with loop count 2, both these Samplers would run 1*3*2= 6 times.
Please see the below screenshots which explain it further
Once Only Controller
This controller is used when you want to run some specific request only one time even if
you have multiple threads in the thread group. The simplest example which can be
considered is “Fetching a website home page” or “Login into a web application”. Real
Time scenario would want it to happen once only and other requests such as search or
edit/delete something to happen multiple times.
Request which is to be executed only once can be placed under Once Only Controller.
Refer below screenshot for your reference. Once Only Controller settings bypass the
parent thread group settings.
Recording Controller
Just like Simple Controller, Recording Controller does not modify any sequence of the
requests that are being sent to the server. It is used with HTTP(S) Script Recorder. All
the requests which are recorded with this Non-Test Element are saved under recording
controller.
You need to specify the target controller to save the recorded requests made to the
server.
Recording controller and HTTP(s) script recorder comes into picture when testers don’t
have the information on the requests URLs and parameters. They can simply record and
capture all the requests hitting their servers. This works both for mobile as well as web
projects.
Throughput Controller
This controller is also used to control the execution flow. As seen in the below clipping,
this controller is further divided into two parts:
Percent Execution – This selection will make Jmeter execute only a certain percentage
of total iterations for Samplers placed under this controller. You can also check the “Per
User” checkbox to control this at a user level.
For Example, Thread Group is configured to have 10 users and loop count is 5.
Therefore, total iterations are 50. If % execution is set to 50%, then all the Samplers
under the Throughput Controller will make 25 iterations only (50% of 50).
Total Executions – This lets users specify the number of iterations directly for the
samplers contained under this controller.
Interleave Controller
This controller lets you increase the scope of your performance testing by modifying
sequence in n number of ways so that you get to test the load on the server when
applications are hitting request with different sequences. Interleave Controller makes
alternative selections with the samplers under it.
In case there are other controllers like Simple Controller kept under this block, interleave
controller gives the privilege to select one sampler each from a container for each
iteration. To explain it further, consider the below test plan.
So, Jmeter will iterate alternatively between the controllers for each iteration. Therefore,
sequence will be mentioned below
Request1 -> Request3 -> Request 5 – Request2 -> Request4 -> Request6
Please see the below JMeter Results with Number of Threads = 2 and Loop Count = 2.
In above example, Jmeter is Interleaving with next loop/iteration. There are settings in
Interleave Controller that can make JMeter send alternate requests with every new
thread.
Please see the results below. This is useful in scenarios where you want sequential or
concurrent load hitting servers with the different sequence of the request from the same
machine.
Random Controller
It works almost like Interleave controller but does not pick the samplers in order. It just
selects the sub controllers and the samplers inside them on a random basis.
If Controller
If Controller works in a similar way an IF expression is treated in any programming
language. The condition is validated first and then the components under this container
execute if the condition is TRUE, otherwise, the elements outside the IF Controller will
get executed.
Example: Continuing with one of the earlier examples involving Interleave Controller,
now one of the two Interleave controller has been kept under the IF Controller and
Condition “$(COUNT) < 10 is added as a condition. Variable COUNT is defined in config
element “User Defined Variables” and is given a value equal to 11. Configuration is
described below
Defining condition in If Controller settings. Jmeter will validate this condition and if
TRUE, will execute the blocks under this container.
As you can see from below test plan, only Request3, Request4 and Request5 will
execute in case IF Condition is evaluated as False.
While Controllers
This controller executes the components under it till the condition becomes false.
For example, say we have a condition while($(count)<10), it will execute the child
elements until the condition is false. To test this, there should be a counter which gets
increment or decrement on each iteration and then condition is evaluated. Config
Element “Counter” can be used to serve this purpose.
Configure “Counter” config element to start at 1 and then increment at each iteration.
See below for more clarification:
We will touch upon rest of the controllers in our video tutorials.
Real Time Example
Let’s consider a scenario in which testing team does not know the URLs and parameters
of all the request hitting the server under test. Now, to load test the server they need to
record the requests and then modulate the load sequentially or concurrently and create
scenarios out of them to do the performance testing.
Steps
Add a template in Jmeter for Recording Controller
Set the proxy on your browser and choose the same port in HTTP(s) Script
Recorder
Record the requests
Modify the Test Plan and increase the load
Group the transactions accordingly
Try different combinations of request
JMeter'secording Template
JMeter also has some predefined template. Select “Template” option from File Menu.
After selecting the template to choose “Recording” option in the window which opens
and click on Create.
As soon as you select this Recording Template, you would see some components
already added in the Test plan.
Click on. jmx option to export the recorded request to Jmeter. Now from Jmeter, open
the. jmx file and see the requests and the parameters associated with each request.