Ruben Getting Started
Ruben Getting Started
6
In part 2, we do a (simple) exercise
… to get you started on monday
8
Skillset to using JMeter
JMeter is tech – you need some technical skills to survive
9
Skillset to using JMeter
Testing web application require “web knowledge”
10
Skillset to using JMeter
JMeter is tech – you need some technical skills to survive
11
Introduction to JMeter
13
How JMeter works
Data is collected
Server sends
for statistical
replies back
purposes
14
Distributed (scaled) setup
Load generators
15
JMeter in a nutshell
16
JMeter in a nutshell
• SOAP
• JMS
• POP3(S)/IMAP(S)
17
JMeter in a nutshell – 2
18
JMeter in a nutshell – 2
übercool
Pack & Go
19
20
The JMeter Toolbar
21
JMeter lingo
22
JMeter functionality linguistics
The Big Picture
• Thread Group
• Number of concurrent users, ramp up time, number of
iterations, etc
• Controllers
• The test plan logic
• Samplers
• Testing something, and getting a response
24
Elements of a test plan - 2
You do not really need these – but they will save your ass
• Configuration elements
• Default values, setups, variables, test data feeds, etc
• Pre processors
• Amend data before sending a request
• Post processors
• Extract data after receiving a response
25
Elements of a test plan - 3
Our nice to have building blocks
• Listener
• Reports and graphs
• Timers
• Emulate human behavior / think time
• Assertions
• Control and compare responses
26
A test plan example
• • Configuration elements
• Thread Group
• Controller + sampler
• Sampler
• Controller
• Override configuration
• Samplers
• Assertion
• Listeners
27
Test plan inheritance
28
Building blocks
29
Samplers
What’s possible to query
• HTTP/HTTS Request
• JDBC Request
• TCP Request
• Reading and sending Mail
• … and many others ...
30
Controllers
Logical building blocks for your test plan
31
Data parameterization
• CSV file(s)
• User defined variables
• Randomized values
• Amending existing data by pre processing
• Extracting data by parsing responses
32
Data parameterization
Pre Processors
• XPath Extractor
• Regular Expression Extractor
• JSON Path Extractor
• Results Expression Extractor
• JSON Format Post Processor
• CSS/JQuery Extractor
• JDBC Post Processor
• JSR233 & BeanShell Post Processor 34
Assertions
Sanity checking your responses
35
Odd – but very important
36
Server Side Resource Monitoring
37
38
Java Application Server Monitoring
39
Java Application Server Monitoring
40
Plugins
41
http://jmeter-plugins.org/
42
Part 2 of the workshop
Practial session
44
What we are going to play with
The 95% usage pattern
45
What you need to participate
(unfortunately, we cannot assist you during the break)
46
Our demonstration application
47
The Spring PetClinic Application
Please go ahead and
play
• List owners and animals
• Add animal visit
• Add owner
• Add animal
• List veterinarians
• Home Page
http://10.0.1.100/petclinic/
• (ERROR)
Download from https://github.com/spring-projects/spring-petclinic
48
Tools for developing a JMeter test plan
49
A few ways to achieve the goal
50
Web browser tools
Avoid if you can, use if you must
• Firefox Addons
(FireBug, HttpFox, FireFox Live HTTP Headers, etc)
• FirefoxDeveloperEdition
• Internet Explorer F12
• Safari Web Inspector
• Chrome Developer Tools
51
A few 3rd party tools of the trade
Use if you can
• Betwixt – github.com/kdzwinel/betwixt
• Burp Suite – www.portswigger.net
• Charles – www.charlesproxy.com
• Fiddler – www.telerik.com/fiddler
• HTTP Scoop – www.tuffcode.com
• ZAP - www.owasp/index.php/ZAP
52
Using Blazemeter Chrome Extension
https://blazemeter.com/
53
Workshop method
54
This is how we are going to work
55
Recording a session with JMeter
56
How we are going to do it
57
Start JMeter
58
Start JMeter
59
Save the test plan as ”recorder”
60
All test plans needs a Thread Group
Threads == Number of Concurrent Users
61
Thread Group Options
62
Thread Group Basics
Pro Tip:
Initial Ramp-up period == number of threads
63
Different Thread Groups
• Thread Group
• jp@gc – Stepping Thread Group
• jp@gc – Ultimate Thread group
• setUp Thread Group
• tearDown Thread Group
64
Stepping Thread Group
65
Ultimate Thread Group
66
We need a recording controller
67
This is where we store our recordings
(an exceptionally boring controller)
68
Let’s set up the Proxy
69
The Test Script Recorder Controller
Consider changing
this to port 9090
Pro Tip:
You can record to any place in the test plan
70
Add include/exclude patterns
71
We want to see what’s happening
Add the View Results Tree Listener
72
The View Results Tree Controller
73
Remember to
SAVE
your plan!
74
Starting the JMeter proxy
75
Recording with your browser
76
Configuring the browser proxy settings
77
Pro Tip:
Use a Proxy selector or switcher
78
What shall we test today?
How will the PetClinic application perform when we
1. Search for an pet owner
2. Add a visit to
one of owners
animals
79
Our Test Plan
81
Step 2 – Click on Find Owners
82
Step 3a – Enter an owners name
83
Step 3b – Click on
84
Step 3b – Click on
85
Step 3b – Click on
86
Step 3b – Structured request
87
Step 3b – Raw or structured?
88
Step 3b – The Response
89
Enough with Step 3b!
Let’s continue!
90
Step 3c – Owner info + pets
91
Step 4 – Add a visit
Click on the Add Visit link
92
Step 4a – fill out the visitation form
93
Step 4b – click on
94
Step 4c – Back to the owner + pet
How this this happen?!
95
Step 4c – HTTP Redirect
Status code 302 + Location: header
96
Step 4c – HTTP Redirect
Status code 302 + Location: header
97
Step 4d – The Real McCoy
Nothing special
98
Step 4d – The Real McCoy
This Is The Place
99
Congratulation!
You have a successful recording!
100
Remember to
save your plan
with a new name!
Name: step1
101
Examples of other proxy tools
102
A few 3rd party tools of the trade
Use if you can
• Betwixt – github.com/kdzwinel/betwixt
• Burp Suite – www.portswigger.net
• Charles – www.charlesproxy.com
• Fiddler – www.telerik.com/fiddler
• HTTP Scoop – www.tuffcode.com
• ZAP - www.owasp/index.php/ZAP
103
Charles – charlesproxy.com
104
Burp Suite – Portswigger.net
105
Removing unwanted junk
106
This is how we are going to work
107
What is unwanted junk?
• JavaScript (.js)
• Images (.png, .jpg, .gif)
• CSS (.css)
108
What to keep and what to junk
.CSS
.JS
.PNG
109
Remove this!
110
What about the HTTP Redirect?
BTW – we do not
need these anymore
111
This is where you should end up
DO IT NOW
112
Congratulation!
You have cleaned your plan!
113
Remember to
save your plan
with a new name!
Name: step2
114
Mangle the test plan
115
This is how we are going to work
116
Our starting point
117
Keep some some config elements
119
Anatomy of the HTTP Reqeust Sampler
120
HTTP POST Request
121
What do all HTTP Request have in
comon?
• HTTP host
• HTTP port
• HTTP protocol
122
Add a HTTP Request Default config
123
HTTP Request Default config element
124
Amending the HTTP Request – part 1
We need to do this for all HTTP Request samplers
125
Amending the HTTP Request – part 2
Drag the sampler directly under the Thread Group
126
Amending the HTTP Request – part 3
Rename the sampler to a logical name (home page)
Pro Tip:
Prefix the name with a sequence number
127
Let’s repeat for all HTTP samplers
128
The web form
129
The search results
130
Parameterization
131
The search result is based on input
132
We need to parameterize
JMeter variables
• Syntax ${VariableName}
• Several sources
• User Defined Variables
• Data from CSV files
• Extracted from sampler responses
133
Adding Test Plan variables
Pro Tip:
Never add variables here, it’s confusing
134
Test data
Rename to
Test Data Variables
Pro Tip:
Add variable sets for the environment(s)
Add variable: OwnerLastName
Value: Davies
135
Environment Variables
136
Parameterizing the search results page
137
Parameterizing the HTTP defaults
138
Pro Tip:
Try to identify all variables before you start
139
Let’s mangle some more
140
Show owner
Parameterize
Remove this IDs
any Session
${OwnerID}
141
Show visitation form
Parameterize this
Move below 400
Remove default values
Rename to
500 Show visitation form
Exercise:
/petclinic/owners/2/pets/2/visits/new
1) Rename form to 500 Show visitation form
2) Remove default values ${OwnerID}
/petclinic/owners/${OwnerID}/pets/${PetID}/visits/new
3) Move to correct position below the Thread Group
4) Parameterize the Path (hint: you already have ${OwnerID})
142
Save visitation
Parameterize this
Move below 500
Remove default values
Rename to
… and these
500 Show visitation form
Exercise:
1) Rename request to 600 Save visitation data
2) Remove default values ${OwnerID}
3) Move to correct position below the Thread Group
4) Parameterize what ever needs to be parameterized
143
Remove the Recording Controller
144
Congratulation!
You have created your Test Plan!
145
Remember to
save your plan
with a new name!
Name: step3
146
Testing the Test Plan
147
This is how we are going to work
148
Debug with Results Tree Sampler
149
Test the plan – press Play
150
Inspect the results
151
Change sampler, and re-run
152
Inspecting the results
153
Why do this happen?
Pro Tip:
You want Follow Redirects most of the time
154
Congratulation!
Your plan is free of bugs!
155
Remember to
save your plan
with a new name!
Name: step4
156
Next steps
157
Where to go next
We will also:
• Extracting data from a response
• Using the extracted data as input to a test step
158
But first – a new exercise
Extend the current test plan by:
Hints:
1. Disable all the test steps you have so far
2. Start a new JMeter, and record the steps
3. Use the recorded steps in your primary test plan
159
Recording stuff
Two interesting screens:
Add Owner and Add New Pet
160
The recorded owner
Rename to
Remove
Copydefault
to the values
other JMeter
700 Add New Owner
Ctrl-C (win) Cmd-C (Mac)
161
The recorded owner
162
The recorded pet
Rename to
Remove default values 800 Add New Pet
Copy to the other JMeter
Ctrl-C (win) Cmd-C (Mac)
163
The recorded pet
164
There is a slight problem here!
Start by disabling
800 Add New Pet to Owner
166
Let’s investigate - 2
Investigating the data
167
Let’s investigate - 3
Extracting data from the results
168
Let’s investigate - 4
Data extraction using Regular Expressions
Pro Tip:
9 out 10 times you want the extraction to
apply to Main samples and sub-samples
169
Let’s investigate - 5
We need a debugger
170
Let’s investigate - 6
Clean-up and run
171
Let’s fix 800 Add New Pet to Owner
${OwnerID}
172
Clean up and run 3 times
173
Congratulation!
Your plan is even cooler now!
174
Remember to
save your plan
with a new name!
Name: step5
175
The last preparations
176
This is what we are going to do
177
Disable / enable stuff
We need to get back to sane configuration
DO IT NOW
178
Dynamic date
179
We want to use the current date
… by using a JMeter function
180
Built in JMeter functions
Kind of function Function name Description
Information threadNum get thread number
Information log log (or display) a message (and return the value)
Information
Java SimpleDateFormat
logn log (or display) a message (empty return value)
String unescape Process strings containing Java escapes (e.g. \n & \t)
183
Dynamic date
${__time(yyyy/MM/DD)}
184
Let’s run this
Clear your plan – and press Play
185
Debug your plan – part II
186
Adding some debugging
… but also prepare the plan for real load testing
Pro Tip:
Always add the debugger on the thread,
or on the plan level – then you have control
187
Let’s run again
188
Let’s run again
${__time(yyyy/MM/DD)}
${__time(yyyy/MM/dd)}
189
Let’s run again
190
Test data from a file
191
petclinic_data.csv
Owner, ID, Owner Name, Pet ID
1,Franklin,1 6,Coleman,7
2,Davis,2 7,Black,9
3,Rodriquez,3 8,Escobito,10
3,Rodriquez,4 9,Schroeder,11
4,Davis,5 10,Estaban,12
5,McTavish,6 10,Estaban,13
6,Coleman,8
192
What we are going to do
193
1. Duplicate our Test Data Variables
194
2. Rename the Config Element
DEVELOPMENT Test Data Variables
195
3. Rename the other Config Element
TEST RUN Test Data Variables
Pro Tip:
use INITIAL
Always declare VALUE
all variables
196
4. Add the CSV data reader
197
5. Configure the CSV Data Set Config
Pro Tip:
Parameterize this – do not hard code name
198
6. Getting ready to test
199
7. Adding another debugging element
200
8. Clean test plan, then Run Plan
201
9. More reports
=> more Listeners
What do you need to measure?
• Response times (over time)
• Response Times Percentiles
• Response Time Distribution
• Response Codes Per Second
• Response Latencies over Time
• Transaction Troughput vs Threads
• Simple Data Writer 202
You add some listeners
• Summary Report
• Response Time over Time
• Response Times Percentiles
• Response Time Distribution
203
Congratulation!
Your plan is done!
204
Remember to
save your plan
with a new name!
Name: final
205
Let’s do some testing!
206
Let’s run and look at reports
Pro Tip:
When performing real testing, disable all
debugging
207
Let’s re-run and look at reports
Summary report
208
Response Time Distribution
The Main View
209
Response Time Distribution
Selecting rows
210
Response Time Distribution
Back to the Main View
211
Response Time Distribution
Settings view
212
Response Time Distribution
Back to the Main View
213
Response Times Over Time
214
Response Time Percentile
215
A real test run example
216
Stepping Thread Group
217
Active Threads Over Time
218
Simple Data Writer
219
Summary Report
Pro Tip:
Look for errors, consistency and outliers
220
Response Times Distribution
221
Response Times Over Times - 200
222
Response Times Over Times - 300
223
Response Times Over Times – ex 300
Pro Tip:
Requests that follow each others are good
224
Response Times Percentiles - 1
Pro Tip:
Start big, then drill down
225
Response Times Percentiles - 2
226
Response Times Percentiles - 3
227
Response Times vs Threads - 1
Pro Tip:
We hate fluctuations!
Fluctuations == Broken software
228
Response times vs Threads – 2
Pro Tip:
Stable response times with increased
threads => we have a stable application
229
Response times vs Threads – 3
230
Response Codes per Second
231
Transaction Throughput vs Threads
Pro Tip:
Look for correlations in the data
232
Transactions per Second
233
Transactions per Second
234
Testing REST APIs
236
Some technical data
237
Some technical data
238
User Defined Variables
For test data
239
User Defined Variables
For the test environment
• HTTPServerName http://10.0.1.100
• HTTPServerPort 80
• HTTPServerProtocol http
• ContextRoot /rest/holiday-requests
GET /rest/holiday-request
POST /rest/holiday-request
PUT /rest/holiday-request
DELETE /rest/holiday-request/${ID}
240
Start a new JMeter project
Copy your last JMX and delete what you do not need
242
The HTTP Request Defaults
GET /rest/holiday-request
POST /rest/holiday-request
PUT /rest/holiday-request
DELETE /rest/holiday-request/${ID}
243
User Defined Variables
For test data
244
User Defined Variables
For the test environment
• HTTPServerName http://10.0.1.100
• HTTPServerPort 80
• HTTPServerProtocol http
• ContextRoot /rest/holiday-requests
GET /rest/holiday-request
POST /rest/holiday-request
PUT /rest/holiday-request
DELETE /rest/holiday-request/${ID}
245
Let’s build the test
246
Fetch Employee Holidays
GET /rest/holiday-requests
247
Let’s run
Remember to add a View Results Tree Listener
248
A better view
249
Create new vacation
POST /rest/holiday-requests
{
"employeeId": 688,
"employeeName": "Stephen Lee",
"startDate": "2014-07-01T00:00:00Z",
"endDate": "2014-07-08T00:00:00Z"
}
250
Create new vacation
POST /rest/holiday-requests
{
"employeeId": ${EmployeeID},
"employeeName": "${EmployeeName}",
"startDate": "${VacationDateStart}",
"endDate": "${VacationDateEnd}"
}
251
Create a new vacation
Uinsg JSON and POST data
252
Let’s run
HTTP Content Type MAY matter
253
Add a HTTP Header Manager
254
Let’s run
Hmmmm……
255
Let’s run
Yay! Success!
256
Using the JSON Path Extractor
Post Processors for the win
257
Use a proper JSON tool
Power JSON Editor
Pro Tip:
Get good tools for bulidng JSONPath, Xpath,
and Regular Expressions
258
Using the Regular Expression Extractor
Post Processors for the win
Pro Tip:
Regular Expr. Extractor is supposedly faster
259
Modify vacations
PUT /rest/holiday-requests
{
"id”:6,
"employeeId": 688,
"employeeName": "Stephen Lee",
"startDate": "2014-07-01T00:00:00Z",
"endDate": "2014-07-08T00:00:00Z"
}
260
Modify vacation
POST /rest/holiday-requests
{
"id":${id},
"employeeId": ${EmployeeId},
"employeeName": ”${EmployeeName}",
"startDate": "2014-07-01T00:00:00Z",
"endDate": "2014-07-08T00:00:00Z"
}
261
Modify vacation request
REST payload are always done as a HTTP body!
262
Let’s run
Yay! Success!
263
List all vacations – once more
GET /rest/holiday-requests
264
The Module Controller
Modularization for the win!
266
Reorganize!
Do a test run to see if it’s still produce the expected results!
267
Include the module Controller
What will we name this step?
Pro Tip:
Continue the number sequence
400 Fetch Employee Holidays
268
Let’s run to see if we are still on track
269
Delete the Employee Holiday request
DELETE /rest/holiday-requests/${id}
270
Another run for success?
JMeter honour HTTP response codes, not the payload!
271
Wrap up with listing all holidays
272
Takk for meg
Ruben Olsen Lærk