o r d e v e l o p e rs

e a rc h i te ct ure f
Softwar

What is software What is therole of How do you define How do you share How do you deliver
architecture? a software architect? software architecture? software architecture? software architecture?

A developer’s guide
to load testing
[email protected]
@simonbrown on Twitter
Software Architecture for Developers (.com)

What is software
architecture?

What is the role of

a software architect?

How do you define

software architecture?

How do you share

software architecture?

How do you deliver

software architecture?
 Who
shou
all o ld d
Simon Brown f o
wa rethi s?
Hands-on soft
architect It sh
ould
ftware Projec
ts Fail arch be t
he
Why So
itec
t
(som
ebo
dy
we g has to d
et p o it
aid and
the that
big
.com buck ’s why
www.codingth
earchitecture s)

The
role
of a
soft
Man
a
non gement war
-f
requ unction f
o
e ar
irem
ents
al Arch chit
Tech
it
defin ecture
ition
Own
ect
n er
sele ology the ship of
ctio big
n Arch pict ger
it ure
evalu ecture Lead
atio ersh
n ip
Coa
Arch ch
coll itectur men ing and
abo torin
ratio e g
Qua
n
assu lity
Defin ranc
e

...architects ar
ition
De
deve sign, e here
www
.cod
This
writ is wha
lo
and pment
test hand A
e ab t we ing
s-on
ingt

help, not to hinder

hearc ou
hite web t on ou

hite softwar
Deli

fail...
cture

a
site rto very
r
.com ...

Why software
projects inva c
luab c e
le for t can be
proj p
ect revent
failu ing
re

soft hands-
A

war o
e ar n
proj chit
can
be i

ect
nvalu
a
ect
ble
for d
rivin
succ g

ess Iterative agile and

techniques

..
a il me problems.
http

are p ro je cts f so lv e so
://w
ww. Webs
codi i
ngth te
Softw s Lon earc

ber of reason
Mon h itec
thly
mix don ture

for a num
of p Use .com
rese r Gr
http ntat
io
oup
://g n an
roup Goo d di
s.go gle scus
sion
o gle. Gro
c om/ up
codi
ngth
earc
hite
ctur
e
The role of a software architect

Management of Ownership of
Architecture
non-functional the bigger Leadership
definition
requirements picture

Technology Architecture Coaching and Quality

selection evaluation mentoring assurance

Design,
Architecture
development
collaboration
and testing

Definition Delivery
Why is quality
important?
Reputation
of the business
(e.g. driven by customer satisfaction)
 Service level agreements
& key performance indicators
(e.g. between suppliers, between systems,
non-functional requirements, etc...)
Reputation
of the development team

It’s important It’s important

that we know that the
software we
what release
we’re releasing
“works”
That’s us!
 What is
performance?
Performance is about how

fast something is
Response Latency and

time throughput

p p s,
w eb a
e t o e m s ,
ca b l y st
li ing s
App s ag
, m es t c .. .
UI s s , e
SO A
 What is
scalability?
Scalability is about

doing more
(more requests, more data,
more users, more messages, ...) y
e n t l
in h er
i t y is c y
a b i l r r e n
Scal co n c u
u t
abo
Vertical Horizontal
scalability scalability
(scale-up) (scale-out)

n en ts
o m po ;
A l l c nt l y
if f ere
a le d a s y
sc is e
e- u p
s c a l
 22:26
22/09/2009

ility Principles
InfoQ: Scalab

Article

ty Princ iples
Scalabili May 21, 2008 Tags Concurren
cy
requests, ex
ecuting more
ents its own
set
on Brown on
Posted by Sim mance & Scalability
po nd ing to more user lot s of work pres
Topics Perfor is co uld be res pable of do ing
hitecture ng . Th are ca
Community Arc of somethi king that so
ftw
doing more mplexities, ma lable software systems.
ility is about are has its co
st level, scalab signing softw for building
sca
At the simple ta. Wh ile de d gu ide lin es
ling more da principles an
work or hand esents some
This article pr
of problems.
rContent
RelatedVendo
ance
tion Perform
sure Applica d Web 2.0 Ap
plications
5 Ways to En
tec tion for REST, SOA, an
siv e Th rea t Pro
Comprehen
r
ject Manage
The Agile Pro
ecklist
The Agile Ch
erview PDF ple,
® Platform Ov e. For exam
Fla sh its to complet e.
Adobe® dual work un amount of tim
tim e tak en for indivi req ue sts in the same
ce ss ing tim e to de cre as e the nd le mo re user
is
1. Decrease pro plication do
es
you are able
to ha
rk that an ap means that ategies.
amount of wo user request realisation str ta and the co
de.
increase the to process a some possible cating the da
One way to e req uir ed ria te an d
the am ou nt of tim
this pr inc ipl e is approp
a pie ce of work, by collo over and ov
er again.
decreasing s of wh ere req uir ed for
d of fet ch ing it
example fetching data the overhea
Here are some sociated with ta to reduce dual steps.
overheads as , cache the da ng them. ng the indivi
: reduce any be collocated rces by pooli and parallelizi rtitions.
Collocation the co de ca n't
using expe ns ive res ou the pr ob lem g rel ate d pa
the data and work by deco
mposing d collocatin . It's also
Caching : if ea d as sociated with a un it of tio nin g the code an mo re co arse- grained
uce the ov erh complet e ble, by pa rti interfac es uting - do
Pooling : red as e the time taken to clo se tog ether as possi ex am ple , making the law of dis tributed comp
on : de cre essing as by, for r the fir st
Parallelizati related proc ote services d to conside
: concentrate t accessing rem decision not a switch an
Partitioning of time spen
g : red uc e the amount al is an ex plicit design are great too
ls for
Remotin ote vs loc se concepts between
rth rem em bering that rem en no t req uired. Yes, the ly if yo u'r e converting
wo
your objects. y are oft , particular abstractions
aren't too
not distribute ns an d lay ers where the d im pa ct performance re tha t the use,
e abstractio complexity an
ised is to en
su granted beca
d to introduc y to increase can be minim t we take for
ftw are de ve lopers, we ten bu t the y ha ve a tendenc wh ich pr oc essing time st of run tim e services tha
pli ca tio ns .
As so onents, er way in the co r ap
software comp layer. Therefore, the oth derstanding enecks in ou
decoupling , it's worth un ing the bottl
tations at ea
ch . In addition uld end up be
data represen no t too much layering po ssi ble that these co
that there' s it's
abstract and el agreement, when
ific service lev your system
have a spec to scale out rs to
unless they im ate ly you'll need ad dit ion al web serve
so far, bu t ult starting up
will get you be as easy as rt to
ion wo rk un it ou t co uld ll arc hit ec ture will sta
2. Pa rtit lar cation, scalin
g era those
with a particu rts of your ov g behind all
e associated ical web appli ver, is that pa server sittin gy.
ing the pr ocessing tim s de plo ym ent. In a typ yo u mi gh t find, howe ple is a sin gle database op t a pa rtitioning strate
Decreas gle proc es m. What od exam s is to ad nt into
u rea ch the limits of a sin d ba lan cin g between the
sy at the sa me time. A go ch an d on e way to do thi
tio nin g tha t single eleme
yo and loa ll get bu approa able chunks
. Parti ir architectur
es
er requests everything wi change your
handle the us on because you have to more manage ay use to en
sure that the
s of contenti bottleneck, into smaller es such as eB
become point t sta rts to become the ce of the architecture t lar ge sit
When tha that single pie ique tha
web servers. breaking up tly the techn . tween
this involves d this is exac ff consistency d balanced be
Put simply, them out an t you trade-o the work loa maintained,
you
ks all ow s you to scale ho ug h yo u may find tha ply be sca led out and ne ed s to be
smaller chun solution, alt s componen
ts can sim state that each instance
is
rti tio nin g is a good uly sta tel es oth er ha nd, there is en ts, wh ere
scale. Pa s. Tr If, on the on
ll that depend tive manner. stateful comp
ur system, we ning in an ac ces of those
u partition yo mponent run multiple instan
As to how yo ins tan ces of the co t wi ll all ow you to have
y with all strategy tha
them, ideall partitioning ta.
a workload rk and/or da
need to find distinct subs
et of the wo versions of
ibl e for a as the early gle -threade
d.
respons logies such
me time. Techno en ts that were sin ht
is ab ou t co nc urren cy
mo re wo rk at the sa
d us to wr ite co mp on
co nc urr en cy isn't thoug
3. Scalability about doing d encourage oblems. If t are
nc urr en cy ; after all, it's og ram mi ng model an s led to co ncurrency pr en cy lea d to systems tha
out co plified pr d thi ncurr
inherently ab provide a sim mponents an s around co t there are so
me
Scalability is attempted to s on other co er hand, too
many guard rd to do, bu
aBeans (EJB) dependencie d. On the oth isn't that ha
Enterprise Jav mp on en ts typically had me co rru pte urr en t pr ogramming
ly, these co easily beco n scale. Co nc
Unfortunate ere data can which they ca
ve systems wh limited in the degree to as possible.
about, you ha e and le systems. as little time
work
l in na tur sca lab by scheduling
essentially se
ria when build ing hold them for g path (e.g.
pr inc ipl es that can help as e ob jec ts, etc), try to of the cri tical processin tial
simple objects, datab off where poten
ks (e.g. local y contention d safely and
try to take an s can be share
yo u do ne ed to hold loc sh are d res ources and ich res ou rce
If on of d wh
ize contenti ll understoo
Try to minim so that it's we
sly). ne up -front,
asynchronou urr en cy ne eds to be do
conc
Any design for ll be. al requiremen
ts
ttlenecks wi the function
scalability bo aim ing for. While bu ild a piece of
at yo u'r e inely ne ed to
must be know
n als are and wh you do genu s.
what your go lly absent . If ents/workflow
4. Requirements need to know that are usua tical compon
so ftw are system, you ts (or sy ste m qualities) ng s up -fr ont for the cri
ul emen of thi
ild a successf ctional requir lowing types
In order to bu the non- fun rstand the fol
ll- known, it's need to unde
are often we hly sca lab le, then you cy , etc ).
t is hig e time, lat en
software tha (i.e. respons volumes, etc
).
performance ers, message
ge and peak concurrent us
Target avera loa d (i.e .
ge and peak scalability. Page 1 of 4
Target avera rmance and
its for perfo
Acceptable lim

y-principles
icles/scalabilit
foq.com/art
http://www.in

http://www.infoq.com/articles/scalability-principles
 What do you
want to know?
d w h y?)
(a n
 s s by
g u e
o nl y or
c a n r a m s
You a t di a g
in g de
look r ce c o
u
Will these software systems s o

perform and scale acceptably?

“
This new system will work
because we have a

spreadsheet
showing performance figures
from a past project
 Testing provides

confidence
 Load testing is one way to

evaluate
your architecture
(if performance and/or scalability is important to you)
 d t e st e a r ly
Loa
You should have a

reason
for load testing
What do you want to

learn
about your system? it?
st is
w f a a le?
H o i t s c
d o e s e ak?
b r
How o es i t
e re d
W h
What do you want to

prove
about your system? s a r e X .
ti m e
o ns e s Y.
p i
Res u g h p u t
s.
r o s e r
Th Z u
l es to
Sca
 What is
load testing?
Load testing is asserting how the architecture
performs under load with a view to
monitoring the response times for key transactions
 Stress testing is asserting what the upper
bounds are for the scalability of the architecture,
understanding how it reacts when stressed
 Soak testing is asserting that the
performance of the architecture remains stable
over longer periods of time
 Load testing client
simulating concurrent Website
user access
p u blic
n a l or H P ,
r , P
Inte T , J av a
, ...
; .N E C R M
in g in t ,
fac ar e P o
y, S h
Rub

Simulate multiple users

with a
typical usage profile,
preferably with an environment as near Other systems, data
to production as possible sources, etc
 Load testing client Messaging
generating messages System
r v ic e,
s S e
i nd ow M Q ,
A , W J M S,
SO E S B, r , . . .
alk , a p te
Biz T a a d
d at
ar k et
m

Generate
representative input
messages, preferably with an Other systems, data
environment as near sources, etc
to production as possible
 e p o n e
S t

Understand what you

want to learn/prove
Performance and scalability characteristics,
typical usage profiles, etc.
g s or
d i t lo
s , au n d
f il e s t a
l o g u n d er
Use ic s t o
n s
l y t t e r
ana g e p at
usa
 p t w o
Ste

Define the test script

Determine the actions to simulate from the test
script and implement with a load testing tool.

t o add
or g et io ns
t f ser t
Don’ e s t as
r equ
t -
pos
 t h re e
S t e p

Schedule and configure

environment
Book testing slots to minimise disruption
and configure/clear data for load testing.
nn i ng
o f ru
w a r e on
Be e s ts
d t s!
loa n L A N
c t io
o d u
pr
 p f o ur
Ste

Determine metrics to
record and monitor
Ensure that the test script captures the
appropriate statistics and determine the
system characteristics to monitor
(e.g. CPU, RAM, IO, etc). o r
ni t
t o mo )
r g et n t ( s
n ’ t fo g clie
Do t e s tin
e l oad
th
 e p f iv e
S t

Execute pre-tests
Test the test scripts and monitor,
refining if necessary.

u ’ l l
er e yo
is wh d at a
This c e on
r el ian m e n t
find n v i r on
t h e e
or
 t e p s i x
S

Execute tests
Clear down the environment, warm it up,
execute tests at varying levels of
concurrent usage.
t e st
s a me
n t he es
r u t i m
to f
Try m b e r o
t e n cy
a nu c o ns is
u r e
o e ns
t
 s e v en
S t e p

Analyse results
Calculate useful statistics (average, maximum
and 95th percentile response times),
draw graphs and make conclusions.

t h e
s in g
oc e s s
- p r v id e
Post u lt s p ro
r es i lity
r a w lex ib
e f
mor
Analysing the
results
 u c c e s s,UR L
readNa m e , s
n s e C o d e , t h on.aspx
l a p s e d , l a bel,respo
ib4c urrencydev
/user/Log
cur rencydev/u on.
aspx
ser/Logon. spx
a

imeStamp, e porate us e r 1 - 1 , t r ue,http:// 1-1,true,http://ib4 currencydev/user/Log ge.aspx

nding-pa
t 0 9 9 1 0 4
ge,200,Cor
, 7 7
pa 200,Corpor
ate user
-1,true,ht
rd,
tp://ib4
ev/user/la
r 1
,01. Logon username and passwo on,302,Corporate use rue,http://ib4curre elationship.aspx
ncyd
1242921 . Enter formati r 1-1,t v/view/r
318,684,02 morable in porate use currencyde aspx x
1242921099 1,1057,03. Enter me landing page,200,Cor 1-1,true,http://ib4 encydev/user/Logon. ydev/user/Logon.asp
1 0 0 1 0 o g o n u s e r 4 c u r r r r e n c . a s p x
1242921 0 , 1 0
l b
,04. Post- ients,200,Corporate er 1-2,true,http://i 1-2,true,http://ib4 rrencydev/user/Logo .aspx
cu n
1 0 1 1 7 l e
1242921 View c rate us e user ://ib4cu ing-pag
191,42,05. ,200,Corpo 0,Corporat ,true,http /user/land
1242921101 4,13,01. Logon page name and password,20 ,Corporate user 1-2 tp://ib4currencydev ionship.aspx
39 r 2 ht at
1242921104 0,788,02. Enter use rable information,30 rate user 1-2,true, urrencydev/view/rel spx
1 1 0 4 4 2 er m e m o 0, C o r p o :/ / i b 4 c o g o n .a on.aspx
124292 2 , 8 72,03. Ent gon landing page,20 e user 1-2,true,http ib4currencydev/user urrencydev/user/Log .aspx
/L
1 1 0 5 2 3
124292 Post-lo rporat
1-3,true,h
ttp:// ://ib4c r/Logon
120,9,04. nts,200,Co ,true,http ncydev/use px
1242921106 1,715,05. View clie ,200,Corporate user ,Corporate user 1-3 rue,http://ib4curre ser/landing-page.as
1 1 0 6 1 4 on p a g e r d , 2 0 0 1 - 3 , t y d e v / u s t . a s px
124292 0 , 1 66,01. Log n a m e and passwo ,302,Corporate user ue,http://ib4currenc dev/view/account-li
1 1 0 9 6 0 u s e r n r y
124292 . Enter ormatio 1-3,t currenc aspx
775,582,02 orable inf orate user http://ib4 ationship. t.aspx
1242921109 7,835,03. Enter mem anding page,200,Corp rate user 1-1,true, urrencydev/view/rel v/recipients/Defaul
37 l o 4c de
1242921110 1,9,04. Post-logon ent details,200,Corp 1-3,true,http://ib ,http://ib4currency nts/Create.aspx i e n
aspx
ts/Create. te.aspx
1 1 1 1 2 2 ie w c l i e u s e r 1 , t r u e e c i p i e ev / r e c i p ea
124292 4 , 1
t - /r
0312,06. V clients,200,Corpora 200,Corporate user 1 tp://ib4currencydev http://ib4currencyd cydev/recipients/Cr
1 1 0 1 2 5
124292 . View page,
user 1-1,t
rue,ht 1,true, 4curren
239,925,05 ts landing te user 1- ,http://ib
1242921111 4,1612,07. Recipien pient,200,Corporate nt type,200,Corpora orate user 1-1,true spx
58 i e rp .a x
1242921111 8,51,08. Create rec ipient - choose paym ient details,200,Co rencydev/user/Logon ydev/user/Logon.asp
2 1 1 1 3 2 0 ea t e r e c er r e c i p // i b 4 c u r b4c u r r e n c go n . a s p x
12429 4 , 268,09. Cr ate recipient - ent user 1-4,true,http: r 1-4,true,http://i currencydev/user/Lo age.aspx
2 1 1 1 3 2 7
12429 0. Cre orate
0,Corporat
e use ://ib4 ding-p
558,1613,1 e,200,Corp ,true,http v/user/lan
1242921113 9,480,01. Logon pag name and password,20 ,Corporate user 1-4 ttp://ib4currencyde onship.aspx
70 r 2 ,h ti
1242921114 7,626,02. Enter use rable information,30 orate user 1-4,true rrencydev/view/rela
2 1 1 1 5 1 9 r m e m o 0 , C o r p / / i b 4 c u
12429 . Ente ge,20 ttp:
857,957,03 landing pa 1-4,true,h
1242921115 3,11,04. Post-logon s,200,Corporate user
82 t
1242921116 2,37,05. View clien
2 1 1 1 6 8 4
12429

Post-process
the raw results
 e l i s a
E x c
o s o ft o r
ic r o l f
M u l t o
usef u l ts
n g r es
h ar ti
c
 g r a ph
r t of
is so
aw th e e d to
Dr y o u n i li ty
if c alab
and s
r s t
unde
 Response time
This shows poor
scalability because
response times have
started to increase
Number of users exponentially

This shows good

Response time
scalability because
response times increase
in a predictable way Number of users
Diagnosing
problems
What happens if you find

performance
problems?
Look at your monitoring data
and log files,
run profilers,
etc...
Maybe the environment

needs tuning
[ Memory | Connection pools | Worker thread pools |
Database indexes & hints | etc ]
The big picture and

the detail
are equally as important
My Java application has

intermittent
poor performance w i th
e t est
a t t h t i o n
Rep e lle c
c o
a r b age b le d
g g en a
in
logg
 My application is slow, but it’s

not using any CPU

t and
h etes
a t t
Repe it o r IO
m on dis k)
r k and
e tw o
(n
My application is

slow Fin d t
i
h
ng
e b
y
ot
our
t le
ap
nec
p l
k
ic a
b
t
y
io n

f il
pro
Gotchas
 16 milliseconds on
Windows

Clock granularity
can vary by platform
What’s the

time Ar e y o u
?
su r e ?
How much does
this all cost?
 1-2 days
for a simple load testing script,
a few simple test runs and
some data analysis
 1-2 weeks
for a realistic load testing script, some
representative test runs,
some data analysis and diagnostics
Let’s wrap up
Few
software teams do any
sort of load testing
Even a little testing can increase

confidence ,
particularly if done

early
[email protected]
@simonbrown on Twitter

ni n g .
l i s te
s f or ?
a nk i on s
Th u e s t
q
Any

http://www.codingthearchitecture.com
o r d e v e l o p e rs
e a rc h i te ct ure f
Softwar