COMP9334

Capacity Planning of Computer Systems

and Networks

Week 1B: Queuing networks.

Operational analysis

COMP9334 1
Last lecture
• Solve capacity planning by solving a number of
performance analysis problems

• Performance metrics
• Response time, waiting time
• Throughput

• Single server FIFO queue

• A server = A processing unit

T1,2024 COMP9334 2
This lecture
• Queueing networks

• Operational analysis
• Fundamental laws relating the basic performance metrics

T1,2024 COMP9334 3
Modelling computer systems

• Single server queue considers only a component within

a computer system
• A component can be a CPU, a disk, a transmission channel

• A request may require multiple resources

• E.g. CPU, disk, network transmission

• We model a computer system with multiple resources by

a Queueing Networks (QNs)

T1,2024 COMP9334 4
Pictorial representation of single server queues

Waiting line
server

Arriving
customers
Queue

Requests waiting
to be processed CPU Finished
Arriving
requests requests

T1,2024 COMP9334 5
Pictorial representation of queues

Systems with m servers

Waiting line
m

T1,2024 COMP9334 6
 A simple database server
The server has a CPU and a disk.
Open queueing network

External arrivals

Workload intensity
A transaction specified
may visit the CPUby arrival
and rate times.
disk multiple

Unbounded
T1,2024 number of customers in the system
COMP9334 7
 Database servers for batch jobs
• Example: Batch processing system
• E.g. For summarization data from databases

Database server for batch jobs

• No on-line transactions

Running batch jobs overnight

E.g. producing managerial reports
T1,2024 COMP9334 8
 Open vs. closed queueing networks (1)

Open queueing network

Open queueing
network
• External arrivals
• Workload intensity
specified by arrival
rate
External arrivals
Database server for batch jobs
Workload intensity specified by arrival rate

Unbounded number of customers in the system Closed queueing network

• No external arrivals
In equilibrium, flow in = flow out • Workload intensity
) throughput = arrival rate specified by customer
population
Page 26

Running batch jobs overnight

COMP9334
E.g. producing
T1,2024
managerial reports 9
 Open vs. closed queueing networks (2)
Open queueing network
Open queueing network
• Possibly unbounded
#customers
• For stable equilibrium
Throughput = arrival
rate
External arrivals
Database server for batch jobs
Workload intensity specified by arrival rate
Closed queueing network
Unbounded number of customers in the system • Known #customers
In equilibrium, flow in = flow out
• Throughput depends on
) throughput = arrival rate #customers etc.
Page 26

Running batch jobs overnight

COMP9334
E.g. producing
T1,2024
managerial reports 10
 Open vs. closed queueing networks - Terminology
Open queueing network
Work in an open
queueing network is
called transactions

External arrivals
Database server for batch jobs
Workload intensity specified by arrival rate
Work in a closed
Unbounded number of customers in the system queueing network is
In equilibrium, flow in = flow out
called jobs
) throughput = arrival rate
Page 26

Running batch jobs overnight

COMP9334
E.g. producing
T1,2024
managerial reports 11
 DB server - mixed model
• The server has both
• External transactions
• Batch jobs
Mixed queueing network

Service
Different techniques areLevel
neededAgreements
to analyse open and
closed Transaction
queueing networks
Maximum Average Minimum
T1,2024 Group COMP9334 Response Time (sec) Throughput 12
owed into the database. If the MPL is too MPL is already met, all remaining transactions are queued
will suffer, since not all DBMS resources up in an external queue. The application can then control
DB server – Multi-programming level
On the other hand, if the MPL is too high, the order in which transactions are executed by scheduling
ent control on scheduling. The question of the external queue.
e MPL to achieve both goals simultaneously
em, not•justSome database
for databases but inserver
system de-
Herein we management
study this problemsystems (DBMS)
in the context
workloads,set bothanviaupper
extensivelimit on the number
experimenta-
active transactions within the incoming
ng theoreticofanalysis. MPL=4
transactions external
he two most critical
system factors in adjusting the
queue
mber of resources that the workload utilizes
ity of the• transactions’
This upper limitdemands.
service is calledWemulti- DBMS
programming
ack based controller, augmentedlevelby(MPL)
queue- Figure 1. Simplified view of the mechanism used in
odels for automatically adjusting the MPL. external scheduling. A fixed limited number of trans-
y our methods to the specific problem of ex- actions (MPL=4) are allowed into the DBMS simul-
tion of transactions. We find that external taneously. The remaining transactions are held back
n be nearly as effective as internal prioriti- in an external queue. Response time is the time from
any negative consequences, when the MPL when a transaction arrives until it completes, includ-
tely. ing time spent queueing externally to the DBMS.

• A help page from SAP explaining MPL

on Examples of recent work on external scheduling come
• http://dcx.sap.com/1200/en/dbadmin_en12/running-s-3713576.html
from many areas including storage servers, web servers, and
ays web•applications
Picture from Schroder
are largely et al. “How
dependent to determine
database a good Jin
servers. For example, multi-
et al. [9] develop an ex-
programming
atabase, where the majority level
of the for externalternal
request scheduling”
scheduling front-end to provide proportional sharing
NSF grants CCR-0133077, CCR-0311383, 0313148, among the requests at a storage service utility. Blanquer et
T1,2024 COMP9334
gh Digital Greenhouse Grant. al. [4] study external scheduling for quality of service 13
pro-
Operational analysis (OA)

• “Operational”
• Collect performance data during day-to-day operation
• Operation laws
• Applications:
• Use the data for building queueing network models
• Perform bottleneck analysis
• Perform modification analysis

T1,2024 COMP9334 14
 Single-queue example (1)

server
#requests = A #requests = C

B
In an observational period of T, server busy for time B
A requests arrived, C requests completed

A, B and C are basic measurements

Deductions: Arrival rate l = A/T

Output rate X = C/T
Utilisation U = B/T
Mean service time per completed request = B/C
T1,2024 COMP9334 15
Motivating example
• Given
• Observation period = 1 minute
• CPU
• Busy for 36s.
• 1790 requests arrived
• 1800 requests completed
• Find
• Mean service time per completion = 36/1800 = 0.02s
• Utilisation = 36/60 = 60%
• Arrival rate = 1790/60 = 29.83 requests /s
• Output rate = 1800/60 = 30 requests/s

T1,2024 COMP9334 16
Utilisation law

• The operational quantities are inter-related

• Consider
• Utilisation U = B / T
• Mean service time per completion S = B / C
• Output rate X = C / T

• Utilisation law – Can you relate U, S and X?

• U=SX

• Utilisation law is an example of operational law.

T1,2024 COMP9334 17
Application of OA

• Don’t have to measure every operational quantities

• Measure B to deduce U - don’t have to measure U
• Consistency checks
• If U ¹ S X, something is wrong
• Operational laws can be used for performance analysis
• Bottleneck analysis (Lecture 2A)
• Mean value analysis (Later in the course)

T1,2024 COMP9334 18
Equilibrium assumption
• OA makes the assumption that
• C=A
• Or at least C » A
• This means that
• The devices and system are in equilibrium
• Arrival rate of requests to a device = Output rate of requests for that
device = Throughput of the device
• The above statement also applies to the system, i.e. replace the word
“device” by “system”

T1,2024 COMP9334 19
OA for Queueing Networks (QNs)

The computer
system has K
devices, labelled
as 1,…,K.

The convention
is to add an
additional
device 0 to
represent the
outside world.

T1,2024 COMP9334 20
 OA for QNs (cont’d)

• We measure the basic operational quantities for each

device (or other equivalent quantities) over a time of T
• A(j) = Number of request arriving at device j
• B(j) = Busy time for device j
• C(j) = Number of completed requests for device j
• In addition, we have
• A(0) = Number of arrivals at the system
• C(0) = Number of completions for the system
• Question: What is the relationship between A(0) and C(0) for a
closed QNs?

T1,2024 COMP9334 21
 Visit ratios
• A job arriving at the system may require multiple visits to a
device in the system
• Example: If a transaction arriving at the system requires 3 visits to
the disk (= device j), what is the ratio of C(j) to C(0)?

• We expect C(j)/C(0) = 3.

• V(j) = Visit ratio of device j

= Number of times a job (transaction) visits device j

• We have V(j) = C(j) / C(0)

Trans C C C C
C Location
of the trans.
over time D D D
D Time

T1,2024 COMP9334 22
Forced Flow Law

Since

The forced flow law is

T1,2024 COMP9334 23
 Service time versus service demand

• Ex: A job requires two disk accesses to be completed. One

disk access takes 20ms and the other takes 30ms.

• Service time = the amount of processing time required per

visit to the device
• The quantities “20ms” and “30ms” are the individual service times.

• D(j) = Service demand of a job at device j is the total service

time required by that job
• The service demand for this job = 20ms + 30 ms = 50ms

T1,2024 COMP9334 24
Service demand
• Service demand can be expressed in two different
ways
• Ex: A job requires three disk accesses to be completed. One
disk access takes 20ms and the others take 30ms and
28ms.
• What is D(j)? 20+30+28 = 78ms.
• What are V(j) and S(j)?
• Recall that S(j) = mean service time of device j
• V(j) = 3. S(j) = 78/3 = 26ms.

• Service demand D(j) = V(j) S(j)

T1,2024 COMP9334 25
Service demand law (1)

Given D(j) = V(j) S(j)

Since

• What is X(j) S(j)?

• It is U(j)

Service demand law

T1,2024 COMP9334 26
Service demand law (2)
• Service demand law D(j) = U(j) / X(0)
• You can determine service demand without knowing the visit ratio
• Over measurement period T, if you find
• B(j) = Busy time of device j
• C(0) = Number of requests completed
• You’ve enough information to find D(j)

• The importance of service demand

• You will see that service demand is a fundamental quantity you
need to determine the performance of a queueing network
• You will use service demand to determine system bottleneck in
Lecture 2A

T1,2024 COMP9334 27
Server example exercise

Measurement time = 1 hr

# I/O per Utilisation

second
Disk 1 32 0.30
Disk 2 36 0.41
Disk 3 50 0.54

CPU 0.35

Total # jobs=13680

What is the service time of Disk 2?

What is the service demand of Disk 2?
What is its visit ratio?

T1,2024 COMP9334 28
Server example solution

Measurement time = 1 hr

# I/O per Utilisation

second
Disk 1 32 0.30
Disk 2 36 0.41
Disk 3 50 0.54

CPU 0.35

Total # jobs=13680

Service time = U2/X2 = 0.41/36 = 11.4ms

System throughput = 13680/3600 = 3.8 jobs/s
Service demand = 0.41/3.8 = 108ms
Visit ratio = 36/3.8 = 108 / 11.4 = 9.47
T1,2024 COMP9334 29
 Little’s law (1)
• Due to J.C. Little in 1961
• A few different forms
• The original form is based on stochastic models
• An important result which is non-trivial
• All the other operational laws are easy to derive, but Little’s
Law’s derivation is more elaborate.

• Consider a single-server device

• Navg = Average number of requests in the device
• When we count the number of requests in a device, we include
the one being served and those in the queue waiting for service

T1,2024 COMP9334 30
Little’s Law (2)

• X = Throughput of the device

• Ravg = Average response time of the requests
• Navg = Average number of requests in the device
• Little’s Law (for OA) says that

Navg = X * Ravg
We will argue the validity of Little’s Law using a simple
example.

T1,2024 COMP9334 31
 Consider the single sever queue example from Week 1A
Request index Arrival time Service time Departure time
1 2 2 4
2 6 4 10
3 8 4 14
4 9 3 17

Let us use blocks of height 1 to show the time span of the

requests, i.e. width of each block = response time of the request

3
4
2
3
1 time
1 2
2 4 6 10 14 17
T1,2024 COMP9334 32
3
4
2
3
1 time
1 2
2 4 6 10 14 17
Assuming that in the measurement time interval [0,20]
these 4 requests arrive and depart from this device, i.e. the device
is in equilibrium.

Total area of the blocks

= Response time of request 1 + Response time of request 2 +
Response time of request 3 + Response time of request 4
= Average response time over the measurement interval *
Number of requests completed over the measurement interval

This is one interpretation. Let us look at another.

T1,2024 COMP9334 33
3
4
2
3
1 time
1 2
2 4 6 10 14 17
Let us assume these blocks are “plasticine” and let them fall
to the ground. Like this.

3
4
2
3 4
1 time
1 2 3 4
2 4 6 10 14 17
There is an interpretation of the height of the graph.
T1,2024 COMP9334 34
 Request index Arrival time Service time
1 2 2
2 6 4
3 8 4
4 9 3

Requests
waiting
3 to be
4 processed
2
3 4 Request
1 being
1 2 3 4 processed
2 4 6 10 14 17 time
Interpretation: Height of the graph = #requests in the device
E.g. Number of requests in [9,10] = 3
E.g. Number of requests in [11,12] = 2 etc.
T1,2024 COMP9334 35
3 waiting
4 requests
2
3 4 Request
1 being
1 2 3 4 processed
2 4 6 10 14 17 time

Again, consider the measurement time interval of [0,20].

Area under the graph in [0,20]

= Height of the graph in [0,1] + Height of the graph in [1,2] + …
Height of the graph in [19,20]
= #reqs in [0,1] + #reqs in [1,2] + … + #reqs in [19,20]
= Average number of requests in [0,20] in the device * 20

T1,2024 COMP9334 36
3
4
2
3
1 time
1 2
2 4 6 10 14 17
Area = Average response time over [0,T] *
Number of requests completed in [0,T]

3 waiting
4 requests
2
3 4
Request
1
1 2 3 4 being processed

2 4 6 10 14 17 time
Area = Average number of requests in [0,T] * T
T1,2024 COMP9334 37
 Deriving Little’s Law

Area = Average response time of all jobs *

Number of requests completed in [0,T] (Interpretation #1)
= Average #requests in [0,T] * T (Interpretation #2)

Since Number of requests completed in [0,T] / T

= Device throughput in [0,T]

We have Little’s Law.

Average number of requests in [0,T]

= Average response time of all reqs * Device throughput in [0,T]

T1,2024 COMP9334 38
Using Little’s Law (1)

queue server

• A device consists of a server and a queue

• The device completes on average 8 requests per second
• On average, there are 3.2 requests in the device
• What is the response time of the device?
• Mean throughput X = 8 requests/s
• Mean number of requests Navg = 3.2 requests
• By Little’s Law, average response time = Navg/X = 3.2 / 8 =
0.4 s
T1,2024 COMP9334 39
Intuition of Little’s Law
• Little’s Law
• Mean #requests = Mean response time * Mean throughput
• If #requests in the device ⬆ , then response time ⬆
• And vice versa

T1,2024 COMP9334 40
Applicability of Little’s Law
• Little’s Law can be applied at many different levels
• Little’s law can be applied to a device
• Navg(j) = Ravg(j) * X(j)

• A “box” with K devices

• Box 3 in the next slide has 4 devices
• Navg(j) = #requests in device j
• Average number of requests in the box Navg = Navg(1) + …. +
Navg(K)
• Average response time of the box = Ravg
• Response time of a box = Departure time from the box – Arrival time
to the box
• We can also apply it to an entire system
• Navg = Ravg * X(0)

T1,2024 COMP9334 41
T1,2024 COMP9334 42
 Using Little’s Law (2)
queue server

• The device completes on average 8 requests per second

• On average, there are
• 3.2 requests in the device
• 2.4 requests in the queue
• 0.8 requests in the server
• What is the mean waiting time and mean service time?
• Hint: You need to draw “boxes” around certain parts of the
device and interpret the meaning of response time for that
box.
T1,2024 COMP9334 43
 Using Little’s Law (2)
queue server

• The device completes on average 8 requests per second

• On average, there are
• 3.2 requests in the device
• 2.4 requests in the queue
• 0.8 requests in the server
• What is the mean waiting time and mean service time?
• Mean throughput X = 8 requests/s
• Mean waiting time = 2.4 / 8 = 0.3 s
• Mean service time = 0.8 / 8 = 0.1 s

T1,2024 COMP9334 44
 References
• Operational analysis
• Lazowska et al, Quantitative System Performance, Prentice Hall, 1984.
(Classic text on performance analysis. Now out of print but can be download
from http://www.cs.washington.edu/homes/lazowska/qsp/
• Chapters 3 and 5 (For Chapter 5, up to Section 5.3 only)
• Alternative 1: You can read Menasce et al, “Performance by design”, Chapter
3. From beginning of Chapter 3 to Section 3.2.4.
• Alternative 2: You can read Harcol-Balter, Chapter 6. The treatment is more
rigorous. You can gross over the discussion mentioning ergodicity.

• Little’s Law (Optional)

• I presented an intuitive “proof”. A more formal proof of this well known Law is
in Bertsekas and Gallager, “Data Networks”, Section 3.2

• Revision questions based on this week’s lecture are available from course
web site

T1,2024 COMP9334 45