Computing Infrastructures

Operational laws 09/03/20

Danilo Ardagna

Credits: Raffaela Mirandola,

Jane Hilston, Ed Lazowska,
Marco Gribaudo, John Zahorian
Model Definition

Existing
system Parameterize the Model

Performance
Model
System Design

Define the goal

Create the Model

Queuing Network Evaluate the Model

Performance indices
Model Definition

Existing
system Parameterize the Model

Performance
Model
System Design

Define the goal

Create the Model

Queuing Network Evaluate the Model

Performance indices
Operational laws

• Operational laws are simple equations which may be used as an

abstract representation or model of the average behaviour of
almost any system

• The laws are very general and make almost no assumptions

about the behaviour of the random variables characterising the
system

• Another advantage of the laws is their simplicity: this means that

they can be applied quickly and easily
Operational laws

Requests
Arrival
System Requests
satisfied

• Operational laws are based on observable variables - values

which we could derive from watching a system over a finite
period of time
• We assume that the system receives requests from its
environment
• Each request generates a job or customer within the system
• When the job has been processed the system responds to the
environment with the completion of the corresponding request
Operational laws
Requests
Arrival System
Requests
satisfied

System
Arrivals Departures

• Operational laws are based on observable variables - values

which we could derive from watching a system over a finite
period of time
• We assume that the system receives requests from its
environment
• Each request generates a job or customer within the system
• When the job has been processed the system responds to the
environment with the completion of the corresponding request
 Observations and measurements

If we observed such an abstract system we might measure the following

quantities:

§ T, the length of time we observe the system

§ A, the number of request arrivals we observe
§ C, the number of request completions we observe
§ B, the total amount of time during which the system is busy (B ≤ T)
§ N, the average number of jobs in the system

System
Arrivals Departures
Four important quantities

From these observed values we can derive the following four

important quantities:

§ λ = A/T , the arrival rate

§ X = C /T , the throughput or completion rate
§ U = B/T, the utilisation
§ S = B/C, the mean service time per completed job

System
Arrivals Departures
Job flow balance

• We will assume that the system is job flow balanced. This means
that the number of arrivals is equal to the number of
completions during an observation period, i.e. A = C

• This is a testable assumption because an analyst can always test

whether the assumption holds
• it can be strictly satisfied by careful choice of measurement
interval

• Note that if the system is job flow balanced the arrival rate will
be the same as the completion rate, that is:

λ=X
Operational laws
System
Requests
subsystem
Arrival subsystem

Requests
subsystem
satisfied

• A system may be regarded as being made up of a number of

devices or resources

• Each of these may be treated as a system in its own right from

the perspective of operational laws

• An external request generates a job within the system; this job

may then circulate between the resources until all necessary
processing has been done; as it arrives at each resource it is
treated as a request, generating a job internal to that resource
Operational laws

• A system may be regarded as being made up of a number of

devices or resources

• Each of these may be treated as a system in its own right from

the perspective of operational laws

• An external request generates a job within the system; this job

may then circulate between the resources until all necessary
processing has been done; as it arrives at each resource it is
treated as a request, generating a job internal to that resource
 Observations and measurements

If we observed such an abstract system we might measure the following

quantities:

§ T, the length of time we observe the system

§ Ak, the number of request arrivals we observe for resource k
§ Ck, the number of request completions we observe at resource k
§ Bk, the total amount of time during which the resource k is busy (Bk ≤ T)
§ Nk, the average number of jobs in the resource k (queueing or being
served)
just add k...

arrival, completions, busy, number of jobs

 Four important quantities

From these observed values we can derive the following four

important quantities for resource k:

§ λk = Ak/T , the arrival rate

§ Xk = Ck /T , the throughput or completion rate
§ Uk = Bk/T, the utilisation
§ Sk = Bk/Ck, the mean service time per completed job

arrival rate
thoughput or completion rate
utilization
service rate
Number of users
Example

Let us observe the k-th resource

and show in the graph the total number
of users in k (both waiting for service and
actually served)
Time (t)

T = 26 s Arrival rate: lk = Ak/T = 7/26 req/s

Arrivals number Throughput: Xk = Ck/T = 7/26 req/s
Ak = 7 Utilization: Uk = Bk/T = 20/26 = 0.77 = 77%
Completions number Average service time: Sk = Bk/Ck= 20/7 s
Ck = 7
Utilization law

Let us recall the following definitions for a resource k:

Throughput: Xk = Ck/T
Service time Sk = Bk/Ck k for each element of the system.
Utilization: Uk = Bk/T

From:

Xk Sk = Ck/T * Bk/Ck = Bk/T=Uk

we can derive (utilization law):

Uk = XkSk
Utilization law

Let us recall the following definitions for a resource k:

Throughput: Xk = Ck/T
Service time Sk = Bk/Ck
Utilization: Uk = Bk/T

From:

Xk Sk = Ck/T * Bk/Ck = Bk/T=Uk

we can derive (utilization law):

Uk = XkSk utilization = throughput * service time

Utilization law

The law is valid also for the entire system:

U = XS
also for the entire system
Example

• Let us consider a resource k serving 40 requests/s, each of them

requiring on average 0.0225 s

• From the utilization law we have:

Uk=XkSk=40 x 0.0225 = 0.9 -->90%

• Let us consider a gas station serving 2 cars per minute dedicating to

each of them 12 s for refuelling

• From the utilization law we have:

U=XS=2 cars/min x 12s/car = 2 cars/min x 0.2 min/cars = 0.4 -->40%
Example

• Let us consider a resource k serving 40 requests/s, each of them

requiring on average 0.0225 s

• From the utilization law we have:

Uk=XkSk=40 x 0.0225 = 0.9 -->90%

• Let us consider a gas station serving 2 cars per minute dedicating to

each of them 12 s for refuelling

• From the utilization law we have:

U=XS=2 cars/min x 12s/car = 2 cars/min x 0.2 min/cars = 0.4 -->40%
Little Law

Little’s law:
N = XR

Little law can be applied to the entire system as well as to some

subsystems

If the system throughput is X requests/sec, and each request

remains in the system on average for R seconds, then for each unit
of time, we can observe on average XR requests in the system
Derivation of Law
Derivation of Little Little’s Law
the area W between the arrival
W = total time in system and completion functions
(in job-seconds) represents the accumulated time
in system during that interval,
measured in request-seconds

Total Arrivals
This represents
the number of requests
#Jobs

present in the system at

this instant of time

Total Completions

Time T
CSE 597 - Lecture 5 11
Derivation of Little Law

W denotes the accumulated time in system (jobs- sec)

if there are 3 requests in the system during a 2 second period, then W is 6
request-seconds

N is the average number of requests in the system: N=W/T

R is the average system residence time per request: R=W/C

We can write:

N = W/T = C/T * W/C = X*R, so

N = XR
Derivation of Little Law

W denotes the accumulated time in system (jobs- sec)

if there are 3 requests in the system during a 2 second period, then W is 6
request-seconds

N is the average number of requests in the system: N=W/T

R is the average system residence time per request: R=W/C

We can write:

N = W/T = C/T * W/C = X*R, so

N = XR
Example

• Consider a disk that serves 40 requests/seconds (X = 40 req/s)

and suppose that on average there are 4 requests (N = 4) present
in the disk system (waiting to be served or in service)

• Little’s law tells us that the average time spent at the disk by a
request must be 4/40 = 0.1 seconds

• If we know that each request requires 0.0225 seconds of disk

service we can then deduce that the average waiting time (time
in the queue) is 0.0775 seconds
Example

• Consider a disk that serves 40 requests/seconds (X = 40 req/s)

and suppose that on average there are 4 requests (N = 4) present
in the disk system (waiting to be served or in service)

• Little’s law tells us that the average time spent at the disk by a
request must be 4/40 = 0.1 seconds

• If we know that each request requires 0.0225 seconds of disk

service we can then deduce that the average waiting time (time
in the queue) is 0.0775 seconds
 11/03/20

Application of Little’s Law at different levels

Terminals

CPU

Disks
Little’s Law, level 1

It can be applied to the single server Disk (without the queue)

(Red Box)
N(1) in this case represents the percentage of time in which the
server Disk is busy, so it corresponds to Udisk
R(1) represents the requests average service time
X(1) represents the rate of serving requests

we are not considering the queue time.

Let us apply Little law:

X(1) =40 req/sec, S =R(1) =0.0225 sec,

Little law N(1) =X(1) R(1) =0.9 probability that the disk is busy. It's 0 or 1 or between.

U(1) =X(1) S, U(1) = 90% here the utilization is a specific case of the little law.
Application of Little’s Law at different levels

Terminals

CPU

Disks
 Little’s Law, level 2

Let us include now the queue (Blue Box)

N(2) is the number of users in the service center (waiting+ inservice)
R(2) is the time spent in the service center (waiting time + service time)
X(2) is the throughput of the server Disk and corresponds to X(1)

We know that X(1)=X(2) = 40 req/s.

Let us suppose to have obtained, through observations, the
following information: N(2) = 4.

From Little’s law we have: R(2) =N(2)/X(2)= 0.1 s.

Since R(1) = 0.0225 s, we can compute the waiting time as:

R(2) − R(1) = 0.0775 s.
 Application of Little’s Law at different levels
residence time =
response time Residence time corresponds
that the users see.
to our conventional notion of
response time: the period of
Terminals time from when a user
submits a request until that
user's response is returned

CPU

Disks
 Little’s Law, level 3

Let us consider the central subsystem (Green Box)

N(3) represents the total number of users in the subsystem (e.g., requests
of web pages/s)
R(3) represents the average time spent in the subsystem by each request
X(3) is the subsystem throughput (e.g., number of web pages/s)

If from observations we know that X(3) = 0.5 job/sec and

requests number is N(3) = 7.5

From Little law we have that

N = XR
R(3) = N(3)/X(3)=15 s
Application of Little’s Law at different levels

Terminals

CPU

Disks
 Little’s Law, level 4

Let us now consider the complete system (Orange Box)

N(4) is the total number of users in the system (which is fixed since we
have a closed system)
R(4) is the total amount of time spent in service, waiting and at the
“terminals” client side (think time, e.g., the time a user spend to
read a web page and to elaborate a request)
X(4) is the rate with which the requests reach the systems from the
terminals client and it corresponds to X(3)

Let us suppose that there are N(4) = 10 users, and the think time is Z = 5 s.
We know that the time spent in the system is R(3) = 15 s.
Now, since R(4) = R(3) + Z, we can derive from Little law that
N(4) = X(4)(R(3) + Z),
and we can compute the time is spent at the terminals (think time Z) and the
X(4) = 0.5 job/s other is in the CPU + DISKs = R3 -> R4 = R3 + Z
 Interactive Response Time Law

• Back when most processing was done on shared mainframes,

think time, Z, was quite literally the length of time that a
programmer spent thinking before submitting another job

• More generally in interactive systems, jobs spend time in the

system not engaged in processing, or waiting for processing: this
may be because of interaction with a human user, or may be for
some other reason

• The think time represents the time between processing being

completed and the job becoming available as a request again
Think time: identify a user, and measure the amount of time between two access to the system from a user.
Example

For example, if we are studying a cluster of workstations with a

central file server to investigate the load on the file server, the
think time might represent the average time that each workstation
spends processing locally without access to the file server.

At the end of this non-processing period (from the file server point
of view) the job generates a fresh request.
 little law.
Interactive Response Time Law

N(4) = «ready» + «not ready» (Z) users

R(4) = R(3) + Z
N = X(3) (R(3) + Z) = X (R+Z)

Terminals

CPU

Disks
 Interactive Response Time Law

• Interactive Response Time Law:

R = N/X − Z

• The response time in an interactive system is the residence time

minus the think time

• Note that if the think time is zero, Z = 0 and R = N , then the

interactive response time law simply becomes Little’s Law

this is the case where we take in account the think time of the users.

R = N/X - Z
Interactive Response Time Law: Example

• Suppose that the library catalogue system has 64 interactive

users connected via Browsers, that the average think time is 30
seconds, and that system throughput is 2 interactions/second.

• What is the response time?

• The interactive response time law tells us that the response time
must be 64/2 − 30 = 2 seconds
Operational laws

System
Requests
subsystem
Arrival subsystem

Requests
subsystem
satisfied

• In an observation interval we can count not only completions

external to the system, but also the number of completions at
each resource within the system

• Let Ck be the number of completions at resource k

• We define the visit count, Vk , of the k-th resource to be the

ratio of the number of completions at that resource to the
number of system completions Vk = Ck/C
Visit count: example

For example, if, during an observation interval, we measure 10

system completions and 150 completions at a specific disk, then on
the average each system-level request requires 15 disk operations.
 42

Visits

Note that:

• If Ck > C, resource k is visited several times (on average) during

each system level request. This happens when there are loops in
the model
• If Ck < C, resource k might not be visited during each system
level request. This can happen if there are alternatives (i.e.
caching of disks)
• If Ck = C, resource k is visited (on average) exactly once every
request
Forced Flow Law

The forced flow law captures the relationship between the different
components within a system. It states that the throughputs or
flows, in all parts of a system must be proportional to one another.

Xk = VkX

The throughput at the k-th resource is equal to the product of the

throughput of the system and the visit count at that resource.

Rewriting Ck = VkC and applying Xk = Ck/T , we can derive the forced

flow law:
Ck = VkC
Ck / T= Vk C/ T
Xk = VkX
 Forced Flow Law example

Consider a robotic workcell within a computerised manufacturing system

which processes widgets.

Suppose that processing each widget requires 4 accesses to the lathe and 2
accesses to the press.

We know that the lathe processes 8 widgets in a minute and we want to know
the throughput of the press.

The throughput of the workcell will be proportional to the lathe throughput,

i.e. X = Xlathe/Vlathe = 8/4 = 2.

The throughput of the press will be Xpress = X * Vpress = 2 * 2 = 4.

Thus the press throughput is 4 widgets per minute.

 Forced Flow Law example

Consider a robotic workcell within a computerised manufacturing system

which processes widgets.

Suppose that processing each widget requires 4 accesses to the lathe and 2
accesses to the press.

We know that the lathe processes 8 widgets in a minute and we want to know
the throughput of the press.

The throughput of the workcell will be proportional to the lathe throughput,

i.e. X = Xlathe/Vlathe = 8/4 = 2.

The throughput of the press will be Xpress = X * Vpress = 2 * 2 = 4.

Thus the press throughput is 4 widgets per minute.

Utilisation Law

• If we know the amount of processing each job requires at a

resource then we can calculate the utilisation of the resource

• Let us assume that each time a job visits the k-th resource the
amount of processing, or service time it requires is Sk

• Note that service time is not necessarily the same as the

residence time of the job at that resource: in general a job
might have to wait for some time before processing begin

• The total amount of service that a system job generates at the

k-th resource is called the service demand, Dk:

Dk =SkVk
Utilisation Law

• The utilisation of a resource, the percentage of time that the k-

th resource is in use processing to a job, is denoted Uk .

• Utilisation Law:

Uk = XkSk = (XVk) Sk = DkX

• The utilisation of a resource is equal to the product of: 1) the

throughput of that resource and the average service time at that
resource, 2) the throughput at system level and the average
service demand at that resource
from Dk=SkVk
and Xk=VkX
Uk =DkX
Utilisation Law

• The utilisation of a resource, the percentage of time that the k-

th resource is in use processing to a job, is denoted Uk .

• Utilisation Law:

Uk = XkSk = (XVk) Sk = DkX

• The utilisation of a resource is equal to the product of: 1) the

throughput of that resource and the average service time at that
resource, 2) the throughput at system level and the average
service demand at that resource

Uk =DkX
General Residence Time Law

• One method of computing the mean residence or response time

per job in a system is to apply Little's Law to the system as a
whole

• However, if the mean number of jobs in the system, N, or the

system level throughput, X, are not known an alternative method
can be used

• Applying Little's Law to the k-th resource we see that Nk = XkRk ,

where Nk is the mean number of jobs at the resource and Rk is
the average residence time of the resource

• From the Forced Flow Law we know that Xk = XVk. Thus we can
deduce that:
Nk/X = VkRk
General Residence Time Law

The total number of jobs in the system is clearly the sum of the
number of jobs at each resource, i.e. N = N1 +…+ NM if there are M
resources.

From Little's Law R = N/X and so, from Nk/X = VkRk

General Residence Time Law

R= S k VkRk

The average response time of a job in the system will be the sum of
the product of its average residence time at each resource and the
number of visits it makes to that resource
General Residence Time Law: Example

A program running on a computer server requires 126 bursts of CPU

time and makes 75 I/O requests to disk A and 50 I/O requests to
disk B.

On average each CPU burst requires 30 milliseconds (waiting +

processing time).

Monitoring has shown that the throughput of disk A is 15 requests

per second and the average number in the buffer is 4 whilst at disk
B the throughput is 10 requests per second and the average number
in the buffer is 3.
Model derivation and parameterization

Disk A

CPU

Disk B

VCPU= 126
XDiskA=15 req/sec NDiskA= 4+1 RCPU= 30 msec
VDiskA=75
XdiskB=10 req/sec NdiskB=3+1
VdiskB=50
Model Evaluation

Using Little's Law we calculate the residence time at each of the

disks (remembering that the number in the system is the number in
the buffer +1):
– RdiskA = NdiskA/XdiskA = (5/15)x1000= 5000/15 ms

– RdiskB = NdiskB/XdiskB = (4/10)x1000=4000/10 ms

Then
– R = RCPUVCPU +RdiskAVdiskA +RdiskBVdiskB
= 30 *126 +(5000/15)*75 +(4000/10)*50
= 3780 + 25000 + 20000
= 48780 ms
Model Evaluation

Using Little's Law we calculate the residence time at each of the

disks (remembering that the number in the system is the number in
the buffer +1):
– RdiskA = NdiskA/XdiskA = (5/15)x1000= 5000/15 ms

– RdiskB = NdiskB/XdiskB = (4/10)x1000=4000/10 ms

Then
– R = RCPUVCPU +RdiskAVdiskA +RdiskBVdiskB
= 30 *126 +(5000/15)*75 +(4000/10)*50
= 3780 + 25000 + 20000
= 48780 ms
Operational laws

• Operational laws are simple equations which may be used as an

abstract representation or model of the average behaviour of
almost any system

• The laws are very general and make almost no assumptions

about the behaviour of the random variables characterising the
system

• Another advantage of the laws is their simplicity: this means that

they can be applied quickly and easily
Operational laws summary

§ T: observation interval
§ λk = Ak/T , the arrival rate
§ Xk = Ck /T , the throughput or completion rate
§ Uk = Bk/T, the utilisation
§ Sk = Bk/Ck, the mean service time per completed job
§ Vk = Ck/C, visit count of the k-th resource
§ Dk =SkVk, the total amount of service that a system job
generates at the k-th resource
Operational laws summary

Utilisation Law:

Uk =XkSk Uk=DkX
Little’s law:

N = XR

Forced flow law:

Xk = VkX

Response time law:

R = N/X -Z