Chapter 6

Queueing Models

Banks, Carson, Nelson & Nicol

Discrete-Event System Simulation
 Purpose

 Simulation is often used in the analysis of queueing models.

 A simple but typical queueing model shown in fig:

 Queueing models (either solved mathematically or analyzed

through simulation model) provide the analyst with a powerful tool
for designing and evaluating the performance of queueing
systems.
 Typical measures of system performance:
 Server utilization, length of waiting lines, and delays of customers
 Queuing theory and simulation analysis are used to predict
performance measure as input parameter.
Input parameter: arrival rate, service demanded, service rate,
2
arrangement of server
Outline
 Discuss some well-known models (not development
of queueing theories):
 General characteristics of queues,
 Meanings and relationships of important performance
measures,
 Estimation of mean measures of performance.
 Effect of varying input parameters,
 Mathematical solution of some basic queueing models.

3
Characteristics of Queueing Systems
 Key elements of queueing systems:
 Customer: refers to anything that arrives at a facility and requires
service, e.g., people, machines, trucks, emails.
 Server: refers to any resource that provides the requested
service, e.g., repairpersons, retrieval machines, runways at
airport.

4
1. Calling Population
[Characteristics of Queueing System]

 Calling population: the population of potential customers,

may be assumed to be finite or infinite.
 Finite population model: if arrival rate depends on the number of
customers being served and waiting, e.g., model of one
corporate jet, if it is being repaired, the repair arrival rate
becomes zero.
 Infinite population model: if arrival rate is not affected by the
number of customers being served and waiting, e.g., systems
with large population of potential customers.

5
2. System Capacity
[Characteristics of Queueing System]

 System Capacity: a limit on the number of customers

that may be in the waiting line or system.
 Limited capacity, e.g., an automatic car wash only has room for
10 cars to wait in line to enter the mechanism.
 Unlimited capacity, e.g., concert ticket sales with no limit on the
number of people allowed to wait to purchase tickets.

6
3. Arrival Process
[Characteristics of Queueing System]

 For infinite-population models:

 In terms of interarrival times of successive customers.
 Random arrivals: interarrival times usually characterized by a
probability distribution.
 Most important model for random arrival is Poisson arrival process
(with rate ), where An represents the interarrival time between
customer n-1 and customer n, and is exponentially distributed (with
mean 1/).
 Scheduled arrivals: interarrival times can be constant or constant
plus or minus a small random amount to represent early or late
arrivals.
 e.g., patients to a physician or scheduled airline flight arrivals to an
airport.
 At least one customer is assumed to always be present, so the
server is never idle, e.g., sufficient raw material for a machine.
7
3. Arrival Process
[Characteristics of Queueing System]

 For finite-population models:

 Customer is pending when the customer is outside the queueing
system, e.g., machine-repair problem: a machine is “pending”
when it is operating, it becomes “not pending” the instant it
demands service form the repairman.
 Runtime of a customer is the length of time from departure from
the queueing system until that customer’s next arrival to the
queue, e.g., machine-repair problem, machines are customers
and a runtime is time to failure.
 Let A1(i), A2(i), … be the successive runtimes of customer i, and
S1(i), S2(i) be the corresponding successive system times:

8
4. Queue Behavior and Queue Discipline
[Characteristics of Queueing System]
 Queue behavior: the actions of customers while in a queue
waiting for service to begin, for example:
 Balk: leave when they see that the line is too long,
 Renege: leave after being in the line when its moving too slowly,
 Jockey: move from one line to a shorter line.

 Queue discipline: the logical ordering of customers in a queue

that determines which customer is chosen for service when a
server becomes free, for example:
 First-in-first-out (FIFO)
 Last-in-first-out (LIFO)
 Service in random order (SIRO)
 Shortest processing time first (SPT)
 Service according to priority (PR).

9
5. Service Times and Service Mechanism
[Characteristics of Queueing System]
 Service times of successive arrivals are denoted by S1,
S2, S3.
 May be constant or random.
 {S1, S2, S3, …} is usually characterized as a sequence of
independent and identically distributed random variables, e.g.,
exponential, Weibull, gamma, lognormal, and truncated normal
distribution.
 A queueing system consists of a number of service
centers and interconnected queues.
 Each service center consists of some number of servers, c,
working in parallel, upon getting to the head of the line, a
customer takes the 1st available server.

10
Service Times and Service Mechanism
[Characteristics of Queueing System]

 Example 6.1: consider a discount warehouse where

customers may:
 Serve themselves before paying at the cashier:

11
Service Times and Service Mechanism
[Characteristics of Queueing System]
 Wait for one of the three clerks:

 Batch service (a server serving several customers

simultaneously), or customer requires several servers
simultaneously.
 Example 6.2 ( book)
12
Queueing Notation
[Characteristics of Queueing System]

 A notation system for parallel server queues: A/B/c/N/K

 A represents the interarrival-time distribution,
 B represents the service-time distribution,
 c represents the number of parallel servers,
 N represents the system capacity,
 K represents the size of the calling population.

13
 Computer Science,
Informatik 4 Communication and
Distributed Systems

Queueing Notation – Kendall

Notation

 A notation system for parallel server queues:

• A. represents the interarrival-time
A/B/c/N/K
• distribution
• B. represents the service-time
• N distribution
• cK represents
representsthe number
the size ofofthe
parallel
calling servers represents the
population
system
• N, K are capacity
usually dropped, if they are
 Common symbols for A
infinity
• MB Markov, exponential
and
distribution
Constant,
• D deterministic Erlang
• Ek distribution of order k
• H Hyperexponential
 • G
Examples distribution General,
• M/M/1/arbitrary
/ same as M/M/1: Single-server with unlimited
capacity and call- population. Interarrival and service
times are exponentially distributed
• G/G/1/5/5: Single-server with capacity 5 and call-population
5.

Chapter 8. Queueing
Models Dr. Mesut Güneş 14
Queueing Notation
[Characteristics of Queueing System]
 Primary performance measures of queueing systems:
 Pn: steady-state probability of having n customers in system,
 Pn(t): probability of n customers in system at time t,
 : arrival rate,
 e: effective arrival rate,
 : service rate of one server,
 : server utilization,
 An: interarrival time between customers n-1 and n,
 Sn: service time of the nth arriving customer,
 Wn: total time spent in system by the nth arriving customer,
 WnQ: total time spent in the waiting line by customer n,
 L(t): the number of customers in system at time t,
 LQ(t): the number of customers in queue at time t,
 L: long-run time-average number of customers in system,
 LQ: long-run time-average number of customers in queue,
 w: long-run average time spent in system per customer,
 wQ: long-run average time spent in queue per customer.
15
 Computer Science,
Informatik 4 Communication and
Distributed Systems

Long-run Measures of Performance of Queueing

Systems

 Goal of this section

• Major measures of performance for a general
G/G/c/N/K queueing system
• How these measures can be estimated from
simulation runs

 Two types of estimators

• Sample average
• Time-integrated sample average

Chapter 8. Queueing
Models Dr. Mesut Güneş 16
17
 Computer Science,
Informatik 4 Communication and
Distributed Systems

Time-Average Number in System

L

Number
of customers in
the
system

Tim
e

Chapter 8. Queueing
Models Dr. Mesut Güneş 18
Time-Average Number in System L
[Characteristics of Queueing System]

 Consider a queueing system over a period of time T,

 Let Ti denote the total time during [0,T] in which the system
contained exactly i customers, the time-weighted-average number
in a system is defined by:
 
1 T 
Lˆ 
T 
i 0
iTi   i i 
i 0 
T
 Consider the total area under the function is L(t), then,

1 1 T
Lˆ 
T 
i 0
iTi 
T  L(t )dt
0

 The long-run time-average # in system, with probability 1:

1 T
Lˆ 
T  L(t )dt  L
0
as T  

19
Time-Average Number in System L
[Characteristics of Queueing System]
 The time-weighted-average number in queue is:

1 1 T
ˆ
LQ  
T i 0
Q
iTi  
T 0
LQ (t )dt  LQ as T  

 G/G/1/N/K example: consider the results from the queueing

system (N > 4, K > 3).
Lˆ  [0(3)  1(12)  2(4)  3(1)] / 20
 23 / 20  1.15 cusomters

0, if L(t)  0
LQ (t )  
 L(t )  1, if L(t)  1

0(15)  1( 4)  2(1)
LˆQ   0.3 customers
20

20
Average Time Spent in System Per
Customer w [Characteristics of Queueing System]

 The average time spent in system per customer, called

the average system time, is: 1
N
wˆ 
N  W
i 1
i

where W1, W2, …, WN are the individual times that each of the N
customers spend in the system during [0,T].
 For stable systems: w ˆ  w as N  
 If the system under consideration is the queue alone:
1 N Q
wˆ Q   Wi  wQ as N  
N i 1
 G/G/1/N/K example (cont.): the average system time is
W1  W2  ...  W5 2  (8  3)  ...  (20  16)
wˆ    4.6 time units
5 5
21
The Conservation Equation
[Characteristics of Queueing System]

 Conservation equation (a.k.a. Little’s law)

Average # in
Lˆ  ˆwˆ Average
system System time
Arrival rate

L  w as T   and N  
 Holds for almost all queueing systems or subsystems (regardless
of the number of servers, the queue discipline, or other special
circumstances).
 G/G/1/N/K example (cont.): On average, one arrival every 4 time
units and each arrival spends 4.6 time units in the system.
Hence, at an arbitrary point in time, there is (1/4)(4.6) = 1.15
customers present on average.

22
Server Utilization
[Characteristics of Queueing System]

 Definition: the proportion of time that a server is busy.

 Observed server utilization, ̂ , is defined over a specified time
interval [0,T].
 Long-run server utilization is .
 For systems with long-run stability: ˆ   as T  

23
Server Utilization
[Characteristics of Queueing System]

 For G/G/1/∞/∞ queues:

 Any single-server queueing system with average arrival
rate  customers per time unit, where average service time
E(S) = 1/ time units, infinite queue capacity and calling
population.
 Conservation equation, L = w, can be applied.
 For a stable system, the average arrival rate to the server,
s, must be identical to .
 The average number of customers in the server is:
1
  T  T0
T
Lˆ s 
T 0 L (t )  LQ (t ) dt 
T

24
Server Utilization
[Characteristics of Queueing System]
 In general, for a single-server queue:
Lˆ s  ˆ  Ls   as T  

and   E ( s ) 


 For a single-server stable queue:  1


 For an unstable queue (), long-run server utilization is 1.

25
Server Utilization
[Characteristics of Queueing System]

 For G/G/c/∞/∞ queues:

 A system with c identical servers in parallel.
 If an arriving customer finds more than one server idle, the
customer chooses a server without favoring any particular
server.
 For systems in statistical equilibrium, the average number of
busy servers, Ls, is: Ls, = E(s) = .
 The long-run average server utilization is:
Ls 
  , where   c for stable systems
c c

26
 Computer Science,
Informatik 4 Communication and
Distributed Systems

Server Utilization and System

Performance

 System performance varies widely for a given utilization


• For example, a D/D/1 queue where E(A) = 1/ and
E(S) = 1/, where:
L = =  , w = E(S) = 1/ LQ = WQ = 0
,  and , server utilization can assume
- By varying
any value between 0 and 1.
- Yet there is never any line.
• In general, variability of interarrival and
service times causes lines to fluctuate in
length.

Chapter 8. Queueing
Models Dr. Mesut Güneş 27
Server Utilization and System Performance
[Characteristics of Queueing System]

 System performance varies widely for a given utilization 

 For example, a D/D/1 queue where E(A) = 1/ and E(S) =
1/, where:
L =  = , w = E(S) = 1/, LQ = WQ = 0.
 By varying  and , server utilization can assume any value
between 0 and 1.
 Yet there is never any line.
 In general, variability of interarrival and service times
causes lines to fluctuate in length.

28
 Computer Science,

Server Utilization and System Informatik 4 Communication and

Distributed Systems

Performance

• Consider the system is

simulated times:
• S1= 9, S2=12, S3 = 9, S4 = 9, S5 = 9,
….

i ⎨

• The occurrence of a
relatively long service
time (S2 = 12) causes a
waiting line to form
temporarily.
Chapter 8. Queueing
Models Dr. Mesut Güneş 29
Server Utilization and System Performance
[Characteristics of Queueing System]
 Example: A physician who schedules patients every 10 minutes and
spends Si minutes with the ith patient:  9 minutes with probability 0.9
Si  
12 minutes with probability 0.1
 Arrivals are deterministic, A1 = A2 = … = -1 = 10.
 Services are stochastic, E(Si) = 9.3 min and V(S0) = 0.81 min2.
 On average, the physician's utilization =  = 0.93 < 1.
 Consider the system is simulated with service times: S1 = 9, S2 = 12,
S3 = 9, S4 = 9, S5 = 9, …. The system becomes:

 The occurrence of a relatively long service time (S2 = 12) causes a

waiting line to form temporarily.
30
Costs in Queueing Problems
[Characteristics of Queueing System]
 Costs can be associated with various aspects of the
waiting line or servers:
 System incurs a cost for each customer in the queue, say at a
rate of $10 per hour per customer.
 The average cost per customer is:
WQj is the time
N
$10 *W jQ
j 1
N
 $10 * wˆ Q customer j spends
in queue

 If ̂ customers per hour arrive (on average), the average cost

per hour is: ˆ
 ˆ customer  $10 * wQ 
    $10 * ˆwˆ Q  $10 * LˆQ / hour
 hour  customer 

 Server may also impose costs on the system, if a group of c

parallel servers (1  c  ∞) have utilization r, each server imposes
a cost of $5 per hour while busy.
 The total server cost is: $5*c

31
Steady-State Behavior of Infinite-Population
Markovian Models M/M/C or M/G/C
 Markovian models: exponential-distribution arrival process
(mean arrival rate = ).
 Service times may be exponentially distributed as well (M) or
arbitrary (G).
 A queueing system is in statistical equilibrium if the probability
that the system is in a given state is not time dependent:
P( L(t) = n ) = Pn(t) = Pn.
 Mathematical models in this chapter can be used to obtain
approximate results even when the model assumptions do not
strictly hold (as a rough guide).
 Simulation can be used for more refined analysis (more faithful
representation for complex systems).

32
Steady-State Behavior of Infinite-Population
Markovian Models
 For the simple model studied in this chapter, the steady-state
parameter, L, the time-average number of customers in the system
is: 
L 
nPn
n 0

 Apply Little’s equation to the whole system and to the queue alone:
 L=W*lamda
L 1
w , wQ  w 
 
LQ  wQ

 G/G/c/∞/∞ example: to have a statistical equilibrium, a necessary

and sufficient condition is /(c) < 1.

33
M/G/1 Queues [Steady-State of Markovian Model]

 Single-server queues with Poisson arrivals & unlimited capacity.

 Suppose service times have mean 1/ and variance 2 and
  =  < 1, the steady-state parameters of M/G/1 queue:

   /  , P0  1  
 2 (1   2  2 )  2 (1   2  2 )
L , LQ 
2(1   ) 2(1   )
1  (1 /  2   2 )  (1 /  2   2 )
w  , wQ 
 2(1   ) 2(1   )

34
M/G/1 Queues [Steady-State of Markovian Model]

 No simple expression for the steady-state probabilities P0, P1, …

 L – LQ =  is the time-average number of customers being
served.
 Average length of queue, LQ, can be rewritten as:
2 2 2
LQ  
2(1   ) 2(1   )

 If  and  are held constant, LQ depends on the variability, 2, of the

service times.

35
M/G/2 Queues [Steady-State of Markovian Model]

 Example: Two workers competing for a job, Able claims to be faster

than Baker on average, but Baker claims to be more consistent,
 Poisson arrivals at rate  = 2 per hour (1/30 per minute).
 Able: 1/ = 24 minutes and 2 = 202 = 400 minutes2:

(1 / 30) 2 [ 24 2  400]
LQ   2.711 customers
2(1  4 / 5)

 The proportion of arrivals who find Able idle and thus experience no delay is P0
= 1- = 1/5 = 20%.
 Baker: 1/ = 25 minutes and 2 = 22 = 4 minutes2:

(1 / 30) 2 [25 2  4]
LQ   2.097 customers
2(1  5 / 6)
 The proportion of arrivals who find Baker idle and thus experience no delay is
P0 = 1- = 1/6 = 16.7%.
 Although working faster on average, Able’s greater service variability
results in an average queue length about 30% greater than Baker’s.

36
M/M/1 Queues [Steady-State of Markovian Model]

 Suppose the service times in an M/G/1 queue are

exponentially distributed with mean 1/, then the variance
is 2 = 1/2.
 M/M/1 queue is a useful approximate model when service
times have standard deviation approximately equal to their
means.
 The steady-state parameters:

   / , Pn  1     n
  2 2
L  , LQ  
   1       1 
1 1  
w  , wQ  
    (1   )        (1   )

37
M/M/1 Queues [Steady-State of Markovian Model]

 Example: M/M/1 queue with service rate  customers

per hour.
 Consider how L and w increase as arrival rate, , increases from 5
to 8.64 by increments of 20%:

l 5.0 6.0 7.2 8.64 10.0

r 0.500 0.600 0.720 0.864 1.000
L 1.00 1.50 2.57 6.35 ∞
w 0.20 0.25 0.36 0.73 ∞

 If   1, waiting lines tend to continually grow in length.

 Increase in average system time (w) and average number in
system (L) is highly nonlinear as a function of .

38
 Computer Science,
Informatik 4 Communication and
Distributed Systems

M/M/1
Queues

 Single-chair unisex hair-styling shop

• Interarrival and service times are
exponentially distributed
•  = 2 customers/hour and µ = 3
customers/hour
   23  2
L     3 2
Customers
 2
1
P0  1   3 w  L   12hour
 2
1 ⎛2⎞ 2
1 1 1 2
wQ  w   1 
P1  ⎜ ⎟ 
3⎝3⎠ 9  hour 3 3
4LQ  4
2 2 ( 
P2 1 ⎛⎜ 2 ⎟
⎞ 4
3⎝3⎠  Customers 2)
) 3(3
3 27 3
P4  1 P  16  4 2
 n 81 L  LQ    3  2 Customers
n0 3

Chapter 8. Queueing
Models Dr. Mesut Güneş 39
Effect of Utilization and Service Variability
[Steady-State of Markovian Model]
 For almost all queues, if lines are too long, they can be reduced
by decreasing server utilization () or by decreasing the service
time variability (2).
 A measure of the variability of a distribution, coefficient of
variation (cv):
V (X )
(cv) 2 
 E ( X ) 2
 The larger cv is, the more variable is the distribution relative to its
expected value

40
Effect of Utilization and Service Variability
[Steady-State of Markovian Model]
 Consider LQ for any M/G/1
queue:
 2 (1   2  2 )
LQ 
2(1   )
  2  1  (cv) 2 
  
 1    2 
  

Corrects the M/M/1

L Q for M/M/1 formula to account
queue for a non-
exponential service
time dist’n

41
Multiserver Queue [Steady-State of Markovian Model]

 M/M/c/∞/∞ queue: c channels operating in parallel.

 Each channel has an independent and identical exponential
service-time distribution, with mean 1/.
 To achieve statistical equilibrium, the offered load () must
satisfy  < c, where /(c) =  is the server utilization.
 Some of the steady-state probabilities:
   / c
1
 c 1 ( /  ) n     c 1  c  
   
P0  
 n 0
       
n!      c!  c    
 
(c ) c 1 P0  P  L ( )  c 
L  c   c 
c(c!)(1   ) 2 1 
L
w
 42
Multiserver Queue [Steady-State of Markovian Model]

 Other common multiserver queueing models:

 M/G/c/∞: general service times and c parallel server. The
parameters can be approximated from those of the M/M/c/∞/∞
model.
 M/G/∞: general service times and infinite number of servers, e.g.,
customer is its own system, service capacity far exceeds service
demand.
 M/M/C/N/∞: service times are exponentially distributed at rate m
and c servers where the total system capacity is N  c customer
(when an arrival occurs and the system is full, that arrival is turned
away).

43
Steady-State Behavior of Finite-Population
Models
 When the calling population is small, the presence of one or
more customers in the system has a strong effect on the
distribution of future arrivals.
 Consider a finite-calling population model with K customers
(M/M/c/K/K):
 The time between the end of one service visit and the next call for
service is exponentially distributed, (mean = 1/).
 Service times are also exponentially distributed.
 c parallel servers and system capacity is K.

44
Steady-State Behavior of Finite-Population
Models
 Some of the steady-state probabilities:
1
 c 1  K    n K K! 
n


P0      
 n  0  n    n c
( K  n )! c! c nc
 



 K    n
   P0 , n  0,1,..., c  1
 n 
Pn     n
 K! 
 ( K  n)!c!c n  c   , n  c, c  1,...K
 
K
L  nP ,
n 0
n w  L / e ,   e / c

where e is the long run effective arrival rate of customers to queue (or entering/exiting service)
K
e   ( K  n )P
n 0
n

45
Steady-State Behavior of Finite-Population
Models
 Example: two workers who are responsible for10 milling
machines.
 Machines run on the average for 20 minutes, then require an
average 5-minute service period, both times exponentially
distributed:  = 1/20 and  = 1/5.
 All of the performance measures depend on P0:
1
 2 1 10  5  n 10 10!  5 
n


P0       
 n 0  n  20  n  2 (10  n)!2!2
n2
 
 20 


 0.065

 Then, we can obtain the other Pn.

 Expected number of machines in system:
10
L  nP
n 0
n  3.17 machines

 The average number of running machines:

K  L  10  3.17  6.83 machines
46
Networks of Queues
 Many systems are naturally modeled as networks of single
queues: customers departing from one queue may be routed
to another.
 The following results assume a stable system with infinite
calling population and no limit on system capacity:
 Provided that no customers are created or destroyed in the
queue, then the departure rate out of a queue is the same as the
arrival rate into the queue (over the long run).
 If customers arrive to queue i at rate i, and a fraction 0  pij  1 of
them are routed to queue j upon departure, then the arrival rate
form queue i to queue j is ipij (over the long run).

47
Networks of Queues
 The overall arrival rate into queue j:
j  aj   p
all i
i ij

Arrival rate Sum of arrival rates

from outside from other queues
the network in network

 If queue j has cj < ∞ parallel servers, each working at rate j, then
the long-run utilization of each server is j=j/(cj) (where j < 1
for stable queue).
 If arrivals from outside the network form a Poisson process with
rate aj for each queue j, and if there are cj identical servers
delivering exponentially distributed service times with mean 1/j,
then, in steady state, queue j behaves likes an M/M/cj queue with
arrival rate  j  a j  
i pij
all i
48
Network of Queues
 Discount store example:
 Suppose customers arrive at the rate 80 per hour and 40%
choose self-service. Hence:
 Arrival rate to service center 1 is 1 = 80(0.4) = 32 per hour
 Arrival rate to service center 2 is 2 = 80(0.6) = 48 per hour.
 c2 = 3 clerks and 2 = 20 customers per hour.
 The long-run utilization of the clerks is:
2 = 48/(3*20) = 0.8
 All customers must see the cashier at service center 3, the
overall rate to service center 3 is 3 = 1 + 2 = 80 per hour.
 If 3 = 90 per hour, then the utilization of the cashier is:
3 = 80/90 = 0.89

49
Summary
 Introduced basic concepts of queueing models.
 Show how simulation, and some times mathematical analysis, can
be used to estimate the performance measures of a system.
 Commonly used performance measures: L, LQ, w, wQ, , and e.
 When simulating any system that evolves over time, analyst must
decide whether to study transient behavior or steady-state behavior.
 Simple formulas exist for the steady-state behavior of some queues.
 Simple models can be solved mathematically, and can be useful in
providing a rough estimate of a performance measure.

50