Dr.

Nawel ARRAR REMITA

November 2, 2023
 Continuous-Time Markov Chains
DTMCs are totaly synchronized, in that the state only changes
at discrete time steps, whereas in CTMCs the state can
change at any time. This makes CTMCs more realistic for
modeling computer systems, where events can occur at any
time. In preparation for CTMCs, we need to dicuss the
exponential distribution and Poisson arrival process.
Let’s consider the residence time Ti at state i in the context
of a continuous-time process fXt gt 0 .
By hypothesis, we have the following property: The residence
time Ti is a continuous random variable with truly positive
values, thus prohibiting instantaneous visits. In fact, it follows
an exponential law with parameter 0 λi < ∞, with λi = 0
when the i is an absorbing state.
 Exponential Distribution
De…nition
We say that a random variable X is distributed exponentially
with rate λ, X E (λ) , if X has the probability density
function:
λe λt , t 0
f (x ) =
0, t < 0
and the cummulative distribution function,
F (x ) = 1 e λt 1t 0 . So F (x ) = e λt , t 0.
In addition,
1 2 1
E [X ] = , E X 2 = 2 , Var (X ) = 2
λ λ λ
h i λ
E e tX = for t < λ.
λ t
 Exponential Distribution

Remark
And the square coe¢ cient of variation of random varaible X is
de…ned as
Var (X )
CVX2 =
E [X ]2
This can be thought of as the scaled or normalized variance.
When X E (λ) , CVX2 = 1. A random variable X is said to
be memoryless if

P (X > s + t/X > s ) = P (X > t ) , 8s, t 0.

 Exponential Distribution

Proposition
Given X1 , X2 , ..., Xn independant r.v of exponential law with
respective parameters λ1 , λ2 , ..., λn . We de…ne
X = min (X1 , X2 , ..., Xn ).
The variables X1 , X2 , ..., Xn represent times before di¤erent
events occur, and then X is the time until the …rst of these
events occurs. We have the following properties:
 Exponential Distribution

Theorem
Given X1 , X2 , ..., Xn i.i.d of E (λi ) , 1 i n. Let
X = min (X1 , X2 , ..., Xn ) . Then
a)
X E (λ1 + λ2 + ... + λn ) ;
b)

λi
P (X Xi ) = , 8i = 1, 2, ..., n;
λ1 + λ2 + ... + λn
c) The event X Xi is independent of the r.v X .
 Processus de Poisson
A Poisson process is a continuous-time Markov chain
fXt , t 0g , which counts the number of arrivals of an event
that occur randomly over time at a constant rate, for example
customers or accidents. The intensity of a Poisson process is
the instantaneous rate λ > 0 with which arrivals occur.
A ‡ow of random events can be described mathematically in
two di¤erent ways:
1. The number of events Xt occurring in [0, t ] and seek to
determine the probability law of this discrete random
variable. The process fXt , t 0g , is called the
"counting process".
2. The times intervals that separate the instants of
occurrence of two consecutive events is called
"inter-arrival time". These are independent and
identically distributed r.v with law E (λ). This process is
called the "birth process".
 Processus de Poisson

Let’s recall the important properties of the Poisson process:

- it is homogeneous, which means that the rate of appearance
(birth) λ is independent of time ;
- it is a law with "independent increases": the numbers of
events occurring X1 and X2 over two disjoint time intervals T1
and T2 , are independent random variables.
- the probability of "simultaneous" occurrence (i.e. between t
and t + ∆t) of several events (birth) is negligible: o∆t.
 Processus de Poisson

De…nition
A Poisson process having rate λ is a sequence of events such
that

(λt )k λt
P (X (t ) = k ) = pk (t ) = e , λ > 0, k 0;
k!
E [X (t )] = λt et Var [X (t )] = λt.

De…nition
These relations de…ne the transitional state of the Poisson
process. No stationary state exists, since
pk = lim pk (t ) = 0, 8k 0.
t !∞
 Processus de Poisson

De…nition
The interval time time Ti , (i 1) that separates any instant
from the next event is a random variable distributed according
to an E (λ) law.
 Particular Markov Processes
Birth and Death Processes
The processes in question can generally be used to write the
temporal evolution of the size of a population of a given type.
They are widely used to model waiting phenomena or systems
subject to repairable failures. They are obtained by
superimposing a birth process and a death process. This
Markov process Xt represents the size of a population at time
t. These are stochastic processes with continuous time and
discrete states space (S = f0, 1, 2, ...g ). They are
characterized by two important conditions:
- Memoryless;
- transitions are only possible to one or other of the
neighboring states,from state n the possible
transitions are n 1 and n + 1 with n 1.
 Birth and Death Process
Let Xt = N (t ) be the size of a population at time t (number
of individuals present). De…ne
pn (t ) = P (N (t ) = n) (during [0, t ]);
and pij (t ) be the probability that at the moment t the number
of individuals is j such that there were already i individuals in
the population. We have that
pi ,j (t ) = P(N (t + s ) = j/N (s ) = i ) does not depend on s
(the process is homogeneous), so
pi ,i +1 (∆t ) = λi ∆t + o∆t,
pi ,i 1 (∆t ) = µi ∆t + o∆t,
pi ,i (∆t ) = 1 (λi + µi )∆t + o∆t,
1, i = j
pi ,j (∆t ) = o∆t if ji j j 2 and pi ,j (0) = δij =
0, i 6= j
with λi > 0, µi > 0 and µ0 = 0.
 Birth and Death Process
Transitory State

Let the transitions probabilities (or states probabilities)

pn (t ) = P(N (t ) = n), 8n 0 during the interval time [(0, t ]
The transitory state is described by

P(t + ∆t ) = P(t ) M

where M is transition matrix. Letting ∆t ! 0, we obtain the

following Kolmogorov equations system:
8
>
> p00 (t ) = λ0 p0 (t ) + µ1 p1 (t )
<
p10 (t ) = λ0 p0 (t ) (λ1 + µ1 )p1 (t ) + µ2 p2 (t )
>
>
: 0
pn (t ) = λn 1 pn 1 (t ) (λn + µn )pn (t ) + µn +1 pn +1 (t ), n 1.
(1)
 Birth and Death Process
Stationory state
In state of equilibrium, also know as steady state, the behavior
of the process is independent of the time and of the initial
state; i,e, pn = lim pn (t ), n = 0, 1, 2...which is the stationary
t !∞
distribution of the process under study. Therefore,
lim p 0 (t ) = 0.
t !∞ n
Then, these probabilities satisfy the following balance
equations or statistical equilibrium system (obtained from
(1) by taking):
8
>
> λ0 p0 = µ1 p1
<
(λ1 + µ1 )p1 = λ0 p0 + µ2 p2
(2)
>
>
:
(λn + µn )pn = λn 1 pn 1 + µn +1 pn +1 , n 1.
∞
To solve this system, we add ∑ pn = 1.
n =0
 Simple Markovian Queueing Systems
The formation of queues is a frequent phenomenon that
occurs whenever service demands exceed the permitted
capacity of service devices.
The general model of a queueing phenomenon (waiting
system) can be summarized as follows: requests (customers)
arrive at a certain location and demand a certain service. If a
service device is free, the incoming request immediately goes
to this device, where it is served. Otherwise, there are two
possibilities: either the request leaves the system (denied
request systems), or it goes into a queue (queuing system). At
some point, the request is selected for service according to a
discipline. On completion of the service, the request leaves the
system.
A queuing system therefore comprises a service area with one
or more service devices (servers), and a waiting area in which a
possible queue is formed.
 Simple Markovian Queueing Systems
Classi…cation of queueing systems
To identify a queuing system, we need to specify the entry
rate, the service mechanism and the waiting discipline.
- The size of a potential customer population (or
the number of sources) can be …nite or in…nite.
- The process (‡ow) of arrivals can be regular or
random. The time between two successive arrivals
follows a probability law.
- The service mechanism includes the number of
servers and the distribution service times.
- Queuing discipline or service discipline, customers
can be selected and served in order of arrival
FCFS, or LCFS, or selected at random
RANDOM.
- Waiting space capacity can be unlimited or not.
 Simple Markovian Queueing Systems
Classi…cation of queueing systems
We use a symbolic notation "Kendall notation" comprising 4
symbols arranged in the order A/B/s/m, where A and B
describe respectively the distribution of times between two
successive arrivals and the distribution of service times, s is
the number of servers (connected in parallel), m is the system
capacity (the number of servers plus the number of waiting
positions). The last symbol can be omitted if m = ∞.
To specify distributions A and B, we introduce the following
symbols:
M Exponential distribution ;
Ek Erlang distribution of degree k ;
Hk Hyperexponential distribution of degree k ;
D Deterministic ;
G General distribution.
 Simple Markovian Queueing Systems
Mathematical analysis

The mathematical study of a queuing system involves the

introduction of an appropriately de…ned stochastic process.
The stationary distribution of the introduced stochastic
process allows us to obtain the system’s performance
characteristics, such as such as:
- the mean number of customers in system n;
- the mean number of customers in the queue nq ;
- the mean waiting time for a customer W ;
- the mean amount of time a customer spends in the system
Ws.
 Simple Markovian Queueing Systems
Mathematical analysis
One of a more important and useful relationship in queueing
theory is what is communly known as Little’s Law:

n = λWs, nq = λW ,

and
λ 1
n = nq + , Ws = W + ,
µ µ
where λ is the rate at which customers enter the system, µ1 is
the mean service time (µ > 0). Another important measure
for a queueing system, the one that measures the degree of
saturation of the system, is the tra¢ c intensity ρ. It is de…ned
by
mean service time
ρ= .
mean time between two successive arrivals
 Queueing Model M/M/1
Model Description

Customers arrive at the system according to a Poisson process

of rate λ > 0, (mean number of customers arriving during a
unit of time) ; i.e., the time interval between two successive
arrivals follows an exponential law with parameter λ > 0. The
service is provided by a single server. When a customer arrives,
if the server is free, it is immediately taken over. Otherwise,
the customer is put on a queue. Queuing capacity is unlimited
(the number of positions is in…nite and no other restrictions
are imposed). Waiting discipline is FCFS. Service times follow
an exponential distribution with parameter µ > 0.
Consequently, the service rate is µ (mean number of
customers served during a unit of time), and the mean time of
service for a customer is µ1 .
 Queueing Model M/M/1
Model Description
Finaly, we assume that the r.v representing the times between
two consecutive arrivals and the service times are mutually
independent.
 Queueing Model M/M/1
Model analysis

The state of the system at date t can be described by the

stochastic process

fN (t ) , t 0g , (3)

which is a birth and death process and states space

S = f0, 1, 2, ...g. The rates of transitions are λn = λ, 8n 0
and µn = µ, for n 1, (µ0 = 0).
 Queueing Model M/M/1
Transitory State
Let pn (t ) = P(N (t ) = n), 8n 0 be the probability of
observing n customers in the system at time t. The
representative transition graph of an M/M/1 queue is as
follows
 Queueing Model M/M/1

For the limiting probabilities lim Pn (t ) = pn , the state

t !∞
balance equations are obtained. Hence, we get
n
λ λ
pn = 1 = (1 ρ ) ρn , n 0,
µ µ
λ
where ρ = µ < 1.
 Queueing Model M/M/1
Mesures de performance

Mean number of customers in the system

ρ
n = µ λ λ = (1 ρ ) .
λ2
Mean number of customers in the queue nq = µ(µ λ)
.
Mean time a customer spends in the system Ws = µ 1 λ .
Mean waiting time for a customer W = µ(µλ λ) = µ(1ρ ρ) .
 Queueing Model M/M/s
Model description

Customers arrive at the system according to a Poisson process

of rate λ > 0. Service is provided by s 1 servers connected
in parallel. When a customer arrives, if one of the servers is
free, the customer immediately begins service. In the opposite
case (all servers are occupied by the service), the customer
takes his place in the queue, common to all servers. Waiting
capacity is unlimited. When a server is free, the customer at
the head of the queue occupies the freed-up server.
Consequently, the waiting discipline is FCFS. Service times are
exponentially distributed with …nite mean 1/µ. Times
between two consecutive arrivals and service times are
mutually independent.
Queueing Model M/M/s
Model description
 Queueing Model M/M/s
Analyse du modèle
The state of the system at date t can be described using the
process (3), whose state space S = f0, 1, 2, ...g. The latter is
a birth and death process whose transition rates are:
λn = λ, 8n 0 and µn = min fn, s g µ, n 1( µ 0 = 0) .
Let the state probabilities pn (t ) = P(N (t ) = n), 8n 0. The
representative graphe of transitions
 Queueing Model M/M/s
Stationary state

As before, for the limit probabilities, from the transition graph

and simpli…cation, we obtain :
8 n
< pn = 1 λ p0 , 1 n < s;
n! µ
n
: pn = 1 1 λ
p0 , n s.
s! sn s µ

where,
2 s 3 1
s 1 n λ
1 λ
p0 = 4 ∑
µ
+ 5 .
n =0 n! µ λ
s! 1 sµ
 Queueing Model M/M/s
Stationary state

We get

n
1 λ
pn = p0 , 1 n<s
n! µ
s
1 λ
pn = ρn s
p0 , n s
s! µ

provided sλµ = ρ < 1.

ρ is the total tra¢ c intensity .
 Queueing Model M/M/s
Stationary state

We can write
pn = ρn s
ps , n s
The probabilité ps that the customer will have to wait for
service if only the number of customers n s, called Erlang
formula, it is given by
s
λ
µ (sρ)s
ps = p0 = p0 .
s! 1 λ s! (1 ρ)
sµ
 Queueing Model M/M/s
Mesures de performance

Mean number of customers in the system

s +1
λ
λ µ ρps
n= µ + 2 p0 = sρ + (1 ρ )
.
λ
ss ! 1 sµ

Mean number of customers in the queue

s +1
λ
µ ρps
nq = 2 p0 = (1 ρ )
.
λ
ss ! 1 sµ

Mean time a customer spends in the system

s
λ
1 1
+ s µ(p1s
µ
Ws = µ + 2 p0 = µ ρ)
.
λ
s µs ! 1 sµ

Mean waiting time for a customer

s
λ
µ ps
W = p0 = .
s !s µ(1 ρ)2 s µ (1 ρ )