MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies

BSc (Hons) in Industrial Statistics

BIRTH AND DEATH PROCESSES

Continuous-time Markov chains

The process {𝑋(𝑡), 𝑡 ≥ 0} is a continuous-time Markov chain if for all 𝑠, 𝑡 ≥ 0 and non-

negative integers 𝑖, 𝑗, 0 ≤ 𝑢 < 𝑠,

𝑃(𝑋(𝑡 + 𝑠) = 𝑗|𝑋(𝑠) = 𝑖, 𝑋(𝑢) = 𝑥(𝑢), 0 ≤ 𝑢 < 𝑠) = 𝑃(𝑋(𝑡 + 𝑠) = 𝑗|𝑋(𝑠) = 𝑖).

That is, a continuous-time Markov chain is a stochastic process having the Markov

property that the conditional distribution of the future 𝑋(𝑡 + 𝑠) given the present 𝑋(𝑠)

and the past 𝑋(𝑢), 0 ≤ 𝑢 < 𝑠, depends only on the present and is independent of the past.

If, in addition, 𝑃(𝑋(𝑡 + 𝑠) = 𝑗|𝑋(𝑠) = 𝑖) is independent of 𝑠, then the continuous-time

Markov chain is said to have stationary or homogeneous transition probabilities.

Let 𝑇𝑖 be the sojourn time, that is the amount of time that the process stays in state 𝑖

before making a transition into a different state, then

𝑃(𝑇𝑖 > 𝑠 + 𝑡|𝑇𝑖 > 𝑠) = 𝑃(𝑇𝑖 > 𝑡), ∀𝑠, 𝑡 ≥ 0.

Hence, the random variable 𝑇𝑖 is memoryless and must thus be exponentially distributed
1
(see Section 3.2) with mean, say 𝜈 .
𝑖

When the process leaves state 𝑖, it next enters state 𝑗 with some probability, say, 𝑝𝑖,𝑗 that

must satisfy

𝑝𝑖,𝑖 = 0, ∀𝑖

∑ 𝑝𝑖,𝑗 = 1 , ∀𝑖.
𝑗

School of Mathematical Sciences (SMS) 4-1 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Birth and death processes

Consider a system whose state at any time is represented by the number of people in the

system at that time. Suppose that whenever there are 𝑛 people in the system, then

(i) new arrivals enter the system at an exponential rate 𝜆𝑛 , and

(ii) people leave the system at an exponential rate 𝜇𝑛 .

Such a system is called a birth and death process. The parameter {𝜆𝑛 }∞
𝑛=0 and {𝜇𝑛 }𝑛=0 are
∞

called, respectively, the arrival (or birth) and departure (or death) rates.

Thus, a birth and death process is a continuous-time Markov chain with states {0, 1, 2, … }

for which transitions from state 𝑖 may go only to either state 𝑖 − 1 or state 𝑖 + 1. The

relationship between the birth and death rates and the state transition rates and

probabilities are

𝜈0 = 𝜆0 ,

𝜈𝑖 = 𝜆𝑖 + 𝜇𝑖 , 𝑖>0

𝑝0,1 = 1,

𝜆𝑖
𝑝𝑖,𝑖+1 = , 𝑖>0
𝜆𝑖 + 𝜇𝑖
𝜇𝑖
𝑝𝑖,𝑖−1 = , 𝑖 > 0.
𝜆𝑖 + 𝜇𝑖

The Poisson process

Consider a birth and death process for which

𝜇𝑛 = 0, 𝜆𝑛 = 𝜆, ∀𝑛 ≥ 0.

This is a process which departures never occur, and the time between successive arrivals
1
is exponential with mean 𝜆. Hence, this is just the Poisson process.

School of Mathematical Sciences (SMS) 4-2 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

A birth process with linear birth rate – Yule process

Consider a population whose member can give birth to new members but cannot die. If

each member act independently of the others and takes an exponentially distributed
1
amount of time, with mean 𝜆, to give birth, then if 𝑋(𝑡) is the population size at time 𝑡,

then {𝑋(𝑡), 𝑡 ≥ 0} is a pure birth process with 𝜆𝑛 = 𝑛𝜆, 𝑛 ≥ 0. This follows since if the

population consists of 𝑛 persons and each gives birth at an exponential rate 𝜆, then the

total rate at which births occur is 𝑛𝜆.

A linear growth model with immigration

Consider each individual in a population can give birth at an exponential rate 𝜆 ; in

addition, there is an exponential rate of increase 𝜃 of the population due to an external

source such as immigration. Deaths are assumed to occur at an exponential rate 𝜇 for

each member of the population. If 𝑋(𝑡) is the population size at time 𝑡, then {𝑋(𝑡), 𝑡 ≥ 0}

is a linear growth model with immigration, such that

𝜇𝑛 = 𝑛𝜇, 𝑛≥1

𝜆𝑛 = 𝑛𝜆 + 𝜃, 𝑛 ≥ 0.

School of Mathematical Sciences (SMS) 4-3 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

The queueing system M/M/1

Suppose that customers arrive at a single-server service station in accordance with a

Poisson process having rate 𝜆 . That is, the times between successive arrivals are
1
independent exponential random variables having mean 𝜆. Upon arrival, each customer

goes directly into service if the server is free; if not, then the customer joins the queue.

When the server finishes serving a customer, the customer leaves the system and the next

customer in line, if there are any waiting, enters the service. The successive service times
1
are assumed to be independent exponential random variables having mean 𝜇.

It is known as the M/M/1 queuing system. The first M refers to the fact that the

interarrival process is Markovian (since it is a Poisson process) and the second refers to

the fact that service distribution is exponential (and, hence Markovian). The 1 refers to

the fact that there is a single server.

If let 𝑋(𝑡) denote the number of customers in the system at time 𝑡 then {𝑋(𝑡), 𝑡 ≥ 0} is a

birth and death process with

𝜇𝑛 = 𝜇, 𝑛≥1

𝜆𝑛 = 𝜆, 𝑛≥0

School of Mathematical Sciences (SMS) 4-4 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

A multi-server exponential queuing system (M/M/s)

Consider an exponential queuing system in which there are 𝑠 servers available, each

serving at rate 𝜇. An entering customer first waits in line and then goes to the first free

server. If there are 𝑛 customers in the system, where 𝑛 ≤ 𝑠, then 𝑛 servers will be busy.

Since each of these servers works at rate 𝜇, the total departure rate will be 𝑛𝜇. On the

other hand, if there are 𝑛 customers in the system, where 𝑛 > 𝑠, then all 𝑠 of the servers

will be busy, and thus the total departure rate will be 𝑠𝜇. Hence, this is a birth and death

process with parameters

𝑛𝜇, 1 ≤ 𝑛 ≤ 𝑠
𝜇𝑛 = {
𝑠𝜇, 𝑛>𝑠

𝜆𝑛 = 𝜆, 𝑛≥0

Expected time to enter state 𝑖 + 1 from state 𝑖, 𝑖 ≥ 0

Consider a general birth and death process with birth rates {𝜆𝑛 } and death rates {𝜇𝑛 },

where 𝜇0 = 0, and let 𝑇𝑖 denote the time, starting from state 𝑖, it takes for the process to

enter state 𝑖 + 1, 𝑖 ≥ 0. Since 𝑇0 is exponential with rate 𝜆0 ,

1
E[𝑇0 ] = .
𝜆0

For 𝑖 > 0, conditioning on whether the first transition takes the process into state 𝑖 − 1

or 𝑖 + 1. That is, let

1, if the first transition from 𝑖 is to 𝑖 + 1

𝐼𝑖 = {
0, if the first transition from 𝑖 is to 𝑖 − 1

School of Mathematical Sciences (SMS) 4-5 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Hence,

1
E[𝑇𝑖 |𝐼𝑖 = 1] = ,
𝜆𝑖 + 𝜇𝑖

1
E[𝑇𝑖 |𝐼𝑖 = 0] = + E[𝑇𝑖−1 ] + E[𝑇𝑖 ]
𝜆𝑖 + 𝜇𝑖

Regardless of whether the first transition is from a birth or death, the time until it occurs

is exponential with rate 𝜆𝑖 + 𝜇𝑖 ; if the first transition is a birth, then the population size is

at 𝑖 + 1, so no additional time is needed; whereas if it is a death, then the population size

becomes 𝑖 − 1 and the additional time needed to reach 𝑖 + 1 is equal to the time it takes

to return to state 𝑖 (this has mean E[𝑇𝑖−1 ]) plus the additional time it takes to reach 𝑖 + 1

(this has mean E[𝑇𝑖 ]). Hence, since the probability that the first transition is a birth is

𝜆𝑖
,
𝜆𝑖 +𝜇𝑖

1 𝜇𝑖
E[𝑇𝑖 ] = + (E[𝑇𝑖−1 ] + E[𝑇𝑖 ])
𝜆𝑖 + 𝜇𝑖 𝜆𝑖 + 𝜇 𝑖

or equivalently,

1 𝜇𝑖
E[𝑇𝑖 ] = + E[𝑇𝑖−1 ], 𝑖 ≥ 1.
𝜆𝑖 𝜆𝑖

Expected time to enter state 𝑗 from state 𝑖, 𝑖 < 𝑗

Consider the birth and death process having parameters 𝜆𝑖 ≡ 𝜆, 𝜇𝑖 ≡ 𝜇. From Example

4.2.6,

1 𝜇 1
E[𝑇𝑖 ] = + E[𝑇𝑖−1 ] = (1 + 𝜇E[𝑇𝑖−1 ])
𝜆 𝜆 𝜆
1
Starting with E[𝑇0 ] = 𝜆,

1 𝜇
E[𝑇1 ] = (1 + )
𝜆 𝜆

1 𝜇 𝜇 2
E[𝑇2 ] = [1 + + ( ) ]
𝜆 𝜆 𝜆

School of Mathematical Sciences (SMS) 4-6 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

and so on. In general,

1 𝜇 𝜇 2 𝜇 𝑖
E[𝑇𝑖 ] = [1 + + ( ) + ⋯ + ( ) ]
𝜆 𝜆 𝜆 𝜆

𝜇 𝑖+1
1−( )
= 𝜆 , 𝑖≥0
𝜆−𝜇

Hence, the expected time to reach state 𝑗, starting at state 𝑘, 𝑘 < 𝑗,

𝑗−1

E[time to go from state 𝑘 to state 𝑗] = ∑ E[𝑇𝑖 ]

𝑖=𝑘

𝜇 𝑘+1 [1 − (𝜇 )𝑗−𝑘 ]
𝑗−𝑘 ( ) 𝜆
= − 𝜆 𝜇
𝜆−𝜇 𝜆−𝜇 1−
𝜆

Variance time to enter state 𝑗 from state 𝑖, 𝑖 < 𝑗

From Example 4.2.6,

1
E[𝑇𝑖 |𝐼𝑖 ] = + (1 − 𝐼𝑖 )(E[𝑇𝑖−1 ] + E[𝑇𝑖 ])
𝜆𝑖 + 𝜇𝑖

Thus,

Var(E[𝑇𝑖 |𝐼𝑖 ]) = (E[𝑇𝑖−1 ] + E[𝑇𝑖 ])2 Var(𝐼𝑖 )

𝜇𝑖 𝜆𝑖
= (E[𝑇𝑖−1 ] + E[𝑇𝑖 ])2
(𝜆𝑖 + 𝜇𝑖 )2

𝜆𝑖
since 𝐼𝑖 is a Bernoulli random variable with parameter 𝑝 = .
𝜆𝑖 + 𝜇𝑖

School of Mathematical Sciences (SMS) 4-7 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Let 𝑋𝑖 denote the time until the transition from 𝑖 occurs, then
1
Var(𝑇𝑖 |𝐼𝑖 = 1) = Var(𝑋𝑖 |𝐼𝑖 = 1) = Var(𝑋𝑖 ) =
(𝜆𝑖 + 𝜇𝑖 )2

Var(𝑇𝑖 |𝐼𝑖 = 0) = Var(𝑋𝑖 + time to get back to 𝑖 + time to reach 𝑖 + 1)

= Var(𝑋𝑖 ) + Var(𝑇𝑖−1 ) + Var(𝑇𝑖 )

Hence,

Var(𝑇𝑖 |𝐼𝑖 ) = Var(𝑋𝑖 ) + (1 − 𝐼𝑖 )(Var(𝑇𝑖−1 ) + Var(𝑇𝑖 ))

And so,

1 𝜇𝑖
E[Var(𝑇𝑖 |𝐼𝑖 )] = + (Var(𝑇𝑖−1 ) + Var(𝑇𝑖 ))
(𝜆𝑖 + 𝜇𝑖 )2 𝜆𝑖 + 𝜇𝑖

By the law of total variance,

Var(𝑇𝑖 ) = E[Var(𝑇𝑖 |𝐼𝑖 )] + Var(E[𝑇𝑖 |𝐼𝑖 ])

1 𝜇𝑖 2
𝜇𝑖 𝜆𝑖
= + (Var(𝑇𝑖−1 ) + Var(𝑇𝑖 )) + (E[𝑇𝑖−1 ] + E[𝑇𝑖 ])
(𝜆𝑖 + 𝜇𝑖 )2 𝜆𝑖 + 𝜇𝑖 (𝜆𝑖 + 𝜇𝑖 )2

1 𝜇𝑖 𝜇𝑖
= + Var(𝑇𝑖−1 ) + (E[𝑇𝑖−1 ] + E[𝑇𝑖 ])2
(𝜆
𝜆𝑖 𝑖 + 𝜇 𝑖 ) 𝜆𝑖 𝜆𝑖 + 𝜇 𝑖

Consider the birth and death process having parameters 𝜆𝑖 ≡ 𝜆, 𝜇𝑖 ≡ 𝜇, from Example
1
4.2.6, starting with Var(𝑇0 ) = 𝜆2 , and using the recursion to compute Var(𝑇𝑖 ). In addition,

to find the variance of the time to reach state 𝑗, starting from state 𝑘, 𝑘 < 𝑗, this can be

expressed as the time to go from 𝑘 to 𝑘 + 1, plus the additional time to go from 𝑘 + 1 to

𝑘 + 2 , and so on. By the Markov property, these successive random variables are

independent, it follows that

𝑗−1

Var(time to go from state 𝑘 to state 𝑗) = ∑ Var(𝑇𝑖 )

𝑖=𝑘

School of Mathematical Sciences (SMS) 4-8 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Chapman-Kolmogorov equations

Let

𝑝𝑖,𝑗 (𝑡) = 𝑃(𝑋(𝑡 + 𝑠) = 𝑗|𝑋(𝑠) = 𝑖)

denote the probability that a process presently in state 𝑖 will be in state 𝑗 a time 𝑡 later.

These quantities are often called the transition probabilities of the continuous-time

Markov chain.

Proposition 4.3.1

For a pure birth process having 𝜆𝑖 ≠ 𝜆𝑗 , when 𝑖 ≠ 𝑗

𝑗 𝑗 𝑗−1 𝑗−1
−𝜆𝑘 𝑡
𝜆𝑟 𝜆𝑟
𝑝𝑖,𝑗 (𝑡) = (∑ 𝑒 ∏ ) − (∑ 𝑒 −𝜆𝑘 𝑡 ∏ ), 𝑖<𝑗
𝜆𝑟 − 𝜆𝑘 𝜆𝑟 − 𝜆𝑘
𝑘=1 𝑟≠𝑘,𝑟=𝑖 𝑘=𝑖 𝑟≠𝑘,𝑟=𝑖

𝑝𝑖,𝑖 (𝑡) = 𝑒 −𝜆𝑖 𝑡

Proof. Let 𝑋𝑘 (𝑡) denote the time the process spends in state 𝑘 before making a transition

into state 𝑘 + 1, 𝑘 ≥ 1. Suppose that the process is currently in state 𝑖, and let 𝑗 > 𝑖. Then

as 𝑋𝑖 is the time it spends in state 𝑖 before moving to state 𝑖 + 1, and 𝑋𝑖+1 is the time it

𝑗−1
then spends in state 𝑖 + 1 before moving to state 𝑖 + 2, and so on, it follows that ∑𝑘=𝑖 𝑋𝑘

is the time it takes until the process enters state 𝑗. Now suppose if the process has not yet

entered state 𝑗 by time 𝑡, then its state at time 𝑡 is smaller than 𝑗, and vice versa. That is,

𝑋(𝑡) < 𝑗 ⇔ 𝑋𝑖 + ⋯ + 𝑋𝑗−1 > 𝑡

Therefore, for 𝑖 < 𝑗, for a pure birth process,

𝑗−1

𝑃(𝑋(𝑡) < 𝑗|𝑋(0) = 𝑖) = 𝑃 (∑ 𝑋𝑘 > 𝑡).

𝑘=𝑖

School of Mathematical Sciences (SMS) 4-9 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Since 𝑋𝑖 , …, 𝑋𝑗−1 are independent exponential random variables with respective rates

𝜆𝑖 , …, 𝜆𝑗−1,

𝑗−1 𝑗−1
𝜆𝑟
𝑃(𝑋(𝑡) < 𝑗|𝑋(0) = 𝑖) = ∑ 𝑒 −𝜆𝑘𝑡 ∏
𝜆𝑟 − 𝜆𝑘
𝑘=𝑖 𝑟≠𝑘,𝑟=𝑖

Replacing 𝑗 by 𝑗 + 1,

𝑗 𝑗
−𝜆𝑘 𝑡
𝜆𝑟
𝑃(𝑋(𝑡) < 𝑗 + 1|𝑋(0) = 𝑖) = ∑ 𝑒 ∏
𝜆𝑟 − 𝜆𝑘
𝑘=𝑖 𝑟≠𝑘,𝑟=𝑖

Hence,

𝑃(𝑋(𝑡) = 𝑗|𝑋(0) = 𝑖) = 𝑃(𝑋(𝑡) < 𝑗 + 1|𝑋(0) = 𝑖) − 𝑃(𝑋(𝑡) < 𝑗|𝑋(0) = 𝑖)

𝑗 𝑗 𝑗−1 𝑗−1
𝜆𝑟 𝜆𝑟
= (∑ 𝑒 −𝜆𝑘𝑡 ∏ ) − (∑ 𝑒 −𝜆𝑘𝑡 ∏ )
𝜆 𝑟 − 𝜆𝑘 𝜆𝑟 − 𝜆𝑘
𝑘=1 𝑟≠𝑘,𝑟=𝑖 𝑘=𝑖 𝑟≠𝑘,𝑟=𝑖

and 𝑝𝑖,𝑖 (𝑡) = 𝑃(𝑋𝑖 > 𝑡) = 𝑒 −𝜆𝑖 𝑡 .

Definition. Instantaneous transition rate

For a pair of states 𝑖 and 𝑗, let

𝑞𝑖,𝑗 = 𝜈𝑖 𝑝𝑖,𝑗 .

Since 𝜈𝑖 is the rate at which the process makes a transition when in state 𝑖 and 𝑝𝑖,𝑗 is the

probability that this transition is into state 𝑗, it follows that 𝑞𝑖,𝑗 is the rate, when in state

𝑖, at which the process makes a transition into state 𝑗. Hence, the quantities 𝑞𝑖,𝑗 are called

the instantaneous transition rates. Since

𝜈𝑖 = ∑ 𝜈𝑖 𝑝𝑖,𝑗 = ∑ 𝑞𝑖,𝑗
𝑗 𝑗

and

𝑞𝑖,𝑗 𝑞𝑖,𝑗
𝑝𝑖,𝑗 = =
𝜈𝑖 ∑𝑗 𝑞𝑖,𝑗

School of Mathematical Sciences (SMS) 4-10 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

It follows that specifying the instantaneous transition rates determines the parameters

of the continuous-time Markov chain.

Lemma 4.3.2

1−𝑝𝑖,𝑖 (ℎ)
(a) lim = 𝜈𝑖
ℎ→0 ℎ

𝑝𝑖,𝑗 (ℎ)
(b) lim = 𝑞𝑖,𝑗 , when 𝑖 ≠ 𝑗
ℎ→0 ℎ

Proof.

(a) Since the amount of time until a transition occurs is exponentially distributed, it

follows that the probability of two or more transitions in a time ℎ is 𝑜(ℎ). Thus 1 −

𝑝𝑖,𝑖 (ℎ), the probability that a process in state 𝑖 at time 0 will not be in state 𝑖 at time

ℎ, equals the probability that a transition occurs within time ℎ plus something small

compared to ℎ. Therefore,

1 − 𝑝𝑖,𝑖 (ℎ) = 𝜈𝑖 ℎ + 𝑜(ℎ)

(b) 𝑝𝑖,𝑗 (ℎ) is the probability that the process goes from state 𝑖 to state 𝑗 at a time ℎ,

equals the probability that a transition occurs in this time multiplied by the

probability that the transition is into state 𝑗, plus something small compared to ℎ.

That is,

𝑝𝑖,𝑗 (ℎ) = ℎ𝜈𝑖 𝑝𝑖,𝑗 + 𝑜(ℎ)

School of Mathematical Sciences (SMS) 4-11 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Lemma 4.3.3 Chapman-Kolmogorov equations

For all 𝑠 ≥ 0, 𝑡 ≥ 0,
∞

𝑝𝑖,𝑗 (𝑡 + 𝑠) = ∑ 𝑝𝑖,𝑘 (𝑡)𝑝𝑘,𝑗 (𝑠)

𝑘=0

Proof. In order for the process to go from state 𝑖 to state 𝑗 in time 𝑡 + 𝑠, it must be

somewhere at time 𝑡 and thus

𝑝𝑖,𝑗 (𝑡 + 𝑠) = 𝑃(𝑋(𝑡 + 𝑠) = 𝑗|𝑋(0) = 𝑖)

= ∑ 𝑃(𝑋(𝑡 + 𝑠) = 𝑗, 𝑋(𝑡) = 𝑘|𝑋(0) = 𝑖)

𝑘=0

= ∑ 𝑃(𝑋(𝑡 + 𝑠) = 𝑗, 𝑋(𝑡) = 𝑘, 𝑋(0) = 𝑖)𝑃(𝑋(𝑡) = 𝑘|𝑋(0) = 𝑖)

𝑘=0

= ∑ 𝑃(𝑋(𝑡 + 𝑠) = 𝑗|𝑋(𝑡) = 𝑘)𝑃(𝑋(𝑡) = 𝑘|𝑋(0) = 𝑖)

𝑘=0

= ∑ 𝑝𝑘,𝑗 (𝑠)𝑝𝑖,𝑘 (𝑡)

𝑘=0

School of Mathematical Sciences (SMS) 4-12 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Theorem 4.3.4 Kolmogorov’s backward equations

For all states 𝑖, 𝑗 and times 𝑡 ≥ 0,

′
𝑝𝑖,𝑗 (𝑡) = ∑ 𝑞𝑖,𝑘 𝑝𝑘,𝑗 (𝑡) − 𝜈𝑖 𝑝𝑖,𝑗 (𝑡)
𝑘≠𝑖

Proof. From Lemma 4.3.3,

∞

𝑝𝑖,𝑗 (𝑡 + ℎ) − 𝑝𝑖,𝑗 (𝑡) = ∑ 𝑝𝑖,𝑘 (ℎ)𝑝𝑘,𝑗 (𝑡) − 𝑝𝑖,𝑗 (𝑡)

𝑘=0

= ∑ 𝑝𝑖,𝑘 (ℎ)𝑝𝑘,𝑗 (𝑡) − [1 − 𝑝𝑖,𝑖 (ℎ)]𝑝𝑖,𝑗 (𝑡)

𝑘≠𝑖

and thus,

𝑝𝑖,𝑗 (𝑡 + ℎ) − 𝑝𝑖,𝑗 (𝑡) 𝑝𝑖,𝑘 (ℎ) 1 − 𝑝𝑖,𝑖 (ℎ)

lim = lim {∑ 𝑝𝑘,𝑗 (𝑡) − [ ] 𝑝𝑖,𝑗 (𝑡)}
ℎ→0 ℎ ℎ→0 ℎ ℎ
𝑘≠𝑖

And hence from Lemma 4.3.2,

′
𝑝𝑖,𝑗 (𝑡) = ∑ 𝑞𝑖,𝑘 𝑝𝑘,𝑗 (𝑡) − 𝜈𝑖 𝑝𝑖,𝑗 (𝑡)
𝑘≠𝑖

Kolmogorov’s backward equations for birth and death process

The backward equations for the pure birth process become

′
𝑝𝑖,𝑗 (𝑡) = 𝜆𝑖 𝑝𝑖+1,𝑗 (𝑡) − 𝜆𝑖 𝑝𝑖,𝑗 (𝑡)

The backward equations for the birth and death process become
′
𝑝0,𝑗 (𝑡) = 𝜆0 𝑝1,𝑗 (𝑡) − 𝜆0 𝑝0,𝑗 (𝑡)

= 𝜆0 (𝑝1,𝑗 (𝑡) − 𝑝0,𝑗 (𝑡))

′
𝜆𝑖 𝜇𝑖
𝑝𝑖,𝑗 (𝑡) = (𝜆𝑖 + 𝜇𝑖 ) [ 𝑝𝑖+1,𝑗 (𝑡) + 𝑝 (𝑡)] − (𝜆𝑖 + 𝜇𝑖 )𝑝𝑖,𝑗 (𝑡)
𝜆𝑖 + 𝜇𝑖 𝜆𝑖 + 𝜇𝑖 𝑖−1,𝑗

= 𝜆𝑖 𝑝𝑖+1,𝑗 (𝑡) + 𝜇𝑖 𝑝𝑖−1,𝑗 (𝑡) − (𝜆𝑖 + 𝜇𝑖 )𝑝𝑖,𝑗 (𝑡), 𝑖>0

School of Mathematical Sciences (SMS) 4-13 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Theorem 4.3.6 Kolmogorov’s forward equations

Under suitable regularity conditions,

′
𝑝𝑖,𝑗 (𝑡) = ∑ 𝑞𝑘,𝑗 𝑝𝑖,𝑘 (𝑡) − 𝜈𝑗 𝑝𝑖,𝑗 (𝑡)
𝑘≠𝑗

Proof. From Lemma 4.3.3,

∞

𝑝𝑖,𝑗 (𝑡 + ℎ) − 𝑝𝑖,𝑗 (𝑡) = ∑ 𝑝𝑖,𝑘 (𝑡)𝑝𝑘,𝑗 (ℎ) − 𝑝𝑖,𝑗 (𝑡)

𝑘=0

= ∑ 𝑝𝑖,𝑘 (𝑡)𝑝𝑘,𝑗 (ℎ) − [1 − 𝑝𝑗,𝑗 (ℎ)]𝑝𝑖,𝑗 (𝑡)

𝑘≠𝑗

and thus,

𝑝𝑖,𝑗 (𝑡 + ℎ) − 𝑝𝑖,𝑗 (𝑡) 𝑝𝑘,𝑗 (ℎ) 1 − 𝑝𝑗,𝑗 (ℎ)

lim = lim {∑ 𝑝𝑖,𝑘 (𝑡) −[ ] 𝑝𝑖,𝑗 (𝑡)}
ℎ→0 ℎ ℎ→0 ℎ ℎ
𝑘≠𝑗

And assuming that can interchange limit with summations, from Lemma 4.3.2,

′
𝑝𝑖,𝑗 (𝑡) = ∑ 𝑞𝑘,𝑗 𝑝𝑖,𝑘 (𝑡) − 𝜈𝑗 𝑝𝑖,𝑗 (𝑡)
𝑘≠𝑗

School of Mathematical Sciences (SMS) 4-14 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Exercise 4.3.7

Show that for a pure birth process,

𝑝𝑖,𝑖 (𝑡) = 𝑒 −𝜆𝑖 𝑡 , 𝑖≥0

𝑡
𝑝𝑖,𝑗 (𝑡) = 𝜆𝑗−1 𝑒 −𝜆𝑗𝑡 ∫ 𝑒 𝜆𝑗 𝑠 𝑝𝑖,𝑗−1 (𝑠) 𝑑𝑠 , 𝑗 ≥𝑖+1
0

School of Mathematical Sciences (SMS) 4-15 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Kolmogorov’s forward equations for birth and death process

The forward equations for the general pure and birth process are

′
𝑝𝑖,0 (𝑡) = ∑ 𝑞𝑘,0 𝑝𝑖,𝑘 (𝑡) − 𝜆0 𝑝𝑖,0 (𝑡)
𝑘≠0

= 𝜇1 𝑝𝑖,1 (𝑡) − 𝜆0 𝑝𝑖,0 (𝑡)

′
𝑝𝑖,𝑗 (𝑡) = ∑ 𝑞𝑘,𝑗 𝑝𝑖,𝑘 (𝑡) − (𝜆𝑗 + 𝜇𝑗 )𝑝𝑖,𝑗 (𝑡)
𝑘≠𝑗

= 𝜆𝑗−1 𝑝𝑖,𝑗−1 (𝑡) + 𝜇𝑗+1 𝑝𝑖,𝑗+1 (𝑡) − (𝜆𝑗 + 𝜇𝑗 )𝑝𝑖,𝑗 (𝑡)

Limiting probabilities

Consider the set of forward equations and let 𝑡 → ∞,

′
lim 𝑝𝑖,𝑗 (𝑡) = lim ∑ 𝑞𝑘,𝑗 𝑝𝑖,𝑘 (𝑡) − 𝜈𝑗 𝑝𝑖,𝑗 (𝑡)
𝑡→∞ 𝑡→∞
𝑘≠𝑗

= ∑ 𝑞𝑘,𝑗 𝜋𝑘∗ − 𝜈𝑗 𝜋𝑗∗

𝑘≠𝑗

where

𝜋𝑗∗ ≡ lim 𝑝𝑖,𝑗 (𝑡)

𝑡→∞

is the limiting probability that a continuous-time Markov chain will be in state 𝑗 at time 𝑡

converging to a limiting value that is independent of the initial state 𝑖.

School of Mathematical Sciences (SMS) 4-16 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Since 𝑝𝑖,𝑗 (𝑡) is a bounded function (being a probability it is always between 0 and 1), it

′
follows that if 𝑝𝑖,𝑗 (𝑡) converges, it must converge to 0. Hence, the balance equations,

0 = ∑ 𝑞𝑘,𝑗 𝜋𝑘∗ − 𝜈𝑗 𝜋𝑗∗

𝑘≠𝑗

𝜈𝑗 𝜋𝑗∗ = ∑𝑘≠𝑗 𝑞𝑘,𝑗 𝜋𝑘∗ , ∀𝑗 (4.1)

and along with

∑𝑗 𝜋𝑗∗ = 1 (4.2)

can be used to solve for the limiting probabilities.

Remark.

It is assumed that the limiting probabilities 𝜋𝑗∗ exist. A sufficient condition for this is that

(i) all states of the Markov chain communicate in the sense that starting in state 𝑖,

there is a positive probability of ever being in state 𝑗, ∀𝑖, 𝑗; and

(ii) the Markov chain is positive recurrent in the sense that, starting in any state, the

mean time to return to that state is finite.

Limiting probabilities for birth and death process

From Eq. (4.1) or equivalently, by equating the rate at which the process leaves a state

with the rate at which it enters that state,

𝜆0 𝜆1 𝜆2 𝜆𝑛−1 𝜆𝑛
0 1 2 3 𝑛 𝑛+1

𝜇1 𝜇2 𝜇3 𝜇𝑛 𝜇𝑛+1

Figure 4.4.1: Rate diagram of a birth and death process (Example 4.4.1)

School of Mathematical Sciences (SMS) 4-17 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

State Rate at which leave = rate at which enter

0 𝜆0 𝜋0∗ = 𝜇1 𝜋1∗

1 (𝜆1 + 𝜇1 )𝜋1∗ = 𝜇2 𝜋2∗ + 𝜆0 𝜋0∗

2 (𝜆2 + 𝜇2 )𝜋2∗ = 𝜇3 𝜋3∗ + 𝜆1 𝜋1∗

∗
(𝜆𝑛 + 𝜇𝑛 )𝜋𝑛∗ = 𝜇𝑛+1 𝜋𝑛+1 ∗
𝑛, 𝑛 ≥ 1 + 𝜆𝑛−1 𝜋𝑛−1

By adding to each equation the equation preceding it,

𝜆0 𝜋0∗ = 𝜇1 𝜋1∗

𝜆1 𝜋1∗ = 𝜇2 𝜋2∗

𝜆2 𝜋2∗ = 𝜇3 𝜋3∗

⋮
∗
𝜆𝑛 𝜋𝑛∗ = 𝜇𝑛+1 𝜋𝑛+1 , 𝑛≥0

Solving in terms of 𝜋0∗ yields,

𝜆0 ∗
𝜋1∗ = 𝜋
𝜇1 0

𝜆1 ∗ 𝜆1 𝜆0 ∗
𝜋2∗ = 𝜋 = 𝜋
𝜇2 1 𝜇2 𝜇1 0

𝜆2 ∗ 𝜆2 𝜆1 𝜆0 ∗
𝜋3∗ = 𝜋 = 𝜋
𝜇3 2 𝜇3 𝜇2 𝜇1 0

𝜆𝑛−1 ∗ 𝜆𝑛−1 𝜆𝑛−2 𝜆𝑛−3 … 𝜆0 ∗

𝜋𝑛∗ = 𝜋𝑛−1 = 𝜋
𝜇𝑛 𝜇𝑛 𝜇𝑛−1 𝜇𝑛−2 … 𝜇1 0

Using the fact that ∑∞ ∗

𝑛=0 𝜋𝑛 = 1,

∞
𝜆𝑛−1 𝜆𝑛−2 𝜆𝑛−3 … 𝜆0
1 = 𝜋0∗ + 𝜋0∗ ∑
𝜇𝑛 𝜇𝑛−1 𝜇𝑛−2 … 𝜇1
𝑛=1

1
𝜋0∗ =
𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
1 + ∑∞
𝑛=1 𝜇 𝜇 𝜇 … 𝜇
1 2 3 𝑛

School of Mathematical Sciences (SMS) 4-18 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

and so, for 𝑛 ≥ 1

𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
𝜋𝑛∗ =
𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
𝜇1 𝜇2 𝜇3 … 𝜇𝑛 (1 + ∑∞ 𝑛=1 𝜇 𝜇 𝜇 … 𝜇 )
1 2 3 𝑛

Hence, for the limiting probabilities to exist, it is necessary that

∞
𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
∑ < ∞.
𝜇1 𝜇2 𝜇3 … 𝜇𝑛
𝑛=1

A machine repair model

Consider a job shop that consists of 𝑀 machines and one serviceman. Suppose that the

amount of time each machine runs before breaking down is exponentially distributed
1
with mean 𝜆, and suppose that the amount of time that it takes for the serviceman to fix a

1
machine is exponentially distributed with mean 𝜇.

(a) What is the average number of machines not in use?

(b) What proportion of time is each machine in use?

If the system is in state 𝑛 whenever 𝑛 machines are not in use, then the preceding is a

birth and death process having parameters

𝜇𝑛 = 𝜇, 𝑛≥1

(𝑀 − 𝑛)𝜆, 𝑛 ≤ 𝑀
𝜆𝑛 = {
0, 𝑛>𝑀

1
𝜋0∗ =
𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
1 + ∑∞
𝑛=1 𝜇 𝜇 𝜇 … 𝜇
1 2 3 𝑛

1 1
= =
𝑀𝜆 ∙ (𝑀 − 1)𝜆 ∙ (𝑀 − 2)𝜆 ∙ … ∙ [𝑀 − (𝑛 − 1)]𝜆 𝜆 𝑛 𝑀!
1 + ∑𝑀
𝑛=1 𝑛 1 + ∑ 𝑀
𝑛=1 𝜇 ) (𝑀 − 𝑛)!
(
𝜇

School of Mathematical Sciences (SMS) 4-19 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
𝜋𝑛∗ =
𝜆0 𝜆1 𝜆2 … 𝜆𝑛−1
𝜇1 𝜇2 𝜇3 … 𝜇𝑛 (1 + ∑∞ 𝑛=1 𝜇 𝜇 𝜇 … 𝜇 )
1 2 3 𝑛

𝜆 𝑛
𝑀!
(𝜇 )
(𝑀 − 𝑛)!
=
𝑀 𝜆 𝑛 𝑀!
∑
1 + 𝑛=1 (𝜇 )
(𝑀 − 𝑛)!

(a) Hence, the average number of machines not in use is given by

𝜆 𝑛 𝑀!
𝑀 ∑𝑀𝑛=0 𝑛 ( )
𝜇 (𝑀 − 𝑛)!
∑ 𝑛𝜋𝑛∗ = 𝑛
𝜆 𝑀!
𝑛=0 1 + ∑𝑀𝑛=1 (𝜇 ) (𝑀 − 𝑛)!

(b) The long-run proportion of time that a given machine is working is given by

𝑃(machine is working)

= ∑ 𝑃(machine is working|𝑛 machines is not working)𝜋𝑛∗

𝑛=0

𝑀
𝑀−𝑛 ∗
=∑ 𝜋𝑛
𝑀
𝑛=0

𝑀
𝑛𝜋𝑛∗
= 1−∑
𝑀
𝑛=0

Queuing theory

Some fundamental quantities of interest for queuing models are

𝐿 the average number of customers in the systems;

𝐿𝑞 the average number of customers waiting in queue;

𝑊 the average amount of time a customer spends in the system;

𝑊𝑞 the average amount of time a customer spends waiting in queue.

School of Mathematical Sciences (SMS) 4-20 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

A stochastic model can be developed based on costs or rewards associated with the

system policies. For example, consider that entering customers are forced to pay money

to the system, then the cost equation is

average rate at which the system earns

= 𝜆𝑎 × average amount an entering customer pays (4.3)

where 𝜆𝑎 is the average arrival rate of entering customers. That is, if 𝑁(𝑡) denotes the

number of customer arrivals by time 𝑡, then

𝑁(𝑡)
𝜆𝑎 = lim
𝑡→∞ 𝑡

For instance, suppose that each customer pays $1 per unit time while in the system, Eq.

= 𝜆𝑎 × average amount an entering customer pays (4.3) yields the Little’s formula,

𝐿 = 𝜆𝑎 𝑊

Similarly, suppose that each pays $1 per unit while in queue, Eq. = 𝜆𝑎 ×

average amount an entering customer pays (4.3) yields

𝐿𝑞 = 𝜆𝑎 𝑊𝑞

By supposing that each customer pays $1 per unit time while in service,

average number of customers in service = 𝜆𝑎 E[𝑆]

where E[𝑆] is the average amount of time a customer spends in service.

Let 𝑁(𝑡) denote the number of customers in the system at any time 𝑡 and 𝜋𝑛∗ be the

limiting or long-run probability that there will be exactly 𝑛 customers in the system.

Then, the two set of limiting probabilities are {𝑎𝑛 , 𝑛 ≥ 0} and {𝑑𝑛 , 𝑛 ≥ 0}, where

𝑎𝑛 = proportion of customers that find 𝑛 in the system when they arrive, and

𝑑𝑛 = proportion of customers leaving behind 𝑛 in the system when they depart

School of Mathematical Sciences (SMS) 4-21 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Proposition 4.5.1 PASTA principle

Poisson arrivals always see time averages. In particular, for Poisson arrivals,

𝜋𝑛∗ = 𝑎𝑛

People arrive at a bus stop according to a Poisson process with rate 𝜆. Bus arrive at the

stop according to a Poisson process with rate 𝜇, with each arriving bus picking up all the

currently waiting people.

Let 𝑊𝑞 be the average amount of time that a person waits at the stop for a bus. Since the

waiting time of each person is equal to the time from when they arrive until the next bus,

which is exponentially distributed with rate 𝜇, then

1
𝑊𝑞 =
𝜇

The average number of people waiting at the bus stop, averaged over all time, is

𝜆
𝐿𝑞 = 𝜆𝑊𝑞 =
𝜇

To illustrate the PASTA principle, let 𝑁𝑖 be the number of people picked up by the 𝑖-th

bus, then 𝑇𝑖 equal to the time between the (𝑛 − 1)-th and the 𝑛-th bus arrival,

E[𝑁𝑖 |𝑇𝑖 ] = 𝜆𝑇𝑖 (4.4)

because the number of people that arrive at the stop in any time interval is Poisson with

a mean equal to 𝜆 times the length of the time interval.

School of Mathematical Sciences (SMS) 4-22 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Since 𝑇𝑖 is exponential with rate 𝜇, taking expectation on both sides of Eq. E[𝑁𝑖 |𝑇𝑖 ] = 𝜆𝑇𝑖

(4.4),

𝜆
E[𝑁𝑖 ] = 𝜆E[𝑇𝑖 ] =
𝜇

Thus, the average number of people picked up by a bus is equal to the time average

number of people waiting for a bus, as follows from the PASTA principle.

M/M/1 queue with impatient customers

Consider a single server queue where customers arrive according to a Poisson process

with rate 𝜆 and where the service time distribution is exponential with rate 𝜇. Suppose

that each customer will only spend an exponential time with rate 𝛼 in queue before

quitting the system. Assume that the impatient times are independent of all else, and that

a customer who enters service always remains until its service is completed. This system

can be modelled as a birth and death process with birth and death rates

𝜆𝑛 = 𝜆, 𝑛≥0

𝜇𝑛 = 𝜇 + (𝑛 − 1)𝛼, 𝑛≥1

Suppose 𝜋𝑠 is the proportion of arrivals that receive service and 𝜆𝑠 be the average rate at

which customers are served, then

𝜆𝑠
𝜋𝑠 =
𝜆

Since the service departure rate is zero when the system is empty and is 𝜇 when the

system is non-empty, 𝜆𝑠 = 𝜇(1 − 𝜋0∗ ) yields that

𝜇
𝜋𝑠 = (1 − 𝜋0∗ )
𝜆

School of Mathematical Sciences (SMS) 4-23 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Remark.

Following Example 4.5.3, to determine the proportion of all events that are of a certain

type 𝑖, 𝑖 = 1, 2, 3, … , 𝑘,

rate at which type 𝑖 events occur

proportion of events that are type 𝑖 =
rate at which all events occur

To determine the average time that a customer spends in the system, for the birth and

death queuing system,

∞

𝐿 = ∑ 𝑛𝜋𝑛∗
𝑛=0

Also, the average arrival rate of customer is

∞

𝜆𝑎 = ∑ 𝜆𝑛 𝜋𝑛∗
𝑛=0

Consequently,

𝐿 ∑∞
𝑛=0 𝑛𝜋𝑛
∗
𝑊= =
𝜆 𝑎 ∑∞𝑛=0 𝜆𝑛 𝜋𝑛
∗

Now consider 𝑎𝑛 , the proportion of arrivals that find 𝑛 in the system. Since arrivals are at

rate 𝜆𝑛 whenever there are 𝑛 is the system, it follows that the rate at which arrivals find

𝑛 in 𝜆𝑛 𝜋𝑛∗ . Hence in a large amount of time 𝑡, approximately 𝜆𝑛 𝜋𝑛∗ 𝑡 of the approximately

𝜆𝑎 𝑡 arrivals will encounter 𝑛. Hence

𝜆𝑛 𝜋𝑛∗ 𝑡 𝜆𝑛 𝜋𝑛∗
𝑎𝑛 = lim =
𝑡→∞ 𝜆𝑎 𝑡 𝜆𝑎

School of Mathematical Sciences (SMS) 4-24 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Now consider the average length of a busy period, where the system alternates between

idle periods when there are no customers in the system and busy periods when there is

at least one customer in the system. An idle period begins when the system is empty and

ends when the next customers arrives. Since the arrival rate when the system is empty is

𝜆0 , it follows that, independent of the past, the length of an idle period is exponential with

rate 𝜆0 . The lengths of successive busy periods are independent and identically

distributed as well, since a busy period always begins when there is one in the system

and ends when the system is empty.

Let 𝐼𝑗 and 𝐵𝑗 denote, respectively, the lengths of the 𝑗-th idle and the 𝑗-th busy period, 𝑗 ≥

1 . In the first ∑𝑛𝑗=1(𝐼𝑗 + 𝐵𝑗 ) time units, the system will be empty for a time ∑𝑛𝑗=1 𝐼𝑗 .

Consequently, the long-run proportion of time in which the system is empty,

𝐼1 + ⋯ + 𝐼𝑛
𝜋0∗ = lim
𝑛→∞ 𝐼1 + ⋯ + 𝐼𝑛 + 𝐵1 + ⋯ + 𝐵𝑛

(𝐼1 + ⋯ + 𝐼𝑛 )
= lim 𝑛
𝑛→∞ (𝐼1 + ⋯ + 𝐼𝑛 ) (𝐵 + ⋯ + 𝐵𝑛 )
+ 1 𝑛
𝑛

E[𝐼]
=
E[𝐼] + E[𝐵]

1
𝜆0
=
1
+ E[𝐵]
𝜆0

1
=
1 + 𝜆0 E[𝐵]

or

1 − 𝜋0∗
E[𝐵] =
𝜆0 𝜋0∗

School of Mathematical Sciences (SMS) 4-25 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023
 MAT3034 Stochastic Processes B Sc (Hons) in Actuarial Studies
BSc (Hons) in Industrial Statistics

Now, suppose 𝑇𝑛 is the amount of time during a busy period that there are 𝑛 in the

system, and E[𝑇𝑛 ] is the average amount of time there are 𝑛 in the system in intervals

between successive busy periods. Since the average time between successive busy period

is E[𝐵] + E[𝐼],

E[𝑇𝑛 ]
𝜋𝑛∗ =
E[𝐼] + E[𝐵]

E[𝑇𝑛 ]𝜋0∗
=
E[𝐼]

Hence,

𝜋𝑛∗
E[𝑇𝑛 ] =
𝜆0 𝜋0∗

School of Mathematical Sciences (SMS) 4-26 April 2023 Semester

These teaching materials are protected under the Copyright Act 1987. Duplication, in any form, including digitally, is prohibited by law and a punishable offence. © 2023