Scribd
Upload
184 views18 pages

STA 348 Introduction To Stochastic Processes

This document summarizes key concepts about limiting probabilities and stationary distributions of continuous-time Markov chains (CTMCs). [1] If a CTMC is ergodic, the probability of being in each state converges to a limiting value as time goes to infinity, which defines the stationary distribution. [2] The stationary distribution satisfies the global balance equations. [3] Several examples illustrate calculating the stationary distribution for birth-death processes, queueing models like M/M/1 and M/M/s, and their performance measures.

Uploaded by

scanny16
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
184 views18 pages

STA 348 Introduction To Stochastic Processes

This document summarizes key concepts about limiting probabilities and stationary distributions of continuous-time Markov chains (CTMCs). [1] If a CTMC is ergodic, the probability of being in each state converges to a limiting value as time goes to infinity, which defines the stationary distribution. [2] The stationary distribution satisfies the global balance equations. [3] Several examples illustrate calculating the stationary distribution for birth-death processes, queueing models like M/M/1 and M/M/s, and their performance measures.

Uploaded by

scanny16
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

STA 348

Introduction to
Stochastic Processes
Lecture 21

1
CMC Limiting Probabilities
 If CMC is ergodic (irreducible & +ve recurrent)
probabilities of being in state j after time t→∞
converge to limiting values Pj  lim Pij (t ) (indep.
t 
of starting state i)
 Limiting probs (stationary distr.) satisfy:
v j Pj   k  j Pk qkj , j &  j
Pj  1
 Pj is long run-proportion CMC is in state j
 If CMC starts from stationary initial distribution Pj,
then P[X(t)=j|X(0)~{Pi}] = Pj
2
Example
 Stationary distr. of Birth & Death process

3
4
Example
 Facility has #M machines and one repairman.
Every machine works for Exp(λ) time before
breaking down, and repair takes Exp(µ) time.
 Find average # of machines not in use

5
6
Example (cont’d)
 Find proportion of time that a specific machine is
working

7
Example
 Shoe shine shop has 2 chairs, one for brushing (1) and
another for polishing (2). Customers arrive with Exp(λ),
and enter only if both chairs are empty. Shoe-shiner
takes Exp(µ1) time for brushing and Exp(µ2) time for
polishing. Find the stationary distribution of this CMC.

8
Example
 A single repairman look after machines 1 & 2. Each
machine works for Exp(λi) time, and takes Exp(µi) time to
fix, for i=1,2. Moreover, machine 1 has priority (if both
are down, repairman with work on machine 1).
What proportion of time is machine 2 down?

9
10
Example
 Car insurance company uses a 4 tier (A,B,C,D) system
for ranking drivers. They have found that the average
driver moves about these tiers according to a Birth &
Death process described in the table following table. If
the annual premium depends on the tier as described in
the table, find the average annual premium for a driver.
Tier λ µ Premium
A 1 0 $600
B 2 3 $700
C 3 2 $900
D 0 1 $1200

11
12
Example (M/M/1 queue)
 Consider a single server queue (with ∞ capacity),
where customers arrive at exponential rate λ and
service times are exponential at rate µ. Find the
condition for the limiting probabilities of this queue to
exist, and solve for them.

13
14
Example
 For the M/M/1 queue with infinite capacity, show
that average # of customers in system (i.e. in
queue + in service) is λ/(µ-λ)

15
1
⁞
Example (M/M/s queue)
s
 Consider a queue with # s of servers where
customers arrive at exponential rate λ and service
times for each server are exponential at rate µ. Find
the condition for the limiting probabilities of this
queue to exist.

16
Example
 A service center consists of two servers, each working at
Exp(2), and customers arrive according with Exp(3). If the
queue has a capacity of at most 3 customers:
 What fraction of potential customers enter the system?
 If we replace the two servers with single server of double rate, i.e.
Exp(4), what fraction of potential customers enter the system?

17
18

You might also like