Scribd
Upload
93 views2 pages

IEOR 6711: Stochastic Models I Fall 2003, Professor Whitt Class Lecture Notes: Tuesday, November 18. Solutions To Problems For Discussion

This document contains lecture notes summarizing solutions to stochastic modeling problems discussed in class. The problems involve modeling systems with exponential holding times as continuous-time Markov chains (CTMCs). Specifically, [1] a salesman traveling between three cities is modeled, and [2] two machines repaired by one repairman are modeled under different priority assumptions. The notes provide the CTMC formulations and solve for the limiting and stationary distributions of the processes.

Uploaded by

scanny16
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
93 views2 pages

IEOR 6711: Stochastic Models I Fall 2003, Professor Whitt Class Lecture Notes: Tuesday, November 18. Solutions To Problems For Discussion

This document contains lecture notes summarizing solutions to stochastic modeling problems discussed in class. The problems involve modeling systems with exponential holding times as continuous-time Markov chains (CTMCs). Specifically, [1] a salesman traveling between three cities is modeled, and [2] two machines repaired by one repairman are modeled under different priority assumptions. The notes provide the CTMC formulations and solve for the limiting and stationary distributions of the processes.

Uploaded by

scanny16
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

IEOR 6711: Stochastic Models I

Fall 2003, Professor Whitt

Class Lecture Notes: Tuesday, November 18.

Solutions to Problems for Discussion

1. A salesman flies back and forth among the cities Atlanta (A), Boston (B) and Chicago
(C) as follows: The salesman stays in each city an exponential length of time, with the mean
being 1/4 month if the city is A or B, but with the mean being 1/5 month if the city is C.
From A, the salesman goes next to B or C with probability 1/2 each; from B, the salesman
goes next to A with probability 3/4, and to C with probability 1/4; from C, the salesman
always goes next to A.
(a) Find the limiting fraction of time that the salesman spends in each city.
———————————————————————-
For this problem formulation, it is natural to use the SMP (semi-Markov process) formu-
lation of a CTMC, involving the embedded DTMC and the mean holding times in each state.
We thus define the embedded transition matrix P directly and the mean holding times 1/νi
directly. We can then obtain the Q matrix by setting Qi,j = νi Pi,j . But we do not need the Q
matrix to solve for the steady-state distribution.
In general, the steady-state probability is

πi (1/νi )
αi = P
j πj (1/νj )

In this case, the steady state probability vector is

(1/2, 1/4, 1/4)

———————————————————————-
(b) What is the average number of trips per year from Boston to Atlanta?
———————————————————————-
The long-run fraction of time spent in Boston is 1/4, by part (a). When in Boston, the
rate of trips from B to A is 3 per months, from the Q matrix constructed in part (a). So the
average number of trips from B to A is: 3 months per year × 3 trips per month = 9 trips per
year.
———————————————————————-

2. Consider two machines that are maintained by a single repairman. Machine i functions
for an exponentially distributed amount of time with mean 1/λi before it fails. The repair
times for each unit are exponential with mean 1/µi . They are repaired in the order in which
they fail.
(a) Let {X(t) : t ≥ 0} be a stochastic process, where X(t) represents the number of working
machines at time t. Is {X(t) : t ≥ 0} a Markov process?
———————————————————————-
This process is not a Markov process. When both machines are down, the next transition
depends on which of the two machines failed first.
———————————————————————-
(b) Formulate a CTMC describing the evolution of the system.
———————————————————————-
Let the states be: 0, 1, 2, (1, 2) and (2, 1). State i means only machine i has failed. State
(i, j) means that both machines have failed, with machine i failing first, and thus currently
being repaired. It is now natural to formulate the CTMC by directly defining the infinitesimal
generator matrix Q. The positive elements of the Q matrix are

Q0,1 = λ1 Q0,2 = λ2
Q1,0 = µ1 Q1,(1,2) = λ2
Q2,0 = µ2 Q2,(2,1) = λ1
Q(1,2),2 = µ1 Q(2,1),1 = µ2

———————————————————————-
(c) Suppose that λ1 = 1, µ1 = 2, λ2 = 3 and µ2 = 4. Find the stationary distribution.
———————————————————————-
By solving αQ = 0, we see that the steady-state distribution is

(44/129, 16/129, 36/129, 24/129, 9/129)

———————————————————————-
(d) Now suppose, instead, that machine 1 is much more important than machine 2, so that
the repairman will always service machine 1 if it is down, regardless of the state of machine 2.
Formulate a CTMC for this modified problem and find the stationary distribution.
———————————————————————-
Now it suffices to let the states be: 0, 1, 2 and {1, 2}, indicating which machines have
failed. When both machines are failed, the order does not matter now, because the repairman
always works on machine 1. The new transition rate is Q{1,2},2 = µ1 . The new steady-state
probabilities are
(20/57, 4/57, 18/57, 15/57)

———————————————————————-

You might also like