0% found this document useful (0 votes)
397 views14 pages

Exercises Markov

This document contains exercises from a textbook on Markov chains for an industrial management course. It lists several problems from Chapter 17 of the textbook "Operations Research: Applications and Algorithms" by Wayne Winston. The problems cover topics like formulating transition probability matrices for Markov chains that model machine breakdowns, vehicle conditions over time, and calculating expected costs over multiple time periods for different replacement strategies. Solutions to the problems are provided in detail.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
397 views14 pages

Exercises Markov

This document contains exercises from a textbook on Markov chains for an industrial management course. It lists several problems from Chapter 17 of the textbook "Operations Research: Applications and Algorithms" by Wayne Winston. The problems cover topics like formulating transition probability matrices for Markov chains that model machine breakdowns, vehicle conditions over time, and calculating expected costs over multiple time periods for different replacement strategies. Solutions to the problems are provided in detail.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Exercise sessions for the module

Decision Making for Industrial Management


(B-KUL-H0T99A)
Prof Johan W. Joubert

Contents
Markov chains, Chapter 17 1
Winston (2004), Section 17.2, Problem A3† . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Winston (2004), Section 17.5, Problem A4† . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Winston (2004), Section 17.5, Problem B13† . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Winston (2004), Section 17.6, Problem A1† . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Winston (2004), Section 17.7, Problem C7† . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Winston (2004), Chapter 17, Review problem B12† . . . . . . . . . . . . . . . . . . . . . . 8
Winston (2004), Section 17.7, Problem C9† . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Winston (2004), Chapter 17, Review problem A3† . . . . . . . . . . . . . . . . . . . . . . 10
Winston (2004), Section 17.6, Problem 9† . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Winston (2004), Chapter 17, Review problem A4† . . . . . . . . . . . . . . . . . . . . . . 12
Winston (2004), Section 17.3, Problem A1† . . . . . . . . . . . . . . . . . . . . . . . . . . 13

[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Markov chains, Chapter 17
Winston (2004), Section 17.2, Problem A3†
A company has two machines. During any day, each machine that is working at the beginning of
the day has a 13 chance of breaking down. If a machine breaks down during the day, it is sent to a
repair facility and will be working two days after it breaks down. (Thus, if a machine breaks down
during day 3, it will be working again at the beginning of day 5). Letting the states be the system
be the number of machines working at the beginning of the day, formulate a transition probability
matrix for this situation.

The graphical representation of the stochastic process for one machine is shown in Figure 1, where
W indicates a working machine at the start of the day, N W a non-working machine at the start of
the day (breaking down the previous day).

2 1
W 3 W 3 NW 1 W
1 2
3 3 2
3

1
NW 1 W 3 NW

t0 t1 t2 t3

Figure 1

The following transition probability matrix has, as its states, the number of machines working
at the start of the day.

0 1 2
 
0 0 0 1
1 2
P = 1 0

3 3
1 4 4
2 9 9 9

1
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.5, Problem A4†
At the beginning of each year, my car is in good, fair, or broken-down condition. A good car will
be good at the beginning of next year with a probability of 0.85, fair with a probability of 0.10, or
broken-down with a probability of 0.05. A fair car will be fair at the beginning of the next year with
a probability of 0.70 or broken-down with a probability of 0.30. It costs $6 000 to purchase a good
car; a fair car can be traded in for $2 000; and a broken-down car has no trade-in value and must
immediately be replaced by a good car. It costs $1 000 per year to operate a good car and $1 500 to
operate a fair car. Should I replace my car as soon as it becomes a fair car, or should I drive my car
until it breaks down? Assume that the cost of operating a car during a year depends on the type of
car on hand at the beginning of the year (after a new car, if any, arrives).

The states of the Markov chain are the condition of the vehicle at the start of the year: G for good,
F for fair and B for being broken down. We investigate two scenarios. The first is to replace it when
it becomes fair. This yields the graphical representation in Figure 2 and the following probability
transition matrix.
G F B
 
G0.85 0.10 0.05
P = F 0.85 0.10 0.05
B 0.85 0.10 0.05

We first find the steady-state probabilities.

G 0.85 G 0.85 G 0.85 G


0.10 0.10 0.10

0.05 F 0.05 F 0.05 F

B B B

t0 t1 tn−1 tn

Figure 2

π = πP
G F B
 
  0.85 0.10 0.05
G
= π1 π2 π3 F 0.85 0.10 0.05
B 0.85 0.10 0.05

which results in π1 = 0.85π1 + 0.85π2 + 0.85π3


π2 = 0.10π1 + 0.10π2 + 0.10π3
π3 = 0.05π1 + 0.05π2 + 0.05π3
and knowing that 1 = π1 + π2 + π3

Solving the three equations simultaneously1 yields π1 = 0.85, π2 = 0.10 and π3 = 0.05. To calculate
the total cost, TCfair , we sum over all possible states, multiplying the (steady-state) probability with
1
Note that the transitions matrix is already in the form where all values in each column are the same. So, you
need not calculate the values but can read them off and use them as-is.

2
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
the expected cost of operating a vehicle in that state. So, for this first scenario, we can now calculate
the total cost.

TCfair = π1 (1000) + π2 (1000 + 4000) + π3 (1000 + 6000)


= 0.85(1000) + 0.10(1000 + 4000) + 0.05(1000 + 6000)
= 1 700

For the second scenario, we consider when we only replace the car when it is broken down. The
graphical representation is given in Figure 3 and the following transition probability matrix.

G F B
 
G 0.85 0.10 0.05
P = F 0.00 0.70 0.30
B 0.85 0.10 0.05

We find the steady-state probabilities.

G 0.85 G 0.85 G 0.85 G


0.1 0.1 0.1
0.05 0.05

0.05 F 0.7 F 0.7 F


0.3 0.3

B B B

t0 t1 tn−1 tn

Figure 3

π = πP
G F B
 
  G0.85 0.10 0.05
= π1 π2 π3 F 0.00 0.70 0.30
B 0.85 0.10 0.05

which results in π1 = 0.85π1 + 0.85π3


π2 = 0.10π1 + 0.70π2 + 0.10π3
π3 = 0.05π1 + 0.30π2 + 0.05π3
and knowing that 1 = π1 + π2 + π3

Solving the three equations simultaneously yields π1 = 0.6375, π2 = 0.25 and π3 = 0.1125. We
calculate the total cost, TCbroken , in the same way.

TCbroken = π1 (1000) + π2 (1500) + π3 (1000 + 6000)


= 0.6375(1000) + 0.25(1500) + 0.1125(1000 + 6000)
= 1800

In conclusion, the expected annual cost is lower at $ 1 700 to trade in and replace your vehicle
when it is in a fair condition than to wait until it is broken down.

3
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.5, Problem B13†
An important machine is known to never last more than four months. During its first month of
operation, it fails 10% of the time. If the machine completes its first month, then it fails during its
second month 20% of the time. If the machine completes its second month of operation, then it will
fail during its third month 50% of the time. If the machine completes its third month, then it is sure
to fail by the end of the fourth month. At the beginning of each month, we must decide whether or
not to replace our machine with a new machine. It costs $500 to purchase a new machine, but if a
machine fails during a month, we incur a cost of $1 000 (due to factory downtime) and must replace
the machine (at the beginning of the next month) with a new machine. Three maintenance policies
are under consideration:
Policy 1 Plan to replace a machine at the beginning of its fourth month of operation.
Policy 2 Plan to replace a machine at the beginning of its third month of operation.
Policy 3 Plan to replace a machine at the beginning of its second month of operation.
Which policy will give the lowest average monthly cost?

We define the states to be the age of the machine at the start of the month. Policy 1 starts with
the transition matrix
0 1 2 3
 
0.1 0.9 0
0 0
10.2 0 0.8 0 
P=  
20.5 0 0 0.5
3 0.1 0.9 0 0

which yields π = 0.244 0.344 0.275 0.137 . This allows us to compute the expected cost per
 

month, which is $431.50.


Policy 2 starts with the transition matrix

0 1 2
 
0.1 0.9 0
0
P = 10.2 0 0.8
2 0.1 0.9 0

which yields π = 0.147 0.476 0.377 and an expected cost per month of $410.50.
 

Policy 3 starts with the transition matrix

0 1
 
0 0.1 0.9
P=
1 0.1 0.9

which yields π = 0.1 0.9 and an expected cost per month of $600.
 

Consequently, policy 2 is the cheapest option, suggesting that the machine should be replaced
at the beginning of its third month of operation.

4
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.6, Problem A1†
The State College admissions office has modelled the path of a student through State College as a
Markov chain:
Fr So J Sen Q G
 
0.1 0.8
Freshman 0 0 0.1 0
Sophmore 0 0.1 0.85 0 0.05 0 
 
Junior 0
 0 0.15 0.8 0.05 0 
P=  
Senior 0 0 0 0.1 0.05 0.85
 
Quits 0 0 0 0 1 0 
Graduates 0 0 0 0 0 1

Each student’s state is observed at the beginning of each fall semester. For example, if a student
is a junior at the beginning of the current fall semester, there is an 80% chance that s/he will be a
senior at the beginning of the next fall semester, a 15% chance that s/he will still be a junior and a
5% chance that s/he will have quit. (We assume that once a student quits, s/he never reenrolls.)
a) If a student enters State College as a freshman, how many years can s/he expect to spend as
a student at State?
b) What is the probability that a freshman graduates?

The graphical representation is given in Figure 4. The transition probability matrix can be split
0.1 0.1 0.15 0.1

Fr 0.8 So 0.85 Jr 0.8 Sr 0.85 G

0.1 0.05 0.05 0.05

Figure 4

into two matrices, Q and R.


a) We can then calculate the fundamental matrix.

Fr So Jr Sr
 
Freshman1.11 0.99 0.99 0.88
Sophmore 0 1.11 1.11 0.99
(I − Q)−1 =  
Junior 0 0 1.18 1.05
Senior 0 0 0 1.11

If you start as a freshman, the expected number of years as a freshman is 1.11, then 0.99 as
a sophomore, 0.99 as a junior and 0.88 years as a senior. Added together, yields 3.97 years.
Yes, this is less than 4 years because many entering freshman quit before their senior year.
b) The probability that a freshman graduate is taken from the matrix

Q G
 
0.254
Freshman 0.746
(I − Q)−1 R =
Sophmore
0.160 0.840


Junior0.111 0.889
Senior 0.056 0.944

which gives a result of 0.748.

5
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.7, Problem C7†
For simplicity, suppose that fresh blood obtained by a hospital will spoil if it is not transfused within
five days. The hospital receives 100 pints of fresh blood daily from a local blood bank. Two policies
are possible for determining the order in which blood is transfused (See Table 1). For example, under

Table 1

Age of blood
(beginning of the day
Chance of transfusion 0 1 2 3 4
Policy 1 0.1 0.2 0.3 0.4 0.5
Policy 2 0.5 0.4 0.3 0.2 0.1

policy 1, blood has a 10% chance of being transfused during its first day at the hospital. Under
policy 2, four-day-old blood has a 10% chance of being transfused.
a) A FIFO (first in, first out) blood-issuing policy issues “old” blood first, whereas a LIFO (last
in, first out) policy issues “young” blood first. Which policy represents a LIFO policy, and
which represents a FIFO policy?
b) For each policy, determine the probability that a new pint of blood will spoil.
c) For each policy, determine the average number of pints of blood in inventory.
d) For each policy, find the average age of transfused blood.
e) Comment on the relative merits of a FIFO policy and a LIFO policy.

a) Policy 1 is the FIFO policy as the oldest blood has the highest probability of being used.
Likewise, Policy 2 is the LIFO policy as the newest addition has the highest probability of
being transfused.
b) For Policy 1, the transition probability matrix is
0 1 2 3 4 Spoiled Used
 
0 0 0.9 0 0 0 0 0.1
10 0 0.8 0 0 0 0.2 
 
20 0 0 0.7 0 0 0.3 
 
PFIFO = 30
 0 0 0 0.6 0 0.4 

4
0 0 0 0 0 0.5 0.5 

 
Spoiled0 0 0 0 0 1 0 
Used 0 0 0 0 0 0 1
We first calculate the fundamental matrix
0 1 2 3 4
 
0 1 0.9 0.72 0.504 0.302
10 1 0.8 0.56 0.336
(I − QFIFO )−1
 
= 20 0
 1 0.7 0.42 

30 0 0 1 0.6 
4 0 0 0 0 1
And then we can calculate
Spoiled Used
 
0 0.151 0.849
1 0.168 0.832
(I − QFIFO )−1 RFIFO
 
 0.21
= 2 0.79 

3 0.3 0.7 
4 0.5 0.5

6
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
We read the probability from the matrix as 0.151, or 15.1%.

0 1 2 3 4 Spoiled Used
 
0 0 0.5 0 0 0 0 0.5
10 0 0.6 0 0 0 0.4 
 
20 0 0 0.7 0 0 0.3 
 
30 0 0 0 0.8 0 0.2 
PLIFO =  
4
0 0 0 0 0 0.9 0.1 

 
Spoiled0 0 0 0 0 1 0 
Used 0 0 0 0 0 0 1

We first calculate the fundamental matrix


0 1 2 3 4
 
0 1 0.5 0.3 0.21 0.168
10 1 0.6 0.42 0.336
(I − QLIFO )−1
 
= 20 0
 1 0.7 0.56  
30 0 0 1 0.8 
4 0 0 0 0 1

And then we can calculate


Spoiled Used
 
0 0.151 0.849
1 0.302
 0.698
(I − QLIFO )−1 RLIFO
 
 0.504
= 2 0.496
3 0.72 0.28 
4 0.9 0.1

We again read the probability from the matrix as 0.151, or 15.1%., the same as for Policy 1!
c) For Policy 1, we calculate the number of units in inventory as 100 multiplied by the row sum
of (I − QFIFO )−1 , which is 100(1+0.9+0.72+0.5+0.30)=342.
For Policy 2, we calculate the number of units in inventory as 100 multiplied by the row sum
of (I − QLIFO )−1 , which is 100(1+0.5+0.3+0.21+0.17)=218. This is quite a few units less than
for Policy 1.
d) The average age is calculated as the total age of the volume used divided by the volume used.
For Policy 1, this is
0.1 × 100 × 0 + 0.2 × 90 × 1 + 0.3 × 72 × 2 + 0.4 × 50 × 3 + 0.5 × 30 × 4
= 2.14 days
0.1 × 100 + 0.2 × 90 + 0.3 × 72 + 0.4 × 50 + 0.5 × 30
For Policy 2, this is
0.5 × 100 × 0 + 0.4 × 50 × 1 + 0.3 × 30 × 2 + 0.2 × 21 × 3 + 0.1 × 17 × 4
= 0.68 days
0.5 × 100 + 0.4 × 50 + 0.3 × 30 + 0.2 × 21 + 0.1 × 17
e) Policy 2 generates less inventory; the average age of the blood is lower, and we still have the
same probability of spoiling blood. In both cases, nearly 85% of the blood is being used.

7
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Chapter 17, Review problem B12†
In the game of craps, we roll a pair of six-sided dice. On the first throw, if we roll a 7 or an 11, we
win right away. If we roll a 2, a 3, or a 12, we lose right away. If we first roll a total of 4, 5, 6, 8, 9,
or 10, we keep rolling the dice until we get either a 7 or the total rolled on the first throw. If we get
a 7, we lose. If we roll the same total as the first throw, we win. Use knowledge of Markov chains
to determine our probability of winning at craps.

The probability of winning is the sum of the probability of throwing a 7 or 11 in the next throw,
denoted by P (7 or 11), and the probability of winning later, denoted by P (later).
6 2 8
P (7 or 11) = + =
36 36 36
Let’s set up a transition matrix starting from the second throw (absorbing chain)

Continue
30−i
Win
i
Lose
6

Continue 36 36 36
P=
 
Win 0 1 0 
Lose 0 0 1

where i denotes the value of the dice pair. Then we calculate

30 − i −1
 
−1 36
(I − Q) = 1 − =
36 6+i
−1
and
 i 6

(I − Q) R = 6+1 6+i

We know the value of i for each throw and can calculate the probability of winning after the first
throw.
3 3 4 4 5 5 3 3 4 4 5 5
+ + + + + = 0.271
36 9 36 10 36 11 36 9 36 10 36 11
The first term consists of the probability of throwing a 4 multiplied by the probability of winning
after throwing that 4 in the first attempt. So the total probability of winning is 36
8
+ 0.271 = 0.493.

8
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.7, Problem C9†
The age-based probability that an American dies during a given year is shown in Table 2. For

Table 2

Age Death probability


0 0.007557
1–4 0.000383
5–9 0.000217
10–14 0.000896
15–24 0.001267
25–34 0.002213
35–44 0.004459
45–54 0.010941
55–64 0.025384
65–84 0.058031
85+ 0.153270

example, a fraction 0.007557 of all babies die during the first year of life. Suppose 100 babies are
born each year, and nobody lives to be older than 110.
a) What is the average age of people in the United States?
b) Suppose all people ages 21–65 work, and all people over age 65 are retired. If we want to pay
each retiree $20 000 per year, how much money must each worker pay in to ensure that during
each year, the retirement plan is self-financing.

Exercise 7

9
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Chapter 17, Review problem A3†
A baseball team consists of 2 stars, 13 starters, and 10 substitutes. For tax purposes, the team
owner must value the players. The value of each player is defined to be the total value of the salary
he will earn until retirement. At the beginning of each season, the players are classified into one of
four categories:
Category 1 : Star (earns $1 million per year)
Category 2 : Starter (earns $400 000)
Category 3 : Substitute (earns $100 000)
Category 4 : Retired (earns no more salary)
Given that a player is a star, starter, or substitute a the beginning of the current season, the
probabilities that he will be a star, starter, substitute, or retired at the beginning of the next season
are as follows:
Star Starter Substitute Retired
 
Star 0.5 0.3 0.15 0.05
Starter 0.2
 0.5 0.2 0.1 
P=  
Substitute0.05 0.15 0.5 0.3 
Retired 0 0 0 1

Determine the value of the team’s players.

We calculate the fundamental matrix as it tells us the number of years expected to be in each state.

Star Starter Substitute


 
Star 3.20 2.51 1.96
(I − Q)−1 = Starter 1.60 3.53 1.89 
Substitute 0.80 1.31 2.76

A star will stay a star for 3.2 years, be a starter for 2.51 years, and be a substitute for 1.96 years before
retirement. A star will earn, on average, 3.2×1 000 000+2.51×400 000+1.96×100 000 = $4 400 000.
Similarly, a starter will earn $3 199 900 and a substitute $1 600 000. For 2 stars, 13 starters and 10
substitutes, the total value of the team is $66 413 000.

10
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.6, Problem 9†
In the gambler’s ruin problem (Winston, 2004, Chapter 17, Example 1), assume p = 0.6.
a) What is the probability that I reach $4?
b) What is the probability that I am wiped out?
c) What is the expected duration of the game?

The transition probability matrix is slightly rearranged to allow the identification of Q and R
matrices.
1 2 3 0 4
 
1 0 0.6 0 0.4 0
20.4 0 0.6 0 0
 
P = 3 0 0.4 0 0 0.6


 
0 0 0 0 1 0
4 0 0 0 0 1

We start by calculating the fundamental matrix

1 2 3
 
1 1.462 1.154 0.692
(I − Q)−1 = 20.769 1.923 1.154
3 0.308 0.769 1.462

from which we can also calculate


0 4
 
0.585 0.415
1
(I − Q)−1 R = 20.308 0.692
3 0.123 0.877

a) We read the ij th probability from the (I − Q)−1 R where i = 2 and j = 4, which is 0.692, or
69.2%.
b) The probability of being wiped out is the ij th probability from the (I − Q)−1 R where i = 2
and j = 0, which is 0.308, or 30.8%.
c) The duration of the game is the row sum for starting with $2, which is 0.769+1.923+1.154=3.846
periods.

11
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Chapter 17, Review problem A4†
The best-selling college statistics text, “The Thrill of Statistics” sells 5 million copies every fall.
Some users keep the book, and some sell it back to the bookstore. Suppose that 90% of all students
who buy a new book sell it back, 80% of all students who buy a once-used book sell it back, and
60% of all students who buy a twice-used book sell it back. If a book has been used four or more
times, the cover falls off, and cannot be sold back.
a) In the steady state, how many new copies of the book will the publisher be able to sell?
b) Suppose that a bookstore’s profit on each type is as follows:
• new book: $6
• once-used book: $3
• twice-used book: $2
• thrice-used book: $1
If the steady-state census is representative of the bookstore’s sales, what will be its average
profit per book?

a) There are multiple ways to look at the problem. Firstly, common sense. If we denote with ni
the number of books that have been used i times, with n1 = 0.9n0 , n2 = 0.8n1 , n3 = 0.6n2
and n0 + n1 + n2 + n3 = 5 000 000. Solving this yields n0 = 1+0.9+0.72+0.432
5 000 000
= 1 638 270,
n1 = 1 474 443, n2 = 1 179 554, and n3 = 707 733.
A second approach would be to calculate the steady-state probabilities. If we assume the states
are the age of the book when sold, then the transition probability matrix can be given as

0 1 2 3
 
0.1 0.9 0
0 0
10.2 0 0.8 0 
P=  
20.4 0 0 0.6
3 1.0 0 0 0

Multiplying P with itself numerous times results in


 
π = 0.328 0.295 0.236 0.142

Granted, with rounding errors, we find a similar result n0 = 0.328 × 5 000 000 = 1 640 000.
b) We calculate the expected profit using π .

Eprofit = (0.328 × 6 + 0.295 × 3 + 0.236 × 2 + 0.142 × 1) 5 000 000


= $17 319 777

12
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •
Winston (2004), Section 17.3, Problem A1†
Each American family is classified as living in an urban, rural, or suburban location. During a given
year, 15% of all urban families move to a suburban location, and 5% move to a rural location; also
6% of all suburban families move to an urban location, and 4% move to a rural location; finally, 4%
of all rural families move to an urban location, and 6% move to a suburban location.
a) If a family now lives in an urban location, what is the probability that it will live in an urban
area two years from now? A suburban area? A rural area?
b) Suppose that at present, 40% of all families live in an urban area, 35% live in a suburban area,
and 25% live in a rural area. Two years from now, what percentage of American families will
live in an urban area?
c) What problems might occur if this model were used to predict the future population distribu-
tion of the United States?

The transition probability matrix is

Urban Suburban Rural


 
Urban 0.8 0.15 0.05
P = Suburban 0.06 0.9 0.04 
Rural 0.04 0.06 0.9

a) We calculate P × P:

Urban Suburban Rural


 
0.651
Urban 0.258 0.091
2
P = Suburban 0.104 0.821 0.075
Rural 0.072 0.114 0.814

We can read the result from the first row of P2 . A family living in an urban location now
will have a 65.1% probability of still living in an urban area two years from now, a 25.8%
probability of living in a suburban area, and a 9.1% probability of living in a rural area.
b) We use the probabilities of the first column, 0.4(0.651) + 0.35(0.104) + 0.25(0.072) = 0.31456
or approximately 31.5% of all American families will live in an urban area.
c) Migration rates change over time based on, for example, due to economic changes. Also,
immigration and emigration are not accounted for in this problem. Therefore, we deal with a
nonstationary Markov chain.

References
Winston, W. L. (2004). Operations Research: Applications and Algorithms. Brooks/Cole, Cengage
Learning, 4th edition.

13
[git] • Branch: main @ 71bc1d9; Author: Johan W. Joubert; Date: 2023-11-06 08:45:37 +0100 •

You might also like