Massachusetts Institute of Technology

6.042J/18.062J, Spring ’10: Mathematics for Computer Science May 12

Prof. Albert R. Meyer revised May 10, 2010, 678 minutes

Solutions to In-Class Problems Week 14, Wed.

Problem 1.
A gambler is placing $1 bets on the “1st dozen” in roulette. This bet wins when a number from
one to twelve comes in, and then the gambler gets his $1 back plus $3 more. Recall that there are
38 numbers on the roulette wheel.
The gambler’s initial stake in $n and his target is $T . He will keep betting until he runs out of
money (“goes broke”) or reachs his target. Let wn be the probability of the gambler winning, that
is, reaching target $T before going broke.
(a) Write a linear recurrence for wn ; you need not solve the recurrence.

Solution. The probability of winning a bet is 12/38. Thus, by the Law of Total Probability ??,

wn = Pr {win starting with $n | won first bet} · Pr {won first bet} + Pr {win starting with $n | lost first bet} · Pr {
= Pr {win starting with $n + 3} · Pr {won first bet} + Pr {win starting with $n − 1} · Pr {lost first bet} ,

so
12 26
wn = wn+3 + wn−1 .
38 38
Letting m ::= n + 3 we get
38 26
wm = wm−3 − wm−4 .
12 12
�

(b) Let en be the expected number of bets until the game ends. Write a linear recurrence for en ;
you need not solve the recurrence.

Solution. By the Law of Total Expectation, Theorem ??,

en = (1 + E [number of bets starting with $n | won first bet]) · Pr {won first bet} + (1 + E [number of bets starting
= (1 + E [number of bets starting with $n + 3]) · Pr {won first bet} + (1 + E [number of bets starting with $n −

so
12 26
en = (en+3 + 1) + (1 + en−1 )
38 38
Letting m ::= n + 3 we get
38 1 − 26/12 38
em = em−3 − · em−4 −
12 26/12 12
�
Creative Commons 2010, Prof. Albert R. Meyer.
2 Solutions to In-Class Problems Week 14, Wed.

Problem 2.
Consider the following random-walk graph:

x y

(a) Find a stationary distribution.

Solution. d(x) = d(y) = 1/2 �

(b) If you start at node x and take a (long) random walk, does the distribution over nodes ever
get close to the stationary distribution? Explain.

Solution. No! you just alternate between nodes x and y. �

Consider the following random-walk graph:

w z 0.1

0.9

(c) Find a stationary distribution.

Solution. d(w) = 9/19, d(z) = 10/19. You can derive this by setting d(w) = (9/10)d(z), d(z) =
d(w) + (1/10)d(z), and d(w) + d(z) = 1. There is a unique solution. �

(d) If you start at node w and take a (long) random walk, does the distribution over nodes ever
get close to the stationary distribution? We don’t want you to prove anything here, just write out
a few steps and see what’s happening.

Solution. Yes, it does. �

Consider the following random-walk graph:

1/2
1/2 1/2
1 a b c d 1

1/2
Solutions to In-Class Problems Week 14, Wed. 3

(e) Describe the stationary distributions for this graph.

Solution. There are infinitely many, with d(b) = d(c) = 0, and d(a) = p and d(d) = 1 − p for any
p. �

(f) If you start at node b and take a long random walk, the probability you are at node d will be
close to what fraction? Explain.

Solution. 1/3. �

Appendix

A random-walk graph is a digraph such that each edge, x → y, is labelled with a number, p(x, y) > 0,
which will indicate the probability of following that edge starting at vertex x. Formally, we simply
require that the sum of labels leaving each vertex is 1. That is, if we define for each vertex, x,

out(x) ::= {y | x → y is an edge of the graph} ,

then �
p(x, y) = 1.
y∈out(x)

A distribution, d, is a labelling of each vertex, x, with a number, d(x) ≥ 0, which will indicate the
probability of being at x. Formally, we simply require that the sum of all the vertex labels is 1, that
is, �
d(x) = 1,
x∈V

where V is the set of vertices.

The distribution, d�, after a single step of a random walk from distribution, d, is given by
�
d�(x) ::= d(y) · p(y, x),
y∈in(x)

where
in(x) ::= {y | y → x is an edge of the graph} .

A distribution d is stationary if d� = d, where d� is the distribution after a single step of a random

walk starting from d. In other words, d stationary implies
�
d(x) ::= d(y) · p(y, x).
y∈in(x)
MIT OpenCourseWare
http://ocw.mit.edu

6.042J / 18.062J Mathematics for Computer Science

Spring 2010

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.