Intro to Markov Chain Monte Carlo

Rebecca C. Steorts
Bayesian Methods and Modern Statistics: STA 360/601

Module 7

1
 Gibbs sampling

Instead of moving into the Metropolis algorithm, we now move

into Gibbs sampling, which is a special case of it.

We will return back to the Metropolis algorithm later and go

through the actual algorithm and implementation in detail!

2
 Two-stage Gibbs sampler

I Suppose p(x, y) is a p.d.f. or p.m.f. that is difficult to sample

from directly.
I Suppose, though, that we can easily sample from the
conditional distributions p(x|y) and p(y|x).
I The Gibbs sampler proceeds as follows:
1. set x and y to some initial starting values
2. then sample x|y, then sample y|x,
then x|y, and so on.

3
 Two-stage Gibbs sampler
0. Set (x0 , y0 ) to some starting value.
1. Sample x1 ∼ p(x|y0 ), that is, from the conditional
distribution X | Y = y0 .
Current state: (x1 , y0 )
Sample y1 ∼ p(y|x1 ), that is, from the conditional distribution
Y | X = x1 .
Current state: (x1 , y1 )
2. Sample x2 ∼ p(x|y1 ), that is, from the conditional
distribution X | Y = y1 .
Current state: (x2 , y1 )
Sample y2 ∼ p(y|x2 ), that is, from the conditional distribution
Y | X = x2 .
Current state: (x2 , y2 )
..
.
Repeat iterations 1 and 2, M times.
This procedure defines a sequence of pairs of random variables
(X0 , Y0 ), (X1 , Y1 ), (X2 , Y2 ), (X3 , Y3 ), . . . 4
 Markov chain and dependence

(X0 , Y0 ), (X1 , Y1 ), (X2 , Y2 ), (X3 , Y3 ), . . .

satisfies the property of being a Markov chain.

The conditional distribution of (Xi , Yi ) given all of the previous

pairs depends only on (Xi−1 , Yi−1 )

(X0 , Y0 ), (X1 , Y1 ), (X2 , Y2 ), (X3 , Y3 ), . . . are not iid samples

(Think about why).

5
 Ideal Properties of MCMC

I (x0 , y0 ) chosen to be in a region of high probability under

p(x, y), but often this is not so easy.
I We run the chain for M iterations and discard the first B
samples (X1 , Y1 ), . . . , (XB , YB ). This is called burn-in.
I Typically: if you run the chain long enough, the choice of B
doesn’t matter.
I Roughly speaking, the performance of an MCMC
algorithm—that is, how quickly the sample averages
1 PN
N i=1 h(Xi i ) converge—is referred to as the mixing rate.
, Y
I An algorithm with good performance is said to “have good
mixing”, or “mix well”.

6
 Toy Example

Suppose we want to sample from the bivariate distribution:

p(x, y) ∝ e−xy 1(x, y ∈ (0, c))

where c > 0, and (0, c) denotes the (open) interval between 0 and
c. (This example is due to Casella & George, 1992.)

7
 Toy Example

I The Gibbs sampling approach is to alternately sample from

p(x|y) and p(y|x).
I Note p(x, y) is symmetric with respect to x and y.
I Hence, only need to derive one of these and then we can get
the other one by just swapping x and y.
I Let’s look at p(x|y).

8
 Toy Example

p(x, y) ∝ e−xy 1(x, y ∈ (0, c))

p(x|y) ∝ p(x, y) ∝ e−xy 1(0 < x < c) ∝ Exp(x|y)1(x < c).1

x x x

I p(x|y) is a truncated version of the Exp(y) distribution

I It is the same as taking X ∼ Exp(y) and conditioning on it
being less than c, i.e., X | X < c.
I Let’s refer to this as the TExp(y, (0, c)) distribution.

1
Under ∝, we write the random variable (x) for clarity.
9
 Toy Example

An easy way to generate a sample from Z ∼ TExp(θ, (0, c)), is:

1. Sample U ∼ Uniform(0, F (c|θ)) where

F (x|θ) = 1 − e−θx

is the Exp(θ) c.d.f.

2. Set Z = F −1 (U |θ) where

F −1 (u|θ) = −(1/θ) log(1 − u)

is the inverse c.d.f. for u ∈ (0, 1).

Verify the last step on your own.

10
Let’s apply Gibbs sampling, denoting S = (0, c).
0. Initialize x0 , y0 ∈ S.
1. Sample x1 ∼ TExp(y0 , S), then sample y1 ∼ TExp(x1 , S).
2. Sample x2 ∼ TExp(y1 , S), then sample y2 ∼ TExp(x2 , S).
..
.
N . Sample xN ∼ TExp(yN −1 , S), sample yN ∼ TExp(xN , S).
Figure 1 demonstrates the algorithm, with c = 2 and initial point
(x0 , y0 ) = (1, 1).

11
Figure 1: (Left) Schematic representation of the first 5 Gibbs sampling
iterations/sweeps/scans. (Right) Scatterplot of samples from 104 Gibbs
sampling iterations.

12
 Example: Normal with semi-conjugate prior

iid
Consider X1 , . . . , Xn |µ, λ ∼ N (µ, λ−1 ). Then independently
consider

µ ∼ N (µ0 , λ−1
0 )
λ ∼ Gamma(a, b)

This is called a semi-conjugate situation, in the sense that the prior

on µ is conjugate for each fixed value of λ, and the prior on λ is
conjugate for each fixed value of µ.

For ease of notation, denote the observed data points by x1:n .

13
We know that for the Normal–Normal model, we know that for
any fixed value of λ,

µ|λ, x1:n ∼ N (Mλ , L−1

λ )

where
λ0 µ0 + λ ni=1 xi
P
Lλ = λ0 + nλ and Mλ = .
λ0 + nλ

2
do this on your own
14
We know that for the Normal–Normal model, we know that for
any fixed value of λ,

µ|λ, x1:n ∼ N (Mλ , L−1

λ )

where
λ0 µ0 + λ ni=1 xi
P
Lλ = λ0 + nλ and Mλ = .
λ0 + nλ

For any fixed value of µ, it is straightforward to derive2 that

λ|µ, x1:n ∼ Gamma(Aµ , Bµ ) (1)

where Aµ = a + n/2 and

Bµ = b + 12 (xi − µ)2 = nσ̂ 2 + n(x̄ − µ)2

P

where σ̂ 2 = n1 (xi − x̄)2 .

P

2
do this on your own
14
To implement Gibbs sampling in this example, each iteration
consists of sampling:

µ|λ, x1:n ∼ N (Mλ , L−1

λ )
λ|µ, x1:n ∼ Gamma(Aµ , Bµ ).

15
 Pareto example

Distributions of sizes and frequencies often tend to follow a “power

law” distribution.
I wealth of individuals
I size of oil reserves
I size of cities
I word frequency
I returns on stocks

16
 Power Law Distribution

The Pareto distribution with shape α > 0 and scale c > 0 has p.d.f.
αcα 1
Pareto(x|α, c) = α+1
1(x > c) ∝ α+1 1(x > c).
x x
This is referred to as a power law distribution, because the p.d.f. is
proportional to x raised to a power. Notice that c is a lower bound
on the observed values. In this example, we’ll see how Gibbs
sampling can be used to perform inference for α and c.

17
Rank City Population
1 Charlotte 731424
2 Raleigh 403892
3 Greensboro 269666
4 Durham 228330
5 Winston-Salem 229618
6 Fayetteville 200564
7 Cary 135234
8 Wilmington 106476
9 High Point 104371
10 Greenville 84554
11 Asheville 85712
12 Concord 79066
.. .. ..
. . .
44 Havelock 20735
45 Carrboro 19582
46 Shelby 20323
47 Clemmons 18627
48 Lexington 18931
49 Elizabeth City 18683
50 Boone 17122
18
 Parameter Interpretations

I α tells us the scaling relationship between the size of cities

and their probability of occurring.
I Let α = 1.
I Density looks like 1/xα+1 = 1/x2 .
I Cities with 10,000–20,000 inhabitants occur roughly
10α+1 = 100 times as frequently as cities with
100,000–110,000 inhabitants.
I c represents the cutoff point—any cities smaller than this were
not included in the dataset.

19
To keep things as simple as possible, let’s use an (improper)
default prior:
p(α, c) ∝ 1(α, c > 0).

Recall from Module 4:

I An improper/default prior is a nonnegative function of the
parameters which integrates to infinity.
I Often (but not always!) the resulting “posterior” will be
proper.
I It is important that the “posterior” be proper, since otherwise
the whole Bayesian framework breaks down.

20
Recall
αcα
p(x|α, c) = 1(x > c) (2)
xα+1
1(α, c > 0) (3)

Let’s derive the posterior:

def
p(α, c|x1:n ) ∝ p(x1:n |α, c)p(α, c)
α,c
n
Y αcα
∝ 1(α, c > 0) 1(xi > c)
α,c xα+1
i=1 i
αn cnα
= Q α+1 1(c < x∗ )1(α, c > 0) (4)
( xi )
where x∗ = min{x1 , . . . , xn }.

As a joint distribution on (α, c),

I this does not seem to have a recognizable form,
I and it is not clear how we might sample from it directly.
21
Let’s try Gibbs sampling!

To use Gibbs, we need to be able to sample α|c, x1:n and c|α, x1:n .

By Equation 4, we find that

αn cnα
p(α|c, x1:n ) ∝ p(α, c|x1:n ) ∝ Q α 1(α > 0)
α α ( xi )
n P

= α exp − α( log xi − n log c) 1(α > 0)
 P

∝ Gamma α  n + 1, log xi − n log c ,
α

and

p(c|α, x1:n ) ∝ p(α, c|x1:n ) ∝ cnα 1(0 < c < x∗ ),

c c

which we will define to be Mono(α, x∗ )

22
 Defining the Mono distribution

For a > 0 and b > 0, define the distribution Mono(a, b) (for

monomial) with p.d.f.

Mono(x|a, b) ∝ xa−1 1(0 < x < b).

Rb
Since 0 xa−1 dx = ba /a, we have
a a−1
Mono(x|a, b) = x 1(0 < x < b),
ba
and for 0 < x < b, the c.d.f. is
Z x
a xa xa
F (x|a, b) = Mono(y|a, b)dy = a = a.
0 b a b

23
To use the inverse c.d.f. technique, we solve for the inverse of F
a
on 0 < x < b: Let u = xba and solve for x.

xa
u= (5)
ba
b u = xa
a
(6)
bu1/a = x (7)

Can sample from Mono(a, b) by drawing U ∼ Uniform(0, 1) and

setting X = bU 1/a .3

3
It turns out that this is an inverse of the Pareto distribution, in the sense
that if X ∼ Pareto(α, c) then 1/X ∼ Mono(α, 1/c).
24
So, in order to use the Gibbs sampling algorithm to sample from
the posterior p(α, c|x1:n ), we initialize α and c, and then
alternately update them by sampling:
P

α|c, x1:n ∼ Gamma n + 1, log xi − n log c
c|α, x1:n ∼ Mono(nα + 1, x∗ ).

25
 Ways of visualizing results

Traceplots. A traceplot simply shows the sequence of samples, for

instance α1 , . . . , αN , or c1 , . . . , cN . Traceplots are a simple but
very useful way to visualize how the sampler is behaving.

26
Figure 2: Traceplot of α

Figure 3: Traceplot of c.

27
Estimated density. We are primarily interested in the posterior on
α, since it tells us the scaling relationship between the size of cities
and their probability of occurring.

By making a histogram of the samples α1 , . . . , αN , we can

estimate the posterior density p(α|x1:n ).

The two vertical lines indicate the lower ` and upper u boundaries
of an (approximate) 90% credible interval [`, u]—that is, an
interval that contains 90% of the posterior probability:


P α ∈ [`, u]x1:n = 0.9.

28
Figure 4: Estimated density of α|x1:n with ≈ 90 percent credible
intervals.

29
Running averages. Panel (d) shows the running average
1 Pk
k i=1 i for k = 1, . . . , N .
α

In addition to traceplots, running averages such as this are a useful

heuristic for visually assessing the convergence of the Markov chain.

The running average shown in this example still seems to be

meandering about a bit, suggesting that the sampler needs to be
run longer (but this would depend on the level of accuracy desired).

30
Figure 5: Running average plot

31
 Survival function

A survival function is defined to be

S(x) = P(X > x) = 1 − P(X ≤ x).

Power law distributions are often displayed by plotting their

survival function S(x), on a log-log plot.

Why? S(x) = (c/x)α for the Pareto(α, c) distribution and on a

log-log plot this appears as a line with slope −α.

The posterior survival function (or more precisely, the posterior

predictive survival function), is S(x|x1:n ) = P(Xn+1 > x | x1:n ).

32
Figure 6(e) shows an empirical estimate of the survival function
1 Pn
(based on the empirical c.d.f., F̂ (x) = n i=1 1(x ≥ xi )) along
with the posterior survival function, approximated by

S(x|x1:n ) = P(Xn+1 > x | x1:n ) (8)

Z
= P(Xn+1 > x | α, c)p(α, c|x1:n )dαdc (9)
N N
1 X 1 X
≈ P(Xn+1 > x | αi , ci ) = (ci /x)αi . (10)
N N
i=1 i=1

This is computed for each x in a grid of values.

[Think about why each line is true on your own].

33
 Figure 6: Empirical vs posterior survival function

How could we get a better empirical approximation?

34