Statistical Rethinking

2023

13. Multilevel Adventures

https://www.atlasobscura.com/articles/cyoa-choose-your-own-adventure-maps
Drawing the Bayesian Owl
1. Theoretical estimand

2. Scientific (causal) model(s)

3. Use 1 & 2 to build statistical model(s)

4. Simulate from 2 to validate 3 yields 1

5. Analyze real data

MODEL
Multilevel Adventures
Return to the start: Start again, reinforce
foundation

Skim & Index: Don’t try to learn the

details; just acquaint yourself with
possibilities

Pick & Choose: Engage only with topics

that interest you

Bayesian Flow: Just enough to keep moving

Multilevel Adventures

7
6 5
Response
Clusters: Kinds of groups in the data

3 4
Features: Aspects of the model

2
1
(parameters) that vary by cluster 2 4 6
Story
8 10 12

7
Cluster Features

6
tanks survival

5
Response
4
stories treatment effect

3
individuals average response

2
departments admission rate, bias

1
0 10 20 30 40 50
Participant
 7
Cluster Features

6
tanks survival

5
Response
stories treatment effect

4
individuals average response

3
departments admission rate, bias

2
1
2 4 6 8 10 12
Story

7
Add clusters: More index variables,

6
more population priors

5
Response
4
Add features: More parameters, more

3
dimensions in each population prior

2
1
0 10 20 30 40 50
Participant
Varying effects as confounds
Varying effect strategy: Unmeasured
features of clusters leave an imprint on the
data that can be measured by (1) repeat
observations of each cluster and (2) partial
pooling among clusters

Predictive perspective: Important source of

cluster-level variation, regularize

Causal perspective: Competing causes or

unobserved confounds
Grandparents Parents
G P
Neighborhood
U
C
Children
 Individual
U
Treatment
X R S

E Y G

P
Participation
 Individual
U
Treatment
X R S

E Y G

P
Participation
 X1 X2

Government G1 W G2 Government
War
 X1 X2

G1 W G2
War
N1 N2
Nation Nation
Varying effects as confounds
Causal perspective: Competing causes
or actual confounds
X1 X2
Advantage over “fixed effect”
approach: Can include other cluster-
level (time invariant) causes G1 W G2
Fixed effects: Varying effects with
variance fixed at infinity, no pooling N1 N2
Don’t panic: Make a generative model
and draw the owl
Practical Difficulties

7
6 5
Response
4
Varying effects are a good default, but…

3 2
(1) How to use more than one cluster

1
2 4 6 8 10 12
type at the same time? Story

7
(2) How to calculate predictions

6 5
Response
(3) How to sample chains efficiently

3 4
(4) Group-level confounding

2
1
0 10 20 30 40 50
Participant
Fertility & behavior
1989 Bangladesh Fertility Survey

data(bangladesh)

1934 women, 61 districts

Outcome: contraceptive use

age, living children, urban/rural

 contraceptive
use
age C district
A D

K U
kids urbanity
1. Causes of interest

contraceptive
use
age C district
A D

K U
kids urbanity
2. Competing causes

contraceptive
use
age C district
A D

K U
kids urbanity
3. Relationships among causes

contraceptive
use
age C district
A D

K U
kids urbanity
4. Unfortunate relationships among causes

contraceptive
use
age C district
A D

K U
kids urbanity
5. A series of unfortunate relationships among causes

contraceptive
use
family district
age C
F A D

K U
kids urbanity
Varying districts

Estimand: C in each district, contraceptive

partially pooled use
age C district
Varying intercept on each district A D
Another chance to understand K U
partial pooling kids urbanity
 Ci ∼ Bernoulli(pi)
contraceptive
logit(pi) = αD[i] use
age C district
αj ∼ Normal(ᾱ, σ) A D
ᾱ ∼ Normal(0,1)
K U
σ ∼ Exponential(1) kids urbanity
 Ci ∼ Bernoulli(pi) Bernoulli because 0/1 outcome
logit(pi) = αD[i] log-odds of C=1 in each district D
αj ∼ Normal(ᾱ, σ) Regularizing prior for districts
ᾱ ∼ Normal(0,1) Average district
σ ∼ Exponential(1) Standard deviation among districts
# simple varying intercepts model
library(rethinking)
data(bangladesh)
d <- bangladesh

dat <- list( Ci ∼ Bernoulli(pi)

C = d$use.contraception,
D = as.integer(d$district) )
logit(pi) = αD[i]
mCD <- ulam(
alist( αj ∼ Normal(ᾱ, σ)
C ~ bernoulli(p),
logit(p) <- a[D],
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
abar ~ normal(0,1),
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
# simple varying intercepts model
library(rethinking)
data(bangladesh)
d <- bangladesh

dat <- list( Ci ∼ Bernoulli(pi)

C = d$use.contraception,
D = as.integer(d$district) )
logit(pi) = αD[i]
mCD <- ulam(
alist( αj ∼ Normal(ᾱ, σ)
C ~ bernoulli(p),
logit(p) <- a[D],
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
abar ~ normal(0,1),
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
# simple varying intercepts model
library(rethinking)
data(bangladesh)
d <- bangladesh

dat <- list( Ci ∼ Bernoulli(pi)

C = d$use.contraception,
D = as.integer(d$district) )
logit(pi) = αD[i]
mCD <- ulam(
alist( αj ∼ Normal(ᾱ, σ)
C ~ bernoulli(p),
logit(p) <- a[D],
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
abar ~ normal(0,1),
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
# simple varying intercepts model
library(rethinking)
data(bangladesh)
d <- bangladesh

dat <- list( Ci ∼ Bernoulli(pi)

C = d$use.contraception,
D = as.integer(d$district) )
logit(pi) = αD[i]
mCD <- ulam(
alist( αj ∼ Normal(ᾱ, σ)
C ~ bernoulli(p),
logit(p) <- a[D],
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
abar ~ normal(0,1),
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
# simple varying intercepts model
library(rethinking)
data(bangladesh)
d <- bangladesh

dat <- list( Ci ∼ Bernoulli(pi)

C = d$use.contraception,
D = as.integer(d$district) )
logit(pi) = αD[i]
mCD <- ulam(
alist( αj ∼ Normal(ᾱ, σ)
C ~ bernoulli(p),
logit(p) <- a[D],
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
abar ~ normal(0,1),
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
# simple varying intercepts model
library(rethinking)
data(bangladesh)
d <- bangladesh

dat <- list( Ci ∼ Bernoulli(pi)

C = d$use.contraception,
D = as.integer(d$district) )
logit(pi) = αD[i]
mCD <- ulam(
alist( αj ∼ Normal(ᾱ, σ)
C ~ bernoulli(p),
logit(p) <- a[D],
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
abar ~ normal(0,1),
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
 > precis(mCD,2)
mean sd 5.5% 94.5% n_eff Rhat4
a[1] -0.99 0.20 -1.31 -0.67 3376 1.00
# simple varying intercepts model a[2] -0.59 0.34 -1.16 -0.04 3847 1.00
library(rethinking) a[3] -0.22 0.50 -0.99 0.59 2918 1.00
data(bangladesh) a[4] -0.18 0.30 -0.64 0.30 4063 1.00
d <- bangladesh a[5] -0.58 0.28 -1.04 -0.16 3931 1.00
a[6] -0.81 0.24 -1.21 -0.43 4513 1.00
dat <- list( C ∼ Bernoulli(p )
a[7] -0.76 0.38 -1.36
i 0.30 -0.98
a[8] -0.51
-0.15 3028 1.00
-0.02 i4220 1.00
C = d$use.contraception, a[9] -0.71 0.34 -1.27 -0.18 3619 1.00
D = as.integer(d$district) )
logit(p ) = α
a[10] -1.14 0.43 -1.87
i 0.43D[i]
a[11] -1.54 -2.26
-0.48 2293 1.00
-0.88 2041 1.00
mCD <- ulam( a[12] -0.61 0.32 -1.12 -0.11 4222 1.00
alist( α ∼ Normal(ᾱ, σ)
a[13] -0.43 0.33 -0.94
j 0.18 0.10
a[14] 0.39
0.10 2602 1.00
0.69 3264 1.00
C ~ bernoulli(p), a[15] -0.56 0.34 -1.11 -0.03 3745 1.00
logit(p) <- a[D], a[16] -0.12 0.36 -0.68 0.46 3857 1.00
vector[61]:a ~ normal(abar,sigma), ᾱ ∼ Normal(0,1)
a[17] -0.75 0.34 -1.30 -0.20 4186 1.00
abar ~ normal(0,1), a[18] -0.64 0.25 -1.04 -0.26 3471 1.00
a[19] -0.50 0.33 -1.03 0.00 3715 1.00
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ ∼ Exponential(1)
a[20] -0.47 0.38 -1.09 0.13 4256 1.00
a[21] -0.50 0.36 -1.08 0.05 4264 1.00
a[22] -0.96 0.38 -1.59 -0.38 3273 1.00
a[23] -0.76 0.39 -1.38 -0.15 3241 1.00
a[24] -1.18 0.43 -1.91 -0.54 2182 1.00
a[25] -0.28 0.24 -0.66 0.09 3609 1.00
 a[36] -0.58 0.36 -1.16 -0.01 3809 1.00
a[37] -0.23 0.38 -0.82 0.40 3096 1.00
a[38] -0.71 0.38 -1.33 -0.11 3188 1.00
a[39] -0.21 0.30 -0.69 0.27 3537 1.00
# simple varying intercepts model a[40] -0.25 0.25 -0.66 0.16 3996 1.00
library(rethinking) a[41] -0.21 0.31 -0.71 0.28 3886 1.00
a[42] -0.24 0.40 -0.87 0.42 4422 1.00
data(bangladesh) a[43] -0.04 0.26 -0.46 0.36 4067 1.00
d <- bangladesh a[44] -0.96 0.33 -1.50 -0.44 3195 1.00
a[45] -0.65 0.28 -1.09 -0.21 4171 1.00
dat <- list( a[46] C ∼ Bernoulli(p )
i
0.00 0.20 -0.32 0.31 i3739 1.00
C = d$use.contraception, a[47] -0.34 0.36 -0.91 0.24 3204 1.00
a[48] -0.08 0.27 -0.50 0.35 3590 1.00
D = as.integer(d$district) )
logit(p ) = α
a[49] i
-0.87 0.48D[i]
-1.63 -0.14 2809 1.00
a[50] -0.30 0.34 -0.82 0.24 4255 1.00
mCD <- ulam( a[51] -0.28 0.29 -0.74 0.17 3392 1.00
alist( a[52] α ∼ Normal(ᾱ, σ)
-0.30
j
0.23 -0.67 0.08 3494 1.00
C ~ bernoulli(p), a[53] -0.43 0.35 -0.98 0.14 3814 1.00
logit(p) <- a[D], a[54] -0.54 0.52 -1.40 0.28 3546 1.00
vector[61]:a ~ normal(abar,sigma),
a[55]
a[56]
ᾱ ∼ Normal(0,1)
-0.77
0.09
0.46 -1.56
0.27 -0.34
-0.07
0.53
3145
2499
1.00
1.00
abar ~ normal(0,1), a[57] -1.07 0.35 -1.67 -0.53 2797 1.00
sigma ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
a[58]
a[59]
σ ∼ Exponential(1)
-0.30
-1.01
0.30 -0.75
0.43 -1.74
0.17
-0.35
3378
2425
1.00
1.00
a[60] -0.99 0.33 -1.53 -0.48 3479 1.00
a[61] -1.06 0.30 -1.54 -0.59 3251 1.00
abar -0.54 0.09 -0.68 -0.40 1734 1.00
sigma 0.52 0.09 0.39 0.67 626 1.01
 120
100
number of women
80
60
40
20
0

1 3 5 7 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 55 58 61
district
 prob use contraception
0.0 0.2 0.4 0.6 0.8 1.0

0
10
20
30
Raw proportions

district
40
50
60
 prob use contraception
0.0 0.2 0.4 0.6 0.8 1.0

0
10
20
30
Posterior means

district
40
50
60
 1.0

89% posterior intervals

prob use contraception
0.8
0.6
0.4
0.2
0.0

0 10 20 30 40 50 60
district
 Sample sizes
1.0

2
prob use contraception
0.8

35 45
0.6
0.4
0.2

13 6
4 10
21 14
0.0

0 10 20 30 40 50 60
district
 No data
1.0

2
prob use contraception
0.8

35 45
0.6
0.4
0.2

13 6
4 10
21 14
0.0

0 10 20 30 40 50 60
district
Varying districts

1.0
2

prob use contraception

0.8
35 45

0.6
0.4
0.2
13 6
4 10
21 14

0.0
Partial pooling shrinks districts 0 10 20 30
district
40 50 60

with low sampling towards mean

contraceptive
Better predictions use
age C district
No inference yet A D

K U
kids urbanity
Varying districts + urban
What is the effect of urban living? contraceptive
use
District features are potential district
group-level confounds
age C
A D
Total effect of U passes through K
K U
kids urbanity
Do not stratify by K!
 Ci ∼ Bernoulli(pi)
logit(pi) = αD[i] + βD[i]Ui
contraceptive
αj ∼ Normal(ᾱ, σ) use
age C district
βj ∼ Normal(β̄, τ) A D
ᾱ, β̄ ∼ Normal(0,1) K U
kids urbanity
σ, τ ∼ Exponential(1)
 Ci ∼ Bernoulli(pi)
logit(pi) = αD[i] + βD[i]Ui

αj ∼ Normal(ᾱ, σ) Regularizing prior for rural

βj ∼ Normal(β̄, τ) Regularizing prior for urban effect

ᾱ, β̄ ∼ Normal(0,1) Averages

σ, τ ∼ Exponential(1) Standard deviations
dat <- list(
C = d$use.contraception, Ci ∼ Bernoulli(pi)
D = as.integer(d$district),
U = ifelse(d$urban==1,1,0) ) logit(pi) = αD[i] + βD[i]Ui
# total U
mCDU <- ulam( αj ∼ Normal(ᾱ, σ)
alist(
C ~ bernoulli(p),
logit(p) <- a[D] + b[D]*U, βj ∼ Normal(β̄, τ)
vector[61]:a ~ normal(abar,sigma),
vector[61]:b ~ normal(bbar,tau),
c(abar,bbar) ~ normal(0,1), ᾱ, β̄ ∼ Normal(0,1)
c(sigma,tau) ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ, τ ∼ Exponential(1)
dat <- list(
C = d$use.contraception, Ci ∼ Bernoulli(pi)
D = as.integer(d$district),
U = ifelse(d$urban==1,1,0) ) logit(pi) = αD[i] + βD[i]Ui
# total U
mCDU <- ulam( αj ∼ Normal(ᾱ, σ)
alist(
C ~ bernoulli(p),
logit(p) <- a[D] + b[D]*U, βj ∼ Normal(β̄, τ)
vector[61]:a ~ normal(abar,sigma),
vector[61]:b ~ normal(bbar,tau),
c(abar,bbar) ~ normal(0,1), ᾱ, β̄ ∼ Normal(0,1)
c(sigma,tau) ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ, τ ∼ Exponential(1)
dat <- list(
C = d$use.contraception, Ci ∼ Bernoulli(pi)
D = as.integer(d$district),
mean sd 5.5% 94.5% n_eff Rhat4
U = ifelse(d$urban==1,1,0) )
bbar 0.62 0.16 0.37 0.87 logit(p
678 1.01
i ) = αD[i] + βD[i]Ui
# total U abar -0.70 0.09 -0.85 -0.56 1123 1.00
mCDU <- ulam( tau 0.57 0.24 0.17 0.94 αj ∼
45 1.11 Normal(ᾱ, σ)
alist( sigma 0.49 0.09 0.34 0.64 330 1.02
C ~ bernoulli(p),
logit(p) <- a[D] + b[D]*U, βj ∼ Normal(β̄, τ)
vector[61]:a ~ normal(abar,sigma),
vector[61]:b ~ normal(bbar,tau),
c(abar,bbar) ~ normal(0,1), ᾱ, β̄ ∼ Normal(0,1)
c(sigma,tau) ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
σ, τ ∼ Exponential(1)
 tau n_eff = 45

2.0
1.5
1.0
0.5

0 200 400 600 800 1000

tau n_eff = 45
PAUSE
More priors, more problems
Ci ∼ Bernoulli(pi) Priors inside priors: “centered”
logit(pi) = αD[i] + βD[i]Ui

αj ∼ Normal(ᾱ, σ)

βj ∼ Normal(β̄, τ)

ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
More priors, more problems
Ci ∼ Bernoulli(pi)
logit(pi) = αD[i] + βD[i]Ui
zj = (αj − ᾱ)/σ
αj ∼ Normal(ᾱ, σ)

βj ∼ Normal(β̄, τ)

ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
More priors, more problems
Ci ∼ Bernoulli(pi)
logit(pi) = αD[i] + βD[i]Ui
zj = (αj − ᾱ)/σ
αj ∼ Normal(ᾱ, σ)

βj ∼ Normal(β̄, τ) αj = ᾱ + zα,j × σ
ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
More priors, more problems
Ci ∼ Bernoulli(pi)
logit(pi) = αD[i] + βD[i]Ui
zj = (αj − ᾱ)/σ
αj ∼ Normal(ᾱ, σ)

βj ∼ Normal(β̄, τ) αj = ᾱ + zα,j × σ
ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
zα,j ∼ Normal(0,1)
 Centered varying intercepts Non-centered varying intercepts

Ci ∼ Bernoulli(pi) Ci ∼ Bernoulli(pi)
logit(pi) = αD[i] + βD[i]Ui logit(pi) = αD[i] + βD[i]Ui

αj ∼ Normal(ᾱ, σ) αj = ᾱ + zα,j × σ
βj = β̄ + zβ,j × τ
βj ∼ Normal(β̄, τ)
zα,j ∼ Normal(0,1)
ᾱ, β̄ ∼ Normal(0,1) zβ,j ∼ Normal(0,1)
σ, τ ∼ Exponential(1) ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
 Ci ∼ Bernoulli(pi)
mCDUnc <- ulam(
alist( logit(pi) = αD[i] + βD[i]Ui
C ~ bernoulli(p),
logit(p) <- a[D] + b[D]*U,
# define effects using other parameters αj = ᾱ + zα,j × σ
save> vector[61]:a <<- abar + za*sigma,
save> vector[61]:b <<- bbar + zb*tau, βj = β̄ + zβ,j × τ
# z-scored effects
vector[61]:za ~ normal(0,1), zα,j ∼ Normal(0,1)
vector[61]:zb ~ normal(0,1),
# ye olde hyper-priors zβ,j ∼ Normal(0,1)
c(abar,bbar) ~ normal(0,1),
c(sigma,tau) ~ exponential(1)
) , data=dat , chains=4 , cores=4 )
ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
 mean sd 5.5% 94.5% n_eff Rhat4
bbar 0.62 0.16 0.37 0.86 1513 1.00 Ci ∼ Bernoulli(pi)
abar -0.70 0.09 -0.84 -0.56 1457 1.00
tau 0.55 0.23 0.17 0.92 368 1.01 logit(pi) = αD[i] + βD[i]Ui
sigma 0.49 0.09 0.36 0.64 753 1.00
αj = ᾱ + zα,j × σ
tau n_eff = 368
βj = β̄ + zβ,j × τ
zα,j ∼ Normal(0,1)
zβ,j ∼ Normal(0,1)

ᾱ, β̄ ∼ Normal(0,1)
σ, τ ∼ Exponential(1)
 prob use contraception prob use contraception
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

0
0

10
10
rural

20
20

30
30

district
district

40
40

50
50

60
60

urban
 prob use contraception prob use contraception
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

0
0

2
11
2

10
10
rural

21

20
20

8
10

14

30
30
7

district
district
26

40
40
6

3
5
4

50
50

4
20

10

60
60

urban
 2 2 2 4
1.0

1.0
prob use contraception

prob use contraception

6 11 5
0.8 26

0.8
10
20
0.6

0.6
0.4

0.4
0.2

0.2
7 10 8
21 7 4 3
0.0

0.0
14
0 10 20 30 40 50 60 0 10 20 30 40 50 60
district district

rural
4
3
Density

urban
prior
2
1
0

0.0 0.2 0.4 0.6 0.8 1.0 1.2

posterior standard deviation
 posterior means

0.7
0.6
prob C (urban)
0.5
0.4
0.3
0.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7

prob C (rural)
 0.7
0.6
prob C (urban)
0.5
0.4

50% compatibility regions

0.3
0.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7

prob C (rural)
 0.7
0.6
prob C (urban)
0.5
0.4

50% compatibility regions

0.3
0.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7

prob C (rural)
Multilevel adventures
Clusters: Kinds of groups in the

0.7
data (districts)

0.6
prob C (urban)
Features: Aspects of the model

0.5
(parameters) that vary by cluster

0.4
(rural, urban)

0.3
There is useful information to

0.2
transfer across features 0.1 0.2 0.3 0.4 0.5 0.6 0.7
prob C (rural)
Course Schedule
Week 1 Bayesian inference Chapters 1, 2, 3
Week 2 Linear models & Causal Inference Chapter 4
Week 3 Causes, Confounds & Colliders Chapters 5 & 6
Week 4 Overfitting / MCMC Chapters 7, 8, 9
Week 5 Generalized Linear Models Chapters 10, 11
Week 6 Ordered categories & Multilevel models Chapters 12 & 13
Week 7 More Multilevel models Chapters 13 & 14
Week 8 Multilevel models & Gaussian processes Chapter 14
Week 9 Measurement & Missingness Chapter 15
Week 10 Generalized Linear Madness Chapter 16

https://github.com/rmcelreath/stat_rethinking_2023