Evaluating DSGE Models: Predictive Checks and

Odds

Frank Schorfheide
University of Pennsylvania, CEPR, NBER

June 19, 2013

Two Questions
• Question 1: Does model M1 fit better than model M2 ?
• Question 2: Are there patterns in the data that are inconsistent with
model M1 ?
• Answer 1: use model odds.
• Answer 2: use predictive checks.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Application
• Based on Chang, Doh, and Schorfheide (JMCB 2007):
“Non-stationary Hours in a DSGE Model”
• Many researchers doubt that hours worked are stationary as we have
observed apparent changes in labor-supply patterns over recent
decades.
• We present a modified stochastic growth model in which hours
worked have a stochastic trend, generated by a non-stationary labor
supply shock.
• Based on output and hours data we evaluate the stochastic growth
models.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Model Specifications
• We consider four versions of the stochastic growth model:
• In M0 and M1 firms can choose the employment level at the given
wage rate without any adjustment cost.
• In A0 and A1 , on the other hand, it is costly for firms to adjust the
employment level.
• In A0 and M0 the labor supply shock is a stationary AR(1) process,
whereas it is modeled as random walk in A1 and M1 .

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Households
• The representative household maximizes the expected discounted
lifetime utility from consumption Ct and hours worked Ht :
"∞ #
(Ht+s /Bt+s )1+1/ν
X 
t+s
Et β ln Ct+s − . (1)
s=0
1 + 1/ν

• The log utility in consumption implies a constant long-run labor

supply in response to a permanent change in technology. The
short-run (Frisch) labor supply elasticity is ν. The labor supply
shock is denoted by Bt .
• Per-period budget constraint faced by the household is

Ct + Kt+1 − (1 − δ)Kt = Wt Ht + Rt Kt . (2)

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Firms
• Firms rent capital, hire labor services, and produce final goods
according to the following Cobb-Douglas technology:
 2 !
α 1−α Ht
Yt = (At Ht ) Kt 1−ϕ· −1 . (3)
Ht−1

• The stochastic process At represents the exogenous labor

augmenting technical progress. The last term captures the cost of
adjusting labor inputs: ϕ ≥ 0.
• The firms maximize expected discounted future profits

∞
" #
X
t+s
Et β λt+s (Yt − Wt Htd − Rt Ktd ) , (4)
s=0

where λt is the marginal value of a unit consumption to a

household, which is treated as exogenous to the firm.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Equilibrium Conditions
• In equilibrium λt = 1/Ct and the goods, labor, and capital markets
clear:

Yt = Ct + Kt+1 − (1 − δ)Kt , Htd = Ht , and Ktd = Kt .

• We assume that the log production technology evolves according to

a random walk with drift:

ln At = γ + ln At−1 + a,t , a,t ∼ iidN (0, σa2 ). (5)

The level of technology in period 0 is denoted by A0 .

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Preference Shock
• In models M0 and A0 , the labor supply shock follows a stationary
AR(1) process:

M0 : ln Bt = ρb ln Bt−1 +(1−ρb ) ln B0 +b,t , b,t ∼ iidN (0, σb2 ),

(6)

where 0 ≤ ρb < 1 and ln B0 is the unconditional mean of ln Bt .

• In model M0 and A0 the innovation b,t only has a transitory effect.
Alternatively, in models M1 and A1 the labor supply shock evolves
according to a random walk:

M1 : ln Bt = ln Bt−1 + b,t , b,t ∼ iidN (0, σb2 ) (7)

and we use ln B0 to denote the initial level of ln Bt .

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Detrending
• It is well known that in models M0 and A0 hours are stationary and
that output, consumption, and capital grow according to the
technology process At . Hence, one can induce stationarity with the
following transformation:

M0 : et = Yt ,
Y et = Ct ,
C et+1 = Kt+1 .
K
At At At
• In models M1 and A1 , on the other hand, the labor supply shock Bt
induces a stochastic trend into hours as well as output,
consumption, and capital. To obtain a stationary equilibrium these
variables have to be detrended according to:

M1 : e t = Ht ,
H et = Yt ,
Y et = Ct ,
C et+1 = Kt+1 .
K
Bt At Bt At Bt At Bt

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

State-Space Representation
• We fit the DSGE models to observations on the log level of real per
capita output and hours worked, denoted by the 2 × 1 vector yt .
• Let t = [a,t , b,t ]0 and
• define the vector of structural model parameters as
θ = [α, β, γ, δ, ν, ln A0 , ln B0 , ρb , σa , σb ]0 .
• It is well known that log-linearized DSGE models have a state space
representation:
yt = Γ0 + Γ1 s1,t + Γ2 s2,t + Γ3 t (8)
s1,t = Φ1 s1,t−1 + Ψ1 t (9)
s2,t = s2,t−1 + Ψ2 t . (10)

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

State-Space Representation
• The trend in (8) captures the effect of the drift in the random walk
technology process At .
• Equation (9) represents the law of motion for the state variables of
the detrended model,
• and (10) describes the evolution of trends: s2,t = ln At − γt in
models M0 and A0 and s2,t = [ln At − γt, ln Bt ]0 in M1 and A1 .
• The Kalman filter can be used to compute the likelihood function
p(Y |θ for the state space system (8) - (10).

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Initialization of Kalman Filter
• To initialize the Kalman filter a distribution for the state vector in
period t = 0 has to be specified.
• We factorize the initial distribution as p(s1,0 )p(s2,0 ) and set the first
component equal to the unconditional distribution of s1,t , whereas
the second component, composed of the distribution of ln A0 (for
M0 , A0 ) and [ln A0 , ln B0 ]0 (for M1 , A1 ), respectively, is absorbed
into the specification of our prior p(θ).

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Data
• Paper uses three different data sets comprised of quarterly U.S. real
per capita GDP and hours worked from 1954:Q2 to 2001:Q4.
• For Data Set 1 we use real GDP from the DRI-Global Insight
database (GDPQ) and divide it by population of age 20 or older
(PM20+PF20). Hours worked is measured as average weekly hours
of all people in the non-farm business sector compiled by the Bureau
of Labor Statistics (EEU00500005). We multiply the hours series by
the employment ratio, which is the number of people employed
(LHEM, DRI-Global Insight) divided by population (PM20+PF20).
• The observations from 1954:Q2 to 1958:Q4 are treated as
pre-sample to quantify prior distributions.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Hours Worked

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior
• We assume all parameters to be a priori independent.
• By and large, the prior means are chosen based on a pre-sample of
observations from 1954:Q2 to 1958:Q4.
• The prior mean of the labor share α is 0.66 and that for the
quarter-to-quarter growth rate of productivity, γ, is 0.5%.
• The prior for β is centered at 0.995. Combined with the prior mean
of γ, this corresponds to an annualized real return of about 4%.
• The depreciation rate δ lies between 1.8% and 3.3% per quarter.
• The 90% probability interval for the Frisch labor supply elasticity ν
ranges from 0.3 to 1.8.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior
• We specify a prior for the adjustment cost parameter ϕ as follows.
• In order to recruit labor ∆H, firms can either search for workers,
incurring adjustment costs ϕ( ∆HH
)2 Y , or pay head hunters for finding
workers.
• In the latter case the head hunters service fee is ζW ∆H where ζ is
the fraction of the salary of the job to fill.
• It is known that the head hunters tend to charge about 1/3 to 2/3 of
quarterly earnings of a worker (i.e., ζ = 1/3 to 2/3).
• At the margin, the recruiting costs should be the same:
ϕ( ∆H
H
)2 Y = ζW ∆H.
• With the labor share of 1/3 (= WH Y
) for a size of one percent increase
of employment, ∆H H
= 1%, we obtain a range of 22 to 44 for ϕ.
• We use a fairly diffuse prior distribution that is centered at 33 and
has a standard deviation of 15.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior
• The presence of adjustment costs dampens the effect of technology
and labor supply shocks on output and hours worked.
• In order to guarantee that the adjustment cost specifications have a
priori similar implications for the volatility of the endogenous variable
as M0 and M1 we use slightly different priors for the standard
deviations of the structural shocks.
• Under M0 and M1 the priors for σa and σb are centered at 0.010,
whereas under A0 and A1 they are centered at 0.015.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior
• For M0 and A0 the prior mean of ln B0 is constructed by matching
average hours worked over the pre-sample period with the steady
e ∗ , evaluated at the prior mean values of
state level of hours worked H
the remaining structural parameters.
• For M1 and A1 the prior mean of ln B0 is obtained by equating
e ∗ . Similarly,
hours worked in 1958:Q4 with the steady state level B0 H
∗ e ∗,
we select the prior mean of ln A0 by matching A0 Y e and A0 B0 Y
respectively, with the level of output in 1958:Q4.
• The prior standard deviations for ln A0 and ln B0 are 0.2. Finally, for
M0 and A0 the 90% probability interval for the autoregressive
parameter ρb ranges from 0.825 to 0.977, implying a fairly persistent
labor supply process.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior

Parameter Density Data Set Model Para (1) Para (2)

α Beta all all 0.660 0.020
β Beta all all 0.995 0.002
γ Normal all all 0.005 0.005
δ Beta all all 0.025 0.005
ν Gamma all all 1.000 0.500
ρb Beta all M0 , A0 0.900 0.050
σa InvGamma all M0 , M1 0.010 1.000
all M1 , A1 0.015 1.000
σb InvGamma all M0 , M1 0.010 1.000
all M1 , A1 0.015 1.000
ln A0 Normal 1 M0 , A0 5.647 0.200
1 M1 , A1 5.674 0.200
ln B0 Normal 1 M0 , A0 3.236 0.200
1 M1 , A1 3.209 0.200
ϕ Gamma all A0 , A1 33.00 15.00

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior

Parameter Domain Density Data Set Model Para (1) Para (2)
Alternative Prior P1
ln B0 R Normal 1 M1 3.209 2.000
2 M1 6.405 2.000
3 M1 6.309 2.000
Alternative Prior P2
ln B0 R Normal 1 M1 3.209 0.020
2 M1 6.405 0.020
3 M1 6.309 0.020
Alternative Prior P3
ρb [0, 1) Beta all M0 0.980 0.005
Alternative Prior P4
ρb [0, 1) Beta all M0 0.800 0.100

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Posterior Odds and Marginal Data Densities
• Posterior model probabilities can be computed as follows:

πi,0 p(Y |Mi )

πi,T = P , j = 1, . . . , 2, (11)
j πj,0 p(Y |Mj )

• where
Z
p(Y |M) = p(Y |θ, M)p(θ|M)dθ (12)

• Note:
T
X Z
ln p(Y1:T |M) = ln p(yt |θ, Y1:t−1 , M)p(θ|Y1:t−1 , M)dθ
t=1

• Posterior odds and Bayes Factor

π1,T π1,0 p(Y |M1 )

= × (13)
π2,T π2,0 p(Y |M2 )
|{z} | {z }
Prior Odds Bayes Factor

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Computation of Marginal Data Densities
• Geweke’s modified harmonic mean estimator.
• Chib and Jeliazkov’s estimator

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Modified Harmonic Mean
• Harmonic mean estimators are based on the following identity
Z
1 f (θ)
= p(θ|Y )dθ, (14)
p(Y ) p(Y |θ)p(θ)
R
where f (θ)dθ = 1.
• Conditional on the choice of f (θ) an obvious estimator is
" nsim
#−1
1 X f (θ(s) )
p̂G (Y ) = , (15)
nsim s=1
p(Y |θ(s) )p(θ(s) )

where θ(s) is drawn from the posterior p(θ|Y ).

• Geweke (1999):
= τ −1 (2π)−d/2 |Vθ |−1/2 exp −0.5(θ − θ̄)0 Vθ−1 (θ − θ̄)
 
f (θ)
n o
× (θ − θ̄)0 Vθ−1 (θ − θ̄) ≤ Fχ−1
2 (τ ) . (16)
d

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Chib and Jeliazkov
• Rewrite Bayes Theorem:

p(Y |θ)p(θ)
p(Y ) = . (17)
p(θ|Y )
• Thus,

p(Y |θ̃)p(θ̃)
p̂CS (Y ) = , (18)
p̂(θ̃|Y )

where we replaced the generic θ in (17) by the posterior mode θ̃.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Chib and Jeliazkov
• Use output of Metropolis-Hastings Algorithm.
• Proposal density for transition θ 7→ θ̃: q(θ, θ̃|Y ).
• Probability of accepting proposed draw:
( )
p(θ̃|Y )/q(θ, θ̃|Y )
α(θ, θ̃|Y ) = min 1, .
p(θ|Y )/q(θ̃, θ|Y )

• NoteZthat
α(θ, θ̃|Y )q(θ, θ̃|Y )p(θ|Y )dθ
Z ( )
p(θ̃|Y )/q(θ, θ̃|Y )
= min 1, q(θ, θ̃|Y )p(θ|Y )dθ
p(θ|Y )/q(θ̃, θ|Y )
Z ( )
p(θ|Y )/q(θ̃, θ|Y )
= p(θ̃|Y ) min , 1 q(θ̃, θ|Y )dθ
p(θ̃|Y )/q(θ, θ̃|Y )
Z
= p(θ̃|Y ) α(θ̃, θ|Y )q(θ̃, θ|Y )dθ

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Chib and Jeliazkov
• Posterior density at the mode can be approximated as follows

1
Pnsim
nsim s=1α(θ(s) , θ̃|Y )q(θ(s) , θ̃|Y )
p̂(θ̃|Y ) = PJ , (19)
J −1 j=1 α(θ̃, θ(j) |Y )

• {θ (s) } are posterior draws obtained with the the M-H Algorithm;
• {θ (j) } are additional draws from q(θ̃, θ|Y ) given the fixed value θ̃.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Log Marginal Data Densities

Data Set Prior M0 M1 A0 A1 VAR(4)

1 B 1176.33 1178.45 1182.10 1180.21 1180.49
P1 1176.81
P2 1178.61
P3 1177.64
P4 1174.85

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Predictive Checks
• See, for instance, Gelman, Carlin, Stern, and Rubin (1995),
Lancaster (2003), Geweke (2005).
• Prior predictive check: does the model have a chance explaining
salient features of the data?
• Posterior predictive check: tries to assess the “absolute” fit of the
model – similar to classical specification test.
• Recall: posterior odds are designed for relative model comparisons.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Prior Predictive Checks
• Let Y rep be a sample of observations of length T that we could have
observed in the past or that we might observe in the future.
• Let’s construct a predictive distribution based on our prior
knowledge for Y rep :
Z
p(Y rep ) = p(Y rep |θ) p(θ) dθ
|{z}
Prior
• Let S(Y ) be a sample statistic of interest. From p(Y rep ) we can
derive the predictive distribution of p(S).
• Compute the observed value of S based on the actual data and
assess how far it lies in the tails of its predictive distribution.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Posterior Predictive Checks
• Let Y rep be a sample of observations of length T that we could have
observed in the past or that we might observe in the future.
• Let’s construct a predictive distribution based on our posterior
knowledge for Y rep :
Z
p(Y rep ) = p(Y rep |θ) p(θ|Y ) dθ
| {z }
Posterior
• Let S(Y ) be a sample statistic of interest. From p(Y rep ) we can
derive the predictive distribution of p(S).
• Compute the observed value of S based on the actual data and
assess how far it lies in the tails of its predictive distribution.

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Implementation
For s = 1 to nsim :
1 Generate a draw θ(s) from prior (posterior).
2 Simulate data Y (s) from model conditional on θ(s) .
3 Compute S(Y (s) ).

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Sample Moments of Hours – Posterior Predictive
Distribution

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds

Joint Distribution of ϕ and ρb

Frank Schorfheide Evaluating DSGE Models: Predictive Checks and Odds