Economics Letters 91 (2006) 312 – 320

www.elsevier.com/locate/econbase

The method of endogenous gridpoints for solving dynamic

stochastic optimization problems
Christopher D. Carroll
Department of Economics, The Johns Hopkins University, Baltimore MD, 21218-2685, USA

Received 21 December 2004; received in revised form 29 July 2005; accepted 6 September 2005

Abstract

This paper introduces a solution method for numerical dynamic stochastic optimization problems that avoids
rootfinding operations. The idea is applicable to many microeconomic and macroeconomic problems, including
life cycle, buffer-stock, and stochastic growth problems. Software is provided.
D 2005 Elsevier B.V. All rights reserved.

Keywords: Dynamic optimization; Precautionary saving; Stochastic growth model; Endogenous gridpoints; Liquidity
constraints

JEL classification: C6; D9; E2

1. The Problem

Consider a consumer whose goal is to maximize discounted utility from consumption

X
T
max bst uðCs Þ ð1Þ
s¼t

for a CRRA utility function u(C) = C 1  q /(1  q).1

E-mail address: [email protected].

URL: http://econ.jhu.edu/people/ccarroll. Software archive at http://econ.jhu.edu/people/ccarroll/EndogenousArchive.zip.
1
Putting leisure in the utility function is straightforward but would distract from the paper’s point.

0165-1765/$ - see front matter D 2005 Elsevier B.V. All rights reserved.
doi:10.1016/j.econlet.2005.09.013
 C.D. Carroll / Economics Letters 91 (2006) 312–320 313

The consumer’s problem will be specialized below to two cases: a standard micro-economic problem
with uninsurable idiosyncratic shocks to labor income, and a standard representative agent problem with
shocks to aggregate productivity (the dmicroT and the dmacroT models).2
The consumer’s initial condition is defined by two state variables: M t is dmarket resourcesT (macro
interpretation: capital plus current output) or dcash-on-handT (micro interpretation: net worth plus current
income), while P t is permanent labor productivity in both interpretations.
The transition process for M t is broken up, for convenience of analysis, into three steps. Assets at the
end of the period are market resources minus consumption, equal to
At ¼ Mt  Ct ; ð2Þ
and capital at the beginning of the next period is what remains after a depreciation factor 1 is applied,
Ktþ1 ¼ At 1; ð3Þ
where 1 = (1  d)in the usual macro notation and 1 = 1 in the micro interpretation.
The final step can be thought of as the transition from the beginning of period t + 1, when capital Kt + 1
but has not yet been used to produce output, and the middle of that period, when output has been
produced and incorporated into resources:
uLtþ1
zfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflffl{
Mtþ1 ¼ etþ1 Htþ1 Ptþ1 W tþ1 þ Ktþ1 Rtþ1 ð4Þ
where Wtþ 1 is the wage rate; H t + 1 is an iid transitory shock (e.g., unemployment) normalized to satisfy
E t [H t + n ] = 18n N 0 (usually H t = 18t in the macro interpretation); and e t indicates labor effort (or labor
supply), which for purposes of this paper is fixed at e t = 1, but in general could be allowed to vary. The
disarticulation of the flow of income into labor and capital components is useful in thinking separately
about the effects of productivity growth (captured by HP) and capital accumulation (K).
Permanent labor productivity (in either interpretation) evolves according to
Ptþ1 ¼ Gtþ1 Pt Wtþ1 ð5Þ
for a permanent shock that satisfies E t [W t + n ] = 18n N 0 and G t is exogenous and perfectly predictable
(see below for varying interpretations of G).
Defining lower case variables as the upper-case variable scaled by the level of permanent labor
productivity, e.g. a t = A t / P t ,we have
at ¼ mt  ct ð6Þ
while with a bit of algebra the state transition becomes
mtþ1 ¼ et Htþ1 W tþ1 þ ðat 1=Gtþ1 Wtþ1 Þ Rtþ1 : ð7Þ
|fflfflffl{zfflfflffl} |fflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflffl}
ultþ1 ¼ktþ1
The interest and wage factors are assumed not to depend on anything other than capital and
productive labor input; together with the iid assumption about the structure of the shocks, this implies
that the problem has a Bellman equation representation (henceforth boldface indicates functions)
Vt ðMt ; Pt Þ ¼ max fuðCt Þ þ bEt ½Vtþ1 ðMtþ1 ; Ptþ1 Þg ð8Þ
Ct
subject to the transition equations.

2
Different aspects of the setup of the problem will strike micro and macroeconomists as peculiar; with patience, it should
become clear how the problem as specified can be transformed into more familiar forms.
314 C.D. Carroll / Economics Letters 91 (2006) 312–320

Defining t + 1 u G t + 1W t + 1, consider the related problem

8 2 39
> ¼mtþ1 >
< zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ 7=
6 1q
vt ðmt Þ ¼ max uðct Þ þ bEt 4Ktþ1 vtþ1 Wtþ1 ltþ1 þ Rtþ1 at 1=Ktþ1 Þ 5 : ð9Þ
ct >
: |fflfflfflfflfflffl{zfflfflfflfflfflffl} > ;
ktþ1

Assume that there is some last period T in which

VT ðMT ; PT Þ ¼ PT1q vT ðMT =PT Þ ð10Þ
for some well-behaved vT (we will be more specific about the terminal value function below). In this
case it is easy to show that the solution to the dnormalizedT problem defined by Eq. (9) yields the solution
to the original problem via Vt = P1t  q vt for any t b T.3
Now define an end-of-period value function dGothic vT as
h i
vt ðat Þ ¼ bEt K1q
tþ1 vtþ1 ðW tþ1 l tþ1 þ 1R tþ1 a t =K tþ1 Þ ð11Þ

with derivative
h i
vat ðat Þ ¼ bEt K1q m
tþ1 vtþ1 ðW tþ1 ltþ1 þ Rtþ1 at 1=Ktþ1 ÞRtþ1 1=Ktþ1
 
¼ 1bEt Kq m
tþ1 vtþ1 ðW tþ1 ltþ1 þ Rtþ1 at 1=Ktþ1 ÞRtþ1 ð12Þ
and Eqs. (11) and (6) imply that Eq. (9) can be rewritten using vt as
vt ðmt Þ ¼ max fuðmt  at Þ þ vt ðat Þg; ð13Þ
fat g

and the envelope theorem can be applied

vm
t ðmt Þ ¼ uVðct Þ ð14Þ
while the first order condition yields the Euler equation
uVðct Þ ¼ Vat ðat Þ ¼ 1bEt ½uVðKtþ1 ctþ1 ÞRtþ1 : ð15Þ

2. Recursion

Generically, problems like this can be solved by specifying a final-period decision rule cT and a
procedure for recursion (obtaining ct from ct + 1). Here we specify the recursion; below we specify
choices for the terminal decision rule.

2.1. A standard solution method

The absence of a closed-form solution means that optimal decision functions (e.g. the consumption
function) must be constructed by calculating their values at a finite grid of possible values of the state
variables. Call some ordered set of such values li 2 !
l ufl1 ; l2 ; . . . ; lI g.

3
See Carroll (2004) for a proof.
 C.D. Carroll / Economics Letters 91 (2006) 312–320 315

With c t + 1 in hand, the usual solution procedure is to specify a Y l and, for each element l i , to use a
numerical rootfinding routine to find the v i that satisfies Eq. (15),

uVðvi Þ ¼ vta ðli  vi Þ: ð16Þ

The points {l i ,v i } are then used to construct an interpolating approximation to c t . (Choice of

interpolation method is separable from the point of this paper; see Judd (1998) for a discussion of
choices). Given the interpolated c t function the solution for earlier periods is found by recursion.
One of the most computationally burdensome steps in this approach is the numerical solution of
Eq. (16) for each specified state gridpoint. Even if efficient methods are used for constructing the
expectations (cf. the parameterized-expectations method of denHaan and Marcet (1990)) and shrewd
choices are made for the points to include in Y
l , for each gridpoint a numerical rootfinding operation still
must evaluate a substantial number of candidate values for the control variable before finding values that
satisfy Eq. (16) to an acceptable degree of precision.

2.2. Endogenous gridpoints solution method

This paper’s key contribution is to introduce an alternative approach that does not require numerical
rootfinding. The trick is to begin with end-of-period assets a t and to use the end-of-period marginal
value function vat ,the first order condition, and the budget constraint to construct the unique values of
middle-of-period m t generated by those at values.
Specifically, define an exogenous, time-invariant ordered set of values of a t collected in
a 2Ya ufa1 ; a2 ; . . . ; aI g. For each end-of-period state a i the marginal value Vat ðai Þ is easy to calculate;
inverting the consumption first order condition, the a’s generate
 
vi ¼ uV1 Vat ðai Þ : ð17Þ

Note that the budget constraint implies that

ui ¼ ai þ vi : ð18Þ

We now have a collection of {l i , v i } pairs in hand and can interpolate as before to generate an
approximation to c t . This completes the recursion.
The key distinction between this approach and the standard one is that the gridpoints for the policy
functions are not predetermined; instead they are endogenously generated from a predetermined grid of
values of end-of-period assets (hence the method’s name). One reason the method is efficient is that
expectations are never computed for any grid-point not used in the final interpolating function; the
standard method may compute expectations for many unused gridpoints.

3. Macro specialization

We first specialize to a macroeconomic stochastic growth model. Assuming aggregate production is

Cobb–Douglas in capital and labor F(K, P) = K qP 1  q, after normalizing by productivity P (and
316 C.D. Carroll / Economics Letters 91 (2006) 312–320

assuming a constant value G for the labor productivity growth factor), under the usual assumptions of
perfect competition, etc., if there is no aggregate transitory shock (H t + 1 = 1) we have
e1
Rtþ1 ¼ 1 þ ektþ1 ð19Þ
e
W tþ1 ¼ ð1  eÞktþ1 ð20Þ
and market resources are the sum of capital and production,
mtþ1 ¼ ktþ1 Rtþ1 þ W tþ1 ð21Þ
e
¼ ktþ1 þ ktþ1 : ð22Þ

c
2.00

1.75

1.50

1.25

Perfect Foresight Rep Agent →

1.00
← With Perm Shocks
0.75

0.50

0.25

m
1. 2. 3. 4. 5.
c
2.00

1.75 c=m →

1.50
Liq Constr Micro c (m) →
1.25

← Unemp Micro c (m)

1.00

0.75

0.50

0.25

m
1. 2. 3. 4. 5.

Fig. 1. Macro- and micro-consumption functions.

 C.D. Carroll / Economics Letters 91 (2006) 312–320 317

Table 1
Parameter values
Parameters common to all models
q 2 Relative risk aversion
b 0.96 Annual discount factor
e 1 Labor supply/effort (fixed)
Y {0.90, 1.00, 1.10} Permanent shock realizations
W 
Pr Y
W {0.25, 0.50, 0.25} Permanent shock probabilities

Macro model parameters

1 0.90 Depreciation factor
G 1.01 Exogenous aggregate productivity growth factor
e 0.36 Capital share in production

Micro model parameters

1 1 Depreciation factor
G 1.03 Trend individual wage growth factor
R 1.04 Real interest rate
W 1.00 Wage rate
Y
N  {0.90, 1.00, 1.10} Transitory shock realizations for employed

Pr YN jHN0 {0.25, 0.50, 0.25} Transitory shock probabilities for employed

Parameter unique to unemployment model

i 0.005 Probability of unemployment spell

We specify the terminal consumption function as

cT ðmÞ ¼ m; ð23Þ
which is very far from the converged infinite horizon consumption rule, but easy to verify as satisfying
the assumption (10) imposed earlier. More efficient choices are available, but for our purposes simplicity
trumps efficiency.
An arbitrary specification of the process for permanent productivity shocks is a three point
distribution defined by Y
W ¼ f0:9; 1:0; 1:1g with probabilities PrðY WÞ ¼ f0:25; 0:50; 0:25g.4
The top panel of Fig. 1 plots the converged consumption function that emerges from this solution
method for the benchmark set of parameter values specified in Table 1,along with the consumption
Y Y
function for the standard perfect foresight version of the model W ¼ Pr W ¼ f1gÞ.

4. Micro specialization

In the microeconomic literature, the usual approach is to take aggregate interest and wage rates as
exogenous, and to focus on transitory (H) and permanent (W) shocks to idiosyncratic labor productivity.

4
With careful choice of points and weights, small-dimensional discrete representations like this do a good job of
approximating commonly used continuous distributions like a log normal, cf. Judd (1998). An empirically realistic choice
would have a much lower variance than the specification here.
318 C.D. Carroll / Economics Letters 91 (2006) 312–320

We again start the recursion with c T (m) = m, and the permanent shocks are retained exactly as specified
for the macro problem.5

4.1. Life cycle models

Life cycle models specify a stereotypical pattern of lifetime income growth defined by G t where t is
age rather than time and T is the maximum possible lifespan;6 mortality uncertainty can be accom-
modated by age-varying values of b.

4.2. Buffer stock models

If R, W, G and b are constant, 1 = 1, and the impatience condition

RbE½ðGWÞq b1 ð24Þ

holds, Deaton (1991) and Carroll (2004) show that the problem defines a contraction mapping so that the
consumption functions defined by the problem converge from any well-behaved initial starting function
c T (m); the converged function is defined as
cðmÞ ¼ lim cT n ðmÞ: ð25Þ
n!l

We solve for the converged consumption function for two versions.

4.2.1. Version with unemployment

Assume that in future periods there is a small probability p that income will be zero (corresponding to
a substantial spell of unemployment):

0 with probability pN0

Htþ1 ¼ ð26Þ
Ntþ1 =ð1  pÞ with probability ð1  pÞ

Y Y
where N ¼ f0:9; 1:0; 1:1g and PrðNHN0Þ ¼ f0:25; 0:50; 0:25g (the same structure of non-unemploy-
ment transitory shocks as for the permanent shocks).
Carroll (2004) shows that in this model,

lim ct ðmt Þ ¼ 0: ð27Þ

mt !0

This implies that the minimum value in Y a should be a 1 = 0, which will generate {l 1,v 1} = {0., 0.} as
the first point in the set of interpolating points. The resulting converged c(m) is shown as the thin solid

5
An empirically realistic calibration for micro data would exhibit a permanent variance perhaps 100 times greater than an
appropriate macro calibration; but appropriate calibration is not the point of this paper.
6
This is the context in which the assumption that cT (m) = m actually makes economic sense, as distinct from merely providing
a convenient starting point for recursion.
 C.D. Carroll / Economics Letters 91 (2006) 312–320 319

locus in the bottom panel of Fig. 1; see the software for details of how the remaining values in Y
a were
chosen.

4.2.2. Version with liquidity constraints

Microeconomic models often include a liquidity constraint in addition to the usual transition
equations, and capturing the constraint often induces much additional code.
Dealing with a liquidity constraint using the method of endogenous gridpoints is simple. The key
observation is that when the constraint is on the cusp of binding, the marginal value of consumption is
equal to the marginal value of saving exactly zero (assuming the constraint is of the form that requires a
to be nonnegative; generalization to more elaborate kinds of constraints is straightforward). If the first
value in the ordered set Ya is a 1 = 0, then the method will produce
1
 a 
v1 ¼ l1 ¼ uV Vt ð0Þ ; ð28Þ
and if we define ĉ (m) as the function produced by interpolation among the points generated by Y
t a , the
consumption function imposing the constraint will be
ct ðmÞ ¼ minðm; ĉc t ðmÞÞ: ð29Þ
If the consumption function is defined as a piecewise linear spline interpolation among the {l, v}
points, the constraint can be handled simply by adding the point {l 0, v 0} = {0, 0} to the set of points that
constitute the interpolation data.
The converged solution is shown as the bold locus in the bottom panel of Fig. 1.

5. Conclusion

The method of endogenous gridpoints can be extended to problems with multiple state variables and
multiple controls, e.g. a micro consumer with a portfolio choice problem, or a labor supply decision; or a
macro consumer with a utility function that exhibits habit formation (see Carroll (2000) for examples).
The method is useful both because it is simpler than the standard method and because it reduces
computational demands.

Acknowledgements

I am grateful to Nicola Fuchs-Schündeln, Wouter den Haan, Michael Haliassos, Ken Judd, Albert
Marcet, Dimitri Mavridis, Victor Rios-Rull, John Rust, Eric Young, and participants in the 2002
meetings of the Society for Computational Economics for discussions of the literature and existing
solution techniques. Excellent research assistance has been provided by Zhou Lu and Marc Chan. I am
especially grateful to Michael Reiter for improvements to the Matlab code. All errors are my own.

References

Carroll, Christopher D., 2000. Lecture notes on solving microeconomic dynamic stochastic optimization problems Manuscript,
Johns Hopkins University. Available at http://econ.jhu.edu/people/ccarroll/solvingmicrodsops.pdf.
320 C.D. Carroll / Economics Letters 91 (2006) 312–320

Carroll, Christopher D., 2004. Theoretical foundations of buffer stock saving NBER Working Paper Number 10867. http://
econ.jhu.edu/people/ccarroll/BufferStockProofsNew.pdf.
Deaton, Angus S., 1991. Saving and liquidity constraints. Econometrica 59, 1221 – 1248.
denHaan, Wouter J., Marcet, Albert, 1990. Solving the stochastic growth model by parameterizing expectations. Journal of
Business and Economic Statistics 8 (1), 31 – 34. available at. http://ideas.repec.org/a/bes/jnlbes/v8y1990i1p31-34.html.
Judd, Kenneth L., 1998. Numerical Methods in Economics. The MIT Press, Cambridge, MA.