These lecture notes are incomplete. Various sections still need polishing.

This
should be taken into account when using and working with them.

Applied Intertemporal Optimization

- draft -

Klaus Wälde

University of Würzburg, CESifo, UCL Louvain la Neuve

www.waelde.com

Toolbox2006_8b.tex - January 19, 2007

These lecture notes are permanently being updated. The most recent version
is available at www.waelde.com/aio

@BOOK{Waelde06,
author = {Klaus Wälde},
year = 2006,
title = {Applied Intertemporal Optimization},
publisher = {Lecture Notes, University of Würzburg},
address = {Available at www.waelde.com/aio}
}
 Acknowledgments

This (soon-to-be) book is the result of many lectures given in various places,
including the Bavarian Graduate Program in Economics, the Universities of Dort-
mund, Dresden, Frankfurt, Louvain-la-Neuve, Munich and Würzburg and the Euro-
pean Commission in Brussels. I would therefore …rst like to thank students for their
questions and discussions. These conversations showed what the most important as-
pects in understanding model building and in formulating maximization problems
are. I also pro…ted from many discussions with economists and mathematicians from
many places. I especially would like to thank Ken Sennewald for many insights into
the more subtle issues of stochastic continuous time processes. I hope I succeeded in
incorporating these insights into this book.
The basic structure of this book is replicated here:

1 Introduction

Part I Deterministic models in discrete time

2 OLG models and di¤erence equations
3 In…nite horizon models

Part II Deterministic models in continuous time

4 Di¤erential equations
5 Optimal control theory - Hamiltonians
6 Dynamic programming in continuous time

Part III Stochastic models in discrete time

7 Stochastic di¤erence equations and applications
8 Multi-period models

Part IV Stochastic models in continuous time

9 Stochastic di¤erential equations and rules for di¤erentiating
10 Dynamic programming

11 On notation and variables

This book can be used for a …rst year master class, a PhD course or for studying
for oneself. In fact, given the detailed step-by-step approach to problems, it should be
easier to understand solutions with this text than with other usually more condensed
presentations.
A …rst year master class on economic methods (or third year undergraduate stud-
ies, i.e. …rst semester “Hauptstudium”) would cover part I and part II, i.e. maxi-
mization and applications in discrete and continuous time under certainty. A lecture
series of 14 times 90 minutes (corresponding to one semester course in the German

1
system) is usually enough. It is useful to complement the lectures, however, by ex-
ercise classes which can use the same amount of time. Exercises are included at the
end of each chapter. A PhD class would review parts of part I and part II and cover
in full part III and part IV. It also requires around 14 times 90 minutes and exercise
classes help even more given the more complex material.

2
Contents
1 Introduction 1

I Deterministic models in discrete time 5

2 OLG models and di¤erence equations 5
2.1 The general setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1 Optimal consumption . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2 Optimal consumption with prices . . . . . . . . . . . . . . . . 8
2.2.3 Some useful de…nitions . . . . . . . . . . . . . . . . . . . . . . 9
2.3 The idea behind the Lagrangian . . . . . . . . . . . . . . . . . . . . . 12
2.3.1 Where the Lagrangian comes from I . . . . . . . . . . . . . . . 12
2.3.2 How to understand shadow prices . . . . . . . . . . . . . . . . 14
2.4 A general equilibrium analysis . . . . . . . . . . . . . . . . . . . . . . 15
2.4.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.2 Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.3 Goods market equilibrium and accumulation identity . . . . . 17
2.4.4 The reduced form . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.5 Properties of the reduced form . . . . . . . . . . . . . . . . . . 19
2.5 More on di¤erence equations . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.1 Two useful proofs . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.2 The simplest di¤erence equation . . . . . . . . . . . . . . . . . 21
2.5.3 A slightly less but still simple di¤erence equation . . . . . . . 23
2.5.4 Fix points and stability . . . . . . . . . . . . . . . . . . . . . . 24
2.5.5 An example: Deriving a budget constraint . . . . . . . . . . . 25
2.6 Exercises chapter 2 and further reading . . . . . . . . . . . . . . . . . 27

3 In…nite horizon models 31

3.1 The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Solving by Lagrange . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 The envelope theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.1 The theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.2 Illustration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.3 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Solving by dynamic programming . . . . . . . . . . . . . . . . . . . . 34
3.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.5.1 Individual utility maximization . . . . . . . . . . . . . . . . . 37
3.5.2 Optimal R&D e¤ort . . . . . . . . . . . . . . . . . . . . . . . 39
3.6 A general equilibrium analysis . . . . . . . . . . . . . . . . . . . . . . 42
3.6.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.6.2 Firms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.6.3 Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3
 3.6.4 Aggregation and reduced form . . . . . . . . . . . . . . . . . . 43
3.6.5 Steady state and transitional dynamics . . . . . . . . . . . . . 44
3.6.6 The intertemporal budget constraint . . . . . . . . . . . . . . 45
3.7 A central planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.7.1 Optimal factor allocation . . . . . . . . . . . . . . . . . . . . . 46
3.7.2 Where the Lagrangian comes from II . . . . . . . . . . . . . . 47
3.8 Exercises Chapter 3 and further reading . . . . . . . . . . . . . . . . 48

II Deterministic models in continuous time 52

4 Di¤erential equations 52
4.1 Some de…nitions and theorems . . . . . . . . . . . . . . . . . . . . . . 52
4.1.1 De…nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.2 Two theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2 Analysing ODEs through phase diagrams . . . . . . . . . . . . . . . . 53
4.2.1 One dimensional systems . . . . . . . . . . . . . . . . . . . . . 53
4.2.2 Two-dimensional systems I - An example . . . . . . . . . . . . 56
4.2.3 Two-dimensional systems II - The general case . . . . . . . . . 58
4.2.4 Types of phase diagrams and …xpoints . . . . . . . . . . . . . 62
4.2.5 Multidimensional systems . . . . . . . . . . . . . . . . . . . . 64
4.3 Linear di¤erential equations . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.1 Rules on derivatives . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.2 Forward and backward solutions of a linear di¤erential equation 66
4.3.3 Di¤erential equations as integral equations . . . . . . . . . . . 67
4.3.4 Example 1: Budget constraints . . . . . . . . . . . . . . . . . 68
4.3.5 Example 2: Forward solutions . . . . . . . . . . . . . . . . . . 70
4.3.6 Example 3: A growth model . . . . . . . . . . . . . . . . . . . 71
4.4 Linear di¤erential equation systems . . . . . . . . . . . . . . . . . . . 72
4.5 Exercises chapter 4 and further reading . . . . . . . . . . . . . . . . . 72

5 Optimal control theory - Hamiltonians 76

5.1 An introductory example . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.2 Deriving laws of motion . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.2.1 Solving by using the Lagrangian . . . . . . . . . . . . . . . . . 77
5.2.2 Hamiltonians as a shortcut . . . . . . . . . . . . . . . . . . . . 79
5.3 Boundary conditions and su¢ cient conditions . . . . . . . . . . . . . 79
5.3.1 Free value of state variable at endpoint . . . . . . . . . . . . . 79
5.3.2 Fixed value of state variable at endpoint . . . . . . . . . . . . 80
5.3.3 Su¢ cient conditions . . . . . . . . . . . . . . . . . . . . . . . 80
5.4 The in…nite horizon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4.1 The maximization problem . . . . . . . . . . . . . . . . . . . . 81
5.4.2 The transversality condition . . . . . . . . . . . . . . . . . . . 81
5.4.3 The boundedness condition . . . . . . . . . . . . . . . . . . . 82
5.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

4
 5.5.1 Free value at end point - Adjustment costs . . . . . . . . . . . 83
5.5.2 Fixed value at end point - Adjustment costs II . . . . . . . . . 85
5.5.3 In…nite horizon - Optimal consumption paths . . . . . . . . . 86
5.5.4 The central planner and capital accumulation . . . . . . . . . 89
5.5.5 The matching approach to unemployment . . . . . . . . . . . 92
5.6 The present value Hamiltonian . . . . . . . . . . . . . . . . . . . . . . 95
5.6.1 Problems without (or with implicit) discounting . . . . . . . . 95
5.6.2 Deriving laws of motion . . . . . . . . . . . . . . . . . . . . . 96
5.6.3 The link between CV and PV . . . . . . . . . . . . . . . . . . 97
5.7 Exercises chapter 5 and further reading . . . . . . . . . . . . . . . . . 98

6 Dynamic programming in continuous time 103

6.1 Household utility maxmization . . . . . . . . . . . . . . . . . . . . . . 103
6.2 Comparing dynamic programming to Hamiltonians . . . . . . . . . . 106
6.3 Dynamic programming with two state variables . . . . . . . . . . . . 106
6.4 An example: Nominal and real interest rates and in‡ation . . . . . . 109
6.4.1 Firms, the central bank and the government . . . . . . . . . . 109
6.4.2 Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.4.3 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.5 Exercises chapter 6 and further reading . . . . . . . . . . . . . . . . . 113
6.6 Looking back . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

III Stochastic models in discrete time 117

7 Stochastic di¤erence equations and applications 117
7.1 Basics on random variables . . . . . . . . . . . . . . . . . . . . . . . . 117
7.1.1 Some concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.1.2 An illustration . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.2 Examples for random variables . . . . . . . . . . . . . . . . . . . . . 118
7.2.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . 119
7.2.2 Continuous random variables . . . . . . . . . . . . . . . . . . 119
7.2.3 Higher-dimensional random variables . . . . . . . . . . . . . . 119
7.3 Expected values, variances, covariances and all that . . . . . . . . . . 120
7.3.1 De…nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.3.2 Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.3.3 Functions on random variables . . . . . . . . . . . . . . . . . . 122
7.4 Examples of stochastic di¤erence equations . . . . . . . . . . . . . . . 123
7.4.1 A …rst example . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.4.2 A more general case . . . . . . . . . . . . . . . . . . . . . . . 127
7.5 A simple OLG model . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.5.1 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.5.2 Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.5.3 Firms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
7.5.4 Individual decision problem . . . . . . . . . . . . . . . . . . . 129

5
 7.5.5 Aggregation and the reduced form for the CD case . . . . . . 131
7.5.6 Some analytical results . . . . . . . . . . . . . . . . . . . . . . 132
7.6 Risk-averse and risk-neutral households . . . . . . . . . . . . . . . . . 134
7.7 Pricing of contingent claims and assets . . . . . . . . . . . . . . . . . 138
7.7.1 The value of an asset . . . . . . . . . . . . . . . . . . . . . . . 138
7.7.2 The value of a contingent claim . . . . . . . . . . . . . . . . . 139
7.7.3 Risk-neutral valuation . . . . . . . . . . . . . . . . . . . . . . 140
7.8 Natural volatility I (*) . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.8.1 The basic idea . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.8.2 A simple stochastic model . . . . . . . . . . . . . . . . . . . . 142
7.8.3 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.9 Exercises chapter 7 and further reading . . . . . . . . . . . . . . . . . 144

8 Multi-period models 148

8.1 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . 148
8.2 Household utility maximization . . . . . . . . . . . . . . . . . . . . . 150
8.3 A central planner in an oil-importing economy . . . . . . . . . . . . . 151
8.4 Asset pricing in a one-asset economy . . . . . . . . . . . . . . . . . . 152
8.4.1 The model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.4.2 Optimal behaviour . . . . . . . . . . . . . . . . . . . . . . . . 153
8.4.3 The pricing relationship . . . . . . . . . . . . . . . . . . . . . 154
8.4.4 More real results . . . . . . . . . . . . . . . . . . . . . . . . . 155
8.5 Asset pricing with many assets (*) . . . . . . . . . . . . . . . . . . . 157
8.5.1 A many-sector economy . . . . . . . . . . . . . . . . . . . . . 157
8.5.2 The household´s maximization problem (*) . . . . . . . . . . 157
8.5.3 The fundamental pricing rule . . . . . . . . . . . . . . . . . . 159
8.5.4 The CAP formula . . . . . . . . . . . . . . . . . . . . . . . . . 160
8.6 Matching and unemployment . . . . . . . . . . . . . . . . . . . . . . 160
8.7 How to do without dynamic programming . . . . . . . . . . . . . . . 160
8.7.1 Individual utility maximization . . . . . . . . . . . . . . . . . 160
8.7.2 Contingent claims pricing . . . . . . . . . . . . . . . . . . . . 161
8.7.3 Sticky prices (*) . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.7.4 Optimal employment with adjustment cost (*) . . . . . . . . . 164
8.8 An explicit time path for a boundary condition . . . . . . . . . . . . 165
8.9 Exercises chapter 8 and further reading . . . . . . . . . . . . . . . . . 166

IV Stochastic models in continuous time 170

9 Stochastic di¤erential equations and rules for di¤erentials 170
9.1 Some basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
9.1.1 Stochastic processes . . . . . . . . . . . . . . . . . . . . . . . 170
9.1.2 Stochastic di¤erential equations . . . . . . . . . . . . . . . . . 173
9.1.3 The integral representation of stochastic di¤erential equations 176
9.2 Functions of stochastic processes . . . . . . . . . . . . . . . . . . . . . 177

6
 9.2.1 Why all this? . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9.2.2 Computing di¤erentials for Brownian motion . . . . . . . . . . 178
9.2.3 Computing di¤erentials for Poisson processes . . . . . . . . . . 180
9.2.4 Brownian motion and a Poisson process . . . . . . . . . . . . . 182
9.2.5 Application - Option pricing . . . . . . . . . . . . . . . . . . . 183
9.2.6 Application - Deriving a budget constraint . . . . . . . . . . . 185
9.3 Solving stochastic di¤erential equations . . . . . . . . . . . . . . . . . 186
9.3.1 Some examples for Brownian motion . . . . . . . . . . . . . . 186
9.3.2 A general solution for Brownian motions . . . . . . . . . . . . 189
9.3.3 Di¤erential equations with Poisson processes . . . . . . . . . . 191
9.4 Expectation values . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.4.1 The idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.4.2 Simple results . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
9.4.3 Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
9.4.4 A more general approach to computing moments . . . . . . . 199
9.5 Exercise to chapter 9 and further reading . . . . . . . . . . . . . . . . 203

10 Dynamic programming 209

10.1 Optimal saving under Poisson uncertainty . . . . . . . . . . . . . . . 209
10.1.1 The setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
10.1.2 Three dynamic programming steps . . . . . . . . . . . . . . . 210
10.1.3 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
10.1.4 Optimal consumption and portfolio choice . . . . . . . . . . . 213
10.1.5 Other ways to determine c~ . . . . . . . . . . . . . . . . . . . . 217
10.1.6 Expected growth . . . . . . . . . . . . . . . . . . . . . . . . . 219
10.2 Matching on the labour market: where value functions come from . . 220
10.2.1 A household . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
10.2.2 The Bellman equation and value functions . . . . . . . . . . . 221
10.3 Optimal saving under Brownian motion . . . . . . . . . . . . . . . . . 221
10.3.1 The setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
10.3.2 Three dynamic programming steps . . . . . . . . . . . . . . . 222
10.3.3 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
10.4 Capital asset pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
10.4.1 The setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
10.4.2 Optimal behaviour . . . . . . . . . . . . . . . . . . . . . . . . 225
10.4.3 Capital asset pricing . . . . . . . . . . . . . . . . . . . . . . . 227
10.5 Natural volatility II (*) . . . . . . . . . . . . . . . . . . . . . . . . . . 228
10.5.1 An RBC model . . . . . . . . . . . . . . . . . . . . . . . . . . 228
10.5.2 A natural volatility model . . . . . . . . . . . . . . . . . . . . 230
10.5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
10.6 Exercises chapter 10 and further reading . . . . . . . . . . . . . . . . 230

11 Notation and variables, references and index 236

7
12 Exams 247

(*) These sections are still work in progress. See the note on page 2.

8
1 Introduction
The objective of this course is to provide a toolbox for solving maximization problems
and for working with their solutions in economic models. Maximization problems can
be formulated in discrete or continuous time, under certainty or uncertainty. Various
maximization methods will be used, ranging from “solving by inserting”, via La-
grangian and Hamiltonian approaches to dynamic programming (Bellman equation).
Dynamic programming will be used for all environments, discrete, continuous, certain
and uncertain, the Lagrangian for most of them. Solving by inserting is also very use-
ful in discrete time setups. The Hamiltonian approach is used only for deterministic
continuous time setups. An overview is given on the next page.
Part I deals with discrete time models under certainty, Part II covers continuous
time models under certainty. Part III deals with discrete time models under uncer-
tainty and Part IV, logically, analyzes continuous time models under uncertainty.
The course is called “applied intertemporal optimization“, where the emphasis is
on “applied”. This means that each type of maximization problem will be illustrated
by some example from the literature. These will come from micro- and macroeco-
nomics, from …nance, environmental economics and game theory. This also means,
maybe more importantly, that there is little emphasis on formal scrutiny. This course
is about computing solutions. A more formal treatment (especially of the stochastic
part) can be found elsewhere in the literature and many references will be given.

Anybody who has a nice maximization problem as part of a published

paper and who wants it to be presented in this book, is very wel-
come to send it as both a .tex and .pdf …le to [email protected]. If
it complements or updates existing examples in an interesting way, I
would be very happy to include it. Given the idea of this book, how-
ever, the solution of maximization problems must be presented step
by step such that an understanding requires only short reading for an
economist with experience in solving maximization problems. Clearly,
the original author of the maximization problem will be stated in the
corresponding chapter.

Given that certain maximization problems are solved many times - e.g. utility
maximization of a household …rst under certainty in discrete and continuous time and
then under uncertainty in discrete and continuous time - and using many methods,
the “rules how to compute solutions” can be easily understood: First, the discrete
deterministic two-period approach provides the basic intuition or feeling for a solution.
Next, in…nite horizon problems add one dimension of complexity by “taking away”
the simple boundary condition of …nite horizon models. In a third step, uncertainty
adds expectations operators and so on. By gradually working through increasing steps
of sophistication and by linking back to simple but structurally identical examples,
intuition for the complex setups is built up as much as possible. This approach then
allows to …nally understand the beauty of e.g. Keynes-Ramsey rules in continuous
time under Poisson uncertainty, Brownian motion or with Levy processes.

1
 Even though I would like this to be a nice, clean and …nished textbook,
this is unfortunately not yet the case. Some sections are incomplete
or contain only sketches of ideas. This is also due to the fact that
these notes are permanently expanding in an attempt to cover further
interesting material. The sections which are still work in progress are
marked by an asterisk (*).

An overview of maximization methods

In order to get quick access to various model building blocks, the following tables
can be used as guides. The tables show the numbers of the sections where the methods
are presented. Columns indicate whether the environment is of a deterministic or
stochastic nature, rows distinguish between setups in discrete and continuous time.

Lagrangian deterministic stochastic

discrete 2.3, 3.2, 3.7 -
continuous 5.2.1, 5.5.3 -

Hamiltonian deterministic stochastic

discrete - -
continuous 5 -

Solution by
deterministic stochastic
inserting
discrete 2.2.1 8.7
continuous - -

Dynamic
deterministic stochastic
programming
discrete 3.4 8
continuous 6 10

Figure 1 Methods for solving maximization problems

An overview of model building tools

The simplest approach to dynamic modelling is de…nitely a two-period setup.

Surprisingly often, this yields many useful insights despite - or probably because of
- its simple structure. This simple structure allows very often to obtain analytical
solutions, in deterministic and stochastic setups, even when utility functions are not
Cobb-Douglas. Some examples are therefore provided also here, both for their own
sake, but also as introduction to more complex dynamic systems.

2
 2 period GE
deterministic stochastic
OLG model
discrete 2.4 7.5
continuous - -

Figure 2 The simplest approach to dynamic models

In addition to solving maximization problems, this book also shows (i) how the
structure of maximization problems is determined by the economic environment an
agent …nds itself in and (ii) how to go from solutions of maximization problems
to properties of the resulting dynamic system. The former point simply states that
budget constraints of households or …rms are endogenous and depend on technological
fundamentals of the model.
endogenous
deterministic stochastic
budget constraints
discrete 2.5.5, 3.6.3 -
continuous 4.3.4 9.2.6

how to …nd the

deterministic stochastic
reduced form
discrete 2.4.4, 3.6.4 7.5.5
continuous - -

Figure 3 Things to be taken care of in model building

An overview of economic contents

After having gone through part I and part II, students will be familiar with the
two-period overlapping-generations (OLG) model, the optimal saving central planner
model in discrete and continuous time, the matching approach to unemployment,
the decentralized optimal growth model and (coming soon) an optimal growth model
with money, allowing to understand the di¤erence between nominal and real interest
rates in general equilibrium.
In part III and IV, economic insights will be extended and cover portfolio choices,
growth under uncertainty ...

Even more motivation for this book

Why teach a course based on this book? Is it not boring to go through all these
methods? In a way, the answer is yes. We all want to understand certain empirical
regularities or understand potential fundamental relationships and make exciting new
empirically testable predictions. In doing so, we also all need to understand existing
work and eventually present our own ideas. It is probably much more boring to
be hindered in understanding existing work and be almost certainly excluded from

3
presenting own ideas if we always spend long time trying to understand how certain
results were derived. How did this author get from equation (1) and (2) to equation
(3)? The major advantage of economic analysis over other social sciences is its strong
foundation in formal models. These models allow to discuss questions much more
precisely as expressions like “marginal utility”, “time preference rate” or “Keynes-
Ramsey rule”reveal a lot of information in very short time. It is therefore extremely
useful to …rst spend some time in getting to know these methods and then try to do
what economics is really about: understand the real world.
But, before we really start, there is also a second reason - at least for some
economists - to go through all these methods: They contain a certain type of truth.
A proof is true or false. The derivation of some optimal behaviour is true or false. A
prediction of general equilibrium behaviour of an economy is truth. Unfortunately,
it is only truth in an analytical sense, but this is at least some type of truth. Better
than none.

4
Part I
Deterministic models in discrete
time
2 OLG models and di¤erence equations
Given the idea of this book to start from simple structures and extending them to the
more complex ones, this chapter starts with the simplest intertemporal problem, a
two-period decision framework. This simple setup already allows to illustrate the basic
dynamic trade-o¤s. Aggregating over individual behaviour, assuming an overlapping-
generations (OLG) structure, and putting individuals in general equilibrium provides
an understanding of the issues involved in these steps and in identical steps in more
general settings. Some revision of properties of di¤erence equations concludes this
chapter.

2.1 The general setup

Let there be an individual living for two periods, in the …rst it is young, in the second
it is old. Let her utility function be given by

Ut = U cyt ; cot+1 U (ct ; ct+1 ) ; (2.1)

where consumption when young and old are denoted by cyt and cot+1 or ct and ct+1 ;
respectively, when no ambiguity arises. The individual earns labour income wt in
both periods. It could also be assumed that she earns labour income only in the …rst
period (e.g. when retiring in the second) or only in the second period (e.g. when
going to school in the …rst). Here, with st denoting savings, her budget constraint in
the …rst period is
ct + st = wt (2.2)
and in the second it reads

ct+1 = wt+1 + (1 + rt+1 ) st :

Interests paid on savings in the second period are given by rt+1 : All quantities are
expressed in units of the consumption good (i.e. the price of the consumption good
is set equal to one. See ch. 2.2.2 for an extension where the price of the consumption
good is pt :).
This budget constraint says something about the assumptions made on the timing
of wage payments and savings. What an individual can spend in period two is princi-
pal and interest of her savings st of the …rst period. There are no interest payments in
period one. This means that wages are paid and consumption takes place at the end
of period 1 and savings are used for some productive purposes (e.g. …rms use it in the
form of capital for production) in period 2. Therefore, returns rt+1 are determined

5
by economic conditions in period 2 and have the index t + 1: Timing is illustrated in
the following …gure.

wt ct wt+1 ct+1
st = wt ct (1 + rt+1 )st

t t+1
Figure 4 Timing in OLG models

These two budget constraints can be merged into one intertemporal budget con-
straint by substituting out savings,
1 1
wt + (1 + rt+1 ) wt+1 = ct + (1 + rt+1 ) ct+1 : (2.3)
It should be noted that by not restricting savings to be non-negative in (2.2) or by
equating the present value of income on the left-hand side with the present value
of consumption on the right-hand side in (2.3), we assume perfect capital markets:
individuals can save and borrow any amount they desire. Equation (2.13) provides a
condition under which individuals save.
Adding the behavioural assumption that individuals maximize utility, the eco-
nomic behaviour of an individual is completely described and one can derive her
consumption and saving decisions. The problem can be solved by a Lagrange ap-
proach, by simply inserting (as e.g. in ch. 7.5.4), or in other ways, to be presented
shortly.
The maximization problem reads maxct ; ct+1 (2.1) subject to the intertemporal bud-
get constraint (2.3). The household’s control variables are ct and ct+1 : As they need
to be chosen such that they satisfy the budget constraint, they can not be chosen
independently of each other. Wages and interest rates are exogenously given to the
household. When choosing consumption levels, the reaction of these quantities to the
decision of our household is assumed to be zero - simply because the household is too
small to have an e¤ect on economy-wide variables. The corresponding Lagrangian
reads
1 1
L = U (ct ; ct+1 ) + wt + (1 + rt+1 ) wt+1 ct (1 + rt+1 ) ct+1 ,
where is a constant called the Lagrange multiplier.
The …rst-order conditions are
Lct = Uct (ct ; ct+1 ) = 0;
1
Lct+1 = Uct+1 (ct ; ct+1 ) [1 + rt+1 ] = 0;
1 1
L = wt + (1 + rt+1 ) wt+1 ct + (1 + rt+1 ) ct+1 = 0:

6
Clearly, the last …rst-order condition is identical to the budget constraint. Note that
there are three variables to be determined, consumption for both periods and the
Lagrange multiplier .1 Having at least three conditions is a necessary, though not
su¢ cient (they might, generally speaking, be linearly dependent) condition for this
to be possible.
The …rst two …rst-order conditions can be combined to give

Uct (ct ; ct+1 ) = (1 + rt+1 ) Uct+1 (ct ; ct+1 ) = : (2.4)

Marginal utility from consumption today on the left-hand side must equal marginal
utility tomorrow, corrected by the interest rate, on the right-hand side. The economic
meaning of this correction can best be understood when looking at a version with
nominal budget constraints (see ch. 2.2.2).
What we learn from this maximization here is that its solution (2.4) gives us
an equation, that needs to hold when behaving optimally, which links consumption
ct today to consumption ct+1 tomorrow. This equation together with the budget
constraint (2.3) provides a two-dimensional system: two equations in two unknowns,
ct and ct+1 . These equations therefore allow in principle to compute these endogenous
variables as a function of exogenously given wages and interest rates. The next section
provides an example where this is indeed the case.

2.2 Examples
2.2.1 Optimal consumption
This …rst example allows us to indeed solve explicitly for consumption levels in both
periods. Let households have a Cobb-Douglas utility function represented by

Ut = ln ct + (1 ) ln ct+1 : (2.5)

Utility from consumption in each period, instantaneous utility, is given by the log-
arithm of consumption. As ln c has a positive …rst and negative second derivative,
higher consumption increases instantaneous utility but at a decreasing rate. The
parameter captures the importance of instantaneous utility in the …rst relative to
instantaneous utility in the second. Overall utility Ut is maximized subject to the
constraint we know from above,
1
Wt = ct + (1 + rt+1 ) ct+1 ; (2.6)

where we denote the present value of labour income by Wt wt + (1 + rt+1 ) 1 wt+1 :

Again, the consumption good is chosen as numeraire good and its price equals unity.
Wages are therefore expressed in units of the consumption good.
The Lagrangian for this problem reads
1
L= ln ct + (1 ) ln ct+1 + Wt ct (1 + rt+1 ) ct+1 :
1
Note that equals the marginal utility of one extra unit of income (cf. section 2.3).

7
The …rst-order conditions are
1
Lct = (ct ) = 0;
1 1
Lct+1 = (1 ) (ct+1 ) [1 + rt+1 ] = 0;

and the budget constraint (2.6) following from L = 0: Dividing …rst-order conditions
gives 1 ct+1
ct
= 1 + rt+1 and therefore

1
ct = (1 + rt+1 ) ct+1 :
1
This equation corresponds to our optimality rule (2.4) derived above for the more
general case. Inserting into the budget constraint (2.6) yields

1 1 1
Wt = + 1 (1 + rt+1 ) ct+1 = (1 + rt+1 ) ct+1
1 1
which is equivalent to
ct+1 = (1 ) (1 + rt+1 ) Wt : (2.7)
It follows that
ct = W t : (2.8)
Apparently, optimal consumption decisions imply that consumption when young is
given by a share of the present value Wt of life-time income at time t of the individual
under consideration. Consumption when old is given by the remaining share 1
plus interest payments, ct+1 = (1 + rt+1 ) (1 ) Wt : Assuming preferences as in (2.5)
makes modelling sometimes easier than with more complex utility functions. The
drawback is that the share of lifetime income consumed and thereby the savings
decision is independent of the interest rate, which appears implausible. A way out
is given by the CES utility function which also allows for closed-form solutions of
consumption (cf. ch. 7.5.4). More generally speaking, such a simpli…cation should be
justi…ed if some aspect of an economy that are fairly independent of savings are the
focus of some analysis. For an example concerning bequests and wealth distributions,
see Kleiber, Sexauer and Wälde (2005).

2.2.2 Optimal consumption with prices

Consider now again the utility function (2.1) and maximize it subject to the con-
straints pt ct + st = wt and pt+1 ct+1 = wt+1 + (1 + rt+1 ) st : The di¤erence to the
introductory example in ch. 2.1 consists in the introduction of an explicit price pt
for the consumption good. The …rst-period budget constraint now equates nominal
expenditure for consumption (pt ct is measured in, say, Euro, Dollar or Yen) plus nom-
inal savings to nominal wage income. The second period constraint equally equates
nominal quantities. How does an optimal consumption rule as in (2.4) now look like?
The Lagrangian is, using now an intertemporal budget constraint with prices,
1 1
L = U (ct ; ct+1 ) + wt + (1 + rt+1 ) wt+1 pt ct (1 + rt+1 ) pt+1 ct+1 :

8
The …rst-order conditions for ct and ct+1 are

Lct = Uct (ct ; ct+1 ) pt = 0;

1
Lct+1 = Uct+1 (ct ; ct+1 ) (1 + rt+1 ) pt+1 = 0

and the intertemporal budget constraint. Combining them gives

Uct (ct ; ct+1 ) Uct+1 (ct ; ct+1 ) Uct (ct ; ct+1 ) pt

= 1 , = 1: (2.9)
pt pt+1 [1 + rt+1 ] Uct+1 (ct ; ct+1 ) pt+1 [1 + rt+1 ]

This equation says that marginal utility of consumption today relative to marginal
utility of consumption tomorrow equals the relative price of consumption today and
tomorrow. This optimality rule is identical for a static 2-good decision problem
where optimality requires that the ratio of marginal utilities equals the relative price.
The relative price here is expressed in a present value sense as we compare marginal
utilities at two points in time. The price pt is therefore divided by the price tomorrow,
discounted by next period’s interest rate, pt+1 [1 + rt+1 ] 1 :

2.2.3 Some useful de…nitions

In order to be able to discuss results in subsequent sections easily, we review some
de…nitions here that will be useful. We are mainly interested in the intertemporal
elasticity of substitution and the time preference rate. We start with the

Marginal rate of substitution (MRS)

Let utility be given by u (c1 ; c2 ; ::::; cn ) : The MRS between good i and good j
given a consumption bundle (c1 ; c2 ; ::::; cn ) (:) is then de…ned by

@u (:) =@ci
M RSij (:) :
@u (:) =@cj

It gives the additional amount of consumption of good j that is required to keep the
utility level at u (c1 ; c2 ; ::::; cn ) when the amount of i is decreased marginally.
Why this is so can easily be shown: Consider the total di¤erential of u (c1 ; c2 ; ::::; cn ) ;
keeping all consumption levels apart from ci and cj …x. This yields

@u (:) @u (:)
du (c1 ; c2 ; ::::; cn ) = dci + dcj :
@ci @cj

The overall utility level u (c1 ; c2 ; ::::; cn ) does not change if

dcj @u (:) @u (:)

du (:) = 0 , = = M RSij (:) :
dci @ci @cj

Equivalent terms

9
 As a reminder, the equivalent term to the MRS in production theory is the mar-
@f (:)=@xi
ginal rate of technical substitution M RT Sij (:) = @f (:)=@xj
where the utility function
was replaced by a production function and consumption ck was replaced by factor
inputs xk .
More on an economy wide level, there is the marginal rate of transformation
@G(:)=@yi
M RTij (:) = @G(:)=@y j
where the utility function was now replaced by a transformation
function G (maybe better known as production possibility curve) and the yk are
output of good k. The marginal rate of transformation gives the increase in output
of good j when output of good i is marginally decreased.
(Intertemporal) elasticity of substitution
Though our main interest is a measure of intertemporal substitutability, we …rst
de…ne the elasiticty of substition in general. The most common de…nition for two
d(ci =cj ) pi =pj
consumption goods is ij d(pi =pj ) ci =cj
: how much does relative consumption ci =cj
adjust when relative prices pi =pj change (proportionally to the current relative-price
p =p
relative-consumption ratio cii =cjj )? This de…nition can be expressed alternatively (see
ex. 5 for details) in a way which is more useful here. Here, the elasticity of substitution
is de…ned by the derivative of the log of relative consumption with respect to the log
of the marginal rate of substitution,
d ln (ci =cj )
ij : (2.10)
d ln M RSij
Inserting the MRS gives
d ln (ci =cj ) uc =uc d (ci =cj )
ij = = i j
d ln uci =ucj ci =cj d uci =ucj
The advantage of an elasticity when compared to a normal derivative as e.g. the MRS
is that an elasticity is measureless. It is expressed in percentage changes. (This can
be seen in the following example and in exercise number 5a.) It can both be applied
to static utility or production functions or to intertemporal utility functions.
The intertemporal elasticity of substitution for a utility function u (ct ; ct+1 ) is then
simply the elasticity of substitution of consumption at two points in time,
uct =uct+1 d (ct =ct+1 )
t;t+1 :
ct =ct+1 d uct =uct+1
Using as example the Cobb-Douglas utility function from (2.5), we obtain
= c1t+1 d (ct =ct+1 )
ct
t;t+1 = = 1;
ct =ct+1 d = 1
ct ct+1

where the last step used

1
d (ct =ct+1 ) 1 d ct+1 =ct 1 1
= = :
d =1 d (ct+1 =ct ) (ct+1 =ct )2
ct ct+1

10
 The time preference rate

Intuitively, the time preference rate is the rate at which future instantaneous
utilities are discounted. To illustrate, imagine a discounted income stream
2
1 1
x0 + x1 + x2 + :::
1+r 1+r
where discounting takes place at the interest rate r: Replacing income xt by instanta-
neous utility and the interest rate by , would be the time preference rate. Formally,
the time preference rate is the marginal rate of substitution of instantaneous utilities
(not of consumption levels) minus one,2

TPR M RSt;t+1 1:

As an example, consider the following standard utility function which we will use
very often in later chapters,

1 t 1
U0 = t=0 u (ct ) ; ; > 0: (2.11)
1+
Let be a positive parameter and the implied discount factor, capturing the idea of
impatience: By multiplying instantaneous utility functions u (ct ) by t , future utility
is valued less than present utility. This utility function generalizes (2.5) in two ways:
First and most importantly, there is a much longer planning horizon than just two
periods. In fact, the individual’s overall utility U0 stems from the sum of discounted
instanteneous utility levels u (ct ) over periods 0; 1; 2; ... up to in…nity. The idea
behind this objective function is not that individuals live forever but that individuals
care about the well-being of subsequent generations. Second, the instantaneous utility
function u (ct ) is not logarithmic as in (2.5) but of a more general nature where one
would usually assume positive …rst and negative second derivatives, u0 > 0; u00 < 0.
The marginal rate of substitution is then
@U0 (:) =@u (cs ) (1= (1 + ))s
M RSs;s+1 (:) = = =1+ :
@U0 (:) =@u (cs+1 ) (1= (1 + ))s+1
The time preference rate is therefore given by :
Now take as example the utility function (2.5). Computing the MRS minus one,
we have
2 1
= 1= : (2.12)
1 1
The time preference rate is positive if > 0:5: This makes sense for (2.5) as one
should expect that future utility is valued less than present utility.

Does consumption increase over time?

2
An alternative formulation implying the same de…nition is used by Buiter (1981, p. 773). He
de…nes the pure rate of time preference as “the marginal rate of substitution between consumption”
in two periods “when equal amounts are consumed in both periods, minus one.”

11
 This de…nition of the time preference rate allows us to provide a precise answer
to the question whether consumption increases over time. We simply compute the
condition under which ct+1 > ct by using (2.7) and (2.8),
ct+1 > ct , (1 ) (1 + r) Wt > Wt ,
1+
1+r > ,r> ,r> :
1 1
Consumption increases if the interest rate is higher than the time preference rate.
The time preference rate of the individual (being represented by ) determines how
to split the present value Wt of total income into current and future use. If the interest
rate is su¢ ciently high to overcompensate impatience, i.e. if (1 ) (1 + r) > in
the …rst line, consumption rises.
Note that even though we computed the condition for rising consumption for our
special utility function (2.5), the result that consumption increases when the interest
rate exceeds the time preference rate holds for more general utility functions as well.
We will get to know various examples for this in subsequent chapters.

Under what conditions are savings positive?

Savings are given by

1
st = wt ct = wt wt + wt+1 = w 1
1+r 1+r
where the last equality assumed an invariant wage level, wt = wt+1 . Savings are
positive if and only if

st > 0 , 1 > () 1 + r > ()

1+r 1
2 1
r> () r > (2.13)
1
This means that savings are positive if interest rate is larger than time preference
rate. Clearly, this result does not necessarily hold for wt+1 > wt :

2.3 The idea behind the Lagrangian

2.3.1 Where the Lagrangian comes from I
The maximization problem

Let us consider a maximization problem maxx1 ;x2 F (x1 ; x2 ) subject to some con-
straint g(x1; x2 ) = b: (The presentation here is strongly inspired by Intriligator, 1971,
p. 28 - 30). If we rewrite the constraint as x2 = h (x1 ) (which is possible if the
function g (:) is not too complex, i.e. not e.g. of the type g (:) = ex1 x2 + x1 x2 ), the
maximisation problem can be written as
max F (x1; h (x1 )) :
x1

12
 The derivatives of implicit functions

As we will use derivatives of implicit functions here and in later chapters, we

brie‡y show how to compute them. Consider a function f (x1 ; x2 ; :::; xn ) = 0: Its
di¤erential is
@f (:) @f (:) @f (:)
df (:) = dx1 + dx2 + ::: + dxn = 0:
@x1 @x2 @xn
When we keep x3 to xn constant, we can solve this to get
dx2 @f (:) =@x1
= : (2.14)
dx1 @f (:) =@x2
This equation can be seen as an application of the implicit function theorem which
- stated simply - says that the function f (x1 ; x2 ; :::; xn ) = 0 implicitly de…nes (un-
der suitable assumptions concerning the properties of f (:)) a functional relationship
x2 = g (x1 ; x3 ; x4 ; :::; xn ) : Hence, the derivative (2.14) is the derivative of this implicit
function g (:) with respect to x1 when all other variables are held constant. Most of
the time, we will compute derivatives of implicit functions stemming from functions
with two variables, i.e. f (:) = f (x1 ; x2 )

Solution of the maximization problem

This has a standard optimality condition

@F @F dh
+ = 0: (2.15)
@x1 @x2 dx1
Taking into consideration that from the implicit function theorem applied to the
constraint,
dh dx2 @g(x1 ; x2 )=@x1
= = ; (2.16)
dx1 dx1 @g(x1 ; x2 )=@x2
@F @F=@x2 @g(x1 ;x2 )
the optimality condition (2.15) can be written as @x1 @g=@x2 @x1
= 0: Now de…ne
@F=@x2
the Lagrange multiplier @g=@x2
and obtain

@F @g(x1 ; x2 )
= 0: (2.17)
@x1 @x1
As can be easily seen, this is the …rst-order condition of the Lagrangian

L = F (x1; x2 ) + [b g (x1; x2 )]

with respect to x1 :
Undertaking the same steps for x2 would yield the second …rst-order condition
@F @g(x1 ; x2 )
= 0:
@x2 @x2
We have thereby shown where the Lagrangian comes from.

13
2.3.2 How to understand shadow prices
The idea

We can now also give an interpretation of the meaning of the multipliers . Start-
ing from the de…nition of in (2.17) and rewriting them according to
@F=@x2 @F @F
= = ;
@g=@x2 @g @b
we see that equals the change in F as a function of b:3 By how much does F increase
(e.g. your utility) when your constraint b (your bank account) is relaxed? How much
does the social welfare function change when the economy has more capital? How
much do pro…ts of …rms change when the …rm has more workers?

A derivation

A more rigorous derivation is as follows (cf. Intriligator, 1971, ch. 3.3). Compute
the derivative of the maximized Lagrangian with respect to b,
@L(x1 (b) ; x2 (b)) @
= (F (x1 (b); x2 (b)) + (b) [b g (x1 (b); x2 (b))]
@b @b
@x @x 0 @x1 @x2
= Fx1 1 + Fx2 2 + (b) [b g(:)] + (b) 1 gx1 gx2
@b @b @b @b
= (b)

The last equality results from …rst-order conditions and the fact that the budget
constraint holds.
As L(x1 ; x2 ) = F (x1 ; x2 ) due to the budget constraint holding with equality,
@L(x1 ; x2 ) @F (x1 ; x2 )
(b) = =
@b @b
Shadow prices

Sometimes to often, is referred to as shadow price. One can think of price

in the sense of a price in a currency e.g. in Euro only if the objective function is
some nominal expression like pro…ts or GDP, otherwise it is a price expressed e.g. in
utility terms. This can be nicely seen in the following example. Consider a central
planner that maximizes social welfare u (x1 ; x2 ) subject to technological and resource
constraints,
max u (x1 ; x2 )
subject to

x1 = f (K1; L1 ) ; x2 = g (K2; L2 ) ;
K1 + K2 = K; L1 + L2 = L:
3 @f (x1 ;:::;xn ) lim4xi !0 f (x1 ;:::;xi +4xi ;:::;xn ) f (x1 ;:::;xn )
The term @x2 cancels out as by de…nition @xi = lim4xi !0 4xi :

14
Using as multipliers p1 ; p2 ; wK and wL ; the Lagrangian reads

L = u (x1 ; x2 ) + p1 [f (K1; L1 ) x1 ] + p2 [g (K2 ; L2 ) x2 ]

+ wK [K K1 K2 ] + w2 [L L1 L2 ]

and …rst-order conditions are

@L @u @L @u
= p1 = 0; = p2 = 0;
@x1 @x1 @x2 @x2
@L @f @L @g
= p1 wK = 0; = p2 wK = 0;
@K1 @K1 @K2 @K2
@L @f @L @g
= p1 wL = 0; = p2 wL = 0:
@L1 @L1 @L2 @L2
Here we see that the …rst multiplier p1 is not a price expressed in some currency
but the derivative of the utility function with respect to good 1, i.e. marginal utility.
By contrast, if we looked at the multiplier wK only in the third …rst-order condition,
we would then conclude that it is a price. Inserting then the …rst …rst-order condition
shows however that it really stands for the increase in utility when the capital stock
used in production of good 1 rises,
@f @u @f @u
wK = p1 = = :
@K1 @x1 @K1 @K1
Hence wK and all other multipliers are prices in utility units.
It is now also easy to see that all shadow-prices are prices expressed in some
currency if the objective function was not utility but, e.g. GDP. Such a maximization
problem could read max p1 x1 + p2 x2 subject to the constraints as above.

2.4 A general equilibrium analysis

We will now analyze many households jointly and see how their consumption and
saving behaviour a¤ects the evolution of the economy as a whole. We will get to
know the Euler theorem and how it is used to sum factor incomes to yield GDP.
We will also understand how the interest rate in the household’s budget constraint is
related to marginal productivity of capital and the depreciation rate. All this jointly
yields time paths of aggregate consumption, the capital stock and GDP.
A model in general equilibrium is described by fundamentals of the model and
market and behavioural assumptions. Fundamentals are technologies of …rms, prefer-
ences of households and factor endowments. Adding clearing conditions for markets
and behavioural assumptions for agents completes the description of the model.

2.4.1 Technologies
The …rms

15
 Let there be many …rms who use a technology

Yt = Y (Kt ; L) (2.18)

which is characterized by constant returns to scale. Letting …rms act under perfect
competition, the …rst-order conditions from pro…t maximization, where pro…ts are
given by t = Yt wtK K wtL L, read

@Yt @Yt
= wtK ; = wtL : (2.19)
@Kt @Lt
They equate in each period t the marginal productivity of capital to the factor price
wtK for capital and the marginal productivity of labour to labour’s factor reward wtL .

Euler’s theorem

Euler’s theorem shows that for a linear-homogenous function f (x1 ; x2 ; :::; xn ) the
sum of partial derivatives times the variables with respect to which the derivative was
computed equals the original function f (:) ;

@f (:) @f (:) @f (:)

f (x1 ; x2 ; :::; xn ) = x1 + x2 + ::: + xn : (2.20)
@x1 @x2 @xn
Provided that the technology used by …rms to produce Yt has constant returns to
scale, we obtain from this theorem that
@Yt @Yt
Yt = Kt + Lt : (2.21)
@Kt @Lt
Using the optimality conditions (2.19) of the …rm for the applied version of Euler’s
theorem (2.21) yields
Yt = wtK K + wtL L: (2.22)
Total output in this economy, Yt ; is identical to total factor income. This result is
usually given the economic interpretation that under perfect competition all revenue
of …rms is used to pay factors of production. As a consequence, pro…ts t of …rms are
zero.

2.4.2 Households
Individual households

Households live again for two periods. The utility function is therefore as in (2.5)
and given by
Ut = ln cyt + (1 ) ln cot+1 : (2.23)
It is maximized subject to the intertemporal budget constraint
1 o
wt = cyt + (1 + rt+1 ) ct+1 :

16
This constraint di¤ers slightly from (2.6) in that people work only in the …rst period
and retire in the second. Hence, there is labour income only in the …rst period on
the left-hand side. Savings of the …rst period are used to …nance consumption in the
second period.
Given that the present value of lifetime wage income is wt ; we can conclude from
(2.7) and (2.8) that individual consumption expenditure and savings are given by

cyt = wt ; cot+1 = (1 ) (1 + rt+1 ) wt ; (2.24)

y
st = wt ct = (1 ) wt : (2.25)

Aggregation

We assume that in each period L individuals are born and die. Hence, the number
of young and the number of old is L as well. As all individuals within a generation
are identical, aggregate consumption within one generation is simply the number of,
say, young times individual consumption. Aggregate output in t is therefore given by
Ct = Lcyt + Lcot : Using the expressions for individual consumption from (2.24) and
noting the index t (and not t + 1) for the old yields

Ct = Lcyt + Lcot = ( wt + (1 ) (1 + rt ) wt 1 ) L:

2.4.3 Goods market equilibrium and accumulation identity

The goods market equilibrium requires that supply equals demand, Yt = Ct + It ;
where demand is given by consumption plus gross investment. Next period’s capital
stock is - by an accounting identity - given by Kt+1 = It + (1 ) Kt such that net
investment amounts to the change in the capital stock, Kt+1 Kt = It Kt ; given by
gross investment It minus depreciation, where is the depreciation rate. Replacing
gross investment by the goods market equilibrium, we obtain the resource constraint

Kt+1 = (1 ) Kt + Y t Ct : (2.26)

For our OLG setup, it is useful to slightly rewrite this constraint,

Yt + (1 ) Kt = Ct + Kt+1 : (2.27)

In this formulation, it re‡ects the goods market equilibrium where the left-hand side
shows supply as current production and capital held by the old. The old sell capital
as it is of no use for them, given that they will not be able to consume anything
next period. Demand for the aggregate good is given by aggregate consumption (i.e.
consumption of the young plus consumption of the old) plus the capital stock to be
held next period by the currently young.

2.4.4 The reduced form

Once all maximization problems are solved, the objective consists in understanding
properties of the model, i.e. understand its predictions. This is usually done by …rst

17
simplifying the structure of the system of equations coming out of the model as much
as possible. In the end, after inserting and reinserting, a system of n equations in n
unknowns results.
Ideally, there is only one equation left and this equation gives an explicit solution
of the endogenous variable. In static models, an example would be LX = L; i.e.
employment in sector X is given by a utility parameter times total exogenous
labour supply L: This would be an explicit solution. If one is left with just one
equation but one obtains on an implicit solution, one would obtain something like
f (LX ; ; L) = 0:

Deriving the reduced form

We now derive, given the results we obtained by now, how large the capital stock
in the next period is. Splitting aggregate consumption into consumption of the young
and consumption of the old and using the output-factor reward identity (2.22) for the
resource constraint in the OLG case (2.27), we obtain

wtK Kt + wtL L + (1 ) Kt = Cty + Cto + Kt+1 :

De…ning the interest rate rt as the di¤erence between factor rewards wtK for capital
and the depreciation rate ;
rt wtK ; (2.28)
we …nd
rt Kt + wtL L + Kt = Cty + Cto + Kt+1 :
The interest rate de…nition (2.28) shows the net income of capital owners per unit of
capital. They earn the gross factor rewards wtK but, at the same time, they experience
a loss from depreciation. Net income therefore only amounts to rt : As the old consume
the capital stock plus interest cot L = (1 + rt )Kt 4 , we obtain

Kt+1 = wt L Cty = st L: (2.29)

which is the aggregate version of the savings equation (2.25). Hence, we have found
that savings st of young at t is the capital stock at t + 1:
Note that equation (2.29) is often present on “intuitive” grounds. The old in
period t have no reason to save as they will not be able to use their savings in t + 1:
Hence, only the young will save and the capital stock in t + 1; being made up from
savings in the previous period, must equal to the savings of the young.

The one-dimensional di¤erence equation

In our simple dynamic model considered here, we obtain indeed the ideal case
where we are left with only one equation that gives us the solution for one variable,
4
Now we see that the interest rate in (2.24) must refer to period t + 1.

18
the capital stock. Inserting the individual savings equation (2.25) into (2.29) gives
with the …rst-order condition (2.19) of the …rm

@Y (Kt ; L)
Kt+1 = (1 ) wtL L = (1 ) L: (2.30)
@L
The …rst equality shows that a share 1 of labour income becomes capital in the
next period. Interestingly, the depreciation rate does not have an impact on the
capital stock in period t + 1. Economically speaking, the depreciation rate a¤ects
wealth of the old but - with logarithmic utility - not saving of the young.

2.4.5 Properties of the reduced form

Equation (2.30) is a di¤erence equation in Kt : All other quantities in this equation
are constant. This equation determines the entire path of capital in this dynamic
economy, provided we have an initial condition K0 . We have therefore indeed solved
the maximization problem and reduced the general equilibrium model to one single
equation. From the path of capital, we can compute all other variables which are of
interest for our economic questions.
Whenever we have reduced a model to its reduced form and have obtained one or
more di¤erence equations (or di¤erential equations in continuous time), we would like
to understand the properties of such a dynamic system. The procedure is in principle
always the same: We …rst ask whether there is some solution where all variables (Kt
in our case) are constant. This is then called a steady state analysis. Once we have
understood the steady state (if there is one), we want to understand how the economy
behaves out of the steady state, i.e. what its transitional dynamics are.

Steady state

In the steady state, the capital stock K is constant and determined by

@Y (K ; L)
K = (1 ) L: (2.31)
@L
All other variables like aggregate consumption, interest rates, wages etc. are constant
as well. Consumption when young and when old can di¤er as in a setup with …nite
lifetimes the interest rate in the steady state does not need to equal the time preference
rate of households.

Transitional dynamics

Dynamics of the capital stock are illustrated in …gure 5. The …gure plots the
capital stock in period t on the horizontal axis. The capital stock in the next period,
Kt+1 ; is plotted on the vertical axis. The law of motion for capital from (2.30) then
shows up as the curve in this …gure. The 45 line equates Kt+1 to Kt :
We start from our initial condition K0 . Equation (2.30) or the curve in this …gure
then determine the capital stock K1 : This capital stock is then viewed as Kt such

19
that, again, the curve gives us Kt+1 ; which is, given that we now started in 1; the
capital stock K2 of period 2: We can continue doing so and see graphically that the
economy approaches the steady state K which we had computed in (2.31).
N
Kt+1
Kt = Kt+1

Kt+1 = (1 )w(Kt )L
6
-

- 6

N
0
K0 Kt
Figure 5 Convergence to the steady state

Summary

We started with a description of technologies in (2.18), preferences in (2.23) and

factor endowment given by K0 : With behavioural assumptions concerning utility and
pro…t maximization and perfect competition on all markets plus a description of mar-
kets in 2.4.3 and some “juggling of equations”, we ended up with a one-dimensional
di¤erence equation (2.30) which describes the evolution of the economy over time and
in the long-run steady state. Given this formal analysis of the model, one could now
start answering economic questions.

2.5 More on di¤erence equations

2.5.1 Two useful proofs
The following result will be useful in what follows. As the proof of this result also
has an esthetic value, there will be a second proof of another result to be done in the
exercises.

Lemma 1 For any a 6= 1;

T i 1 aT T i 1 aT +1
i=1 a =a ; i=0 a =
1 a 1 a
Proof. The left hand side is given by
T i
i=1 a = a + a2 + a3 + : : : + aT 1
+ aT : (2.32)

Multiplying this sum by a yields

T i
a i=1 a = a2 + a3 + : : : + aT + aT +1 : (2.33)

20
Now subtract (2.33) from (2.32) and …nd

T i 1 aT
(1 a) i=1 a =a aT +1 , T
i=1 a
i
=a : (2.34)
1 a

Lemma 2
T i 1 1 aT
i=1 ia = a T aT +1
1 a 1 a

The proof is left as exercise 7.

2.5.2 The simplest di¤erence equation

The simplest di¤erence equation is

xt+1 = axt ; a > 0: (2.35)

This equation appears too simple to be worth being analysed. We do it here as we

get to know the standard steps in analysing di¤erence equations which we will also
use for more complex di¤erence equations. The objective here is therefore not this
di¤erence equation an sich but what is done with it.

Solution by inserting

The simplest way to solve it is simply to insert and reinsert this equation su¢ -
ciently often. Doing it three times gives

x1 = ax0 ;
x2 = ax1 = a2 x0 ;
x3 = ax2 = a3 x0 :

When we look at this solution for t = 3 long enough, we see that the general solution
is
x t = at x 0 :
This could formally be proven by a proof by induction.

Examples for solutions

The sequence of xt given by this solution, given di¤erent initial conditions x0 ;

are shown in the following …gure for a > 1: The parameter values chosen are a = 2,
x0 2 f0:5; 1; 2g and t runs from 0 to 10.

21
 14

12

10

0
0 2 4 6 8 10
e x pw a .m

Figure 6 Solution to a di¤erence equation for a > 1

Long-run behaviour

We can now ask whether xt approaches a constant when time goes to in…nity.
This gives 8 9 8
< 0 = < a<1
t
lim xt = x0 lim a = x 0 , a=1 :
t !1 t !1 : ; :
1 a>1
Hence, xt approaches a constant only for a < 1:

A graphical analysis

For more complex di¤erence equations, it often turns out to be useful to analyze
their behaviour in a phase diagram. Even though this simple di¤erence equation
can be understood easily analytically, we will illustrate its properties in the following
…gure. This allows to understand how analyses of this type can be undertaken also
for more complex di¤erence equations.

N N
xt+1 xt+1

xt+1 = axt
-
xt = xt+1
xt+1 = axt 6
xt = xt+1

-6
N

x0 xt x0 xt

22
Figure 7 A phase diagram for a < 1 in the left and a > 1 in the right panel

The left panel shows how xt evolves over time, starting at x0 : In this case of a < 1;
we see how xt approaches zero. When we graphically illustrate the case of a > 1; the
evolution of xt is as shown in the right panel.

2.5.3 A slightly less but still simple di¤erence equation

We now consider a slightly more general di¤erence equation. Compared to (2.35), we
just add a constant b in each period,

xt+1 = axt + b; a > 0: (2.36)

Solution by inserting

We solve again by inserting. In contrast to the solution for (2.35), we start from
an initial value of xt : Hence, we imagine we are in t (t as today) and compute what
the level of x will be tommorrow and the day after tommorrow etc. We …nd for xt+2
and xt+3 that

xt+2 = axt+1 + b = a [axt + b] + b = a2 xt + b [1 + a] ;

xt+3 = a3 xt + b 1 + a + a2 :

This suggests that the general solution is

n 1 i an 1
xt+n = an xt + b i=0 a = an x t + b :
a 1
The last equality used the …rst lemma from ch. 2.5.1.

Limit for n ! 1 and a < 1

The limit for n going to in…nity and a < 1 is given by

an 1 b
lim xt+n = lim an xt + b = : (2.37)
n !1 n !1 a 1 1 a
A graphical analysis

The left panel of the following …gure studies the evolution of xt for the stable case,
i.e. where 0 < a < 1 and b > 0: Starting today in t with xt ; we end up in x . As
we chose a smaller than one and a positive b; x is positive as (2.37) shows. We will
return to the right panel in a moment.

23
 N N xt+1 = axt + b
xt+1 xt+1 -

xt = xt+1
? xt = xt+1
6
?
xt+1 = axt + b -

- 6

b - 6

N
x xt x xt

Figure 8 Phase diagrams for a more general di¤erence equation

2.5.4 Fix points and stability

De…nition

We can use these examples to de…ne two concepts that will be useful also at later
stages.

De…nition 1 (Fixpoint) A …xpoint x of a function f (x) is de…ned by

x = f (x ) : (2.38)

For di¤erence equations of the type xt+1 = f (xt ) ; the …xpoint x of the function
f (xt ) is the point where x stays constant. This is usually called the long-run equilib-
rium point of some economy or its steady or stationary state. Whenever an economic
model, represented by its reduced form, is analysed, it is generally useful to …rst try
and …nd out whether …xpoints exist and what its economic properties are.
For the di¤erence equation of last section, we obtain
b
xt+1 = xt x () x = ax + b () x = :
1 a
Local and global stability

Once a …xpoint has been identi…ed, one can ask whether it is stable.

De…nition 2 (Global stability) A …xpoint x is globally stable if, starting from an

initial value x0 6= x ; xt converges to x .

The concept of global stability usually refers to initial values x0 that are econom-
ically meaningful. An initial physical capital stock that is negative would not be
considered to be economically meaningful.
unstable
De…nition 3 (Local stability and instability) A …xpoint x is if,
locally stable
diverges from
starting from an initial value x + "; where " is small, xt x.
converges to

24
 For illustration purposes consider the …xpoint x in the left panel of …g. 8 - it is
globally stable. In the right panel of the same …gure, it is unstable. As can easily
be seen, the instability follows from simply letting the xt+1 line intersect the 45 -line
from below. In terms of the underlying di¤erence equation (2.36), this requires b < 0
and a > 1:
Clearly, economic systems can be much more complex and generate several …x-
points. Imagine the link between xt+1 and xt is not linear as in (2.36) but nonlinear,
xt+1 = f (xt ) : Unfortunately for economic analysis, a nonlinear relationship is the
much more realistic case. The next …gure provides an example for some function
f (xt ) that implies an unstable xu and a locally stable …xpoint xs .
N
xt+1

?
-
- 6

N
x01 xu x02 xs x03 x(t)

Figure 9 A locally stable …xpoint xs and an unstable …xpoint xu

2.5.5 An example: Deriving a budget constraint

A frequently encountered di¤erence equation is the budget constraint. We have
worked with budget constraints at various points before but we have hardly spent
thoughts about their origin. We more or less simply wrote them down. Budget
constraints, however, are tricky objects, at the latest when we think about general
equilibrium setups. What is the asset we save in? Is there only one asset or are
there several? What is the price of this asset? How does it relate to the price of the
consumption good, i.e. do we express the value of assets in real or nominal terms?
This section will derive a budget constraint. We assume that there is only one
asset. The price of one unit of the asset will be denoted by vt : Its relation to the
price pt of the consumption good will be left unspeci…ed, i.e. we will discuss the
most general setup which is possible for the one-asset case. It di¤ers from the budget
constraint in ch. 3.6.3 where there was also only one asset, capital K; but the price of
one unit of capital was equal to the price of the output good and this price was unity
as the output good was the numeraire. The budget constraint with these simplifying
assumptions was relatively simple - as can be seen in (3.33). Here, we will see that a
budget constraint will become pretty complex as soon as the price of the asset is not
normalized. This complexity, however, is needed when it comes e.g. to capital asset
pricing - see further below in ch. 8.4.

25
 Deriving a budget constraint is in principle straightforward. One just de…nes
wealth of the household (taking into account which types of assets the household can
hold for saving purposes and what their prices are), computes the di¤erence between
wealth “today”and “tommorrow”(this is where the di¤erence equation aspect comes
in) and uses an equation which relates current savings to current changes in the
number of assets. In a …nal step, one will naturally …nd out how the interest rate
showing up in budget constraints relates to more fundamental quantities like value
marginal products and depreciation rates.
In our one-asset case, wealth at of a household is given by the number kt of stocks
the household owns (say the number of machines she owns) times the price vt of one
stock (or machine), at = vt kt : Computing the …rst di¤erence yields

at+1 at = vt+1 kt+1 vt kt

= vt+1 (kt+1 kt ) + (vt+1 v t ) kt : (2.39)

Wealth changes depend on the acquisition vt+1 (kt+1 kt ) of new assets and on
changes in the value of assets that are already held, (vt+1 vt ) kt .
Savings are given by capital income (net of depreciation losses) and labour income
minus consumption expenditure, wtK kt vt kt + wt pt ct . Savings in t are used for
buying new assets in t for which the period-t price needs to be paid,

wtK kt vt kt + wt pt ct
= kt+1 kt : (2.40)
vt
We can rewrite this equation slightly, which will simplify the interpretation of subse-
quent results, as
wK kt + wt pt ct
kt+1 = (1 ) kt + t :
vt
Wealth in the next period expressed in number of stocks (and hence not in nominal
terms) is given by wealth which is left over from the current period, (1 ) kt , plus
new acquisitions of stocks which amount to gross capital plus labour income minus
consumption expenditure divided by the price of one stock. This is a di¤erence
equation in kt but not yet a di¤erence equation in wealth at :
Inserting this (2.40) into (2.39) yields
vt+1 K
at+1 at = wt kt vt kt + wt pt ct + (vt+1 vt ) kt
vt
vt+1 wtK vt+1
= at at + wt pt ct + 1 at ,
vt vt vt
vt+1 wK
at+1 = 1+ t at + wt pt ct : (2.41)
vt vt

What does this equation tell us? Each unit of wealth at (say Euro, Dollar, Yen
...) gives 1 units at the end of the period as % is lost due to depreciation
plus “dividend payments” wtK =vt . Wealth is augmented by labour income minus
consumption expenditure. This end-of-period wealth is expressed in wealth at+1 at

26
the beginning of the next period by dividing it through vt (which gives the number
kt of stocks at the end of the period) and multiplying it by the price vt+1 of stocks in
the next period. We have thereby obtained a di¤erence equation in at :
This general budget constraint is fairly complex, however, which implies that in
practice it is often expressed di¤erently. One possibility consists in choosing the
capital good as the numeriare good and setting vt 1 8t: This simpli…es (2.41) to

kt+1 = (1 + rt ) kt + wt pt ct ;

which is very similar to (3.33) from ch. 3.6.3. The simpli…cation in this expression
consists also in the de…nition of the interest rate rt as rt wtK :
An alternative consists in equalizing the price of the capital good to the price of the
consumption good, pt = vt . This is often the case in general equilibrium approaches
- think e.g. of growth models - when there is an aggregate technology which is used
both for producing the consumption and the investment good. The constraint (2.41)
then simpli…es to
wt
art+1 = (1 + rt ) art + ct ;
pt
at wK
where real wealth and the interest rate are de…ned by art pt
and rt t
pt
,
respectively.
If one is not willing to …x any prices or relative prices - which should indeed not
be done if one is interested e.g. in capital asset pricing issues, to be treated later in
ch. 8.4 - one can write the somehow cumbersome budget constraint (2.41) in a more
elegant way. To this end, divide by vt+1 and use the de…nition of at to get the number
of stocks held tommorrow as
wtK wt pt
kt+1 = 1+ kt + ct : (2.42)
vt vt vt
Rearranging such that expenditure is on the left- and disposable income on the right-
hand side yields
pt ct + vt kt+1 = vt kt + wtK vt kt + wt :
This equation has a simpler form than (2.41) and it also lends itself to a simple in-
terpretation: On the left-hand side is total expenditure in period t; consisting of con-
sumption expenditure pt ct plus expenditure for buying the number of capital goods,
kt+1 , the household wants to hold in t + 1: As this expenditure is made in t; total
expenditure for capital goods amounts to vt kt+1 . The right-hand side is total dispos-
able income which splits into income vt kt from selling all capital “inherited”from the
previous period, capital income wtK vt kt and labour income wt : This is the form
budget constraints are often expressed in capital asset pricing models. Note that this
is in principle also a di¤erence equation in kt :

2.6 Exercises chapter 2 and further reading

Rates of substition are discussed in many books on microeconomics; see e.g. Mas-
Colell, Whinston and Green (1995) or Varian (1992). The OLG model goes back to

27
Samuelson. For presentations in textbooks, see e.g. Blanchard and Fischer (1989),
Azariadis (1993) or de la Croix and Michel (2002). The presentation of the Lagrangian
is inspired by Intriligator (1971). Treatments of shadow prices are available in many
other textbooks (Dixit, 1989, ch. 4; Intiriligator, 1971, ch. 3.3). More extensive
treatments of di¤erence equations and the implicit function theorem can be found in
many introductory “mathematics for economists”books.

28
 Exercises chapter 2
Applied Intertemporal Optimization
Optimal consumption in two-period discrete time models

1. Optimal choice of household consumption

Consider the following maximization problem,
1
max Ut = v (ct ) + v (ct+1 ) (2.43)
ct ;ct+1 1+
subject to
1 1
wt + (1 + r) wt+1 = ct + (1 + r) ct+1 :

(a) What is the optimal consumption path?

(b) Under what conditions does consumption rise?

2. Capital market restrictions

Now consider the following budget constraint. This is a budget constraint that
would be appropriate if you want to study the education decisions of households.
The parameter b amounts to schooling costs. Inheritance of this individual
under consideration is n.

Ut = ln ct + (1 ) ln ct+1

subject to
1 1
b + n + (1 + r) wt+1 = ct + (1 + r) ct+1 :

(a) What is the optimal consumption pro…le under no capital market restric-
tions?
(b) Assume loans for …nancing education are not available, hence savings need
to be positive, st 0. What is the consumption pro…le in this case?

3. Optimal investment
Consider a monopolist investing in its technology. Technology is captured by
marginal costs ct . The chief accountant of the …rm has provided the manger of
the …rm with the following information,

= 1 + R 2; t = p (xt ) xt ct xt It ; ct+1 = ct f (I1 ) :

Assume you are the manager. What is the optimal investment sequence I1 , I2 ?

29
4. A particular utility function
Consider the utility function U = ct + ct+1 ; where 0 < < 1: Maximize U
subject to an arbitrary budget constraint of your choice. Derive consumption
in the …rst and second period. What is strange about this utility function?
5. Intertemporal elasiticity of substitution
Consider the utility function U = ct1 + ct+1
1
:

(a) What is the intertemporal elasticity of substitution?

(b) How can the de…nition in (2.10) of the elasticity of substitution be trans-
formed into the maybe better known de…nition
d ln (ci =cj ) pi =pj d (ci =cj )
ij = = ?
d ln uci =ucj ci =cj d (pi =pj )

6. General equilibrium
Consider the Diamond model for a Cobb-Douglas Production function of the
form Y = K L1 and a logarithmic utility function.

(a) Derive the di¤erence equation for Kt :

(b) Draw a phase diagram.
(c) What are the steady state consumption level and capital stock?

7. Sums

(a) Proof the statement of the second lemma in ch. 2.5.1,

T i 1 1 aT
i=1 ia = a T aT +1 :
1 a 1 a
The idea is identical to the …rst proof in ch. 2.5.1.
(b) Show that (exam)
k 1 k 1 s s ck4 k
s=0 c4 = :
c4
Both parameters obey 0 < c4 < 1 and 0 < v < 1: Hint: Rewrite the sum
as ck4 1 ks=01 ( =c4 )s and observe that the …rst lemma in ch. 2.5.1 holds for
a which are larger or smaller than 1.

8. Di¤erence equations
Consider the following linear di¤erence equation system

yt+1 = a yt + b; a < 0 < b;

(a) What is the …xpoint of this equation?

(b) Is this point stable?
(c) Draw a phase diagram.

30
3 In…nite horizon models
This chapter looks at decision processes where the time horizon is longer than two
periods. In most cases, the planning horizon will be in…nity. In such a context,
Bellman’s optimality principle is very useful. Is it, however, not the only way how to
solve maximization problems with in…nite time horizon. For comparison purposes, we
therefore start with the Lagrange approach, as in the last section. Bellman’s principle
will be introduced afterwards when intuition for the problem and relationships will
have been increased.

3.1 The problem

The objective function is given by the utility function of an individual,
1 t
Ut = =t u (c ) ; (3.1)
where again
1
(1 + ) ; >0 (3.2)
is the discount factor and is the positive time preference rate. We know this utility
function already from the de…nition of the time preference rate, see (2.11). The
utility function is to be maximized subject to a budget constraint. The di¤erence to
the formulation in the last section is that consumption does not have to be determined
for two periods only but for in…nitely many. Hence, the individual does not choose
one or two consumption levels but an entire path of consumption. This path will be
denoted by fc g ; where 2 [t; 1[ : Note that the utility function is a generalization of
the one used above in (2.1), but is assumed to be time separable. The corresponding
two period utility function was used in exercise set 1, cf. equation (2.43).
The budget constraint can be expressed in the intertemporal version by
1 ( t) 1 ( t)
=t (1 + r) e = at + =t (1 + r) w ; (3.3)
where e = p c : It states that the present value of expenditure equals current wealth
at plus the present value of labour income w . Labour income w and the interest rate
r are exogenously given to the household, her wealth level at is given by history. The
only quantity that is left to be determined is therefore the path fc g : Maximizing
(3.1) subject to (3.3) is a standard Lagrange problem.

3.2 Solving by Lagrange

The Lagrangian reads
h i
1 t 1 ( t) 1 ( t)
L= =t u (c ) + =t (1 + r) e at =t (1 + r) w ;
where is the Lagrange mulitplier. It is constant as before. First order conditions
are
t 0 ( t)
Lc = u (c ) + [1 + r] p = 0; t < 1; (3.4)
L = 0;

31
where the latter is, as in the OLG case, the budget constraint. Again, we have as
many conditions as variables to be determined.
Do these …rst-order conditions tell us something? Take the …rst-order condition
for period and for period + 1. They read
t 0 ( t)
u (c ) = [1 + r] p ;
+1 t 0 ( +1 t)
u (c +1 ) = [1 + r] p +1 :

Dividing them gives

1 u0 (c ) p u0 (c ) p
0
= (1 + r) () 0
= 1 : (3.5)
u (c +1 ) p +1 u (c +1 ) (1 + r) p +1

Rearranging allows an intuitive interpretation: Comparing the instantaneous gain in

utility u0 (c ) with the future gain, discounted at the time preference rate, u0 (c +1 ) ;
must yield the same ratio as the price p that has to be paid today relative to the price
that has to be paid in the future, also appropriately discounted to its present value
price (1 + r) 1 p +1 : This interpretation is identical to the two-period interpretation
in (2.9) in ch. 2.2.2. If we normalize prices to unity, (3.5) is just the expression we
obtained in the solution of the OLG example (2.43).

3.3 The envelope theorem

We saw how the Lagrangian can be used to solve optimization problems with many
time periods. In order to understand how dynamic programming works, it is use-
ful to understand a theorem which is frequently used when employing the dynamic
programming method: the envelope theorem.

3.3.1 The theorem

In general, the envelope theorem says

Theorem 1 Let there be a function g (x; y) : Choose x such that g (:) is maximized
for a given y: (Assume g (:) is such that a smooth interior solution exists.) Let f (y)
be the resulting function of y;

f (y) max g (x; y) :

x

Then the derivative of f with respect to y equals the partial derivative of g with respect
to y, if g is evaluated at that x = x(y) that maximizes g;
d f (y) @ g (x; y)
= :
dy @y x=x(y)

@
Proof. f (y) is constructed by @x g (x; y) = 0: This implies a certain x = x (y) ;
provided that second order conditions hold. Hence, f (y) = maxx g (x; y) = g (x (y) ; y) :Then,
d f (y)
dy
= @ g(x(y);y)
@x
d x(y)
dy
+ @ g(x(y);y)
@y
: The …rst term of the …rst term is zero.

32
3.3.2 Illustration
The plane depicts the function g (x; y). The maximum of this function with respect
to x is shown as maxx g (x; y), which is f (y). Given this …gure, it is obvious that the
derivative of f (y) with respect to y is the same as the partial derivative of g (:) with
respect to y at the point where g (:) has its maximum with respect to x: The partial
@g
derivative @y is the derivative in the direction of y (“Richtungsableitung”). Choosing
the highest point of g (:) with respect to x, this “Richtungsableitung” must be the
same as dfdy(y) at the back of the …gure.

f (y )

max g(x , y )
x

g(x , y )

df (y ) ∂g(x(y ), y ) dx ∂g(x(y ), y )
= +
dy ∂x dy ∂y
∂g(x(y ), y )
=
∂y y

Figure 10 Illustrating the envelope theorem5

3.3.3 An example
There is a central planner of an economy. The social welfare function is given by
U (A; B), where A and B are consumption goods. The technologies available for
producing these goods are A = A (cLA ) and B = B (LB ) and the economy’s resource
constraint is LA + LB = L:
5
This …gure is contained in the …le envelope.ds4. It was originally created in Mathematica, using
the command Plot3D[-(x-.5*y)^ 2-(y-2)^ 2,fx,0,2g,fy,0,4g, fAxes -> noaxes, PlotRange ->f-8,5g,
Boxed -> Falseg]

33
 The planner is interested in maximizing the social welfare level and allocates
labour according to maxLA U (A (cLA ) ; B (L LA )) : The optimality condition is

@U 0 @U 0
Ac B = 0: (3.6)
@A @B
This makes LA a function of c,
LA = LA (c) : (3.7)
The central planner now asks what happens to the social welfare level when the
technology parameter c increases and she still maximizes the social welfare function,
i.e. (3.6) continues to hold. Without using the envelope theorem, the answer is

d
U (A (cLA (c)) ; B (L LA (c)))
dc
0 0 0
= UA (:) A [LA (c) + cL0A (c)] + UB (:) B [ L0A (c)] = UA (:) A LA (c) > 0

where the last equality follows from inserting the optimality condition (3.6). Eco-
nomically, this result means that the e¤ect of a better technology on overall welfare
is given by the direct e¤ect in sector A: The indirect e¤ect through reallocation of
labour vanishes as due to the …rst-order condition (3.6) the marginal contribution of
each worker is identical across sectors. Clearly, this only holds for a small change in
c:
If one is interested in …nding an answer by using the envelope theorem, one would
start by de…ning a function V (c) maxLA U (A (cLA ) ; B (L LA )) : Then, according
to the envelope theorem,

d @ 0 0
V (c) = U (A (cLA ) ; B (L LA )) = UA (:) A LA = UA (:) A LA (c) > 0:
dc @c LA =LA (c) LA =LA (c)

Apperently, both approaches yield the same answer. Applying the envelope theorem
gives the answer faster.

3.4 Solving by dynamic programming

We will not get to know how dynamic programming works. Let us study a slightly
more general maximization problem than the one in ch. 3.1. We use the same utility
function as in (3.1).6 The constraint, however, will be represented in a more general
way than in (3.3). We stipulate that wealth of the household evolves according to

xt+1 = f (xt ; ct ) : (3.8)

We denote wealth by xt to make clear that this constraint can represent also other
aspects than a budget constraint of a household. In fact, this di¤erence equation
6
All intertemporal utility functions in this book will use exponential discounting. This is clearly
a special case. Models with non-exponential or hyperbolic discounting imply fundamentally di¤erent
dynamic behaviour and time inconsistencies.

34
can be non-linear (e.g. in central planner problem where the constraint is a resource
constraint like (3.48)) or linear (e.g. a budget constraint like (3.14)). We treat here
this general case …rst before we go to more speci…c examples further below.
The consumption level ct and - more generally speaking - other variables whose
value is directly chosen by individuals, e.g. investment levels or shares of wealth
held in di¤erent assets, are called control variables. Variables which are not under
the direct control of individuals are called state variables. In many maximization
problems, state variables depend indirectly on the behaviour of individuals as in
(3.8). State variables can also be completely exogenous like e.g. the TFP level in an
exogenous growth model.
Optimal behaviour is de…ned by maxfc g Ut subject to (3.8), where the value of
this optimal behaviour or optimal program is denoted by

V (xt ) max Ut subject to xt+1 = f (xt; ct ) : (3.9)

fc g

V (xt ) is called the value function. It is a function of the state variable xt and not
of the control variable ct . The latter point is easy to understand if one takes into
account that the control variable ct is, when behaving optimally, a function of the
state variable xt :
The value function V (:) could also be a function of time t (e.g. in problems with
…nite horizon) but we will not discuss this further as it is of no importance in the
problems we will encounter. Generally speaking, xt and ct could be vectors and time
could then be part of the state vector xt : The value function is always a function of
the states of the system or the maximization problem.
Given this description of the maximization problem, solving by dynamic program-
ming essentially requires to go through three steps. This three-step approach will be
followed here, in continuous time later, and also in models with uncertainty.

DP1: Bellman equation and …rst-order conditions

The …rst step establishes the Bellman equation and computes …rst-order condi-
tions. The objective function Ut in (3.1) is additively separable which means that it
can be written in the form
Ut = u(ct ) + Ut+1 : (3.10)
Bellman’s idea consists in rewriting the maximization problem in the optimal program
(3.9) as
V (xt ) max fu(ct ) + V (xt+1 )g (3.11)
ct

subject to
xt+1 = f (xt ; ct ) :
Equation (3.11) is known as Bellman equation. In this equation, the problem
with potentially in…nitely many control variables fc g was broken down in many
problems with one control variable ct . Note that there are two steps involved: First,
the additive separability of the objective function is used. Second, more importantly,

35
Ut is replaced by V (xt+1 ). This says that we assume that the optimal problem for
tomorrow is solved and we worry about the maximization problem of today only.
We can now compute the …rst-order condition which is of the form
@f (xt ; ct )
u0 (ct ) + V 0 (xt+1 ) = 0: (3.12)
@ct
In principle, this is the solution of our maximization problem. Our control variable
ct is by this expression implicitly given as a function of the state variable, ct = c (xt ) ;
as xt+1 by the constraint (3.8) is a function of xt and ct . Hence, as V is well-de…ned
above, we have obtained a solution.
As we know very little about the properties of V at this stage, however, we need
to go through two further steps in order to eliminate V from this …rst-order condition
and obtain a condition that uses only functions (like e.g. the utility function or the
technology for production in later examples) of which properties like signs of …rst and
second derivatives are known.

DP2: Evolution of the costate variable

The second step of the dynamic programming approach starts from the “maxi-
mized Bellman equation”. The maximized Bellman equation is obtained by replacing
the control variables in the Bellman equation, i.e. the ct in (3.11) in the present case,
by the optimal control variables as given by the …rst-order condition, i.e. by c (xt )
resulting from (3.12). Logically, the max operator disappears (as we insert the c (xt )
which imply a maximimum) and the maximized Bellman equation reads

V (xt ) = u (c (xt )) + V (f (xt ; c(xt ))) :

The derivative with respect to xt reads

dc (xt ) dc (xt )
V 0 (xt ) = u0 (c (xt )) + V 0 (f (xt ; c (xt ))) fxt + fc :
dxt dxt
This step shows why it matters that we use the maximized Bellman equation here:
Now control variables are a function of state variable and we need to compute the
derivative of ct with respect to xt when computing the derivative of the value function
V (xt ) : Inserting the …rst-order condition gives

V 0 (xt ) = V 0 (f (xt ; ct (xt ))) fxt = V 0 (xt+1 ) fxt (3.13)

This equation is a di¤erence equation for the costate variable V 0 (xt ). This derivative
of the value function is the shadow price of the state variable xt : It says how much
an additional unit of the state variable (say e.g. of wealth) is valued: As V (xt ) gives
the value of optimal behaviour between t and the end of the planning horizon, V 0 (xt )
says by how much this value changes when xt is changed marginally. Hence, equation
(3.13) describes how the shadow price of the state variable changes over time when
the agent behaves optimally. If we had used the envelope theorem, we would have
immediately ended up with (3.13) without having to insert the …rst-order condition.

36
 DP3: Inserting …rst-order conditions

Now insert the …rst-order condition (3.12) twice into (3.13) to replace the unknown
value function and to …nd an optimality condition depending on u only. This will
then be the Euler equation. We do not do this here explicitly as many examples will
go through this step in detail in what follows.

3.5 Examples
3.5.1 Individual utility maximization
The individual’s budget constraint is given in the dynamic formulation (in contrast
to an intertemporal version (3.3)) by

at+1 = (1 + rt ) (at + wt ct ) : (3.14)

Note that this dynamic formulation corresponds to the intertemporal version in the
sense that (3.3) implies (3.14) and (3.14) with some limit condition implies (3.3). The
…rst part of this statement can easily be seen: Write (3.3) for the next period as
1 ( t 1) 1 ( t 1)
=t+1 (1 + r) e = at+1 + =t+1 (1 + r) w : (3.15)

Express (3.3) as
1 ( t) 1 ( t)
et + =t+1 (1 + r) e = at + wt + =t+1 (1 + r) w ,
1 1 ( t 1) 1 1 ( t 1)
et + (1 + r) =t+1 (1 + r) e = at + wt + (1 + r) =t+1 (1 + r) w ,
1 ( t 1) 1 ( t 1)
=t+1 (1 + r) e = (1 + r) (at + wt et ) + =t+1 (1 + r) w :

Insert (3.15) and …nd (3.14). The second part is part of the exercises.
The budget constraint (3.14) can be found e.g. in Blanchard and Fischer (1989, p.
280). The timing as implicit in (3.14) is illustrated in the following …gure. All events
take place at the beginning of the period. Our individual owns a certain amount
of wealth at at teh beginning of t and receives here wage income wt and spends ct
on consumption also at the beginning. Hence, savings st can be used during t for
production and interest are paid on st which in turn gives at+1 at the beginning of
period t + 1.

st = at + wt − ct at +1 = (1 + rt )st
ct ct +1

t
t +1

Figure 11 The timing in an in…nite horizon discrete time model

37
 The consistency of (3.14) with technologies in general equilibrium is not self-
evident. We will encounter more conventional budget constraints of the type (??)
further below. For understanding how to apply dynamic programming methods,
however, this budget constraint is perfectly …ne.
The objective of the individual is to maximize her utility function (3.1) subject
to the budget constraint by choosing a path of consumption levels ct ; denoted by
fc g ; 2 [t; 1] : The value of the optimal program fc g is, given her initial endow-
ment with at ,
V (at ) = max Ut (3.16)
fc g

It is called value function.

DP1: Bellman equation and …rst-order conditions
We know that the utility function can be written as Ut = u (ct ) + Ut+1 : Now
assume that the individual behaves optimally as from t + 1: Then we can insert the
value function. The utility function reads Ut = u (ct ) + V (at+1 ) : Inserting this in
the value function, we obtain the recursive formulation
V (at ) = max fu (ct ) + V (at+1 )g ; (3.17)
ct

known as the Bellman equation.

Again, this breaks down a many-period problem into a two-period problem: The
objective of the individual was maxfc g (3.1) subject to (3.14), as shown by the value
function in equation (3.16). The Bellman equation (3.17), however, is a two period
decision problem, the trade-o¤ between consumption today and more wealth tomor-
row (under the assumption that the function V is known). This is what is known as
Bellman’s principle of optimality: Whatever the decision today, subsequent decisions
should be made optimally, given the situation tomorrow. History does not count,
apart from its impact on the state variable(s).
We now derive a …rst-order condition for (3.17). It reads
d d d at+1
u (ct ) + V (at+1 ) = u0 (ct ) + V 0 (at+1 ) = 0:
dct dct dct
Since dat+1 =dct = (1 + rt ) by the budget constraint (3.14), this gives
u0 (ct ) (1 + rt ) V 0 (at+1 ) = 0: (3.18)
Again, this equation makes consumption a function of the state variable, ct = ct (at ) :
Following the …rst-order condition (3.12) in the general example, we wrote ct = c (xt ) ;
i.e. consumption ct changes only when the state variable xt changes. Here, we write
ct = ct (at ) ; indicating that there can be other variables which can in‡uence consump-
tion other than wealth at : An example for such an additional variable in our setup
would be the wage rate wt or interest rate rt showing up in (3.14). If we had de…ned
the value function as having arguments at ; wt and rt ; i.e. V = V (at ; wt ; rt ) ; we would
have obtained a …rst order condition which would allow us to write something like
ct = c (at ; wt ; rt ; :::) : As this is much more cumbersome from a notational point of
view, the shortcut ct = ct (at ) is often preferred in this book.

38
 DP2: Evolution of the costate variable

The derivative of the maximized Bellman equation reads, using the envelope the-
orem,
@at+1
V 0 (at ) = V 0 (at+1 ) : (3.19)
@at
We compute the partial derivative of at+1 with respect to at as the functional rela-
tionship of ct = ct (at ) should not (because of the envelope theorem) be taken into
account.
From the budget constraint we know that @a@at+1 t
= 1 + rt : Hence, the evolution of
the shadow price under optimal behaviour is described by

V 0 (at ) = (1 + rt ) V 0 (at+1 ) :

DP3: Inserting …rst-order conditions

We can insert the …rst-order condition (3.18) on the RHS. We can also rewrite the
…rst-order condition (3.18), by lagging it one period, as (1 + rt 1 ) V 0 (at ) = u0 (ct 1 )
and can insert this on the LHS. This gives
1 1
u0 (ct 1 ) (1 + rt 1 ) = u0 (ct ) , u0 (ct ) = (1 + rt ) u0 (ct+1 ) : (3.20)

This is the same result as the one we have obtained when we used the Lagrange
method in equation (3.5).
It is also the same result as for the 2-period saving problem which we found in
OLG models - see e.g. (2.9) or (2.43) in the exercises. This might be suprising as
the planning horizons di¤er considerably between a 2- and an in…nite-period decision
problem. Apperently, whether we plan for two periods or many more, the change
between two periods is always the same when we behave optimally. It should be kept
in mind, however, that consumption levels (and not changes) do depend on the length
of the planning horizon.

3.5.2 Optimal R&D e¤ort

Let us turn to a second example, originally inspired by Grossman and Shapiro (1986).
At some known point in time T , a research project has to be …nished. This research
project has a certain value at point T and we denote it by D like dissertation. In
order to reach this goal, a path of a certain length L needs to be gone through (“das
Ziel ist das Ziel”), think of L as a certain number of pages, a certain number of papers
or - probably better - a certain quality of a …xed number of papers. Going through
this process of walking and writing is costly, it causes e¤ort e at each point in time
t: Summing over these cost - think of them as hours worked per day - yields
eventually the desired amount of pages,
T
=t f (e ) = L; (3.21)

39
where f (:) is the page of quality production function: More e¤ort means more output,
f 0 (:) > 0; but any increase in e¤ort implies a lower increase in output, f 00 (:) < 0:
The objective function of our student is given by
T t T t
Ut = D =t e : (3.22)

The value of the completed dissertation is given by D and its present value is obtained
by discounting at the discount factor : Total utility Ut stems from this present value
minus the present value of research cost e . The maximization problem consists in
maximizing (3.22) subject to the constraint (3.21) by choosing an e¤ort path fe g :
The question now arises how these costs are optimally spread over time. How many
hours should be worked per day?
An answer can be found by using the Lagrange-approach with (3.22) as the objec-
tive function and (3.21) as the constraint. We will use here the dynamic programming
approach, however. Before we can apply it, we need to derive a dynamic budget con-
straint. We therefore de…ne
t 1
Mt =1 f (e )

as the amount of the pages that have already been written by today. This then implies

Mt+1 Mt = f (et ): (3.23)

The increase in the number of completed pages between today and tommorrow de-
pends on e¤ort-induced output f (e ) today. We can now apply the three dynamic
programming steps.

DP1: Bellman equation and …rst-order conditions

The value function is given by V (Mt ) = maxfe g Ut subject to the constraint. The
objective function written recursively reads
T (t+1) 1 T t T (t+1) 1 T t
Ut = D =t e = D et + =t+1 e
T (t+1) T (t+1)
= D =t+1 e et = Ut+1 et :

Assuming that the individual behaves optimally as from tomorrow, this reads Ut =
et + V (Mt+1 ) and the Bellman equation reads

V (Mt ) = max f et + V (Mt+1 )g : (3.24)

et

The …rst-order conditions is 1 + V 0 (Mt+1 ) dMdet+1

t
= 0; which, using the dynamic
budget constraint, becomes

1 = V 0 (Mt+1 )f 0 (et ): (3.25)

Again as in (3.12), implicitly and with (3.23), this equation de…nes a functional
relationship between the control variable and the state variable, et = e (Mt ) :

40
 DP2: Evolution of the costate variable

To provide some variation, we will now go through the second step of dynamic
programming without using the envelope theorem. Consider the maximized Bellman
equation, where we insert et = e (Mt ) and (3.23) into Bellman equation (3.24),

V (Mt ) = e (Mt ) + V (Mt + f (e (Mt ))) :

The derivative with respect to Mt is

d [Mt + f (e (Mt ))]

V 0 (Mt ) = e0 (Mt ) + V 0 (Mt + f (e (Mt )))
dMt
0 0 0 0
= e (Mt ) + V (Mt+1 ) [1 + f (e (Mt )) e (Mt )] :

Using the …rst-order condition (3.25) simpli…es this derivative to V 0 (Mt ) = V 0 (Mt+1 ) :
Expressed for t + 1 gives
V 0 (Mt+1 ) = V 0 (Mt+2 ) (3.26)

DP3: Inserting …rst-order conditions

The …nal step inserts the …rst-order condition (3.25) twice to replace V 0 (Mt+1 )
and V 0 (Mt+2 ) ;
1 f 0 (et+1 )
(f 0 (et )) 1 = (f 0 (et+1 )) 1 , 0 = : (3.27)
f (et )
The interpretation is now simple. As f 00 (:) < 0 and < 1; e¤ort et is increasing under
optimal behaviour, i.e. et+1 > et : Optimal writing of a dissertation implies more work
every day.

What about levels?

The optimality condition in (3.27) speci…es only how e¤ort e changes over time,
it does not provide information on the level of e¤ort required every day. This is
a property of all solutions of intertemporal problems. They only give information
about changes of levels, not about levels themselves. The basic idea how to obtain
information about levels can be easily illustrated, however.
Assume f (et ) = et ; with 0 < < 1: Then (3.27) implies (with t being replaced
by ) e +11 =e 1 = , e +1 = 1=(1 ) e : Solving this di¤erence equation yields
( 1)=(1 )
e = e1 ; (3.28)

where e1 is the (at this stage still) unknown initial e¤ort level. Inserting this solution
into the constraint (3.21) yields, starting in = 1 on the …rst day,

T ( 1)=(1 ) T ( 1) =(1 )
=1 f e1 = =1 e1 = L:

41
This gives us the initial e¤ort level as (the sum can be solved by using the proofs in
ch. 2.5.1)
1=
L
e1 = T ( 1) =(1 )
:
=1
With (3.28), we have now also computed the level of e¤ort every day.
Behind these simple steps, there is a general principle. Solutions to intertemporal
problems are di¤erence or di¤erential equations. Any di¤erence (or also di¤erential)
equation when solved gives a unique solution only if an initial or boundary condition is
provided. Here, we have solved the di¤erence equation in (3.27) assuming some initial
condition e1 : The meaningful initial condition then followed from the constraint (3.21).
Hence, in addition to the optimality rule (3.27), we always need some additional
constraint which allows to compute the level of optimal behaviour. We return to this
point when looking at problems in continuous time in ch. 5.3.

3.6 A general equilibrium analysis

We have so far analyzed maximization problems of households in partial equilibrium.
In two-period models, we have analyzed how households can be aggregated and what
we learn about the evolution of the economy as a whole. We will now do the same
for in…nite horizon problems.
We will, however, not simply aggregate over households and then put things to-
gether as we did in ch. 2.4, but we will rather present a model as it should be pre-
sented: First, technologies are speci…ed. This shows what is technologically feasible
in this economy. Which goods are produced, how can be saved, is there uncertainty
in the economy? Second, preferences of the household are presented and the budget
constraint of households is derived from the technologies presented before. This is the
reason why technologies should be presented before households: budget constraints
are endogenous and depend on knowledge what households can do.

3.6.1 Technologies
The technology is a simple Cobb-Douglas technology
Yt = At Kt Lt1 : (3.29)
This good can be used for consumption and investment and equilibrium on the goods
market requires
Yt = Ct + It :
Gross investment It is turned into net investment by taking depreciation into account,
Kt+1 = (1 ) Kt + It :
Taking these two equations together give the resource constraint of the economy,
Kt+1 = (1 ) Kt + Y t Ct : (3.30)
As this constraint is simply a consequence of technologies and market clearing, it is
identical to the one used in the OLG setup in (2.26).

42
3.6.2 Firms
Firms maximize pro…ts by employing optimal quantities of labour and capital, given
the technology in (3.29). First order conditions are

@Yt @Yt
= wtK ; = wtL (3.31)
@Kt @Lt
as in (2.19), where we have again chosen the consumption good as numeraire.

3.6.3 Households
Preferences of households are described as in the intertemporal utility function (3.1).
As the only way households can transfer savings from one period to the next is by
buying investment goods, an individual’s wealth is given by the “number of machines”
kt , she owns. Clearly, adding up over all individual wealth stocks gives the total
capital stock, L kt = Kt : Wealth kt increases over time if the household spends less
on consumption than what she earns through capital plus labour income, corrected
for the loss in wealth each period caused by depreciation,

kt+1 kt = wtK kt kt + wtL ct ,

kt+1 = 1 + wtK kt + wtL ct :

If we now de…ne the interest rate to be given by

rt wtK ; (3.32)

we obtain our budget constraint

kt+1 = (1 + rt ) kt + wtL ct : (3.33)

Note that the “derivation” of this budget constraint was simpli…ed as compared
to ch. 2.5.5 as the price vt of an asset is, as we measure it in units of the consumption
good, by 1.
Given the preferences and the constraint, the solution to this maximization prob-
lem is given by (see exercise 5)

u0 (ct ) = (1 + rt+1 ) u0 (ct+1 ) : (3.34)

Structurally, this is the same expression as in (3.20). The interest rate, however, refers
to t+1, due to the change in the budget constraint. Remembering that = 1= (1 + ),
this shows that consumption increases as long as rt+1 > .

3.6.4 Aggregation and reduced form

Aggregation

43
 To see that individual constraints add up to the aggregate resource constraint,
we simply need to take into account as in (2.21) that individual income adds up to
output Yt ; wtK Kt + wtL Lt = Yt :
The optimal behaviour of all household taken together can be gained from (3.34)
by summing over all households. This is done analytically correctly by …rst applying
the inverse function of u0 to this equation and then summing individual consumption
levels over all households. Applying the inverse function again gives
u0 (Ct ) = (1 + rt+1 ) u0 (Ct+1 ) ; (3.35)
where Ct is aggregate consumption in t:
Reduced form
We now need to understand how our economy evolves in general equilibrium. Our
…rst equation is (3.35), telling us how consumption evolves over time. This equation
contains consumption and the interest rate as endogenous variables.
Our second equation is therefore the de…nition of the interest rate in (3.32) which
we combine with the …rst-order condition of the …rm in (3.31) to yield
@Yt
rt = : (3.36)
@Kt
This equation contains the interest rate and the capital stock as endogenous variable.
Our …nal equation is the resource constraint (3.30), which provides a link between
capital and consumption. Hence, (3.35), (3.36) and (3.30) gives a system in three
equations and three unknowns. When we insert the interest rate into the optimality
condition for consumption , we obtain as our reduced form
@Yt+1
u0 (Ct ) = 1 + @K u0 (Ct+1 ) ;
t+1 (3.37)
Kt+1 = (1 ) Kt + Y t Ct :
This is a two-dimensional system of non-linear di¤erence equations which gives a
unique solution for the time path of capital and consumption, provided we have two
initial conditions K0 and C0 :

3.6.5 Steady state and transitional dynamics

Steady state
When trying to understand a system like (3.37), the easiest is to understand the
steady state. In a steady state, all variables are constant and we obtain
@Y @Y
1= 1+ , = + ; C=Y K;
@K @K
where we used the de…nition of in (3.2). In the steady state, the marginal produc-
tivity of capital is given by the time preference rate plus depreciation rate. Consump-
tion equals output minus depreciation, i.e. minus replacement investment. These two
equations determine two variables K and C.

44
 Transitional dynamics

Understanding transitional dynamics is not as straightforward as understanding

the steady state. Its analysis follows the same idea as in continuous time, however,
and we will analyze transitional dynamics in detail there.

3.6.6 The intertemporal budget constraint

Consider the budget constraint derived in (??) as just used in (3.33),

at+1 = (1 + rt ) at + wt et :

What is the intertemporal version of this budget constraint?

To …nd this out, we now solve it recursively. Rewrite it as
at+1 + et wt at+i+1 + et+i wt+i
at = ; at+i = :
1 + rt 1 + rt+i
Inserting su¢ ciently often yields
at+2 +et+1 wt+1
1+rt+1
+ et wt at+2 + et+1 wt+1 et wt
at = = +
1 + rt (1 + rt+1 ) (1 + rt ) 1 + rt
at+3 +et+2 wt+2
1+rt+2
+ et+1 wt+1
et wt
= +
(1 + rt+1 ) (1 + rt ) 1 + rt
at+3 + et+2 wt+2 et+1 wt+1 et wt
= + +
(1 + rt+2 ) (1 + rt+1 ) (1 + rt ) (1 + rt+1 ) (1 + rt ) 1 + rt
at+i et+i wt+i
= ::: = lim + 1
i=0 :
i!1 (1 + rt+i 1 ) (1 + rt+1 ) (1 + rt ) (1 + rt+i ) (1 + rt+1 ) (1 + rt )
The expression in the last line is hopefully instructive but not very stringent. We can
write it in a more stringent way as
at+i 1 et+i wt+i
at = lim i 1 + i=0 i
i!1
s=0 (1 + rt+i ) s=0 (1 + rt+s )

where indicates a product, i.e. is=0 (1 + rt+i ) = 1+rt for i = 0 and is=0 (1 + rt+i ) =
(1 + rt+i ) (1 + rt ) for i > 0: For i = 1; is=0 (1 + rt+i ) = 1 by de…nition.
Letting the limit be zero, we obtain
1 et+i wt+i 1 e w 1 e w
at = i=0 i
= =t t =t
s=0 (1 + rt+s ) s=0 (1 + rt+s ) R
where the last but one equality substituted t + i by : We can write this as
1 e 1 w
=t = at + =t : (3.38)
R R
With a constant interest rate, this reads
1 ( t+1) 1 ( t+1)
=t (1 + r) e = at + =t (1 + r) w :

45
Equation (3.38) is the intertemporal budget constraint that results from a dynamic
budget constraint as speci…ed in (??). Comparing it with (3.3) shows that the present
values on both sides discount one time more than in (3.3). The economic di¤erence
lies in whether we look at values at the beginning or the end of a period.
The budget constraint (??) and (3.33) are the “natural”budget constraint in the
sense that it can be derived easily as above in (??) and in the sense that it easily
aggregates to economy wide resource constraints. We will therefore work with them in
what follows. The reason for not working with them right from the beginning is that
the intertemporal version (3.3) has some intuitive appeal. Showing its shortcomings
explicitly should make clearer why - despite its intuitive appeal - it is not so useful
in general equilibrium setups.

3.7 A central planner

3.7.1 Optimal factor allocation
One of the most often solved maximization problems in economics is the central
planner solution. The choice of a central planner given a social welfare function and
technological constraints provides information about the …rst-best factor allocation.
This is a benchmark for many analyses in normative economics. We consider the
probably simplest case of optimal factor allocation in a dynamic setup.

The maximization problem

Let preferences be given by

1 t
Ut = =t u (C ) ; (3.39)

where C is aggregate consumption of all households at a point in time : This function

is maximized subject to a resource accumulation constraint which reads

K +1 = Y (K ; L ) + (1 )K C : (3.40)

The production technology is given by a neoclassical production function Y (K ; L )

with standard properties.

The Lagrangian

This setup di¤ers from the ones one got to know before in that there is an in…nity
of constraints in (3.40). This constraint holds for each point in time t: As a
consequence, the Lagrangian is formulated with in…nitly many Lagrangian multipliers,
1 t 1
L= =t u (C ) + =t f [K +1 Y (K ; L ) (1 ) K + C ]g : (3.41)
t
The …rst part of the Lagrangian is standard, 1=t u (C ), it just copies the ob-
jective function. The second part consists in a sum from t to in…nity, one constraint

46
for each point in time, each multiplied by its own Lagrange multiplier : In order to
make the maximization procedure clearer, we rewrite the Lagrangian as
1 t
L= =t u (C ) + s =t2 [K +1 Y (K ; L ) (1 )K + C ]
+ s 1 [Ks Y (Ks 1 ; Ls 1 ) (1 ) Ks 1 + Cs 1 ]
+ s [Ks+1 Y (Ks ; Ls ) (1 ) Ks + Cs ]
1
+ =s+1 [K +1 Y (K ; L ) (1 )K + C ];

where we simply explicitly wrote out the sum for s 1 and s:

Now maximize the Lagrangian both with respect to the control variable C ; the
multipliers and the state variables K . Maximization with respect to K might
appear unusual at this stage; we will see a justi…cation for this in the next chapter.
First order conditions are
s t 0
LCs = u (Cs ) + s = 0 () s = u0 (Cs ) s t ; (3.42)
@Y s 1 @Y
LKs = s 1 s +1 = 0 () =1+ ; (3.43)
@Ks s @Ks
L s = 0: (3.44)
s 1 t
u0 (Cs 1 ) @Y
Combining the …rst and second …rst-order condition gives u0 (Cs ) s t = 1+ @K s
:
This is equivalent to
u0 (Cs ) 1
0
= 1 : (3.45)
u (Cs+1 ) 1+ @Y @Ks+1

This expression has the same interpretation as e.g. (3.5) or (3.20). When we replace s
by ; this equation with the constraint (3.40) is a two-dimensional di¤erence equation
system which allows to determine the paths of capital and consumption, given two
boundary conditions, which the economy will follow when factor allocation is opti-
mally chosen. The steady state of such an economy is found by setting Cs = Cs+1
and Ks = Ks+1 in (3.45) and (3.40).

3.7.2 Where the Lagrangian comes from II

Let us now see how we can derive the same expression as in (3.45) without using the
Lagrangian. This will allow us to give an intuitive explanation for why we maximized
the Lagrangian in the last chapter with respect to both the control and the state
variable.

Maximization without Lagrange

Insert the constraint (3.40) into the objective function (3.39) and …nd
1 t
Ut = =t u (Y (K ; L ) + (1 )K K +1 ) ! max
fK g

This is now maximized by choosing a path fK g for capital. Choosing the state
variable implicitly pins down the path fC g of the control variable consumption and

47
one can therefore think of this maximization problem as one where consumption is
optimally chosen.
Now rewrite the objective function as
s 2 t
Ut = =t u (Y (K ; L ) + (1 )K K +1 )
s 1 t
+ u (Y (Ks 1 ; Ls 1 ) + (1 ) Ks 1 Ks )
s t
+ u (Y (Ks ; Ls ) + (1 ) Ks Ks+1 )
1 t
+ =s+1 t u (Y (K ; L ) + (1 )K K +1 ) :

Choosing Ks optimally implies

s 1 t 0
u (Y (Ks 1 ; Ls 1 ) + (1 ) Ks 1 Ks )
s t 0 @Y (Ks ; Ls )
+ u (Y (Ks ; Ls ) + (1 ) Ks Ks+1 ) +1 = 0:
@Ks

Reeinserting the constraint (3.40) and rearranging gives

@Y (Ks ; Ls )
u0 (Cs 1 ) = u0 (Cs ) 1 + :
@Ks

This is the standard optimality condition for consumption which we obtained in

(3.45). As s can stand for any point in time between t and in…nity, we could replace
s by t, or + 1:

Back to the Lagrangian

When we now go back to the maximization procedure where the Lagrangian was
used, we see that the partial derivative of the Lagrangian with respect to Kt in
(3.43) captures how t changes over time. The simple reason why the Lagrangian is
maximized with respect to Kt is therefore that an aditional …rst-order condition is
needed as t needs to be determined as well.
In static maximization problems with two consumption goods and one constraint,
the Lagrangian is maximized by choosing consumption levels for both consumption
goods and by choosing the Lagrange multiplier. In the Lagrange setup above, we
choose in (3.42) to (3.44) both endogenous variables Kt and Ct plus the multiplier t
and thereby determine optimal paths for all three variables. Hence, it is a technical
- mathematical - reason that Kt is “chosen”, three unknowns simply require three
…rst-order conditions to be determined, but economically, the control variable is Ct
which is “economically chosen” while the state variable Kt adjusts indirectly as a
consequence of the choice of Ct :

3.8 Exercises Chapter 3 and further reading

The dynamic programming approach was developed by Bellman (1957). Maximiza-
tion using the Lagrange method is widely applied by Chow (1997).

48
 Exercises chapter 3
Applied Intertemporal Optimization
Dynamic programming in discrete deterministic time

1. The envelope theorem I

Let the utility function of an individual be given by

U = U (C; L) ;

where consumption C increases utility and supply of labour L decreases utility.

Let the budget constraint of the individual be given by

wL = C:

Let the individual maximize utility with respect to consumption and the amount
of labour supplied.

(a) What is the optimal labour supply function (in implicit form)? How much
does an individual consume? What is the indirect utility function?
(b) Under what conditions does an individual increase labour supply when
wages rise? (No analytical solution)
(c) Assume higher wages lead to higher labour supply. Does disutility arising
from higher labour supply overcompensate utility from higher consump-
tion? Does utility rise if there is no disutility from working? Start from the
indirect utility function derived in a) and apply the proof of the envelope
theorem and the envelope theorem itself.

2. The envelope theorem II

(a) Compute the derivative of the Bellman equation (3.19) without using the
envelope theorem. Hint: Compute the derivative with respect to the state
variable and then insert the …rst-order condition.
(b) Do the same with (3.26)

3. The additively separable objective function

(a) Show that the objective function can be written as in (3.10).

(b) Find out whether (3.10) implies the objective function. (It does not.)

4. Intertemporal and dynamic budget constraints

49
 (a) Show that the intertemporal budget constraint

T 1 1 T 1 1
=t k=t e = at + =t k=t i (3.46)
1 + rk 1 + rk

implies the dynamic budget constraint

at+1 = (at + it et ) (1 + rt ) : (3.47)

(b) Under which conditions does the dynamic budget constraint imply the
intertemporal budget constraint?
(c) Now consider at+1 = (1 + rt ) at + wt et : What intertemporal budget
constraint does it imply?

5. The standard saving problem

t
Consider the objective function from (3.1), Ut = 1=t u (c ), and maximize
it by chosing a consumption path fc g subject to the constraint (3.33), kt+1 =
(1 + rt ) kt + wtL ct : The solution is given by (3.34).

(a) Solve this problem by dynamic programming methods.

(b) Solve this by using the Lagrange approach. Choose a multiplier t for an
in…nite sequence of constraints.

6. A benevolent central planner

You are the omniscient omnipotent benevolent central planner of an economy.
You want to maximize a social welfare function
1 t
Ut = =t u (C )

for your economy by choosing a path of aggregate consumption levels fC g

subject to a resource constraint

Kt+1 Kt = Y (Kt ; Lt ) Kt Ct (3.48)

(a) Solve this problem by dynamic programming methods.

(b) Solve this by using the Lagrange approach. Choose a multiplier t for an
in…nite sequence of constraints.
(c) Discuss how the central planner solution is related to the decentralized
solution from exercise 5.

7. Environmental economics
Imagine you are an economist only interested in maximizing the present value of
your endowment. You own a renewable resource, for example a piece of forest.
The amount of wood in your forest at a point in time t is given by xt . Trees
grow at b(xt ) and you harvest at t the quantity ct .

50
 (a) What is the law of motion for xt ?
(b) What is your objective function if prices at t per unit of wood is given by
pt , your horizon is in…nity and you have perfect information?
(c) How much should you harvest per period when the interest rate is constant?
Does this change when the interest rate is time-variable?

8. Exam question (winter term 2005/06, Würzburg)

Sie überlegen sich, für die Wirtschaftswissenschaftliche Fakultät am Residen-
zlauf teilzunehmen. Dieser …ndet in M Monaten statt. Sie wissen, daßSie dafür
trainieren müssen, was Ihnen Anstrengung a0 abverlangt und Ihren Nutzen u (:)
reduziert. Gleichzeitig ziehen Sie einen Nutzen aus einer kurzen Laufzeit l. Die
Laufzeit ist umso kürzer, je größ er Ihre Anstrengung.

(a) Formulieren Sie ein Maximierungsproblem mit 2 Perioden. Sie trainieren

heute und erzielen damit eine gewisse Laufzeit in M Monaten. Spezi…zieren
Sie eine angemessene Nutzenfunktion und einen Zusammenhang zwischen
Laufzeit und Trainingsanstrengung a0 .
(b) Lösen Sie das Maximierungsproblem durch Herleitung einer Bedingung
erster Ordnung. Interpretieren Sie diese.

9. Exam question (winter term 2005/06, Würzburg)

Betrachten Sie folgende Zielfunktion eines zentralen Planers,
1 t
U0 = t=0 u (Ct ) : (3.49)

Die Nebenbedingung sei gegeben durch

Kt+1 = Y (Kt ; Lt ) + (1 ) Kt Ct : (3.50)

(a) Erläutern Sie Zielfunktion und Nebenbedingung verbal.

(b) Lösen Sie das Maximierungsproblem, indem Sie (3.50) in (3.49) einsetzen
0 (C )
und nach Kt maximieren. Zeigen Sie, daßals Ergebnis uu0 (C t
t+1 )
= 1+
@Y (Kt+1 ;Lt+1 )
@Kt+1
resultiert und interpretieren Sie dieses verbal.
(c) Erläutern Sie nun, wieso bei der Verwendung der Lagrangefunktion (siehe
Übung) auch bezüglich Kt maximiert wird, obwohl Kt eine Zustandsvari-
able ist.

51
Part II
Deterministic models in continuous
time
4 Di¤erential equations
There are many excellent textbooks on di¤erential equations. This chapter will there-
fore be relatively short. Its objective is more to recall basic concepts taught in other
courses and to serve as a background for later applications.

4.1 Some de…nitions and theorems

4.1.1 De…nitions
The following de…nitions are standard and follow Brock and Malliaris (1989).
De…nition 4 An ordinary di¤erential equation system (ODE system) is of the type
dx (t)
x_ (t) = f (t; x (t)) ; (4.1)
dt
where t lies between some starting point and in…nity, t 2 [t0 ; 1] ; x can be a vector,
x 2 Rn and f maps from Rn+1 into Rn : When x is not a vector, i.e. for n = 1; (4.1)
obviously is an ordinary di¤erential equation.
An autonomous di¤erential equation is an ordinary di¤erential equation where
f (:) is independent of time t;
dx
x_ (t) = f (x (t)) : (4.2)
dt
The di¤erence between a di¤erential equation and a “normal”equation obviously
lies in the fact that di¤erential equations contain derivatives of variables like x_ (t).
An example of a di¤erential equation which is not an ODE is the partial di¤erential
equation. A linear example is
@u (x; y) @u (x; y)
a (x; y) + b (x; y) = c (x; y) ;
@x @y
where a (:) ; b (:) ; c (:) and u (:) are functions with “nice properties”. The di¤erence
obviously lies in the presence of two derivatives. While in an ODE, there is one deriv-
ative (often with respect to time), a partial di¤erential equation contains derivatives
with respect to several variables.
De…nition 5 An initial value problem (Anfangswertproblem) is described by
x_ = f (t; x); x (t0 ) = x0; t 2 [t0 ; T ] :
A boundary value problem is of the form
x_ = f (t; x) ; x (T ) = xT ; t 2 [t0 ; T ]

52
4.1.2 Two theorems
Theorem 2 Existence (Brock and Malliaris, 1989)
If f (t; x) is continuous function on rectangle L = f(t; x)j jt t0 j a; jx x0 j bg
then there exists a continuous di¤erentiable solution x (t) on interval jt t0 j a that
solves initial value problem

x_ = f (t; x) ; x (t0 ) = x0 : (4.3)

This theorem only proves that a solution exists. It is still possible that there are
many solutions.

Theorem 3 Uniqueness
If f and @f =@x are continuous functions on L; the initial value problem (4.3) has a
unique solution for
b
t 2 t0 ; t0 + min a;
max jf (t; x)j

If this condition is met, an ODE with an initial or boundary condition has a

unique solution. More generally speaking, a di¤erential equation system consisting
of n ODEs that satisfy these conditions (which are met in the economic problems we
encounter here) has a unique solution provided that there are n initial or boundary
conditions. Knowing about a unique solution is useful as one knows that changes in
parameters imply unambiguous predictions about changes in endogenous variables.
If the government changes some tax, we can unambiguously predict whether employ-
ment goes up or down.

4.2 Analysing ODEs through phase diagrams

This section will present tools that allow to determine properties of solutions of dif-
ferential equations and di¤erential equation systems. The analysis will be qualitative
in this chapter as most economic systems are too non-linear to allow for an explicit
general analytic solution. Explicit solutions for linear di¤erential equations will be
treated in ch. 4.3.

4.2.1 One dimensional systems

We start with a one-dimensional di¤erential equation x_ (t) = f (x (t)), where x 2 R
and t > 0: This will also allow us to review the concepts of …xpoints, local and global
stability and instability as used already when analysing di¤erence equations in ch.
2.5.4.

Unique …xpoint

Let f (x) be represented by the graph in the following …gure, with x (t) being
shown on the horizontal axis. As f (x) gives the change of x (t) over time, x_ (t) is
plotted on the vertical axis.

53
 N
x(t)
_

N
N

N
N
N
x x(t)

Figure 12 Qualitative analysis of a di¤erential equation

As in the analysis of di¤erence equations in ch. 2.5.4, we …rst look for the …xpoint
of the underlying di¤erential equation. A …xpoint is de…ned similarily in spirit but -
thinking now in continuous time - di¤erently in detail: A …xpoint x is a point where
x (t) does not change; this is the similarity in spirit. Formally, this means x_ (t) = 0
which, from the de…nition (4.2) of the di¤erential equation, requires f (x ) = 0 (and
not f (x ) = x as in (2.38)). Looking at the above graph of f (x) ; we …nd x at the
point where f (x) crosses the horizontal line.
We then inquire the stability of the …xpoint. When x is to the left of x ; f (x) > 0
and therefore x increases, x_ (t) > 0: This increase of x is represented in this …gure by
the arrows on the horizontal axis. Similarily, for x > x ; f (x) < 0 and x (t) decreases.
We have therefore found that the …xpoint x is globally stable and have also obtained
a “feeling”for the behaviour of x (t) ; given some initial conditions.
We can now qualitatively plot the solutions with time t on the horizontal axis.
As the discussion has just shown, the solution x (t) depends on the initial value from
which we start, i.e. on x (0) : For x (0) > x ; x (t) decreases, for x (0) < x ; x (t)
increases: any changes over time are monotonic. There is one solution for each initial
condition. The following …gure shows three solutions of x_ (t) = f (x (t)), given three
di¤erent intitial conditions.

54
 N
x(t)
x03

x02

x01

N
t0 t
Figure 13 Qualitative solutions of an ODE for di¤erent initial conditions I

Multiple …xpoints and equilibria

Of course, more sophisticated functions than f (x) can be imagined. Consider now
a di¤erential equation x_ (t) = g (x (t)) where g (x) is non-monotonic as plotted in the
next …gure.
N
x
N

N
N

N
x1 x2 x3 x4 x

Figure 14 Multiple equilibria

As this …gure shows, there are four …xpoints. Looking at whether g (x) is positive
or negative, we know whether x (t) increases or decreases over time. This allows us
to plot arrows on the horizontal axis as in the previous example. The di¤erence to
before consists in the fact that some …xpoints are unstable and some are stable. Any
small deviation of x from x1 implies an increase or decrase of x: The …xpoint x1 is

55
therefore unstable, given the de…nition in ch. 2.5.4. Any small deviation x2 ; however,
implies that x moves back to x2 : Hence, x2 is (locally) stable. The …xpoint x3 is also
unstable, while x4 is again locally stable: While x converges to x4 for any x > x4 (in
this sense x4 could be called globally stable from the right), x converges to x4 from
the left only if x is not smaller than or equal to x3 :
If an economy can be represented by such a di¤erential equation, one would call
…xpoints long-run equilibria. There are stable equilibria and unstable equilibria and
it depends on the underlying system (the assumptions that implied the di¤erential
equation x_ = g (x)) which equilibrium would be considered to be the economically
relevant one.
As in the system with one …xpoint, we can qualitatively plot solutions of x (t)
over time, given di¤erent initial values for x (0). This is shown in the following …gure
which again highlights the stability properties of …xpoints x1 to x4 :

x4

x3

x2

x1
N

t0 t

Figure 15 Qualitative solutions of an ODE for di¤erent initial conditions II

4.2.2 Two-dimensional systems I - An example

We now extend our qualitative analysis of di¤erential equations to two-dimensional
systems. This latter case allows for an analysis of more complex systems than simple
one-dimensional di¤erential equations. We start here with an example before we
analyse two dimensional systems more generally in the next chapter.

The system

56
 Consider the following di¤erential equation system,

x_ 1 = x1 x2 ; x_ 2 = b + x1 1 x2 ; 0< < 1 < b:

Assume that for economic reasons we are interested in properties for xi > 0:

Fixpoint

The …rst question is as always whether there is a …xpoint at all. In a two-

dimensional system, a …xpoint x = (x1 ; x2 ) is two-dimensional as well. The …xpoint
is de…ned such that both variables do not change over time, i.e. x_ 1 = x_ 2 = 0: If such
a point exists, it must satisfy

x_ 1 = x_ 2 = 0 , (x1 ) = x2 ; x2 = b + (x1 ) 1 :

By inserting the second equation into the …rst, x1 is determined by (x1 ) = b + (x1 ) 1
and x2 follows from x2 = (x1 ) : Analysing the properties of the equation (x1 ) = b +
(x1 ) 1 would then show that x1 is unique: The left-hand side increases monotonically
from 0 to in…nity for x1 2 [0; 1[ while the right-hand side decreases monotonically
from in…nity to b: Hence, there must be an intersection point and there can be only
one as functions are monotonic. As x1 is unique, so is x2 = (x1 ) :

General evolution

Having derived the …xpoint, we now need to understand the behaviour of the
system more generally. What happens to x1 and x2 when (x1 ; x2 ) 6= (x1 ; x2 )? To
answer this question, the concept of zero-motion lines is very useful. A zero-motion
line is a line for a variable xi which marks the points for which the variable xi does
not change, i.e. x_ i = 0: For our two-dimensional di¤erential equation system, we
obtain two zero-motion lines,

x_ 1 0 , x2 x1 ;
(4.4)
x_ 2 0 , x2 b + x1 1 :

In addition to the equality sign, we also analyse here at the same time for which
values xi rises. Why this is useful will soon become clear. We can now plot the
curves where x_ i = 0 in a diagram. In contrast to the one-dimensional graphs in the
previous chapter, we now have the variables x1 and x2 on the axes (and not the change
of one variable on the vertical axis). The intersection point of the two zero-motion
lines gives the …x point x = (x1 ; x2 ) which we derived analytically above.

57
 x2

x&1 = 0
IV

III
II
x& 2 = 0
b

x1

Figure 16 A phase diagram

In addition to showing where variables do not change, the zero-motion lines also
delimit regions where variables do change. Looking at (4.4) again shows (why we
used the and not the = sign and) that the variable x1 increases whenever x2 < x1 .
Similarily, the variable x2 increases, whenever x2 < b + x1 1 : The directions in which
variables change can then be plotted into this diagram by using arrows. In this
diagram, there are two arrows per region as two directions (one for x1 ; one for x2 )
need to be indicated. This is in principle identical to the arrows we used in the
analysis of the one-dimensional systems. If the system …nds itself in one of these four
regions, we know qualitatively, how variable change over time: Variables move to the
southeast in region I, to the northeast in region II, to the northwest in region III and
to the southwest in region IV. We will analyse the implications of such a behaviour
further once we have generalized the derivation of a phase diagram.

4.2.3 Two-dimensional systems II - The general case

After this speci…c example, we will now look at a more general di¤erential equation
system and will analyse it by using a phase diagram.

The system

Consider two di¤erential equations where functions f (:) and g (:) are continuous
and di¤erentiable,
x_ 1 = f (x1 ; x2 ) ; x_ 2 = g (x1 ; x2 ) : (4.5)
For the following analysis, we will need four assumptions on partial derivatives; all of
them are positive apart from fx1 (:) ;

fx1 (:) < 0; fx2 (:) > 0; gx1 (:) > 0; gx2 (:) > 0: (4.6)

58
Note that, provided we are willing to make the assumptions required by the theorems
in ch. 4.1.2, we know that there is a unique solution to this di¤erential equation
system, i.e. x1 (t) and x2 (t) are unambiguously determined given two boundary
conditions.

Fixpoint

The …rst question to be tackled is whether there is an equilibrium at all. Is there

a …xpoint x such that x_ 1 = x_ 2 = 0? To this end, set f (x1 ; x2 ) = 0 and g (x1 ; x2 ) = 0
and plot the implicitly de…ned functions in a graph.

x2

f (x1 , x2 ) = 0

x2*

g (x1 , x2 ) = 0

x1* x1

Figure 17 Zero motion lines with a unique steady state

By the implicit function theorem - see (2.14) - and the assumptions made in (4.6),
one zero motion line is increasing and one is decreasing. If we are further willing to
assume that functions are not monotonically approaching an upper and lower bound,
we know that there is a unique …xpoint (x1 ; x2 ) x :

General evolution

Now we ask again what happens if the state of the system di¤ers from x ; i.e. if
either x1 or x2 or both di¤er from their steady state values. To …nd an answer, we
have to determine the sign of f (x1 ; x2 ) and g (x1; x2 ) for some (x1 ; x2 ) : Given (4.5); x1
would increase for a positive f (:) and x2 would increase for a positive g (:) : For any
known functions f (x1 ; x2 ) and g (x1; x2 ) ; one can simply plot a 3-dimensional …gure
with x1 and x2 on the axes in the plane and with time derivatives on the vertical axis.

59
Figure 18 A three-dimensional illustration of two di¤erential equations and their
zero-motion lines

When working with two-dimensional …gures and without the aid of computers, we
start from the zero-motion line for, say, x1 :

N
x2
-
x1;rx~2 )
(~ -
f (x1 ; x2 ) = 0
-

-
N

x1
Figure 19 Step 1 in constructing a phase diagram

Now consider a point (~ x1 ; x~2 ) : As we know that x_ 1 = 0 on f (x1 ; x2 ) = 0 and that

from (4.6) fx2 (:) > 0; we know that moving from x~1 on the zero-motion line vertically
to (~
x1 ; x~2 ) ; f (:) is increasing. Hence, x1 is increasing, x_ 1 > 0; at (~ x1 ; x~2 ) : As this
line of reasoning holds for any (~ x1 ; x~2 ) above the zero-motion line, x1 is increasing
everywhere above f (x1 ; x2 ) = 0: As a consequence, x1 is decreasing everywhere below
the zero-motion line. This movement is indicated by the arrows in the above …gure.

60
 N
x2
6g(x ; x ) = 0
1 2
r
6
?
(~
x1; x~2 ) ? 6

? 6

N
x1
Figure 20 Step 2 in constructing a phase diagram

Let us now consider the second zero-motion line, x_ 2 = 0 , g (x1 ; x2 ) = 0; and

look again at the point (~
x1 ; x~2 ) : When we start from the point x~2 on the zero motion
line and move towards (~x1 ; x~2 ) ; g (x1 ; x2 ) is decreasing, given the derivative gx1 (:) > 0
from (4.6). Hence, for any points to the left of g (x1 ; x2 ) = 0; x2 is decreasing. Again,
this is shown in the above …gure.

Plots and …xpoints, the solution

We can now represent the directions in which x1 and x2 are moving into one
phase-diagram by plotting one arrow each into each of the four regions limited by the
zero-motion lines. Given that the arrows can either indicate an increase or a decrease
for both x1 and x2 ; there are 2 times 2 di¤erent combinations of arrows. When we
add some representative trajectories, a complete phase diagram results.

x2 x&1 = 0

x&2 = 0

x1

Figure 21 A saddle path

61
4.2.4 Types of phase diagrams and …xpoints
Some de…nitions

As has become clear by now, the partial derivatives in (4.6) are crucial for the
slope of the zero-motion lines and for the direction of movements of variables x1 and
x2 : Depending on the signs of the partial derivatives, various phase diagrams can
occur. These phase diagrams can be classi…ed into four typical groups, depending on
the properties of their …xpoint.

De…nition 6 A …xpoint is called a

9 8 9
center >
> >
> 0 >
>
= < =
saddle point 2
() trajectories pass through the …xpoint
focus >
> >
> all >
>
; : ;
node all
8
>
>
<
and
>
> at least one trajectory, 0
: on variables are monotonic
all trajectories, 1 or 2

A node and a focus can be either stable or unstable.

Some graphs

Here is now an overview of some typical phase diagrams.

(ii) node (Knoten)

- all integral curves pass
through node
node
(iia) (iib)
stable or unstable

Figure 22 Phase diagram for a node

62
 This …rst phase diagram shows a node. A node is a …xpoint through which all
trajectories go and where the time paths implied by trajectories are monotonic. As
drawn here, it is a stable node, i.e. for any initial conditions, the system ends up in
the …xpoint. An unstable node is a …xpoint from which all trajectories start. A phase
diagram for an unstable node would look like the one above but with all directions of
motions reversed.

(iii) focus
- all integral curves move
in spirals around fixpoint.
Either towards this point
or away.
focus
(iii) a and b stable or unstable

Figure 23 Phase diagram for a focus

A phase diagram with a focus looks similar to one with a node. The di¤erence
consists in the non-monotonic paths of the trajectories. As drawn here, x1 or x2 …rst
increase and then decrease on some trajectories.

(iv) centre
- no integral curve passes
through fixpoint
centre
predator-prey-model

Figure 24 Phase diagram for a center

63
 A circle is a very special case for a di¤erential equation system. It is rarely found
in models with optimizing agents. The standard example is the predator-prey model,
x_ = x xy, y_ = y + xy, where , , , and are positive constants. This is
also called the Lotka-Volterra model. No closed form solution has been found so far.

Limitations

It should be noted that a phase diagram analysis allows to identify a saddle point.
If no saddle point can be identi…ed, it is generally not possible to distinguish between
a node, focus or center. In the linear case, more can be deduced from a graphical
analysis. This is generally not necessary, however, as there is a closed form solution.

4.2.5 Multidimensional systems

If we have higher-dimensional problems where x 2 Rn and n > 2; phase diagrams are
obviously di¢ cult to be drawn. In the three-dimensional case, plotting zero motion
surfaces sometimes helps to gain some intuition. A graphical solution will generally,
however, not allow to identify equilibrium properties like saddle-path or saddle-plane
behaviour.

4.3 Linear di¤erential equations

This section will focus on a special case of the general ODE de…ned in (4.1). The
special aspect consists in making the function f (:) in (4.1) linear in x (t) : Doing this,
we obtain a linear di¤erential equation,

x_ (t) = a (t) x (t) + b (t) ; (4.7)

where a (t) and b (t) are functions of time. This is the most general case for a one
dimensional linear di¤erential equation.

4.3.1 Rules on derivatives

Before analysing (4.7) in more detail, we …rst need some other results which will
be useful later. This section therefore …rst presents some rules on how to compute
derivatives. It is by no means intended to be comprehensive or go in any depth. It
presents rules which have shown by experience to be of importance.

Integrals
R
De…nition 7 A function F (x) f (x) dx is the inde…nite integral (Stammfunk-
tion) of a function f (x) if
Z
d d
F (x) = f (x) dx = f (x) (4.8)
dx dx
This de…nition implies that there are in…nitely many integrals of f (x) : If F (x) is
an integral, then F (x) + c; where c is a constant, is an integral as well.

64
 Leibniz rule

We present here a rule for computing the derivative of an integral function. Let
there be a function z (x) with argument x; de…ned by the integral
Z b(x)
z (x) f (x; y) dy;
a(x)

where a (x) ; b (x) and f (x; y) are di¤erentiable functions. Then, the Leibniz rule
says that the derivative of this function with respect to x is
Z b(x)
d 0 0 @
z (x) = b (x) f (x; b (x)) a (x) f (x; a (x)) + f (x; y) dy: (4.9)
dx a(x) @x

The following …gure illustrates this rule.

f (x, y )

z (x + ∆x )

f (x, y )

z (x )
z (x )

a (x ) b(x ) y

Figure 25 Illustration of the di¤erentiation rule

Let x increase by a small amount. Then the integral changes at three margins:
The upper bound, the lower bound and the function f (x; y) itself. As drawn here,
the upper bound b (x) and the function f (x; y) increase and the lower bound a (x)
decreases in x. As a consequence, the area below the function f (:) between bounds
a (:) and b (:) increases because of three changes: the increase to the left because of
a (:) ; the increase to the right because of b (:) and the increase upwards because of
f (:) itself. Clearly, this …gure changes when the derivatives of a (:), b (:) and f (:)
with respect to x have a di¤erent sign than the ones assumed here.

Integration by parts (Partielle Integration)

Proposition 4 For two di¤erentiable functions u (x) and v (x) ;

Z Z
0 0
u (x) v (x) dx = u (x) v (x) u (x) v (x) dx: (4.10)

65
 Proof. We start by observing that
0 0 0
(u (x) v (x)) = u (x) v (x) + u (x) v (x) ;
R
where we used the product rule. Integrating both sides, dx; gives
Z Z
0 0
u (x) v (x) = u (x) v (x) dx + u (x) v (x) dx:

Rearranging gives (4.10).

Equivalently, one can show (see the exercises) that
Z b Z b
b
xydt
_ = [xy]a xydt:
_
a a

4.3.2 Forward and backward solutions of a linear di¤erential equation

We now return to our linear di¤erential equation x_ (t) = a (t) x (t) + b (t) from (4.7).
It will now be solved. A solution to this di¤erential equation is a function x (t) which
satis…es this equation. A solution can be called time path of x; when t is representing
time.

General solution of the non-homogenous equation

The di¤erential equation in (4.7) has, as all di¤erential equations, an in…nity of

solutions. The general solution reads
R
Z R
a(t)dt
x (t) = e x~ + e a(t)dt b (t) dt : (4.11)

Here, x~ is some arbitrary constant. As this constant is arbitrary, (4.11) provides

indeed an in…nity of solutions to (4.7).
To see that (4.11) is a solution to (4.7) indeed, remember the de…nition of what
a solution is. A solution is a time path x (t) which satis…es (4.7). Hence, we simply
need to insert the time path given by (4.11) into (4.7) and check whether (4.7) then
holds. To this end, compute the time derivative of x(t),
R
Z R R R
d a(t)dt
x (t) = e a (t) x~ + e a(t)dt b (t) dt + e a(t)dt e a(t)dt b (t) ;
dt
d
R
where we have used the de…nition of the integral in (4.8), dx f (x) dx = f (x) :
Note
R that we do not have to apply the product or chain rule since, again by (4.8),
d
dx
g (x) h (x) dx = g (x) h (x) : Inserting (4.11) gives

x_ (t) = a (t) x (t) + b (t) :

Hence, (4.11) is a solution to (4.7).

Determining the constant x~

66
 To obtain one particular solution, some value x (t0 ) has to be …xed. Depending
on whether t0 lies in the future or in the past, t0 > t or t0 < t, the equation is solved
forward or backward.
We start with the backward solution, i.e. where t > t0 . Let x (t0 ) = x0 : Then the
solution of (4.7) is
Rt Z t R
a( )d a(u)du
x (t) = e t0 x (t0 ) + e t0 b( )d
t0
Rt Z t R
t
t0 a( )d
=e x (t0 ) + e a(u)du b ( ) d : (4.12)
t0

The forward solution is required if t0 ; which we rename T for ease of distinction,

lies in the future of t, T > t: The solution then reads
RT
Z T R
a( )d
x (t) = e t x (T ) e t a(u)du b ( ) d : (4.13)
t

Veri…cation

We now show that (4.12) and (4.13) are indeed a solution for (4.7). The time
derivative of (4.12) is given by
Rt Rt
Z t R
t
t0 a( )d a(u)du
x_ (t) = e a (t) x (t0 ) + e t b (t) + e a(u)du b ( ) d a (t)
t0
Rt Z t R
a( )d t
= a (t) e t0 x (t0 ) + e a(u)du b ( ) d + b (t)
t0
= a (t) x (t) + b (t) :

The time derivative of (4.13) is

RT
Z T R
a( )d a(u)du
x_ (t) = e t a (t) x (T ) + b (t) e t b ( ) d a (t)
t
RT
Z T R
a( )d a(u)du
= a (t) e t x (T ) e t b( )d + b (t)
t
= a (t) x (t) + b (t) :

4.3.3 Di¤erential equations as integral equations

Any di¤erential equation can be written as an integral equation. While we will work
with the “usual” di¤erential equation representation most of the time, we introduce
the integral representation here as it will be frequently used later when comput-
ing moments in stochastic setups. Understanding the integral version of di¤erential
equations in this deterministic setup allows for an easier understanding of integral
representations of stochastic di¤erential equations later.

The principle

67
 The non-autonomous di¤erential equation x_ = f (t; x) can be written equivalently
as an integral equation. To this end, write this equation as dxR= f (t; x) dt or, after
t
substituting s for t, as dx = f (s; x) ds: Now apply the integral 0 on both sides. This
gives the integral version of the di¤erential equation x_ = f (t; x) which reads
Z t Z t
dx = x (t) x (0) = f (s; x) ds:
0 0

An example

The di¤erential equation x_ = a (t) x is equivalent to

Z t
x (t) = x0 + a (s) x (s) ds: (4.14)
0

Computing the derivative of this equation with respect to time t gives, using (4.9),
x_ (t) = a (t) x (t) again.
The presence of an integral in (4.14) should not lead one to confuse (4.14) with
a solution Rof x_ = a (t) x in the sense of the last section. Such a solution would read
t
x (t) = x0 e 0 a(s)ds .

4.3.4 Example 1: Budget constraints

As an application of di¤erential equations, consider the budget constraint of a house-
hold. As in discrete time in ch. 3.6.6, budget constraints can be expressed in a
dynamic and an intertemporal way. We …rst show here how to derive the dynamic
version and then how to obtain the intertemporal version from solving the dynamic
one.

Deriving a nominal dynamic budget constraint

Let us …rst, following ch. 2.5.5, derive the dynamic budget constraint. We start
from the de…nition of nominal wealth. It is given by a = kv; where k is the households
physical capital stock and v is the value of one unit of the capital stock. By computing
the time derivative, wealth of a household changes according to
_ + k v:
a_ = kv _

If the household wants to save, it can buy capital goods at the end of period t:
Nominal savings of the household in period t are given by s = wk k + w e; the
di¤erence between factor rewards for capital (value marginal product times capital
owned), labour income and expenditure. Dividing savings by the value of a capital
stock at the end of period t gives the number of capital stocks bought
wk k + w e
k_ = : (4.15)
v

68
Inserting into the a_ equation gives
wk + v_
a_ = wk k + w e + k v_ = (wk + v)
_ k+w e= a+w e:
v
De…ning the nominal interest rate as
wk + v_
i= ; (4.16)
v
we have the nominal budget constraint

a_ (t) = i (t) a (t) + w (t) e (t) : (4.17)

Finding the intertemporal budget constraint

We can now obtain the intertemporal budget constraint from solving the dynamic
one in (4.17). Using the forward solution from (4.13), we take a (T ) as the boundary
condition lying with T > t in the future. The solution is then
RT
Z T R
i( )d
a (t) = e t a (T ) e t i(u)du [w ( ) e ( )] d ,
t
Z T Z T
D ( ) w ( ) d + a (t) = D (T ) a (T ) + D( )e( )d ;
t t
R
where D ( ) e t i(u)du de…nes the discount factor. As we have used the forward
solution, we have obtained an expression which easily lends itself to an economic
interpretation. Think of an individual who - at the end of his life - does not want to
leave any bequest, i.e. a (T ) = 0 or, for a positive a (T ) ; does not want to use any
of this a (T ) for own consumption. In both cases, a (T ) = 0 in the above expression.
Then, this intertemporal budget constraint requires that current wealth on the left-
hand side, consisting of the present value of life-time labour income plus …nancial
wealth a (t) needs to equal the present value of current and future expenditure on the
right-hand side.

Real wealth

We can also start from the de…nition of real wealth of the household, measured in
units of the consumption good, whose price is p. Real wealth is then
kv
ar = :
p

The change in real wealth over time is then (apply the log on both sides and derive
with respect to time),
a_ r k_ v_ p_
= + :
ar k v p

69
Inserting the increase in the capital stock obtained from (4.15) gives

a_ r wk k + w e p_ v_(wk k + w e) p 1 v_ p_
r
= + = 1
+
a vk p v vkp v p
k w e v_ r p_ r wk r w e v_ r p_ r
, a_ r = wk + + a a = a + + a a
p p v p v p v p
wk + v_ p_ w e w e
= ar + = rar + : (4.18)
v p p p
Hence, the real interest rate r is - by de…nition -
wk + v_ p_
r= :
v p
Solving the di¤erential equation (4.18) again provides the intertemporal budget con-
straint as in the nominal case above.
Now assume that the price of the capital good equals the price of the consumption
good, v = p: This is the case in an economy where there is one homogenous output
good as in (2.26) or in (8.14). Then, the real interest rate is equal to the marginal
product of capital, r = wk =p:

4.3.5 Example 2: Forward solutions

The capital market no-arbitrage condition

Imagine you own wealth of worth v (t) : You can invest it on a bank account which
pays a certain return r (t) per unit of time or you can buy shares of a …rm which cost
v (t) and which yield dividend payments (t) and are subject to changes v_ (t) in its
worth. In a world of perfect information and assuming that in some equilibrium
agents hold both assets, the two assets must yield identical income streams,

r (t) v (t) = (t) + v_ (t) : (4.19)

This is a linear di¤erential equation in v (t). As just motivated, it can be considered

as a no-arbitrage condition. Note, however, that it is structurally equivalent to (4.16),
i.e. this no-arbitrage condition can be seen to just de…ne the interest rate r (t).
Whatever the interpretation of this di¤erential equation is, solving it forward gives
RT
Z T R
r( )d
v (t) = e t v (T ) + e t r(u)du ( ) d :
t

Letting T go to in…nity, we have

Z 1 R RT
v (t) = e t r(u)du ( ) d + lim e t r( )d
v (T ) :
t T !1

This forward solution stresses the economic interpretation of v (t) : The value of
an asset depends on the future income stream - dividend payments ( ) - that are

70
generated from owning this asset. Note that it is usually assumed that there are no
bubbles, i.e. the limit is zero such that the fundamental value of an asset is given by
the …rst term.
For a constant interest rate and dividend payments and no bubbles, the expression
for v (t) simpli…es to v = =r:
The utility function
Consider an intertemporal utility function as it is often used in continuous time
models, Z 1
U (t) = e [ t] u (c ( )) d : (4.20)
t
This is the standard expression which corresponds to (2.11) or (3.1) in discrete time.
Again, instantaneous utility is given by u (:) : It depends here on consumption only,
where households consume continuously at each instant : Impatience is captured as
before by the time preference rate : Higher values attached to present consumption
are captured by the discount function e [ t] ; whose discrete time analog in (3.1) is
t
:
Using (4.9), di¤erentiating with respect to time gives
Z 1
d
U_ (t) = u (c (t)) + e [ t] u (c ( )) d = u (c (t)) + U (t) :
t dt
Solving this linear di¤erential equation forward gives
Z T
[T t]
U (t) = e U (T ) + e [ t] u (c ( )) d :
t

Letting T go to in…nity, we have

Z T
U (t) = e [ t]
u (c ( )) d + lim e [T t]
U (T ) :
t T !1

The second term is related to the transversality condition.

4.3.6 Example 3: A growth model

Consider an example inspired by growth theory. Let the capital stock of the economy
follow K_ = I K, gross investment minus depreciation gives the net increase of the
capital stock. Let gross investment be determined by a constant saving rate times
output, I = sY (K) and the technology be given by a linear AK speci…cation (as in
e.g. Rebelo, 1991). The complete di¤erential equation then reads
K_ = sAK K = (sA ) K:
Its solution is (see ch. 4.3.2) K (t) = e(sA )t . As in the qualitative analysis above, we
found a multitude of solutions (a “Schar von Lösungen”), depending on the constant
: If we specify an initial condition, say we know the capital stock at t = 0; i.e.
K (0) = K0 , then we can …x the constant by = K0 and our solution …nally reads
K (t) = K0 e(sA )t :

71
4.4 Linear di¤erential equation systems
A di¤erential equation system consists of 2 or more di¤erential equations which are
mutually related to each other. An example is

x_ (t) = Ax (t) + b;

where the vector x (t) is given by x = (x1 ; x2 ; x3 ; :::; xn )0 ; A is an n n matrix with

elements aij and b is a vector b = (b1 ; b2 ; b3 ; :::; bn )0 : Note that elements of A and b
can be functions of time but not functions of x:
Such a system can be solved in various ways, e.g. by determining so-called Eigen-
values and Eigenvectors. These systems either result from economic models directly
or are the outcome of a linearization of some non-linear system around a steady state.
This latter approach plays an important role for local stability analyses (compared to
the global analyses we undertook above with phase diagrams). These local stability
analyses can be performed for systems of almost arbitrary dimension and are there-
fore more general and (for the local sourrounding of a steady state) more informative
than phase diagram analyses.
Please see the references in “Further reading”on many textbooks that treat these
issues.

4.5 Exercises chapter 4 and further reading

There are very many textbooks that treat di¤erential equations and di¤erential equa-
tion systems. Any library search tool will provide very many hits. This chapter ows
insights to Gandolfo (1996) on phase diagram analysis and di¤erential equations and
- inter alia - to Brock and Malliaris (1989), Braun (1975) and Chiang (1984) on dif-
ferential equations. See also Gandolfo (1996) on di¤erential equation systems. The
predator-prey model is treated in various biology textbooks. It can also be found on
many sites in the internet. The Leibniz rule was taken from Fichtenholz (1997) and
can be found in many other textbooks on di¤erentiation and integration.

72
 Exercises chapter 4
Applied Intertemporal Optimization
Using phase diagrams

1. Phase diagram I
Consider the following di¤erential equation system,
x_ 1 = f (x1 ; x2 ) ; x_ 2 = g (x1 ; x2 ) :
Assume
dx2 dx2
fx1 (x1 ; x2 ) < 0; gx2 (x1 ; x2 ) < 0; < 0; > 0:
dx1 f (x1 ;x2 )=0 dx1 g(x1 ;x2 )=0

a) Plot a phase diagram for the positive quadrant.

b) What type of …xpoint can be identi…ed with this setup?
2. Phase diagram II

(a) Plot 2 phase diagrams for

x_ = xy a; y_ = y b; a > 0: (4.21)
by varying the parameter b.
(b) What type of …xpoints do you …nd?

3. Phase diagram III

(a) Plot paths through points marked by \ in the …gure below.

(b) What type of …xpoints are A and B?

y
y& = 0

x& = 0

O
x

73
4. Local stability analysis
Study local stability properties of the …xpoint of the di¤erential equation system
(4.21).

5. Exam question (winter term 2005-06, Würzburg)

Grossman und Helpman (1991) untersuchen ein Wachstumsmodell mit steigen-
der Anzahl der Varianten eines gleichen Produktes. Die reduzierte Form der
Ökonomie wird durch folgendes, zweidimensionales Di¤erenzialgleichungssys-
tem beschrieben:
L
n_ (t) =
a v (t)
1
v_ (t) = v (t)
n (t)
wobei 0 < < 1 und a > 0. Die Variablen v (t) und n (t) bezeichnen jeweils
den Wert der Firma bzw. die Anzahl der Varianten eines gleichen Produktes.
Die positiven Konstanten und L bezeichnen jeweils die Zeitpräferenzrate der
Haushalte bzw. die Anzahl der Erwerbspersonen in der Ökonomie.
Zeichnen Sie das Phasendiagramm für den positiven Quadranten. Bestimmen
Sie den Fixpunkt. Um was für einen Fixpunkt handelt es sich?

6. Solving linear di¤erential equations

Solve y_ (t) + y (t) = ; y (s) = 17 for

(a) t > s;
(b) t < s:
(c) What is the forward, what is the backward solution? How do they relate
to each other?

7. Derivatives of integrals
Compute the following derivatives.
Ry
(a) ddy a f (s) ds;
Ry
(b) ddy a f (s; y) ds;
Rb
(c) ddy a f (y) dy;
R
(d) ddy f (y) dy:
Rb Rb
(e) Show that a xydt_ = [xy]ba a
xydt:
_

8. Intertemporal and dynamic budget constraints

Consider the intertemporal budget constraint which equates the discounted ex-
penditure stream to asset holdings plus a discounted income stream,
Z 1 Z 1
Dr ( ) E ( ) d = A (t) + Dr ( ) I ( ) d ; (4.22)
t t

74
 where Z
Dr ( ) = exp r (s) ds : (4.23)
t
A dynamic budget constraint reads
E (t) + A_ (t) = r (t) A (t) + I (t) : (4.24)

(a) Show that solving the dynamic budget constraint yields the intertemporal
budget constraint if and only if
Z T
lim A (T ) exp r ( ) d = 0: (4.25)
T !1 t

(b) Show that di¤erentiating the intertemporal budget constraint yields the
dynamic budget constraint.

9. A budget constraint with many assets

Consider an economy with two assets whose prices are vi (t). A household owns
ni (t) assets of each type such that total wealth at time t of the household is given
by a (t) = v1 (t) n1 (t) + v2 (t) n2 (t) : Each asset pays a ‡ow of dividends i (t) :
Let the household earn wage income w (t) and spend p (t) c (t) on consumption
per unit of time. Show that the budget constraint of the household is given by
a_ (t) = r (t) a (t) + w (t) p (t) c (t)
where the interest rates are de…ned by
i (t) + v_ i (t)
r (t) (t) r1 (t) + (1 (t)) r2 (t) ; ri (t)
vi (t)
and (t) v1 (t) n1 (t) =a (t) is de…ned as the share of wealth held in asset 1:
10. Optimal saving
Let optimal saving and consumption behaviour (see ch. 5, e.g. eq. (5.5)) be
described by the two-dimensional system
c_ = gc; a_ = ra + w c;
where g is the growth rate of consumption, given e.g. by g = r or g =
(r ) = : Solve this system for time paths of consumption c and wealth a:
11. ODE systems
Study transitional dynamics in a two-country world.

(a) Compute time paths of the number ni (t) of …rms in country i: The laws
of motion are given by (Grossman and Helpman, 1991; Wälde, 1996)
n_ i = nA + nB Li ni (L + ) ; i = A; B; L = L A + LB ; ; > 0:
Hint: Eigenvalues are g = (1 )L > 0 and = (L + ).
A
(b) Plot the time path of n . Choose appropriate initial conditions.

75
5 Optimal control theory - Hamiltonians
One widely used approach to solve deterministic intertemporal optimization problems
in continuous time is to use the so-called Hamiltonian function. Given a certain
maximization problem, this function can be adapted - just like a recipe - to yield a
straightforward solution. The …rst section will provide an introductory example. It
shows how easy it can sometimes be to solve a maximization problem.
It is useful to understand, however, where the Hamiltonian comes from. A list
of examples can never be complete, so it helps to be able to derive the appropriate
optimality conditions in general. This will be done in the subsequent section. Section
5.3 then discusses how boundary conditions for maximization problems look like and
how they can be motivated. The in…nite planning horizon problem is then presented
and solved in section 5.4 which includes a section on transversality and bounded-
ness conditions. Various examples follow in section 5.5. Section 5.6 …nally shows
how to work with present-value Hamiltonians and how they relate to current-value
Hamiltonians (which are the ones used in all previous sections).

5.1 An introductory example

Consider an individual that wants to maximize a utility function as encountered
already in (4.20),
Z T
max e [ t] ln c ( ) d : (5.1)
fc( )g t
The budget constraint of this individual equates changes in wealth, a_ ( ) ; to current
savings, i.e. the di¤erence between capital and labour income, r ( ) a ( ) + w ( ) ; and
consumption expenditure c ( ),
a_ ( ) = r ( ) a ( ) + w ( ) c( ):
This maximization problem can be solved by using the present-value or the current-
value Hamiltonian. We will work with the current-value Hamiltonian here and in
what follows (see section 5.6 on the present-value Hamiltonian). The current-value
Hamiltonian reads
H = ln c ( ) + ( ) [r ( ) a ( ) + w ( ) c ( )] ; (5.2)
where ( ) is called the costate variable; “costate” as it corresponds to the state
variable a ( ) : In maximization problems with more than one state variable, there is
one costate variable for each state variable. The costate variable could also be called
Hamilton multiplier - similar to the Lagrange multiplier. We show further below that
( ) is the shadow price of wealth.
Omitting time arguments, optimality conditions are
@H 1
= = 0; (5.3)
@c c
_ = @H
= r : (5.4)
@a

76
The …rst-order condition in (5.3) is a usual optimality condition: the derivative of the
Hamiltonian (5.2) with respect to the consumption level c must be zero. The second
optimality condition - at this stage - just comes “out of the blue”. Its origin will be
discussed in a second. Applying logs to the …rst …rst-order condition, ln c = ln ;
and computing derivatives with respect to time yields c=c _ = _ = : Inserting into
(5.4) gives the Keynes-Ramsey rule
c_ c_
= r, =r : (5.5)
c c

5.2 Deriving laws of motion

This subsection shows where the Hamiltonian comes from. More precisely, it shows
how the Hamiltonian can be deduced from the optimality conditions resulting from
a Lagrange approach. The Hamiltonian can therefore be seen as a shortcut which is
faster than the Lagrange approach but leads to (it needs to) identical results.

5.2.1 Solving by using the Lagrangian

Consider the objective function
Z T
[ t]
U (t) = e u (y ( ) ; z ( ) ; ) d (5.6)
t

which we now maximize subject to the constraint

y_ ( ) = Q (y ( ) ; z ( ) ; ) : (5.7)

The function Q (:) is left fairly unspeci…ed. It could be a budget constraint of a

household, a resource constraint of an economy or some other constraint. We assume
that Q (:) has “nice properties”, i.e. it is continuous and di¤erentiable everywhere.
The objective function is maximized by an appropriate choice of the path fz( )g of
control variables.
This problem can be solved by using the Lagrangian
Z T Z T
[ t]
L= e u( )d + ( ) [Q ( ) y_ ( )] d :
t t

The utility function u (:) and other functions are presented as u ( ) : This shortens
notation compared to full expressions in (5.6) and (5.7). The intuition behind this
Lagrangian is similar to the one behind the Lagrangian in the discrete time case in
(3.41) in ch. 3.7 where we also looked at a setup with many constraints. The …rst
part is simply the objective function. The second part refers to the constraints. In the
discrete-time case, each point in time had its own constraint with its own Lagrange
multiplier. Here, the constraint (5.7) holds for a continuum of points . Hence,
instead of the sum in the discrete case we now have an integral over the product of
multipliers ( ) and constraints.

77
 This Lagrangian can be rewritten as follows,
Z T Z T
[ t]
L= e u( ) + ( )Q( )d ( ) y_ ( ) d
t t
Z T Z T
= e [ t]
u( ) + ( )Q( )d + _ ( )y( )d [ ( ) y ( )]Tt (5.8)
t t

where the last step integrated by parts and [ ( ) y ( )]Tt is the integral function of
( ) y ( ) evaluated at T minus its level at t. Now assume that we could choose
not only the control variable z ( ) but also the state variable y ( ) at each point in
time. The intuition for this is the same as in discrete time in ch. 3.7.2. We maximize
therefore this Lagrangian with respect to y and z at one particular 2 [t; T ].
For the control variable z; we get a …rst-order condition
[ t] [ t]
e uz ( ) + ( ) Qz ( ) = 0 , uz + e ( ) Qz = 0:

When we de…ne
[ t]
( ) e ( ); (5.9)
we …nd
uz + ( ) Qz = 0: (5.10)
For the state variable y; we obtain
[ t]
e uy ( ) + ( ) Qy ( ) + _ ( ) = 0 ,
[ t] [ t]
uy ( ) ( )e Qy = e _ ( ): (5.11)

Di¤erentiating (5.9) with respect to time gives

_( )= e [ t]
( )+e [ t]
_( )= ( )+e [ t]
_ ( ):

Inserting (5.11), we obtain

_( )= ( ) uy ( ) ( ) Qy ( ) : (5.12)

Equations (5.10) and (5.12) are the two optimality conditions that solve the above
maximization problem jointly with the constraint (5.7). We have three equations
which …x three variables: The …rst condition (5.10) determines the optimal level of
the control variable z: As this optimality condition holds for each point in time ; it
…xes an entire path for z: The second optimality condition (5.12) …xes a time path for
: It makes sure, by letting the costate follow an appropriate path, that the level
of the state variable (which is not instantaneously adjustable as the maximization of
the Lagrangian would suggest) is as if it had been optimally chosen at each instant.
The constraint (5.7), …nally, …xes the time path for the state variable y:

78
5.2.2 Hamiltonians as a shortcut
Let us now see how Hamiltonians can be justi…ed. De…ne the Hamiltonian similar to
(5.2),
H = u( ) + ( )Q( ): (5.13)
In fact, this Hamiltonian shows the general structure of Hamiltonians. Take the
instantaneous utility level (or any other function behind the discount term in the
objective function) and add the costate variable times the right-hand side of the
constraint. Optimality conditions are then

Hz = 0; (5.14)
_ = Hy : (5.15)

These conditions were already used in (5.3) and (5.4) in the introductory example
in the previous chapter and in (5.10) and (5.12): When (5.13) is inserted into (5.14)
and (5.15), this yields equations (5.10) and (5.12). Hamiltonians are therefore just
a shortcut that allow to …nd solutions faster than in the case where Lagrangians are
used. Note for later purposes that both and have time as an argument.
There is an interpretation of the costate variable which we simply state at this
point (see ch. 6.2 for a formal derivation): The derivative of the objective function
with respect to the state variable at t, evaluated on the optimal path, equals the
value of the corresponding costate variable at t. Hence, just as in the static Lagrange
case, the costate variable measures the change in utility as a result of a change in
endowment (i.e. in the state variable). Expressing this formally, de…ne the value
function as V (y (t)) maxfz( )g U (t) ; identical in spirit to the value function in
dynamic programming as we got to know it in discrete time. The derivative of the
value function, the shadow price V 0 (y (t)) ; is then the change in utility when behaving
optimally resulting from a change in the state y (t) : This derivative equals the costate
variable, V 0 (y (t)) = .

5.3 Boundary conditions and su¢ cient conditions

So far, maximization problems were presented without boundary conditions. Usually,
however, boundary conditions are part of the maximization problem. We will now
consider two cases. Both cases will later be illustrated in the phase diagram of section
5.5.

5.3.1 Free value of state variable at endpoint

Many problems are of the form (5.6) and (5.7) were boundary values for the state
variable are given by
y (0) = y0 ; y (T ) free. (5.16)
The …rst condition is the usual initial value condition. The second condition allows
the state variable to be freely chosen for the end of the planning horizon.

79
 We can use the current-value Hamiltonian (5.13) which gives optimality conditions
(5.14) and (5.15). In addition (cf. Feichtinger and Hartl, 1986, p. 20),

(T ) = 0: (5.17)

As is the shadow price of the state variable, this simply means that the value of
the state variable at T needs to be zero.

5.3.2 Fixed value of state variable at endpoint

Now consider (5.6) and (5.7) with one initial and one endpoint condition,

y (0) = y0 ; y (T ) = yT : (5.18)

In order for this problem to make sense, we assume that a feasible solution exists. This
“should”generally be the case, but it is not obvious: Consider again the introductory
example in ch. 5.1. Let the endpoint condition be given by “the agent is very rich in
T ”; i.e. a (T ) =“very large”. If a (T ) is too large, even zero consumption at each point
in time, c ( ) = 0 8 2 [t; T ] ; would not allow wealth a to be as large as required by
a (T ) : In this case, no feasible solution would exist.
We assume, however, that a feasible solution exists. Optimality conditions are
then identical to (5.14) and (5.15), plus initial and boundary values. The di¤erence
from this approach to the one before is that now y (T ) = yT is exogenously given, i.e.
part of the maximization problem, whereas before the corresponding y (T ) = 0 was
endogenously determined as a necessary condition for optimality.

5.3.3 Su¢ cient conditions

So far, we only presented conditions that are necessary for a maximum. We do
not yet know, however, whether these conditions are also su¢ cient. (Think of the
static maximization problem maxx f (x) where f 0 (x ) = 0 is necessary for an interior
maximum but not su¢ cient as f 0 (x + ") could be positive for any " 6= 0.) The
following theorem from Kamien and Schwartz (1991, ch. 3, p. 133) tells us more.

Theorem 5 Necessary conditions are su¢ cient if either

(i) the functions u (:) and Q (:) in (5.6) and (5.7) are concave in y and z and if
(t) 08t < < T .
(ii) Q (:) is linear in y and z for any (t).
(iii) Q (:) is convex and (t) 0 8 t < < T:

For concavitiy, the proof of this theorem employs f f (x x) fx +(u u) fu ;

f = G; Q: There is a further theorem due to Arrow which is also presented by Kamien
and Schwartz (1991, ch. 15, p. 221).

80
5.4 The in…nite horizon
5.4.1 The maximization problem
In the in…nite horizon case, the objective function has the same structure as before in
e.g. (5.6) only that the …nite time horizon T is replaced by an in…nite time horizon
1. The constraints are unchanged and the maximization problem reads
Z 1
max e [ t] u (y ( ) ; z ( ) ; ) d ;
fz( )g t

subject to

y_ ( ) = Q (y ( ) ; z ( ) ; ) ;
y (0) = y0 : (5.19)
R1
We need to assume for this problem that the integral t e [ t] u (:) d converges
for all feasible paths of y ( ) and z ( ), otherwise the optimality criterion must be
rede…ned. This boundedness condition is important only in this in…nite horizon case.
If individuals have …nite horizon and the utility function u (:) is continuous over the
entire planning period, the objective function is …nite and the boundedness problem
disappears. Clearly, making such an assumption is not always innocuous and one
should check, at least after having solved the maximization problem, whether the
objective function converges indeed. This will be done in ch. 5.4.3.
The current-value Hamiltonian as in (5.13) is de…ned by

H = u (:) + ( ) Q (:) ;

and optimality conditions are (5.14) and (5.15), i.e.

@H _ = @H
= 0; :
@z @y
Hence, we have identical optimality conditions to the case of a …nite horizon.

5.4.2 The transversality condition

The solution to the maximization problem with an in…nite horizon also led to a
system of two di¤erential equations. One boundary condition is provided by (5.19)
for the state variable. Hence, again, we need a second condition to pin down the
initial consumption level. In the …nite horizon case, we had conditions of the type
K (T ) = KT or (T ) = 0 in (5.17) and (5.18). The condition used here, which is
related to these conditions, is the transversality condition (TVC),

lim f (t) [y (t) y (t)]g = 0;

t!1

where y (t) is the path of y (t) for an optimal choice of control variables.

81
 A conceptionally di¤erent condition from the transversality condition is the “No-
Ponzi game”condition (4.25)
Z T
lim y (T ) exp r( )d =0
T !1 t

where r @H=@y. Note that this No-Ponzi game condition can be rewritten as
T
lim e (T ) y (T ) = lim (T ) y (T ) = 0; (5.20)
T !1 T !1

T R
where the fact that _ = = r implies e t = 0 e t r( )d was used. The for-
mulations in (5.20) of the No-Ponzi game condition is frequently encountered and is
sometimes even called TVC.
There is a considerable literature on the necessity and su¢ ciency of the TVC.
Some examples include Mangasarian (1966), Arrow (1968), Arrow and Kurz (1970),
Araujo and Scheinkman (1983), Léonard and Long (1992, p. 288-289), Chiang (1992,
p. 217, p. 252) and Kamihigashi (2001). Counterexamples that the TVC is not
necessary are provided by Michel (1982) and Shell (1969).

5.4.3 The boundedness condition

When introducing an in…nite horizon objective function, it was stressed right after
(5.19) that objective functions must be …nite for any feasible paths of the control
variable. Otherwise, overall utility U (t) would be in…nitly large and there would
be no objective for optimizing - we are already in…nitly happy! This “problem” of
unlimited happiness is particularily severe in models where control variables grow
with a constant rate, think e.g. of consumption in a model of growth. A pragmatic
approach to checking whether growth is not too high is to …rst assume that it is not
to high, to then maximize the utility function and to afterwards check whether the
initial assumption is satis…ed. R1
As an example, consider the intertemporal utility function U (t) = t e [ t] u (C ( )) d ;
where > 0 and the instantaneous utility function u (C ( )) is characterized by con-
stant relative risk aversion (CRRA),

C ( )1 1
u (C ( )) = :
1
Assume that consumption grows with a rate of g, where this growth rate results from
utility maximization. Think of this g as representing e.g. the di¤erence between
the interest rate and the time preference rate, corrected by intertemporal elasticity
of substitution, as will be found later e.g. in the Keynes-Ramsey rule (5.33), i.e.
c=c
_ = (r )= g. With this exponential growth of consumption, the utility
function becomes
Z 1
1
U (t) = (1 ) C0 e [ t] e(1 )g[ t] 1 d :
t

82
This integral is bounded only if

(1 )g < 0:

This can formally be seen by computing the integral explicitly and checking under
which conditions it is …nite. Intuitively, this condition makes sense: Instantaneous
utility from consumption grows by a rate of (1 ) g: Impatience implies that future
utility is discounted by the rate : Only if this time preference rate is large enough,
the overall expression within the integral, e [ t] C ( )1 1 ; falls in :
This problem has been discovered a relatively long time ago and received renewed
attention in the 90s when the new growth theory was being developed. A more general
treatment of this problem was undertaken by von Weizsäcker (1965) who compares
utility levels in unbounded circumstances by using “overtaking criteria”.

5.5 Examples
This section presents various examples that show both how to compute optimality
conditions and how to understand the predictions of the optimality conditions. We
will return to a phase-diagram analysis and understand the meaning of …xed and free
values of state variables at the end points.

5.5.1 Free value at end point - Adjustment costs

Let us look at a …rm that operates under adjustment costs. Capital can not be rented
instantaneously on a spot market but its installation is costly. The crucial implication
of this simple generalization of the standard theory of production implies that …rms
“all of a sudden”have an intertemporal and no longer a static optimization problem.
As one consequence, factors of production are then no longer paid their value marginal
product as in static …rm problems.

The model

A …rm maximizes the present value 0 of its future instantaneous pro…ts (t),
Z T
rt
0 = e (t) dt; (5.21)
0

subject to a capital accumulation constraint

K_ (t) = I (t) K (t) : (5.22)

Gross investment I (t) minus depreciation K (t) gives the net increase of the …rm’s
capital stock. Instantaneous pro…ts are given by the di¤erence between revenue and
cost,
(t) = pF (K (t)) (I (t)) :
Revenue is given by pF (K (t)) where the production technology F (:) employs capital
only. Output increases in capital input, F 0 (:) > 0; but to a decreasing extent, F 00 (:) <

83
0. Costs of …rm are given by the cost function (I (t)) : As the …rm owns capital, it
does not need to pay any rental costs for capital. The only costs that are captured
by (:) are adjustment costs. They include both the cost of buying and of installing
capital. The initial capital stock is given by K (0) = K0 . The interest rate r is
exogenous to the …rm.

Solution

The current value Hamiltonian reads

H = pF (K (t)) (I (t)) + (t) [I (t) K]

and optimality conditions are

0
HI = (I (t)) + (t) = 0; (5.23)
_ (t) = r HK (t) = r pF (K (t)) + 0

0
= (r + ) pF (K (t)) ; (5.24)
0
(T ) = 0 , (I (T )) = 0:

The optimality condition for shows that the value marginal product of capital,
pF 0 (K (t)) ; still plays a role and it is still compared to the rental price r of capital,
but there is no longer an equality as in static models of the …rm. We will return to
this point in exercise 2 of ch. 6.
Optimality conditions can be presented in a simpler way (i.e. with less endogenous
variables). First, solve the …rst optimality condition for the costate variable and
compute the time derivative,
_ (t) = 00
(I (t)) I_ (t) :

Second, insert this into the second optimality condition (5.24) to …nd
00
(I (t)) I_ (t) = (r + ) 0
(I (t)) pF 0 (K (t)) : (5.25)

This equation, together with the capital accumulation constraint (5.22), is a two-
dimensional di¤erential equation system that can be solved, given initial conditions
and some boundary condition. As a solution to a di¤erential equation system is
usually a path of time variant variables, this shows that with adjustment costs, …rms
indeed have an intertemporal problem.

An example

Now assume adjustment costs are of the form

(I) = vI + I 2 =2: (5.26)

The price to be paid per unit of capital is given by the constant v and costs of
installation are given by I 2 =2: This quadratic term captures the idea that installation

84
costs are low and do not increase fast, i.e. underproportionally to the new capital
stock, at low levels of I but increase overproptionally when I becomes large. Then,
optimality requires (5.22) and, from inserting (5.26) into (5.25),

I_ (t) = (r + ) (v + I (t)) pF 0 (K (t)) (5.27)

A phase diagram using (5.22) and (5.27) is plotted in the following …gure.

I
K& = 0

I& = 0

K0 K

Figure 26 A …rm with adjustment cost

We now have to select one of this in…nity of paths that start at K0 : What is the
correct investment level I0 ? Given that the value of the state variable at the end
point is freely chosen, the optimal path satis…es
0
(T ) = 0 , (I (T )) = 0 , I (T ) = 0:

Hence, the trajectory where the investment level is zero at T is the optimal one.

5.5.2 Fixed value at end point - Adjustment costs II

To illustrate this problem, consider the example of the last section where we require
a certain stock of capital at T . The phase diagram then looks identical. The higher
the endpoint stock of capital, the more you have to invest. If it is relatively low but
time is long, it might be advisable to …rst approach the steady state and then “fall
back“ to capital stock KT .
Note the turnpike theorem in this context!

85
5.5.3 In…nite horizon - Optimal consumption paths
Let us now look at an example with in…nite horizon. We focus on optimal behaviour
of a consumer. The classic reference is Ramsey (1928) or Arrow and Kurz (1969).
The problem can be posed in at least two ways. In either case, one part of the
problem is the intertemporal utility function
Z 1
U (t) = e [ t] u (c ( )) d : (5.28)
t

Due to the general instantaneous utility function u (c ( )), it is somewhat more general
than e.g. (5.1). The second part of the maximization problem is a constraint limiting
the total amount of consumption. Without such a constraint, maximizing (5.28)
would be trivial (or meaningless): With u0 (c ( )) > 0 maximizing the objective simply
means setting c ( ) to in…nity. The way this constraint is expressed determines in
which way the problem is most straightforwardly solved.

Solving by Lagrange

The constraint to (5.28) is given by a budget constraint. The …rst way in which
this budget constraint can be expressed is, again, the intertemporal formulation,
Z 1 Z 1
Dr ( ) E ( ) d = a (t) + Dr ( ) w ( ) d ; (5.29)
t t
R
where E ( ) = p ( ) c ( ) and Dr ( ) = e t r(u)du : The maximization problem is then
given by: maximize (5.28) by choosing a path fc( )g subject to the budget constraint
(5.29).
We build the Lagrangean with as the time-independent Lagrange multiplier
Z 1
L= e [ t] u (c ( )) d
t
Z 1 Z 1
Dr ( ) E ( ) d a (t) Dr ( ) w ( ) d :
t t

Note that in contrast to section 5.2.1 where a continuum of constraints implied a con-
tinuum of Lagrange multipliers (or, in an alternative interpretation, a time-dependent
multiplier) there is only one constraint here.
The …rst-order conditions are (5.29) and the partial derivative with respect to
consumption c ( ) at one speci…c point in time,
[ t] 0
Lc( ) = e u (c ( )) Dr ( ) p ( ) = 0
1 [ t] 1 1
, Dr ( ) e p( ) = u0 (c ( )) :

Note that this …rst order condition represents an in…nite number of …rst order con-
ditions: one for each point in time between t and in…nity. The integral is not part
of the …rst-order condition as maximization takes place seperatly for each . Note

86
the analogy to maximizing a sum as e.g. in (3.4). The integration variable here
corresponds to the summation index in (3.4). When one speci…c point in time is
chosen (say = 17), all derivatives of other consumption levels with respect to this
speci…c c are zero.
Applying logs yields
Z
r (u) du [ t] ln p ( ) = ln ln u0 (c ( )) :
t

Di¤erentiating with respect to time gives the Keynes-Ramsey rule

u00 (c ( )) p_ ( )
c_ ( ) = r ( ) : (5.30)
u0 (c ( )) p( )
00
The Lagrange multiplier drops out as it is a constant. Note that uu0 (c((c( ))
))
is Arrow’s
measure of absolute risk aversion which is a measure of the curvature of the utility
function.
With a logarithmic utility function, u (c ( )) = ln c ( ), u0 (c ( )) = c(1 ) and
u00 (c ( )) = c( 1)2 and
c_ ( ) p_ ( )
= r( ) : (5.31)
c( ) p( )
Employing the Hamiltonian

In contrast to above, the utility function (5.28) here is maximized subject to the
dynamic (or ‡ow) budget constraint,

p (t) c (t) + a_ (t) = r (t) a (t) + w (t) :

The solution is an exercise.

The interest rate e¤ect on consumption

As an application for the methods we got to know so far, imagine there is a

sudden discovery of a new technology in an economy. All of a sudden, computers
or cell-phones or the internet is available on a large scale in the economy. Imagine
further that this implies a sudden increase in the returns to investment (i.e. the
interest rate r): For any Euro invested, more comes out than before the discovery of
the new technology. What is the e¤ect of this discovery on consumption? To ask this
more precisely: what is the e¤ect of a change in the interest rate on consumption?
To answer this question, we study a maximization problem as the one just solved,
i.e. the objective function is (5.28) and the constraint is (5.29). We simplify the
maximization problem, however, by assuming a a CRRA utility function u (c ( )) =
c ( )1 1 = (1 ) ; a constant interest rate and a price being equal to one (imag-
ine the consumption good is the numeraire). This implies that the budget constraint
reads Z 1 Z 1
r[ t]
e c ( ) d = a (t) + e r[ t] w ( ) d (5.32)
t t

87
and the Keynes-Ramsey rule becomes, from inserting the CRRA u (c ( )) into (5.30),

c_ ( ) r
= : (5.33)
c( )

One e¤ect, the growth e¤ect, is straightforward from (5.33) or also from (5.30). A
higher interest rate, ceteris paribus, increases the growth rate of consumption. The
second e¤ect, the e¤ect on the level of consumption, is less obvious, however. In order
to understand it, we undertake the following steps. This is in principle similar to the
analysis of level e¤ects in ch. 3.5.2.
First, we solve the linear di¤erential equation in c ( ) given by (5.33). Following
ch. 4.3, we …nd
r
c ( ) = c (t) e ( t) : (5.34)
Consumption, starting today in t with a level of c (t) grows exponentially over time
at the rate (r ) = to reach the level c ( ) at some future > t:
In the second step, we insert this solution into the left-hand side of the budget
constraint (5.32) and …nd
Z 1 Z 1 h i1
r r 1 r
e r[ t]
c (t) e ( t)
d = c (t) e (r )[ t] d = c (t)
r e (r )[ t]
:
t t r t

The simpli…cation stems Rfrom the fact that c (t) ; the initial consumption level, can
1
be pulled out of the term t e r[ t] c ( ) d ; representing the present value of current
and future consumption expenditure. Please note that c (t) could be pulled out of
the integral also in the case of a non-constant interest rate. Note also that we do not
need to know what the level of c (t) is, it is enough to know that there is some c (t)
in the solution (5.34), whatever its level.
With a constant interest rate, the remaining integral can be solved explicitely.
First note that r r must be negative. Otherwise consumption growth would
exceed the interest rate and a boundedness condition for the objective function like in
ch. 5.4.3 would eventually be violated. Hence, we assume r > r , (1 )r < :
Clearly, this holds in a steady state where r = : Therefore, we obtain for the present
value of consumption expenditure

1 h r
i1 c (t) c (t)
c (t) r e (r )[ t]
= r =
r t r (1 )r

and, inserted into the budget constraint, this yields

Z 1
(1 )r r[ t]
c (t) = a (t) + e w( )d : (5.35)
t

For the special case of a logarithmic utility function, the fraction in front of the curly
brackets simpli…es to (as = 1), a result obtained e.g. by Blanchard (1985).
After these two steps, we have two results, both visible in (5.35). One result
shows that initial consumption c (t) is a fraction out of wealth of the household.

88
Wealth needs to be understood in a more general sense than usually, however: It
is …nancial wealth a (t) plus, what could be called humanR wealth (in an economic,
1
i.e. material sense), the present value of labour income, t e r[ t] w ( ) d : Going
beyond t today and realizing that this analysis can be undertaken for any point in
time, the relationship (5.35) of course holds on any point of an optimal consumption
path. The second result is a relationship between the level of consumption and the
interest rate, our original question.
We now need to understand the derivative dc (t) =dr in order to exploit (5.35)
more. If we focus only on the term in front of the curly brackets, we …nd for the
change in the level of consumption when the interest rate changes

dc (t) 1
= f:g R 0 , R 1:
dr
The consumption level increases when the interest rate rises if is larger than one, i.e.
if the intertemporal elasticity of substitution 1 is smaller than unity. This is prob-
ably the empirically more plausible case (compared to < 1) on the aggregate level.
See however Vissing-Jørgensen (2002) for micro-evidence where the intertemporal
elasticity of substitution can be much larger than unity. This …nding is summarized
in the following …gure.
N r2 > r 1
lnc(t)

c2 (t) r1

c2 (t) >1
N

t time

Figure 27 An interest rate change and consumption growth and level for >1

5.5.4 The central planner and capital accumulation

The setup

This example studies the classic central planner problem: First, there is a social
welfare function like (5.1), expressed slightly more generally as
Z 1
max e [ t] u (C ( )) d :
fC( )g t

89
The generalization consists in the in…nite planning horizon and the general instanta-
neous utility function u (c ( )) : We will specify it in the most common version,

C1 1
u (C) = ; (5.36)
1
1
where the intertemporal elasticity of substition is constant and given by : The
00 0
coe¢ cient of relative risk aversion (de…ned by u (C) C=u (C)) is also constant and
given by : While the latter aspect is of importance only in models with uncertainty,
it will turn out that this close relationship between this measure of risk-aversion and
the intertemporal elasticity of substitution is not always desirable. A way out are
Epstein-Zin preferences.
Second, there is a resource constraint that requires that net capital investment is
given by the di¤erence between output Y (K; L), depreciation K and consumption
C;
K_ (t) = Y (K (t) ; L) K (t) C (t) : (5.37)
Assuming for simplicity that the labour force L is constant, this completely describes
this central planner problem.
The choice variable of the planner is the consumption level C (t) ; to be determined
for each point in time between today t and the far future 1. The fundamental trade-
o¤ consists in the utility increasing e¤ect of more consumption visible from (5.36) and
the net-investment descreasing e¤ect of more consumption visible from the resource
constraint (5.37). As less capital implies less consumption possibilities in the future,
the trade-o¤ can also be described to consist in more consumption today vs. more
consumption in the future.

The solution

Let us solve this problem by employing the Hamiltonian

H = u (C (t)) + (t) [Y (K (t) ; L) K (t) C (t)] :

First order conditions are

u0 (C (t)) = (t) ; (5.38)

_ (t) = @H
(t) = (t) (t) [YK (K (t) ; L) ]:
@K
Di¤erentiating the …rst-order condition (5.38) with respect to time gives u00 (C (t)) C_ (t) =
_ (t) : Inserting this and (5.38) again into the second condition gives with same rear-
ranging
u00 (C (t)) _
C (t) = YK (K (t) ; L) :
u0 (C (t))
This is almost identical to the optimality rule we obtained on the individual level
in (5.30). The only di¤erence consists in capital C instead of c and YK (K (t) ; L)

90
instead of r ( ) p(
_ )
p( )
: Instead of the real interest rate on the household level, we have
here the marginal productivity of capital minus depreciation on the aggregate level.
If we assumed a logarithmic instantaneous utility function, u00 (C (t)) =u0 (C (t)) =
1=C (t) and the Keynes-Ramsey rule would look similar to (5.5) or (5.31). In our case
of the more general speci…cation in (5.36), we …nd u00 (C (t)) =u0 (C (t)) = =C (t)
such that the Keynes-Ramsey rule reads

C_ (t) YK (K (t) ; L)
= : (5.39)
C (t)
This could be called the classic result on optimal consumption in general equilib-
rium. Consumption grows if marginal productivity of capital exceeds the sum of the
depreciation rate and the time preference rate. The higher the intertemporal elas-
ticity of substition 1= ; the stronger consumption growth reacts to the di¤erences
YK (K (t) ; L) :

A phase diagram analysis

The resource constraint of the economy in (5.37) plus the Keynes-Ramsey rule
in (5.39) represent a two-dimensional di¤erential equation system which, given two
boundary conditions, give a unique solution for time paths C (t) and K (t) : These
two equations can be analyzed in a phase diagram. This is probably the most often
taught phase diagram in economics.
N
C C_ = 0
N

6 ?

N
N

C B K_ = 0
N

-
A ?
6
-
N

K0 K
Figure 28 Optimal central planner consumption

Zero-motion lines for capital and labour, respectively, are given by

K_ (t) 0 , C (t) Y (K (t) ; L) K (t) ; (5.40)

C_ (t) 0 , YK (K (t) ; L) + ; (5.41)

when the inequality signs hold as equalities. Zero motion lines are plotted in the
above …gure.
When consumption lies above the Y (K (t) ; L) K (t) line, (5.40) tells us that
capital increases, below this line, it decreases. When the marginal productivity of

91
capital is larger than + ; i.e. when the capital stock is su¢ ciently small, (5.41) tells
us that consumption increases. These laws of motion are plotted in …gure 28 as well.
This allows us to draw trajectories A; B and C which all satisfy (5.40) and (5.41).
Hence, again, we have a multitude of solutions for a di¤erential equation system.
As always, boundary conditions allow to “pick”the single solution to this system.
One boundary condition is the initial value K0 of the capital stock. The capital stock
is a state variable and therefore historically given at each point in time. It can not
jump. The second boundary condition should …x the initial consumption level C0 :
Consumption is a control or choice variable and can therefore jump or adjust to put
the economy on the optimal path.
The condition which provides the second boundary condition is, formally speaking,
the transversality condition (see ch. 5.4.2). Taking this formal route, one would have
to prove that starting with K0 at the consumption level that puts the economy on
path A would violate the TVC. Similarily, it would have to be shown that path C or
any path other than B violates the TVC as well. Even though not often admitted, this
is not often done in practice. (For an exception, see exercise 5 in ch. 5.) Whenever
a saddle path is found in a phase diagram, it is argued that the saddle path is the
equilibrium path and the initial consumption level is such that the economy …nds itself
on the path which approaches the steady state. While this is a practical approach, it
is also formally satis…ed as this path satis…es the TVC indeed.

5.5.5 The matching approach to unemployment

Matching functions are widely used in e.g. labour economics and monetary economics.
We will present here the background for their use in labour market modeling, similar
to Pissarides (2000, ch. 1).

Aggregate unemployment

The unemployment rate in an economy is governed by two factors: the speed with
which new employment is created and the speed with which existing employment is
destroyed. The number of new matches per unit of time dt is given by a matching
function. It depends on the number of unemployed U , i.e. the number of those
potentially available to …ll a vacancy, and the number of vacancies V , m = m (U; V ).
The number of …lled jobs that are destroyed per unit of time is given by the product
of the separation rate s and employment, sL. Combining both components, the
evolution of the number of unemployed over time is given by

U_ = sL m (U; V ) : (5.42)

De…ning employment as the di¤erence between the size of the labour force N and the
number of unemployed U , L = N U , we obtain

U_ = s [N U] m (U; V ) , u_ = s [1 u] m (u; V =N ) ;

where we divided by the size of the labour force N in the last step to obtain the
unemployment rate u U=N: We also assumed constant returns to scale in the

92
matching function m (:) : Note that the matching function can further be rewritten
as
V V V U
m u; = m 1; u= m ;1 u q( )u (5.43)
N U U V
and we obtain u_ = s [1 u] q ( ) u which is equation (1.3) in Pissarides (2000).
Clearly, from (5.42), one can easily obtain the evolution of employment, using
again the de…nition L = N U;

L_ = m (N L; V ) sL: (5.44)

Firm behaviour

Given this matching process, the choice variable of a …rm i is no longer employment
Li but the number of vacancies Vi it creates. Each vacency implies costs measured
in units of the output good Y: The …rm also employs capital Ki : Hence, the …rm’s
optimization problem reads
Z 1
max e r[ t] fY (Ki ; Li ) rKi wLi Vi g d
fVi ( );Ki ( )g t

subject to
Vi
L_ i = m (N sLi :
L; V )
V
The constraint now says - in contrast to (5.44) - that only a certain share of all
matches goes to the …rm under consideration and that this share is given by the
share of the …rm’s vacancies in total vacancies, Vi =V . Alternatively, this says that
the “probability” that a match in the economy as a whole …lls a vacency of …rm i
is given by the number of matches in the economy as a whole divided by the total
number of vacancies. As under constant returns to scale for m and using the de…nition
of q ( ) implicity in (5.43), m (N L; V ) =V = m (U=V; 1) = q ( ) ; we can write the
…rm’s constraint as
L_ i = q ( ) Vi sLi :
Assuming small …rms, this rate can q ( ) safely be assumed to be exogenous for the
…rm’s maximization problem.
The current-value Hamiltonian for this problem reads

H = Y (Ki ; Li ) rKi wLi Vi + i [q ( ) Vi sLi ] ;

and …rst-order conditions are

HKi = YKi (Ki ; Li ) r = 0; HVi = + iq ( ) = 0;

_ i = r i HL = r i (YL (Ki ; Li ) w i s) = (r + s) i YLi (Ki ; Li ) + w:
i i

The …rst condition for capital is the usual marginal productivity condition applied
to capital input. With CRTS production functions, this condition …xes the capital
to labour ratio for each …rm. This implies that the marginal product of labour is a

93
function of the interest only and therefore identical for all …rms, independent of the
labour stock, i.e. the size of the …rm. This changes the third condition to
_ = (r + s) YL (K; L) + w (5.45)

which means that the shadow prices are identical for all …rms.
The …rst-order condition for vacencies, written as = q ( ) says that the mar-
ginal costs of a vacancy (which in this special case equal average and unit costs)
must be equal to revenue from a vacancy. This expected revenue is given by the
share q ( ) of vacencies that yield a match times the value of a match. The value of
a match to the …rm is given by ; the shadow price of labour. The link between
and the value of an additional unit of labour (or of the state variable, more generally
speaking) is analyzed in ch. 6.2.

General equilibrium

It appears as if the …rst-order condition for vacencies was independent of the

number of vacancies opened by the …rm. In fact, given this structure, the individual
…rm follows a bang-bang policy. Either the optimal number of vacencies is zero or
in…nity. In general equilibrium, however, a higher number of vacancies, decreases the
share q ( ) of existing vacancies that get …lled. Hence, this second condition holds in
general equilibrium and we can compute
_= = q_ ( ) =q ( ) :

The third condition (5.45) can then be written as

q_ ( ) q( )
=r+s [YL (K; L) w] (5.46)
q( )

which is an equation depending on the number of vacancies and employment only.

This equation together with (5.44) is a two-dimensional di¤erential equation system
which determines V and L; provided there are 2 boundary conditions for V and L
and wages and the interest rate are exogenously given. (In a more complete general
equilibrium setup, wages would be determined e.g. by Nash-bargaining. For this, we
need to derive value functions which is done in ch. 10.2).

An example for the matching function

Assume the matching function has CRTS and is of the CD type, m = U V :

q(
_ ) m
_ V_
As q ( ) = m (N L; V ) =V; it follws that q( )
= m V
: Given our Cobb-Douglas
assumption, we can write this as

q_ ( ) U_ V_ V_ U_ V_
= + = (1 ) :
q( ) U V V U V

94
Hence, the vacency equation (5.46) becomes

U_ V_ q( )
+ (1 ) =r+s [YL (K; L) w] ,
U V
V_ (N L) L V
(1 ) =r+s [YL (K; L) w] + s
V V 1 N L (N L)1

where the last equality used q ( ) = (N L) V 1 and (5.42) with m = U V :

Again, this equation with (5.44) is a two-dimensional di¤erential equation system
which determines V and L; just as just described in the general case.

5.6 The present value Hamiltonian

5.6.1 Problems without (or with implicit) discounting
The problem and its solution

Let the maximization problem be given by

Z T
max F (y ( ) ; z ( ) ; ) d (5.47)
z( ) t

subject to
y_ ( ) = Q (y ( ) ; z ( ) ; ) (5.48)
y (t) = yt ; y (T ) free (5.49)
where y ( ) 2 Rn ; z ( ) 2 Rm and Q = (Q1 (y ( ) ; z ( ) ; ) ; Q2 (:) ; ::: ; Qn (:))T : A
feasible (zulässig) path is a pair (y ( ) ; z ( )) which satis…es (5.48) and (5.49). z( )
is the vector of control variables, y( ) is the vector of state variables.
Then de…ne the (present-value) Hamiltonian H as

HP = F ( ) + ( ) Q ( ) ; (5.50)

where ( ) is the costate variable,

( )=( 1 ( ); 2 ( ) ; ::: ; n ( )) (5.51)

Necessary conditions for an optimal solution are

HzP = 0; (5.52)

_( )= HyP ; (5.53)
and
(T ) = 0;
(Kamien and Schwartz, p. 126) in addition to (5.48) and (5.49). In order to have a
maximum, we need second order conditions Hzz < 0 to hold (Kamien and Schwartz).

95
 Understanding its structure

The …rst m equations (5.52) (z ( ) 2 Rm ) solve the control variables z( ) as a

function of state and costate variables, y( ) and ( ). Hence

z( ) = z(y( ); ( )):

The next 2n equations (5.53) and (5.48) (y (t) 2 Rn ) constitute a 2n dimensional

di¤erential equation system. In order to solve it, one needs, in addition to the n initial
conditions given exogenously for the state variables by (5.49), n further conditions
for the costate variables. These are given by boundary value conditions (T ) = 0:
Su¢ ciency of these conditions results from theorem as e.g. in section 5.3.3.

5.6.2 Deriving laws of motion

As in the section on the current-value Hamiltonian, a ”derivation”of the Hamiltonian
starting from the Lagrange function can be given for the present value Hamiltonian
as well.

The maximization problem

Let the objective function be (5.47) that is to be maximized subject to the con-
straint (5.48) and, in addition, a static constraint

G (y ( ) ; z ( ) ; ) = 0: (5.54)

This maximization problem can be solved by using a Lagrangian (cf. Intriligator,

ch. 14. A useful starting point is the derivation of the Lagrangian for the static case;
Intriligator, p. 28 - 30.) The Lagrangian reads
Z T
L= F (:) + ( ) (Q (:) y_ ( )) ( ) G (:); t) d
t
Z T Z T
= F (:) + ( ) Q (:) ( ) G (:) d ( ) y_ ( ) d
t
Rb Rb
Using the integrating by parts rule a xydt _ = a
_ + [xy]ba ; derived from inte-
xydt
0
grating (uv) = u0 v + uv 0 ; integrating the last expression gives
Z T Z T
( ) y_ ( ) dt = _ ( ) y ( ) d + [ ( ) y ( )]T
t

and the Lagrangian reads

Z T
L= F (:) + ( ) Q (:) + _ ( ) y ( ) ( ) G (:) d
t
[ ( ) y ( )]Tt :

96
 This is now maximized with respect to y ( ) and z ( ), both the control and the
state variable. We then obtain conditions that are necessary for an optimum.

Fz (:) + ( ) Qz (:) ( ) Gz (:) = 0 (5.55)

Fy (:) + ( ) Qy (:) + _ ( ) ( ) Gy (:) = 0 (5.56)

The last …rst-order condition can be rearranged to

_( )= ( ) Qy (:) Fy (:) + ( ) Gy (:) (5.57)

These necessary conditions will be the one used regularly in maximization problems.

The shortcut

As it is cumbersome to start from a Lagrangian for each dynamic maximization

problem, one can de…ne the Hamiltonian as a shortcut as

H P = F (:) + ( ) Q (:) ( ) G (:) : (5.58)

Optimality conditions are then

HzP = 0; (5.59)
_( )= HyP : (5.60)
which are the same as (5.55) and (5.57) above and (5.52) and (5.53) in the last section.

5.6.3 The link between CV and PV

If we solve the problem (5.6) and (5.7) via the present value Hamiltonian, we would
start from
H P ( ) = e [ t] G (:) + ( ) Q (:) (5.61)
…rst-order conditions would be
(T ) = 0; (5.62)
@H P
= e [ t] Gz (:) + Qz (:) = 0; (5.63)
@z
@H P
_( )= = e [ t] Gy (:) Qy (:) (5.64)
@y
and we would be done with solving this problem.

Simpli…cation

We can simplify the presentation of …rst-order conditions, however, by rewriting

(5.63) as
Gz ( ) + ( ) Qz ( ) = 0 (5.65)
where we de…ned
[ t]
( )=e ( ): (5.66)

97
Note that the argument of the costate variables is always time (and not time t).
When we use this de…nition in (5:64), this …rst-order condition reads
[ t]
e _( )= Gy (:) Qy (:) :

Replacing the left-hand side by

e [ t]
_( )= _( ) e [ t]
( )= _( ) ( ):

which follows from computing the time derivative of the de…nition (5.66), we get
_ = Gy ( )Qy : (5.67)

Inserting the de…nition (5.66) into the Hamiltonian (5.61) gives

[ t]
e H P (t) = G (:) + Q (:) < > H c (t) = G (:) + Q (:)

which de…nes the link between the present value and the current value Hamiltonian
as
H c (t) = e [ t] H P (t)

Summary

Hence, instead of …rst-order condition (5.63) and (5.64), we get (5.65) and (5.67).
As just shown these …rst-order conditions are equivalent.

5.7 Exercises chapter 5 and further reading

There are many texts that treat Hamiltonians as a maximization device. Some exam-
ples include Dixit (1990, p. 148), Intriligator (1971), Kamien and Schwartz (1991),
Leonard and Long (1992) or, in German, Feichtinger und Hartl (1986).

98
 Exercises chapter 5
Applied Intertemporal Optimization
Hamiltonians

1. Optimal consumption over an in…nite horizon

Solve the maximization problem
Z 1
max e [ t] ln c ( ) d ;
c( ) t

subject to
p (t) c (t) + A_ (t) = r (t) A (t) + w (t) :
by

(a) using the present-value Hamiltonian. Compare the result to (5.31).

(b) Use u (c ( )) instead of ln c ( ).

2. Adjustment costs
Solve the adjustment cost example for

(I) = I:

What do optimality conditions mean? What is the optimal end-point value for
K and I?
3. Consumption over the life cycle
Utility of an individual, born at s and living for T periods is given at time t by
Z s+T
u (s; t) = e [ t] ln (c (s; )) d :
t

The individuals budget constraint is given by

Z s+T
DR ( ) c (s; ) d = h (s; t) + a (s; t)
t

where
Z Z s+T
DR ( ) = exp r (u) du ; h (s; t) = Dr ( ) w (s; ) d :
t t

This deplorable individual would like to know how it can lead a happy life but,
unfortunately, has not studied optimal control theory!

99
 (a) What would you recommend him? Use a Hamiltonian approach and dis-
tinguish between changes of consumption and the initial level. Which in-
formation do you need to determine the initial consumption level? What
information would you expect this individual to provide you with? In other
words, which of the above maximization problems makes sense? Why not
the other one?
(b) Assume all prices are constant. Draw the path of consumption in a (t,c(t))
diagram. Draw the path of asset holdings a(t) in the same diagram, by
guessing how you would expect it to look like. (You could compute it if
you want)

4. Optimal consumption

(a) Derive the optimal allocation of expenditure and consumption over time
for
c ( )1 1
u (c ( )) = ; >0
1
by employing the Hamiltonian.
(b) Show that this function includes the logarithmic utility function for =1
(apply L’Hospital’s rule).
(c) Does the utility function u(c(t)) make sense for > 1? Why (not)?

5. A central planner
(Rebelo 1990 AK-Model)
You are responsible for the future well-being of 360 Million Europeans and
centrally plan the EU by assigning a consumption path to each inhabitant.
Your problem consists in maximizing a social welfare function of the form
Z 1
i
U (t) = e [ t] C 1 1 (1 ) 1d
t

subject to the EU resource constraint

K_ = BK C (5.68)

(a) What are optimality conditions? What is the consumption growth rate?
(b) Under which conditions is the problem well de…ned (boundedness condi-
tion)? Insert the consumption growth rate and show under which condi-
tions the utility function is bounded.
(c) What is the growth rate of the capital stock? Compute the initial con-
sumption level, by using the No-Ponzi-Game condition (5.20).
(d) Under which conditions could you resign from your job without making
anyone less happy than before?

100
6. Optimal consumption levels

(a) Derive a rule for the optimal consumption level for a time-varying interest
rate r (t) : Show that (5.35) can be generalized to
1
c (t) = R 1 R (1 )r(s)
fW (t) + a (t)g ;
ds
t
e t d

where W (t) is human wealth.

(b) What does this imply for the wealth level a (t)?

7. Investment and the interest rate

(a) Use the result of 5 c) and check under which conditions investment is a
decreasing function of the interest rate.
(b) Perform the same analysis for a budget constraint a_ = ra + w c instead
of (5.68).

8. The Ramsey growth model (Cf. Blanchard and Fischer, ch. 2)

Let the resource constraint now be given by

K_ = Y (K; L) C K:

Draw a phase diagram. What is the long-run equilibrium? Perform a stability

analysis graphically and analytically (locally). Is the utility function bounded?

9. Exam question (winter term 2005/06, Würzburg)

Betrachten Sie eine dezentrale (kein Planerproblem) Ökonomie in kontinuier-
licher (nicht diskreter) Zeit. Die Ökonomie ist mit Kapital und Arbeit ausges-
tattet. Arbeit ist …x gegeben durch L, der anfängliche Kapitalbestand ist K0 .
Es besteht die Möglichkeit zur Kapitalakkumulation K_ ( ) = I ( ) K ( ).
Kapital ist der einzige Vermögensgegenstand, d.h. Haushalte können nur durch
Kapitalerwerb sparen. Haushalte haben eine dazu passende Budgetrestriktion
und eine übliche intertemporale Nutzenfunktion mit Zeitpräferenzrate und un-
endlichem Zeithorizont. Firmen produzieren unter vollständigem Wettbewerb.
Beschreiben Sie eine solche Ökonomie formal und leiten Sie die reduzierte Form
her. Folgen Sie dabei der folgenden Struktur:

(a) Wählen Sie eine übliche Technologie.

(b) Bestimmen Sie die optimalen Faktornachfragen.
(c) Die Budgetrestriktion der Haushalte sei gegeben durch

a_ ( ) = r ( ) a ( ) + wL ( ) c( ):

Formulieren Sie das Maximierungsproblem der Haushalte und lösen Sie es.

101
(d) Aggregieren Sie über die Haushalte und beschreiben Sie dadurch die opti-
male Konsumentwicklung aggregierten Konsums.
(e) Formulieren Sie das Gleichgewicht auf dem Gütermarkt.
(f) Zeigen Sie die Konsistenz der Budgetrestriktion der Haushalte mit der
Kapitalakkumulationsgleichung der Ökonomie.
(g) Leiten Sie die reduzierte Form

K_ ( ) = Y ( ) C( ) K( )
@Y ( )
C_ ( ) @K( )
=
C( )
durch Befolgen dieser Schritte her und erläutern Sie diese verbal.

102
6 Dynamic programming in continuous time
This chapter reanalyzes maximization problems in continuous time that are known
from the chapter on Hamiltonians. It shows how to solve them with dynamic pro-
gramming methods. The sole objective of this chapter is to present the dynamic
programming method in a well-known deterministic setup such that its use in a sto-
chastic world in subsequent chapters becomes more accessible.

6.1 Household utility maxmization

We consider a maximization problem that is very similar to the introductory example
for the Hamiltonian in section 5.1 or the in…nite horizon case in section 5.4. Compared
to 5.1, the utility function here is more general and the planning horizon is in…nity.
None of this is important, however, for understanding the di¤erences in the approach
between the Hamiltonian and dynamic programming.

The maximization problem

Utility of the individual is given by

Z 1
[ t]
U (t) = e u (c ( )) d : (6.1)
t

Her budget constrained equates wealth accumulation with savings,

a_ = ra + w pc: (6.2)

As in models of discrete time, the value V (a (t)) of the optimal program is de…ned
by V (a (t)) maxfc( )g U (t) subject to (6.2). When households behave optimally
between today and in…nity by choosing the optimal consumption path fc ( )g ; their
overall utility U (t) is given by V (a (t)) :

A prelude on the Bellman equation

The derivation of the Bellman equation is not as obvious as under discrete time
as under continuous time, even though we do have a today t, we do not have a
clear tommorrow (like a t + 1 in discrete time). We therefore need to construct a
tommorrow by adding a “small” time intervall t to t: Tommorrow would then be
t + t: Note that this derivation is heuristic and more rigorous approaches exist. See
e.g. Sennewald (2006) for further references to the literature.
Following Bellman’s idea, we rewrite the objective function as the sum of two
subperiods,
Z t+ t Z 1
[ t]
U (t) = e u (c ( )) d + e [ t] u (c ( )) d ;
t t+ t

where t is a “small” time intervall. When we approximate the …rst integral (think
of the area below the function u (c ( )) plotted over time ) by u (c (t)) t and the

103
discounting between t and t + t by 1+1 t and we assume that as of t + t we behave
optimally, we can rewrite the value function V (a (t)) = maxfc( )g U (t) as
1
V (a (t)) = max u (c (t)) t + V (a (t + t)) :
c(t) 1+ t
By doing so, we are left with only one choice variable c (t) instead of the entire path
fc ( )g :When we …rst multiply this expression by 1 + t, then divide by t and
V (a(t))
…nally move
n t
to the right hand side, weoget V (a (t)) =
V (a(t+ t)) V (a(t))
maxc(t) u (c (t)) [1 + t] + t
: Taking the limit lim t!0 gives the
Bellman equation,
dV (a (t))
V (a (t)) = max u (c (t)) + : (6.3)
c(t) dt
This equation again shows Bellman’s trick: A maximization problem, consisting
of the choice of a path of a choice variables, was broken down to a maximization
problem where only the level of the choice variable in t has to be chosen. The
structure of this equation can also be understood from a more intuitive perspective:
The term V (a (t)) can best be understood when comparing it to rK; capital income
at each instant of an individual who owns a capital stock K and the interest rate
is r: A household that behaves optimally “owns” the value V (a (t)) from optimal
behaviour and receives a utility stream of V (a (t)) : This “utility income” at each
instant is given by instantaneous utility from consumption plus the change in the
value of optimal behaviour. Note that this structure is identical to the capital-market
no-arbitrage condition (4.19), r (t) v (t) = (t) + v_ (t) - the capital income stream
from holding wealth v (t) on a bank account is identical to dividend payments (t)
plus the change v_ (t) in the market price when holding the same level of wealth in
stocks.
While the derivation just shown is the standard one, we will now present an
alternative approach which illustrates the economic content of the Bellman equation
and which is more straightforward. Given the objective function in (6.1), we can ask
how overall utility U (t) changes over time. To this end, we compute the derivative
dU (t) =dt and …nd (using the Leibniz rule 4.9 from ch. 4.3.1)
Z 1
d
U_ (t) = e [t t]
u (c (t)) + e [ t] u (c ( )) d = u (c (t)) + U (t) :
t dt
Overall utility U (t) reduces as time goes by by the amount u (c (t)) at each instant
(as the integral becomes “smaller” when current consumption in t is lost and we
start an instant after t) and increases by U (t) (as we gain because future utilities
come closer to today when today moves into the future). Rearranging this equation
gives U (t) = u (c (t)) + U_ (t) : When overall utility is replaced by the value function,
we obtain V (a (t)) = u (c (t)) + V_ (a (t)) which corresponds in its structure to the
Bellman equation (6.3).

DP1: Bellman equation and …rst-order conditions

104
 We will now follow the three-step procedure to maximization when using the
dynamic programming approach as we got to know it in discrete time setups is section
3.4. When we compute the Bellman equation for our case, we obtain for the derivative
in (6.3) dV (a (t)) =dt = V 0 (a (t)) a_ which gives with the budget constraint (6.2)

V (a (t)) = max fu (c (t)) + V 0 (a (t)) [ra + w pc]g : (6.4)

c(t)

The …rst-order condition reads

u0 (c (t)) = p (t) V 0 (a (t)) (6.5)

and makes consumption a function of the state variable, c (t) = c (a (t)) :

DP2: Evolution of the costate variable

In continuous time, the second step of the dynamic programming approach to

maximization can be subdivided into two steps. (i) In the …rst, we look at the
maximized Bellman equation,

V (a) = u (c (a)) + V 0 (a) [ra + w pc (a)] :

The …rst-order condition (6.5) together with the maximized Bellman equation de-
termine the evolution of the control variable c (t) and V (a (t)). Again, however, the
maximized Bellman equation does not provide very much insight. Computing the
derivative with respect to a (t) however (and using the envelope theorem) gives an
expression for the shadow price of wealth that will be more useful,

V 0 (a) = V 00 (a) [ra + w pc] + V 0 (a) r , (6.6)

( r) V 0 (a) = V 00 (a) [ra + w pc] :

(ii) In the second step, we compute the derivative of V 0 (a) with respect to time,
giving
dV 0 (a)
= V 00 (a) a_ = ( r) V 0 (a) ;
dt
where the last equality used (6.6). Dividing by V 0 (a) and using the usual notation
V_ 0 (a) dV 0 (a) =dt, this can be written as

V_ 0 (a)
= r: (6.7)
V (a)

This equation describes the evolution of the costate variable V 0 (a), the shadow price
of wealth.

DP3: Inserting …rst-order conditions

105
 The derivative of the …rst-order condition with respect to time is given by (apply
…rst logs)
u00 (c) p_ V_ 0 (a)
c
_ = + :
u0 (c) p V 0 (a)
Inserting (6.7) gives

u00 (c) p_ u00 (c) p_

0
c_ = + r () c_ = r :
u (c) p u0 (c) p

This is the well-known Keynes Ramsey rule.

6.2 Comparing dynamic programming to Hamiltonians

When we compare optimality conditions under dynamic programming with those
obtained when employing the Hamiltonian, we …nd that they are identical. Observing
that our costate evolves according to

V_ 0 (a)
= r;
V 0 (a)

as just shown in (6.7), we obtain the same equation as we had in the Hamiltonian
approach for the evolution of the costate variable, see e.g. (5.4) or (5.38). Comparing
…rst-order conditions (5.3) or (5.14) with (6.5), we see that they would be identical if
we had chosen exactly the same maximization problem. This is not surprising, given
our applied view of optimization: If there is one optimal path that maximizes some
objective function, this one path should result, independently of which maximization
procedure is chosen.
A comparison of optimality conditions is also useful for an alternative purpose,
however. As (6.7) and e.g. (5.4) or (5.38) are identical, we can conclude that the
derivative V 0 (a (t)) of the value function with respect to the state variable, in our case
a, is identical to the costate variable in the current-value Hamiltonian approach,
V 0 (a) = : This is where the interpretation for the costate variable in the Hamiltonian
approach in ch. 5.2.2 came from. There, we said that the costate variable stands
for the increase in the value of the optimal program when an additional unit of the
state variable becomes available; this is exactly what V 0 (a) stands for. Hence, the
interpretation of a costate variable is similar to the interpretation of the Lagrange
multiplier in static maximization problems.

6.3 Dynamic programming with two state variables

As a …nal example for maximization problems in continuous time that can be solved
with dynamic programming, we look at a maximization problem with two state vari-
ables. Think e.g. of an agent who can save by putting savings on a bank account or
by accumulating human capital. Or think of a central planner who can increase total
factor productivity or the capital stock. We look here at the …rst case.

106
 Our agent has a standard objective function
Z 1
max U (t) = e [ t] u (c ( )) d :
t

It is maximized subject to two constraints. They describe the evolution of the state
variables wealth a and human capital h;

a_ = f (a; h; c) ; h_ = g (a; h; c) : (6.8)

We do not give explicit expressions for the functions f (:) or g (:) but one can think of
a standard resource constraint for f (:) as in (6.2) and a functional form for g (:) that
captures a trade-o¤ between consumption and human capital accumulation: Human
capital accumulation is faster when a and h are large but decreases in c: To be precise,
we assume that both f (:) and g (:) increase in a and h but decrease in c:

DP1: Bellman equation and …rst-order conditions

In this case, the Bellman equation reads

dV (a; h)
V (a; h) = max u (c) + = max fu (c) + Va f (:) + Vh g (:)g :
dt

There are simply two partial derivatives of the value function after the u (c) term
times the da and dh term, respectively, instead of one as in (6.4), where there is only
one state variable.
Given that there is still only one control variable - consumption, there is only
one …rst-order condition. This is clearly speci…c to this example. One could think
of a time constraint for human capital accumulation (a trade-o¤ between leisure and
learning - think of the Lucas (1988) model) where agents choose the share of their
time used for accumulating human capital. In this case, there would be two …rst-order
conditions. Here, however, we have just the one for consumption, given by

@f (:) @g (:)
u0 (c) + Va + Vh =0 (6.9)
@c @c
When we compare this condition with the one-state-variable case in (6.5), we see
that the …rst two terms u0 (c) + Va @f@c(:) correspond exactly to (6.5): If we had speci…ed
f (:) as in the budget constraint (6.2), the …rst two terms would be identical to (6.5).
The third term Vh @g(:)
@c
is new and stems from the second state variable: Consumption
now not only a¤ects the accumulation of wealth but also the accumulation of human
capital. More consumption gives higher instantaneous utility but, at the same time,
decreases future wealth and - now new - also the future human capital stock.

DP2: Evolution of the costate variables

107
 As always, we need to understand the evolution of the costate variable(s). In
a setup with two state variables, there are two costate variables, or, economically
speaking, a shadow price of wealth a and a shadow price of human capital h: This
is obtained by partially di¤erentiating the maximized Bellman equation, …rst with
respect to a; then with respect to h: Doing this, we get (employing the envelope
theorem right away)

@f (:) @g (:)
Va = Vaa f (:) + Va + Vha g (:) + Vh ; (6.10)
@a @a
@f (:) @g (:)
Vh = Vah f (:) + Va + Vhh g (:) + Vh : (6.11)
@h @h
As in ch. 6.1, the second step of DP2 consists in computing the time derivatives
of the costate variables and in reinserting (6.10) and (6.11). We …rst compute time
derivatives, inserting (6.8) in the last step,

dVa (a; h)
= Vaa a_ + Vah h_ = Vaa f (:) + Vah g (:) ;
dt
dVh (a; h)
= Vha a_ + Vhh h_ = Vha f (:) + Vhh g (:) :
dt
Inserting then (6.10) and (6.11), we …nd

dVa (a; h) @f (:) @g (:)

= Va Va Vh ;
dt @a @a
dVh (a; h) @f (:) @g (:)
= Vh Va Vh :
dt @h @h
The nice feature of this last step is the fact that the cross derivatives Vah and Vha “dis-
appear”, i.e. can be substituted out again, using the fact that fxy (x; y) = fyx (x; y)
for any twice di¤erentiable function f (x; y). Writing these equations as

V_ a @f (:) Vh @g (:)
= ;
Va @a Va @a
V_ h Va @f (:) @g (:)
= ;
Vh Vh @h @h
allows to give an interpretation that links them to the standard one-state case in
(6.7). The costate variable Va evolves as above only that instead of the interest rate,
we …nd here @f@a(:) + VVha @g(:)
@a
: The …rst derivative @f (:) =@a captures the e¤ect a change
in a has on the …rst constraint; in fact, if f (:) represented a budget constraint as in
(6.2), this would be identical to the interest rate. The second term @g (:) =@a captures
the e¤ect of a change in a on the second constraint; by how much would h increase if
there was more a? This e¤ect is multiplied by Vh =Va ; the relative shadow price of h:
An analagous interpretation is possible for V_ h =Vh :

DP3: Inserting …rst-order conditions

108
 The …nal step consists in computing the derivative of the …rst-order condition (6.9)
with respect to time and replacing the time derivatives V_ a and V_ h by the expressions
from the previous step DP2. The principle of how to obtain a solution therefore
remains unchanged when having two instead of one state variable. Unfortunately,
however, it is generally not possible to eliminate the shadow prices from the resulting
equation which describes the evolution of the control variable. Some economically
suitable assumptions concerning f (:) or g (:) could, however, help.

6.4 An example: Nominal and real interest rates and in‡a-

tion
We present here a simple model which allows to understand the di¤erence between
the nominal and real interest rate and the determinants of in‡ation. This chapter is
intended (i) to provide another example where dynamic programming can be used and
(ii) to show again how to go from a general decentralized description of an economy
to its reduced form and thereby obtain economic insights.

6.4.1 Firms, the central bank and the government

Firms

Firms use a standard neoclassical technology Y = Y (K; L) : Producing under

perfect competition implies factor demand function of
@Y @Y
wK = p ; wL = p : (6.12)
@K @L
The central bank and the government

Studying the behaviour of central banks allows to …ll many books. We present the
behaviour of the central bank in a very simple way - so simple that what the central
bank does here (buy bonds directly from the government) is actually illegal in most
OECD countries. Despite this simple presentation, the general result we obtain later
would hold in more realistic setups as well.
The central bank issues money M (t) in exchange for government bonds B (t) : It
receives interest payments iB from the government on the bonds. The balance of the
central bank is therefore iB + M_ = B.
_ This equation says that bond holdings by the
central bank increase by B_ either when the central bank issues money M_ or receive
interest payments iB on bonds it holds.
The government’s budget constraint reads G + iB = T + B: _ General government
expenditure G plus interest payments iB on government debt B is …nanced by tax
income T and de…cit B: _ We assume that only the central bank holds government
bonds and not private households. Combining the government with the central bank
budget therefore yields
M_ = G T:

109
This equation says that an increase in monetary supply either is used to …nance
government expenditure minus tax income or, if G = 0 for simplicity, any monetary
increase is given to housholds in the form of negative taxes, T = M_ :

6.4.2 Households
Preferences of households are described by a “money-in-utility”function,
Z 1
m( )
e [ t] ln c ( ) + ln d :
t p( )

In addition to utility from consumption c (t), utility is derived from holding a certain
stock m (t) of money, given a price level p (t) : Given that wealth of households consists
of capital goods plus money, a (t) = k (t) + m (t) ; and that holding money pays no
interest, the budget constraint of the household can be shown to read (see exercises)

a_ = i [a m] + w T =L pc:

Tax payments T =L per representative household are lump-sum. If taxes are negative,
T =L represents transfers of the government to households. The interest rate i is
de…ned according to
wK + v_
i :
v
When households choose consumption and the amount of money held optimally,
consumption growth follows (see exercises)
c_ p_
=i :
c p

Money demand is given by (see exercises)

pc
m= :
i

6.4.3 Equilibrium
The reduced form

Equilibrium requires equality of supply and demand on the goods market. This is
obtained if total supply Y equals demand C + I: Letting capital accumulation follow
K_ = I K; we get
K_ = Y (K; L) C K: (6.13)
This equation determines K: As capital and consumption goods are traded on the
same market, this equation implies v = p and the nominal interest rate becomes with
(6.12)
wK p_ @Y p_
i= + = + : (6.14)
p p @K p

110
The nominal interest rate is given by marginal productivity of capital wK =p = @Y =@K
(the “real interest rate”) plus in‡ation p=p.
_
Aggregating over households yields aggregate consumption growth of C=C _ =i
p=p
_ : Inserting (6.14) yields

C_ @Y
= : (6.15)
C @K
Aggregate money demand is given by
pC
M= : (6.16)
i
Given an exogenous money supply rule and appropriate boundary conditions, these
four equations determine the paths of K; C; i and the price level p:
One standard property of models with ‡exibel prices is the dichotomy between
real variables and nominal variables. The evolution of consumption and capital - the
real side of the economy - is completely independent of monetary in‡uences: Equation
(6.13) and (6.15) determine the paths of K and C just as in the standard optimal
growth model without money - see (5.37) and (5.39) in ch. 5.5.4. Hence, when
thinking about equilibrium in this economy, we can think about the real side on the
one hand - independently of monetary issues - and about the nominal side on the
other hand. Monetary variables have no real e¤ect but real variables have an e¤ect
on monetary variables like e.g. in‡ation.
Needless to say that the real world does not have perfectly ‡exible prices such
that one should expect monetary variables to have an impact on the real economy.
This model is therefore a starting point to well understand structures and not a fully
developed model to analyse monetary questions in a very realistic way. Price rigidity
would have to be included before doing this.

A stationary equilibrium

Assume the technology Y (K; L) is such that in the long-run K is constant. As

a consequence, aggregate consumption C is contant as well. Hence, with respect to
real variables (including, in addition to K and C; the real interest rate and output),
we are in a steady state as in ch. 5.5.4.
Depending on exogenous money supply, equations (6.14) and (6.16) determine the
price level and the nominal interest rate. Substituting the nominal interest rate out,
we obtain
p_ pC @Y
= :
p M @K
This is a di¤erential equation which is plotted in the next …gure. As this …gure shows,
provided that M is constant, there is a price level p which implies that there is zero
in‡ation.

111
 N
p_
p

pC @Y
M @K

! !

N
0
p p

@Y
@K

Figure 29 The price level p in a monetary economy

Now assume there is money growth, M_ > 0: The …gure then shows that the
price level p increases as the slope of the line becomes ‡atter. Money growth in an
economy with constant GDP implies in‡ation. By looking at (6.14) and (6.16) again
and by focusing on equilibria with constant in‡ation rates, we know from (6.14)
that a constant in‡ation rate implies a constant nominal interest rate. Hence, by
di¤erentiating the equilibrium (6.16) on the money market, we get

p_ M_
= : (6.17)
p M
In an economy with constant GDP and increasing money supply, the in‡ation rate is
identical to the growth rate of money supply.

A growth equilibrium

Now assume there is (exogenous) technological progress at a rate g such that in

the long run Y_ =Y = C=C
_ _
= K=K = g: Then by again assuming a constant in‡ation
rate (implying a constant nominal interest rate) and going through the same steps
that led to (6.17), we …nd by di¤erentiating (6.16)

p_ M_ C_
= :
p M C
In‡ation is given by the di¤erence between the growth rate of money supply and
consumption growth.

Exogenous nominal interest rates

112
 The current thinking about central bank behaviour di¤ers from the view that the
central bank chooses money supply M as assumed so far. The central bank rather
sets nominal interest rates and money supply adjusts (where one should keep in mind
that money supply is more than just cash used for exchange as modeled here). Can
nominal interest rate setting be analysed in this setup?
Equilibrium is described by equations (6.13) to (6.16). They were understood to
determine the paths of K; C; i and the price level p; given a money supply choice M
of the central bank. If one believes that nominal interest rate setting is more realistic,
these four equations would simply determine the paths of K; C; M and the price level
p; given a nominal interest rate choice i of the central bank. Hence, by simply making
an exogenous variable, M; endogenous and making a previously endogenous variable,
i; exogenous, the same model can be used to understand the e¤ects of higher and
lower nominal interest rates on the economy.
Due to perfect price ‡exibility, real quantities remain una¤ected by the nominal
interest rate. Consumption, investment, GDP, the real interest rate, real wages are all
determined, as before, by (6.13) and (6.15) - the dichotomy between real and nominal
quantities continues given price ‡exibility. By (6.14), a change in the nominal interest
rate a¤ects in‡ation: high (nominal) interest rates imply high in‡ation, low nominal
interest rates imply low in‡ation. From the money market equilibrium in (6.16) one
can then conclude what this implies for money supply, again both for a growing or
a stationary economy. Much more needs to be said about these issues before policy
implications can be discussed. Any analysis in a general equilibrium framework would
however partly be driven by relationships presented here.

6.5 Exercises chapter 6 and further reading

An alternative way, which is not based on dynamic programming, to reach the same
conclusion about the interpretation of the costate variable for Hamiltonian maximiza-
tion as here in ch. 6.2 is provided by Intriligator (1971, p. 352). An excellent overview
and introduction to monetary economics is provided by Walsh (2003).

113
 Exercises chapter 6
Applied Intertemporal Optimization
Dynamic Programming in Continuous
Time

1. The envelope theorem (nicht schon wieder ...)

Compute the derivative (6.6) of the maximized Bellman equation without using
the envelope theorem.

2. A …rm with adjustment costs

Consider again, as in ch. 5.5.1, a …rm with adjustment cost. The …rms objective
is Z 1
max e r[ t] ( ) d :
fI(t);L(t)g t
In contrast to ch. 5.5.1, the …rm now has an in…nite planning horizon and
employs two factors of production, capital and labour. Instantaneous pro…ts
are
= pF (K; L) wL I I ;
where investment I also comprises adjustment costs for > 0: Capital, owned
by the …rm, accumulates according to K_ = sI K: All parameters s; ; ;
are constant.

(a) Solve this maximization problem by using the dynamic programming ap-
proach. You may choose appropriate (numerical or other) values for para-
meters where this simpli…es the solution (and does not destroy the spirit
of this exercise).
(b) Show that in the long-run with adjustment cost and at each point in time
under the absence of adjustment cost, capital is paid its value marginal
product. Why is labour being paid its value marginal product at each
point in time?

3. Money in the utility function

Consider an individual with the following utility function
Z 1
m( )
U (t) = e [ t] ln c( ) + ln d :
t p( )
As always, is the time preference rate and c ( ) is consumption. This utility
function also captures demand for money by including a real monetary stock of

114
 m ( ) =p ( ) in the utility function where m ( ) is the amount of cash and p ( )
is the price level of the economy. Let the budget constraint of the individual be

a_ = i [a m] + w T =L pc:

where a is total wealth consisting of shares in …rms plus money, a = k + m and

i is the nominal interest rate.

(a) Derive the budget constraint by assuming interest payments of i on shares

in …rms and zero interest rates on money.
(b) Derive optimal money demand.

4. Nominal and real interest rates in general equilibrium

Put households of exercise 3 in general equilibrium with capital accumulation
and a central bank whoe chooses money supply M: Compute the real and the
nominal interest rate in a long-run equilibrium.

115
6.6 Looking back
This is the end of part I and II. This is often also the end of a course. This is a good
moment to look back at what has been acomplished. After 14 or 15 lectures and the
same number of exercise classes, the amount of material covered is fairly impressive.
In terms of maximization tools, this …rst part has covered

Solving by inserting

Lagrange methods in discrete and continuous time

Dynamic programming in discrete time and continuous time

Hamiltonian

With respect to model building components, we got to know

how to build budget constraints

how to structure the presentation of a model

how to derive reduced forms

From an economic perspective, the …rst part presented

two-period OLG model

the optimal saving central planner model in discrete and continuous time

the matching approach to unemployment

the decentralized optimal growth model and

an optimal growth model with money

Most importantly, however, the tools having been presented here allow students to
“become independent”. A very large part of the economics literature (acknowledging
that game theoretic approaches have not at all been covered here) is now open and
accessible and the basis for understanding a paper in detail (and not just the overall
argument) and for presenting own arguments in a scienti…c language are laid.
Clearly, models with uncertainty present additional challenges. They will be pre-
sented and overcome in part III and part IV.

116
Part III
Stochastic models in discrete time
The following chapter starts part III of this book. We got to know many optimization
methods for deterministic setups, both in discrete and continuous time. All economic
questions that were analysed were viewed as “su¢ ciently deterministic”. If there was
any uncertainty in the setup of the problem, we simply ignored it or argued that it
is of no importance for understanding the basic properties and relationships of the
economic question. This is a good approach for many economic questions.
Real life has few certain components - death is certain, but when? Taxes are
certain, but how high are they? We know that we all exist - but don’t ask philosophers.
Part III and part IV will take uncertainty in life seriously and incorporate it explicitly
in the analysis of economic problems. We follow the same distinction as in part I and
II; we …rst analyse the e¤ects of uncertainty on economic behaviour in discrete time
setups and then move to continuous time setups in part IV.

7 Stochastic di¤erence equations and applications

Before we look at di¤erence equations in section 7.4, we …rst spend some sections on
reviewing basic concepts related to uncertain environments. These concepts will be
useful at later stages.

7.1 Basics on random variables

Let us …rst have a look at some basics on random variables. This follows Evans,
Hastings and Peacock (2000).

7.1.1 Some concepts

A probabilistic experiment is some occurrence where a complex natural background
leads to a chance outcome. The set of possible outcomes of a probabilistic experiment
is called the possibility space (Möglichkeitsraum, Raum möglicher Ausprägungen). A
random variable (RV) X is a function which maps from the possibility space into a set
of numbers. The set of numbers this RV can take is called the range of this variable
X.
The distribution function F (Verteilungsfunktion) associated with the RV X is a
function which maps from the range into the probability domain [0,1],

F (x) = Prob (X x) :

The probability that X has a realization of x or smaller is given by F (x) :

We now need to make a distinction between discrete and continuous RVs. When
the RV X has a discrete range then f (x) gives …nite probabilities and is called the

117
probability function (Wahrscheinlichkeitsfunktion) or probability mass function. The
probability that X has the realization of x is given by f (x) :
When the RV X is continuous, the …rst derivative of the distribution function F

dF (x)
f (x) =
dx
is called the probability density function f (Dichtefunktion). The probability that the
Rb
realization of X lies between, say, a and b > a is given by F (b) F (a) = a f (x) dx:
Hence the probability that X equals a is zero.

7.1.2 An illustration
Discrete random variable

Consider the probabilistic experiment ’tossing a coin two times’. The possibility
space is given by fHH; HT; T H; T T g. De…ne the RV ’Number of hats’. The range
of this variable is given by f0; 1; 2g : Assuming that the coin falls on either side with
the same probability, the probability function of this RV is given by
8 9 8
< :25 = < 0
f (x) = :5 for x = 1 :
: ; :
:25 2

Continuous random variable

Think of next weekend. You might consider going into some pub to meet friends.
Before you go there, you do not know how much time you will spend there. If you
meet many friends, you stay longer, if you just drink one beer, you leave soon. Hence,
going to a pub on a weekend is a probabilistic experiment with a chance outcome.
The set of possible outcomes with respect to the amount of time spent in a pub
is the possibility space. Our random variable T maps from this possibility space
into a set of numbers with a range from 0 to, let’s say, 4 hours (as the pub closes
at 1 am and you never go there before 9 pm). As time is continous, T 2 [0; 4] is a
continuous random variable. The distribution function F (t) gives you the probability
that you spend a period of length t or shorter in the pub.R 2 The probability that you
spend between 1.5 and two hours in the pub is given by 1:5 f (t) dt where f (t) is the
density function f (t) = dF (t) =dt.

7.2 Examples for random variables

We now look at some examples of RVs that are useful for later applications. As a RV
is completely characterized by its range and its probability or density function, we
will describe RVs by providing this information.

118
7.2.1 Discrete random variables
Discrete uniform distribution
range x 2 fa; a + 1; :::; b 1; bg
probability function f (x) = 1= (b a + 1)
An example for this RV is the die. Its range is 1 to 6, the probability for any
number (at least for fair dice) is 1=6.
The Poisson distribution
range x 2 f0; 1; 2; :::g
x
e
probability function f (x) = x!

Here is some positive parameter. When we talk about stochastic processes in

part IV, this will be calle the arrival rate.

7.2.2 Continuous random variables

Normal distribution
range x 2 ] 1; +1[
2
density function f (x) = p 1 e
1
2 (x )
2 2

The mean and the standard deviation of X are given by and :

Standard normal distribution
This is the normal distribution with mean and standard deviation given by =0
and = 1:
Exponential distribution
range x 2 [0; 1[
x
density function f (x) = e
Again, is some positive parameter.

7.2.3 Higher-dimensional random variables

We have studied so far one-dimensional RVs. In what follows, we will occasionally
work with multidimensional RVs as well. It will su¢ ce for our illustration purposes
here to focus on a two-dimensional normal distribution. Consider two random vari-
ables X1 and X2 : They are (jointly) normally distributed if the density function of
X1 and X2 is given by
1 1
2) (x~21 2 x
~1 x x22 )
~2 +~
f (x1 ; x2 ) = p e 2( 1
; (7.1)
2 1 2
1 2
xi i 12
where x~i = ; = : (7.2)
i 1 2

119
The mean and standard deviation of the RVs are denoted by i and i : The parameter
is called the correlation coe¢ cient between X1 and X2 is de…ned as the covariance
12 devided by the standard deviations (see ch. 7.3.1).
The nice aspects of this two-dimensional normally distributed RV (the same holds
for n-dimensional RVs) is that the density function of each individual RV Xi ; i.e.
the marginal density (Randverteilung), is given by the standard expression which is
independent of the correlation coe¢ cient,
1 1 2
x
~
f (xi ) = p 2
e 2 i : (7.3)
2 i

This implies a very convenient way to go from independent to correlated RVs in

a multi-dimensional setting: When we want to assume independent normally dis-
tributed RVs, we assume that (7.3) holds for each random variable Xi and set the
correlation coe¢ cient to zero. When we want to work with dependent RVs that are
individually normally distributed, (7.3) holds for each RV individually as will but, in
addition, we …x a non-zero coe¢ cient of correlation .

7.3 Expected values, variances, covariances and all that

7.3.1 De…nitions
We repeat here some de…nitions which can be found in many textbooks on statistics.
We focus on continuous random variables. For discrete
Rb random variables, the integral
is replaced by a sum - in very loose notation, a g (x) dx is replaced by bi=a g (xi ),
where a and b are constants which can R be minus or plus in…nity and g (x) is some
function. In the following de…nitions, :dx means the integral over the relevant range
(i.e. from minus to plus in…nity or from the lower to upper bound of the range of the
RV under consideration).
R
Mean EX Rxf (x) dx 2
Variance varX R (x EX) f (x) dx
kth uncentered moment EX k xk f (x) dx
R
kth centered moment E (X EX)kR R (x EX)k f (x) dx
covariance cov(X; Y ) (xp EX) (y EY ) f (x; y) dx dy
correlation coe¢ cient XY cov (X; Y ) = varX varY
independence p (X 2 A; Y 2 B) = P (X 2 A) P (Y 2 B)

Figure 30 Some basic de…nitions

7.3.2 Lemmas
Basic

120
 On expectations

E [a + bX] = a + bEX
RR
z = g (x; y) ) EZ = g (x; y) f (x; y) dx dy

On variances

varX = E (X EX)2 = E X 2 2XEX + (EX)2

= EX 2 2 (EX)2 + (EX)2 = EX 2 (EX)2 (7.4)

var (a + bX) = b2 varX

var (X + Y ) = varX + varY + 2cov (X; Y ) (7.5)

On covariances

cov (X; X) = varX

cov (X; Y ) = E (XY ) EXEY () E (XY ) = EXEY + cov (X; Y )
cov (a + bX; c + dY ) = bd cov (X; Y )

Advanced

We present here a theorem which is very intuitive and highly useful to analytically
study pro- and countercyclical behaviour of endogenous variables in models of business
cycles. The theorem says that if two variables depend “in the same sense” on some
RV (i.e. they both increase or decrease in the RV), then these two variables have a
positive covariance. If e.g. both GDP and R&D expenditure increase in TFP and
TFP is random, then GDP and R&D expenditure are procyclical.

Theorem 6 Let X a random variable and f (X) and g (X) two functions such that
f 0 (X) g 0 (X) R 0 8 x 2 X: Then

cov (f (X) ; g (X)) R 0:

Proof. We only proof the ”> 0”part. We know that

cov (Y; Z) = E (Y Z) EY EZ:

With Y = f (X) and Z = g (X), we have

cov (f (X) ; g (X)) = E (f (X) ; g (X)) Ef (X) Eg (X)

R R R
= f (x) g (x) p (x) dx f (x) p (x) dx g (x) p (x) dx;

where p (x) is the density of X. Hence

R R R
cov (f (X) ; g (X)) > 0 , f (x) g (x) p (X) dx > f (x) p (x) dx g (x) p (x) dx:
µ
The last inequality holds for f 0 (X) g 0 (X) > 0 as shown by Cebyšev and presented in
Mitrinović (1970, Theorem 10, sect. 2.5, p. 40).

121
7.3.3 Functions on random variables
We will occasionally encounter the situation where we compute density functions of
functions of RVs. Here are some examples.

Linearily transforming a normally distributed RV

Consider a normally distributed RV X~N ( ; 2 ) : What is the distribution of the

Y = a + bX? We know from ch. 7.3.2 that for any RV, E (a + bX) = a + bEX and
V ar (a + bX) = b2 V ar (X). As it can be shown that a linear transformation of a
normally distributed RV gives again a normally distributed RV, Y is also normally
distributed with Y ~N (a + b ; b2 2 ) :

An exponential transformation

Consider the RV Y = eX ; with X being normally distributed. The RV Y is

lognormally distributed. A variable is lognormally distributed if its logarithm is
normally distributed, ln Y ~N ( ; 2 ). The mean and variance of this distribution are
given by
+ 12 2 2 2 + 2 2
y = e ; y = e e 1 : (7.6)
Clearly, Y can only have non-negative realizations.

Transformations of log-normal distributions

Let there be two long-normally distributed variables Y and Z. Any transformation

of the type Y ; where is a constant, or products like Y Z are also lognormally
distributed.
To show this, remember that we can express Y and Z as Y = eX1 and Z = eX2 ,
with the Xi being (jointly) normally distributed. Hence, for the …rst example, we
can write Y = e X1 : As X1 is normally distributed, Y is lognormally distributed.
For the second, we write Y Z = eX1 eX2 = eX1 +X2 : As the Xi are (jointly) normally
distributed, their sum is as well and Y Z is lognormally distributed.
Total factor productivity is sometimes argued to be lognormally distributed. Its
logarithm is then normally distributed. See e.g. ch. 7.5.6.

The general case

Consider now a general transformation of the type y = y (x) where the RV X has
some density f (x) : What is the density of Y ? The answer comes from the following

Theorem 7 Let X be a random variable with density f (x) and range [a; b] which can
be ] 1; +1[: Let Y be de…ned by the monotonically increasing function y = y (x) :
Then the density g (y) is given by g (y) = f (x (y)) dx
dy
on the range [y(a); y(b)]:

122
 This theorem can be easily proven as follows. The proof is illustrated in …g. 31.
The …gure plots the RV X on the horizontal and the RV Y on the vertical axis. A
monotonically increasing function y (x) represents the transformation of realizations
x into y:
Y N
Y (b) Y (x)

Y (e
x)

Y (a)

N
a e
x b X
Figure 31 Transforming a random variable

Proof. The transformation of the range is immediately clear from the …gure:
When X is bounded between a and b; Y must be bounded between y (a) and y (b) :
The proof for the density of Y requires some more steps: The probability that y is
smaller than some y (~ x) is identical to the probability that X is smaller than this
x~: This follows from the monotonicity of the function Y (X) : As a consequence, the
distribution function (cumulative density function) of Y is given by G (y) = F (x)
where y = y (x) or, equivalently, x = x (y) : The derivative then gives the density
d d
function, g (y) dy G (y) = dy F (x (y)) = f (x (y)) dx
dy
:

7.4 Examples of stochastic di¤erence equations

We now return to the …rst main objective of this chapter, the description of stochastic
processes through stochastic di¤erence equations.

7.4.1 A …rst example

The di¤erence equation

Possibly the simplest stochastic di¤erence equation is the following

xt = axt 1 + "t ; (7.7)

where the stochastic components are distributed according to some distribution func-
tion on some range which implies a mean and a variance 2 ; "t ~ ( ; 2 ) : We do not
make any speci…c assumption about "t at this point. Note that the stochastic com-
ponents "t are iid (identically and independently distributed) which implies that the
covariance between any two distinct "t is zero, cov("t ; "s ) = 0 8t 6= s. An alternative
representation of xt would be xt = axt 1 + + vt with vt ~ (0; 2 ).

123
 Solving by inserting

In complete analogy to deterministic di¤erence equations in ch. 2.5.3, equation

(7.7) can be solved for xt as a function of time and past realizations of "t ; provided
we have a boundary condition x0 for t = 0. By repeated reinserting, we obtain

x1 = ax0 + "1 ; x2 = a [ax0 + "1 ] + "2 = a2 x0 + a"1 + "2 ;

x3 = a a2 x0 + a"1 + "2 + "3 = a3 x0 + a2 "1 + a"2 + "3

and eventually

xt = at x0 + at 1 "1 + at 2 "2 + ::: + "t

= at x0 + ts=1 at s "s : (7.8)

The mean

A stochastic di¤erence equation does not predict how the stochastic variable xt
for t > 0 actually evolves, it only predicts the distribution of xt for all future points
in time. The realization of xt is random. Hence, we can only hope to understand
something about the distribution of xt . To do so, we start by analysing the mean of
xt for future points in time.
Denote the expected value of xt by x~t ;

x~t E0 xt ; (7.9)

i.e. x~t is the value expected in t = 0 for xt in t > 0: The expectations operator E0
captures our “knowledge” when we compute the expectation for xt . The “0” says
that we have all information for t = 0 and know therefore x0 ; but we do not know "1 ;
"2 ; etc.
By applying the expectations operator E0 to (7.7), we obtain

x~t = a~
xt 1 + :

This is a deterministic di¤erence equation which describes the evolution of the ex-
pected value of xt over time. There is again a standard solution to this equation
which reads
1 at
x~t = at x0 + t
s=1 a t s
= a t
x 0 + ; (7.10)
1 a
where we used ts=1 at s = a0 + a1 + ::: + at 1 = ti=01 ai and, from ch. 2.5.1, ni=0 ai =
(1 an+1 ) = (1 a), for a < 1: This equation shows that the expected value of xt
changes over time as t changes. Note that x~t might increase or decrease (see the
exercises).

The variance

124
 Let us now look at the variance of xt : We obtain an expression for the variance
by starting from (7.8) and observing that the terms in (7.8) are all independent from
each other: at x0 is a constant and the disturbances "s are iid by assumption. The
variance of xt is therefore given by the sum of variances (compare (7.5) for the general
case),

t s 2 a2t
t s t 1 i 21
V ar (xt ) = 0 + s=1 at V ar ("s ) = 2 t
s=1 a2 : = 2
i=0 a2 =
1 a2
(7.11)
We see that it is also a function of t: The fact that the variance becomes larger
the higher t is intuitively clear. The further we look into the future, the “more
randomness”there is: equation (7.8) shows that a higher t means that more random
variables are added up.
Long-run behaviour
When we considered deterministic di¤erence equations like (2.36), we found the
long-run behaviour of xt by computing the solution of this di¤erence equation and
by letting t approach in…nity. This would give us the …xpoint of this di¤erence
equation as e.g. in (2.37). The concept that corresponds to a …xpoint in an uncertain
environment, e.g. when looking at stochastic di¤erence equations like (7.7), is the
limiting distribution. As stated earlier, stochastic di¤erence equations do not tell
us anything about the evolution of xt itself, they only tell us something about the
distribution of xt : It would therefore make no sense to ask what xt is in the long run
- it will always remain random. It makes a lot of sense, however, to ask what the
distribution of xt is for the long run, i.e. for t ! 1:
All results so far were obtained without a speci…c distributional assumption for "t
apart from specifying a mean and a variance. Understanding the long-run distribution
of xt in (7.7) is easy if we assume that the stochastic component "t is normally
distributed for each t; "t ~N ( ; 2 ). In this case, starting in t = 0; the variable x1 is
from (7.7) normally distributed as well. As a weighted sum of two random variables
that are (jointly) normally distributed gives again a random variable with a normal
distribution (where the mean and variance can be computed as in ch. 7.3.2), we
know from (7.7) also that x1 ; x2 ... are all normally distributed. Hence, in order to
understand the evolution of the distribution of xt , we only have to …nd out how the
expected value and the variance of the variable xt evolves.
Neglecting the cases of a 1, the mean x~ of our long-run normal distribution is
given from (7.10) by the …xpoint
1
lim x~t x~1 = ; a < 1:
t!1 1 a
The variance of the long-run distribution is from (7.11)
2 1
lim var (xt ) = :
t!1 1 a2
Hence, the long-run distribution of xt is a normal distribution with mean = (1 a)
and variance 2 = (1 a2 ).

125
 The evolution of the distribution of xt

Given our results on the evolution of the mean and the variance of xt and the fact
that we assumed "t to be normally distributed, we know that xt is normally distributed
at each point in time. We can therefore draw the evolution of the distribution of x
as in the following …gure.

A simple stochastic difference equation

xt = axt −1 + ε t 0.4

initial distribution 0.35

0.3

0.25

0.2

limiting distribution
0.15

0.1

0 1 0.05
2 3
4
5
6
7 0
8
9
10

19
11

17
12

15
13

13
14

11
15
16 9 time
7

17
5

xt 18
3

19
1

Figure 32 Evolution of a distribution over time

Remember that we were able to plot a distribution for each t only because of
properties of the normal distribution. If we had assumed that "t is lognormally or
equally distributed, we would not have been able to easily say something about the
distribution of xt : The means and variances in (7.10) and (7.11) would still have been
valid but the distribution of xt for future t is generally unknown for distributions of
"t other than the normal distribution.

An interpretation for many agents

Imagine the variable xit represents …nancial wealth of an individual i. Equation

(7.7) can then be interpreted to describe the evolution of wealth xit over time,

xit = axit 1 + "it :

Given information in period t = 0, it predicts the density function f (xit ) for wealth
in all future periods, given some initial value xi0 . Shocks are identically and indepen-
dently distributed (iid) over time and across individuals.
Assuming a large number of agents i, more precisely a continuum of agents with
mass n, a law of large numbers can be constructed along the lines of Judd (1985):

126
Let all agents start with the same wealth xi0 = x0 in t = 0. Then the density f (xit )
for wealth of any individual i in t equals the realized wealth distribution in t for
the continuum of agents with mass n: Put di¤erently: when the probability for an
individual to hold wealth between some lower and upper bound in t is given by p, the
share of individuals that hold wealth between these two bounds in t is also given by
p. Total wealth in t is then deterministic (as are all other moments) and is given by
Z 1
xt = n f (xit )xit dxit = n t ; (7.12)
1

where t is average wealth over individuals or expected wealth of any individual i:

The same argument can also be made with a discrete number of agents. Wealth
xit of individual i at t is a random variable. Let the probability that xit is smaller
than x be given by P (xit x) = p: Now assume there are n agents and therefore at
each point in time t there are n independent random variables xit : Denote by n the
number of random variables xit that have a realization smaller than x: The share of
random variables that have a realization smaller than x is denoted by q n=n: It is
then easy to show that E q = p for all n and, more importantly, limn!1 var (q) = 0:
This equals in words the statement based on Judd above: The share in the total
population n is equal to the individual probability if the population is large. This
share becomes deterministic for large populations.
Here is now a formal proof (thanks are due to Christian Kleiber for discussions
of this issue, see Kleiber and Kotz (2003) for more on distributions): De…ne Yi as
the number of realizations below x for one xit ; i.e. Yi I (xit x) where I is the
indicator function which is 1 if condition in parentheses holds and 0 if not. Clearly,
the probability that Yi = 1 is given by p; i.e. Yi is Bernoulli(p) distributed, with
EYi = p and varYi = p(1 p). (The Yi are iid if the xit are iid.) Then the share q
is given by q = ni=1 Yi =n: Its moments are E q = p and varq = p(1 p)=n. Hence,
the variance tends to zero for n going to in…nity. (More technically, q tends to p in
“quadratic mean”and therefore “in probability”. The latter means we have a “weak
law of large numbers”.)

7.4.2 A more general case

Let us now consider the more general di¤erence equation
xt = t xt 1 + "t ;
where the coe¢ cient is now also stochastic
2 2
t~ a; a ; "t ~ ; " :
Analyzing the properties of this process is pretty complex. For details see Vervaat
(1979).
What can be easily done, however, is to analyze the evolution of moments. As
Vervaat has shown, the limiting distribution has the following moments
j j
Exj = k=0 k E k j k
" Exk :

127
Assuming we are interested in the expected value, we would obtain
Ex1 = 1k=0 1k E k "1 k
Exk = 1
0 E 0 1
" Ex0 + 1
1 E 1 0
" Ex1
= E" + E Ex:
E"
Solving for Ex yields Ex = 1 E
= 1 a
:

7.5 A simple OLG model

Let us now return to maximization issues. We do so in the context of the most simple
example of a dynamic stochastic general equilibrium model. It is a straightforward
extension of the deterministic model analyzed in section 2.4.
The structure of the maximization problem of individuals, the timing of when
uncertainty reveals and what is uncertain depends on the fundamental and exogenous
sources of uncertainty. As the fundamental source of uncertainty results here from
the technology used by …rms, technologies will be presented …rst. Once this is done,
we can derive the properties of the maximization problem households or …rms face.

7.5.1 Technology
Let there be an aggregate technology
Yt = At Kt L1t (7.13)
The total factor productivity level At is uncertain. We assume that At is identically
and independently distributed (iid) in each period. The random variable At is pos-
itive, has a mean A and a variance 2 : No further information is needed about its
distribution function at this point,
2
At ~ A; ; At > 0: (7.14)
Assuming that total factor productivity is iid means, inter alia, that there is no
technological progress. One can imagine many di¤erent distributions for At . In
principle, all distributions presented in the last section are viable candidates. Hence,
we can work with discrete distributions or continuous distributions.

7.5.2 Timing
The sequence of events is as follows. At the beginning of period t, the capital stock
Kt is inherited from the last period, given decisions of the last period. Then, total
factor productivity is revealed. With this knowledge, …rms choose factor employment
and households choose consumption (and thereby savings).

ct
At wt
Kt rt

128
Figure 33 Timing of events

Hence, only “at the end of the day”one really knows how much one has produced.
This implies that wages and interest payments are known with certainty at the end
of the period only as well.
The state of the economy in t is completely described by Kt and the realization
of At : All variables of the model are contingent on the state.

7.5.3 Firms
As a consequence of this timing of events, …rms do not bear any risk and they pay
the marginal product of labour and capital to workers and capital owners at the end
of the period,
@Yt
wt = pt ; (7.15)
@Lt
1
@Yt Lt
rt = p t = p t At : (7.16)
@Kt Kt
All risk is therefore born by households through labour and capital income.

7.5.4 Individual decision problem

General approach

We consider an agent that lives for two periods and works and consumes in a
world as just described. Agents consume in both periods and choose consumption
such that they maximize expected utility. In all generality concerning the uncertainty,
she maximizes
max Et fu (ct ) + u (ct+1 )g ; (7.17)
where is the subjective discount factor that measures the agent’s impatience to
consume. The expectations operator has an index t to indicate, similar to E0 in
(7.9), that expectations are based on the knowledge about random variables which
is available in period t. We will see in an instant whether the expectations operator
Et is put at a meaningful place in this objective function. Placing it in front of both
instantaneous consumption from ct and from ct+1 is the most general way of handling
it. We will also have to specify later what the control variable of the household is.
The agent has to choose consumption for the …rst and the second period. When
consumption is chosen and given wage income wt ; savings adjust such that the budget
constraint
wt = ct + st (7.18)
for the …rst period holds. Note that this constraint always holds, despite the uncer-
tainty about the wage. It holds in realizations, not in expected terms. In the second
period, the household receives interest payments on savings made in the …rst period
and uses savings plus interests for consumption,

129
 (1 + rt+1 ) st = ct+1 : (7.19)
One way of solving this problem (for an alternative, see the exercise) is to insert
consumption levels from these two constraints into the objective function (7.17). This
gives
max Et fu (wt st ) + u ((1 + rt+1 ) st )g :
st

This nicely shows that the household in‡uences consumption in both periods by
choosing savings st in the …rst period. In fact, the only control variable the household
can choose is st :
Let us now take into consideration, as drawn in the above …gure, that consumption
takes place at the end of the period after revelation of productivity At in that period.
Hence, the consumption level in the …rst period is determined by savings only and
thereby certain.7 The consumption level in the second period is uncertain, however, as
the next period interest rate rt+1 depends on the realization of At+1 which is unknown
in t when decisions about savings st are made. The objective function can therefore
be written as
max u (wt st ) + Et u ((1 + rt+1 ) st ) :
st

Let us now, for illustration purposes, assume a discrete random variable At with
a …nite number n of possible realizations. Then, this maximization problem can be
written as
n
max u (wt st ) + i=1 i u ((1 + ri;t+1 ) st )
st

where i is the probability that the interest rate ri;t+1 is in state i in t + 1: This is
the same probability as the probability that the underlying source of uncertainty At
is in state i: The …rst-order condition then reads

u0 (wt st ) = n
i=1 i u
0
((1 + ri;t+1 ) st ) (1 + ri;t+1 ) = Et u0 ((1 + rt+1 ) st ) (1 + rt+1 )
(7.20)
Marginal utility of consumption today must equal expected marginal utility of con-
sumption tomorrow corrected by interest and time preference rate. Optimal behav-
iour in an uncertain world therefore means ex ante optimal behaviour, i.e. before
random events are revealed. Ex post, i.e. after resolution of uncertainty, behaviour
is suboptimal as compared to the case where the realization is known in advance:
Marginal utility in t will (with high probability) not equal marginal utility (corrected
by interest and time preference rate) in t + 1. This re‡ects a simple fact of life: “If
I had known before what would happen, I would have behaved di¤erently.”Ex ante,
behaviour is optimal, ex post, probably not. Clearly, if there was only one realization
for At , i.e. 1 = 1 and i = 0 8 i > 1; we would have the deterministic …rst-order
condition we had in exercise 1 in ch. 2.
The …rst-order condition also shows that closed form solutions are possible if
marginal utility is of a multiplicative type. As savings st are known, the only quantity
7
If consumption ct (or savings) was chosen before revelation of total factor productivity, house-
holds would want to consume a di¤erent level of consumption ct after At is known.

130
which is uncertain is the interest rate rt+1 : If the instantaneous utility function u (:)
allows to separate the interest rate from savings, i.e. the argument (1 + ri;t+1 ) st in
u0 ((1 + ri;t+1 ) st ) in (7.20), an explicit expression for st and thereby consumption can
be computed. This will be shown in the example following right now and in exercise
5.

An example - Cobb-Douglas preferences

Assume now the household maximizes a Cobb-Douglas utility function as in (2.5).

In contrast to the deterministic setup in (2.5), however, expectations about consump-
tion levels need to be formed. Preferences are therefore captured by

Et f ln ct + (1 ) ln ct+1 g : (7.21)

When we inserting savings, we can express the maximization problem by

max Et f ln (wt st ) + (1 ) ln ((1 + rt+1 ) st )g

st

= max ln (wt st ) + (1 ) [ln st + Et ln (1 + rt+1 )] : (7.22)

st

The …rst-order condition with respect to savings reads

1
= (7.23)
wt st st
and the optimal consumption and saving levels are

c t = wt ; ct+1 = (1 ) (1 + rt+1 ) wt ; st = (1 ) wt ; (7.24)

just as in the deterministic case (2.24).

Note that we can compute a closed form solution of the same structure as in the
deterministic solutions (2.7) and (2.8), despite the setup with uncertainty. What is
further peculiar about the solutions and also about the …rst-order condition is the fact
that the expectations operator is no longer visible. One could get the impressions that
households do not form expectations when computing optimal consumption paths.
The expectations operator “got lost”in the …rst-order condition only because of the
logarithm, i.e. the Cobb-Douglas nature of preferences. Nevertheless, there is still
uncertainty for an individual being in t : consumption in t + 1 is unknown as it is a
function of the interest-rate in t + 1.

7.5.5 Aggregation and the reduced form for the CD case

We now aggregate over all individuals. Consumption of all young individuals in period
t is given from (7.24),
Cty = N wt = (1 )Yt :
The second equality used (7.15), the fact that with a Cobb-Douglas technology (7.15)
can be written as wt Lt = pt (1 ) Yt ; the normalization of pt to unity and the

131
identity between number of workers and number of young, Lt = N . Note that this
expression would hold identically in a deterministic model. With (7.16), consumption
of old individuals amounts to

Cto = N (1 )(1 + rt )wt 1 = (1 )(1 + rt )(1 )Yt 1 :

The capital stock in period t + 1 is given by savings of the young. One could show
this as we did in the deterministic case in ch. 2.4.2. In fact, adding uncertainty would
change nothing to the fundamental relationships. We can therefore directly write

Kt+1 = N st = N (1 )wt = (1 ) (1 ) Yt ;

where we used the expression for savings for the Cobb-Douglas utility case from (7.24).
Again, we succeeded in reducing the presentation of the model to one equation in one
unknown. This allows us to illustrate the economy by using the simplest possible
phase diagram.
N
Kt+1 K2 = (1 )(1 )A1 K1 L1
K3 = (1 )(1 )A2 K2 L1
K1 = (1 )(1 )A0 K0 L1

K0 K1 K 2 K3 K(t)

Figure 34 Convergence towards a “stochastic steady state”

This phase diagram looks in principle identical to the deterministic case. There
is of course a fundamental di¤erence. The Kt+1 loci are at a di¤erent point in each
period. In t = 0, K0 and A0 are known. Hence, by looking at the K1 loci, we can
compute K1 as in a deterministic setup. Then, however, TFP changes and, in the
example plotted above, A1 is larger than A0 : Once A1 is revealed in t = 1; the new
capital stock for period 2 can be graphically derived. With A2 revealed in t = 2; K3
can be derived an so on. It is clear that this economy will never end up in one steady
state as in the deterministic case as At is di¤erent for each t: As illustrated before,
however, the economy can converge to a stationary stable distribution for Kt :

7.5.6 Some analytical results

The basic di¤erence equation

132
 In order to better understand the evolution of this economy, let us now look at
some analytical results. The logarithm of the capital stock evolves according to
ln Kt+1 = ln ((1 ) (1 )) + ln At + ln Kt + (1 ) ln Lt :
Assuming a constant population size Lt = L, we can rewrite this equation as
2
t+1 = m0 + t + t; t N( ; ); (7.25)
where we used
t ln Kt (7.26)
m0 ln [(1 ) (1 )] + (1 ) ln L (7.27)
and t ln At captures the uncertainty stemming from the TFP level At : As TFP is
iid, so is its logarithm t . Since we assumed the TFP to be lognormally distributed,
its logarithm t is normally distributed. When we remove the mean from the random
variable by replacing according to t = "t + ; where "t N (0; 2 ); we obtain

t+1 = m0 + + t + "t :
The expected value and the variance
We can now compute the expected value of t by following the same steps as in
ch. 7.4.1, noting that the only additional parameter is m0 . Starting in t = 0 with
some initial value 0 ; and solving this equation recursively gives
t
t 1 t 1 t 1 s
t = 0 + (m0 + ) + s=0 "s : (7.28)
1
Computing the expected value from the perspective of t = 0 gives
t
t 1
E0 t = 0 + (m0 + ) ; (7.29)
1
where we used E0 "s = 0 for all s > 0 (and we set "0 = 0 or assumed that expectations
in 0 are formed before the realization of "0 is known). For t going to in…nity, we
obtain
m0 +
lim E0 t = : (7.30)
t!1 1
Note that the solution in (7.28) is neither di¤erence- nor trend stationary. Only in
the limit, we have a (pure) random walk. For a very large t; t in (7.28) is zero and
(7.28) implies
t t 1 = "t 1 :

The variance of t can be computed with (7.28), where again independence of all
terms is used as in (7.11),
2t
t 1 t 1 s t 1 2(t 1 s) 2 1 2
V ar ( t ) = 0 + 0 + V ar s=0 "s = s=0 = 2
(7.31)
1
In the limit, we obtain
2
lim V ar ( t ) = 2
: (7.32)
t!1 1

133
 Relation to fundamental uncertainty

For a more convincing economic interpretation of the mean and the variance, it
is useful to express some of the equations not as a function of properties of the log
of At ; i.e. properties of t ; but directly of the level of At . As (7.6) implies, the mean
and variances of these two variables are related in the following fashion
!
2
2 A
= ln 1 + (7.33)
A

!
2
1 2 (7:33) 1 A
= ln A = ln A ln 1 + (7.34)
2 2 A

We can insert these expressions into (7.29) and (7.31) and study the evolution over
time or directly focus on the long-run distribution of t ln Kt . Doing so by inserting
in (7.30) and (7.32) gives
2 2
1
m0 + ln A 2
ln 1 + A
ln 1 + A
A A
lim E0 t = ; lim V ar ( t ) = 2
:
t!1 1 t!1 1
These equations tell us that uncertainty in TFP as captured by A not only a¤ects
the spread of the long-run distribution of the capital stock but also its mean. More
uncertainty leads to a lower long-run mean of the capital stock. This is interesting
given the standard result of precautionary saving and the portfolio e¤ect (in setups
with more than one asset). (t.b.c.)

7.6 Risk-averse and risk-neutral households

Previous analyses in this book have worked with the represantitive-agent assumption.
This means that we neglected all potential di¤erences accross individuals and assumed
that they are all the same (especially identical preferences, labour income and initial
wealth). We now extend the analysis of the last section by (i) allowing individuals
to give loans to each other. These loans have a riskless endogenous interest rates
r: As before, there is a “normal” asset which pays an uncertain return rt+1 : We
also (ii) assume there are two types of individuals, the risk-neutral ones and the
risk-averse individuals, denoted by i = a; n.8 We want to understand in this world
with heterogeneous agents who owns which assets. We keep the analysis simple by
analysing a partial equilibrium setup.

Households
8
If individuals were identical, i.e. if they had identical preferences and experienced the same
income streams, no loans would be given in equilibrium.

134
 The budget constraints (7.18) and (7.19) of all individuals i now read

wt = cit + sit ; (7.35)

cit+1 = sit 1 + i rt+1 + 1 i

r : (7.36)
Here and in subsequent chapters, i denotes the share of wealth held by individual i
in the risky asset. A household i with time preference rate and thereby discount
factor = 1= (1 + ) maximizes

U i = u wt sit + Eu 1 + i rt+1 + 1 i
r sit ! maxi (7.37)
sit ;

by now choosing two control variables: The amount of resources not used in the …rst
period for consumption, i.e. savings sit ; and the share i of savings held in the risky
asset.
First-order conditions for sit and i , respectively, are

u0 cit = Et u0 cit+1 1 + i rt+1 + 1 i

r ; (7.38)

Eu0 cit+1 [rt+1 r] = 0: (7.39)

Note that the …rst-order condition for consumption (7.38) has the same interpre-
tation, once slightly rewritten, as the interpretation in deterministic two-period or
in…nite horizon models (see (2.9) in ch. 2.2.2 and (3.5) in ch. 3.2). When rewriting
it as ( )
u0 cit+1 1
Et = 1; (7.40)
u0 (cit ) 1= 1 + i rt+1 + 1 i
r
we see that optimal behaviour again requires to equate marginal utilities today and
tomorrow (the latter in its present value) with relative prices today and tomorrow
(the latter also in its present value). Of course, in this stochastic environment, we
need to express everything in expected terms. As the interest rates and consumption
tomorrow are jointly uncertain, we can not bring it exactly in the form as known from
above in (2.9) and (3.5). This will be possible, however, further below in (8.8) in ch.
8.2.
The …rst-order condition for says that expected returns from giving a loan and
holding the risky asset must be identical. Returns consist of the interest rate times
marginal utility. This condition can best be understood when …rst thinking of a
certain environment. In this case, (7.39) would read rt+1 = r : agents would be
indi¤erent between holding two assets only if they receive the same interest rate on
both assets. Under uncertainty and with risk-neutrality of agents, i.e. u0 = const:,
we get Et rt+1 = r : Agents hold both assets only if the expected return from the risky
assets equals the certain return from the riskless asset.
Under risk-aversion, we can write this condition as Et u0 cit+1 rt+1 = Et u0 cit+1 r
(or, given that r is non-stochastic, as Et u0 cit+1 rt+1 = rEt u0 cit+1 ). This says
that agents do not value the interest rate per se but rather the extra utility gained
from holding an asset: An asset provides interest of rt+1 which increases utility

135
by u0 cit+1 rt+1 : The share of wealth held in the risky asset then depends on the
increase in utility from realizations of rt+1 accross various states and the expecta-
tions operator computes a weighted sum of theses utility increases: Et u0 cit+1 rt+1
n 0 i
j=1 u cjt+1 rjt+1 pj ; where j is the state, rjt+1 the interest rate in this state and pj
the probability for state j to occur. An agent is then indi¤erent between holding two
assets when expected utility-weighted returns are identical.

Risk-neutral and risk-averse behaviour

For risk-neutral individuals, the …rst-order conditions become

n n
1 = E [1 + rt+1 + (1 ) r] ; (7.41)

Ert+1 = r: (7.42)
The …rst-order condition on how to invest implies together with (7.41) that the en-
dogenous interest rate for loans is pinned down by the time preference rate,

1= [1 + r] , r = (7.43)

as the discount factor is given by = (1 + ) 1 as stated before (7.37). Reinserting

this result into (7.42) shows that we need to assume that an interior solution for
Ert+1 = exists. This is not obvious for an exogenously given distribution for the
interest rate rt but it is more plausible for a situation where rt is stochastic but
endogenous as in the analysis of the OLG model in the previous chapter.
For risk-averse individuals with u (cat ) = ln cat ; the …rst-order condition for con-
sumption reads
1 1 a a 1 1
a
= E a [1 + rt+1 + (1 ) r] = E a
= a (7.44)
ct ct+1 st st

where we used (7.36) for the second equality and the fact that st as control variable
is deterministic for the third. Hence, as in the last section, we can derive explicit
expressions. Use (7.44) and (7.35) and …nd
1
sat = cat , wt cat = cat , cat = wt :
1+

This gives with (7.35) again and with (7.36)

a a
sat = wt ; cat+1 = [1 + rt+1 + (1 ) r] wt :
1+ 1+
This is our closed-form solution for risk-averse individuals in our heterogeneous-agent
economy.
Let us now look at the investment problem of risk-averse households. The deriva-
tive of their objective function is given by the LHS of the …rst-order condition (7.39).

136
Expressed for logarithmic utility function, and inserting the optimal consumption
result yields
1 1+ rt+1 r
E [rt+1 r] = E a a
cat+1 1 + rt+1 + (1 )r
1+ rt+1 r 1+ X
= E a E : (7.45)
1 + (rt+1 r) + r a+ X
The last step de…ned X rt+1 r as a RV and a as a constant.

Who owns what?

It can now easily be shown that (7.45) implies that risk-averse individuals will
not allocate any of their savings to the risky asset, i.e. a 0: The derivative of the
expression EX= (a + a X) with respect to a is negative

d X X2
aE =E < 0 8 a:
d a + aX (a + a X)2

The sign of this derivative can also easily be seen from (7.45) as an increase in a
implies a larger denominator. Hence, when plotted, the …rst-order condition is down-
ward sloping in a : By guessing we …nd that, with (7.42), a = 0 satis…es the …rst-order
condition for investment,

a X
=0)E = EX = 0:
a + aX
Hence, the …rst-order condition is zero for a = 0: As the …rst-order condition is
monotonously decreasing, a = 0 is the only value for which it is zero,
X a
E =0, = 0:
a + aX
This is illustrated in the following …gure.

N
E a+x a x
N

a
0

137
 a
Figure 35 The …rst-order condition for the share of savings held in the risky asset

The …gure shows that expected utility of a risk-averse individual is the higher,
the lower the share of savings held in the risky asset. If we exclude short-selling,
risk-averse individuals will allocate all of their savings to loans, i.e. a = 0. They
give loans to risk-neutral individuals who in turn pay a certain interest rate equal to
the expected interest rate. All risk is born by risk-neutral individuals.

7.7 Pricing of contingent claims and assets

7.7.1 The value of an asset
The question is what an individual is willing to pay in t = 0 for an asset that is
worth pj (which is uncertain) in t = 1. This is inspired by Brennan (1979, JOF). As
the assumption underlying the pricing of assets is that individuals behave optimally,
the answer is found by considering a maximization problem where individuals solve
a saving and a portfolio problem.

The individual

Let preferences of an individual be given by

E fu (c0 ) + u (c1 )g :

This individual can invest in the risky and in a riskless investment form. It earns
labour income w in the …rst period and does not have any other source of income.
This income w is used for consumption and savings. Savings are allocated to several
risky assets j and a riskless bond. The …rst period budget constraint therefore reads
n
w = c0 + j=1 mj pj0 +b

where the number of shares j are denoted by mj ; their price by pj0 and the amount
of wealth held in the riskless asset is b. The individual consumes all income in the
second period which implies the budget constraint
n
c1 = j=1 mj pj1 + (1 + r) b;

where r is the interest rate on the riskless bond.

We can summarize this maximization problem as

max u (c0 ) + Eu ((1 + r) (w c0 j mj pj0 ) + j mj pj1 ) :

c0 ;mj

This expression illustrates that consumption in period 1 is given by capital income

from the riskless interest rate r plus income from assets j. It also shows that there is
uncertainty only with respect to period one consumption. This is because consump-
tion in period zero is a control variable. The consumption in period 1 is uncertain as
prices of assets are uncertain.

138
 Solution and asset pricing

Computing …rst-order conditions gives

u0 (c0 ) = (1 + r) Eu0 (c1 )

This is the standard …rst-order condition for consumption. The …rst-order condition
for the number of assets to buy reads for any asset j

E [u0 (c1 ) (1 + r) ( pj0 ) + pj1 ] = 0 , E [u0 (c1 ) pj1 ] = (1 + r) pj0 Eu0 (c1 ) :

The last step used the fact that the interest rate and pj0 are known in 0 and can there-
fore be pulled out of the expectations operatior on the right-hand side. Reformulating
this condition gives
1 u0 (c1 )
pj0 = E 0 pj1 (7.46)
1 + r Eu (c1 )
which is an expression that can be given an interpretation as a pricing equation. The
price of an asset j in period zero is given by the discounted expected price in period
one. The price in period one is weighted by marginal utilities.

7.7.2 The value of a contingent claim

What we are now interested in are prices of contingent claims. Generally speaking,
contingent claims are claims that can be made only if certain outcomes occur. The
simplest example of the contingent claim is an option. An option gives the buyer the
right to buy or sell an asset at a set price on or before a given date. We will consider
current contingent claims whose value can be expressed as a function of the price of
the underlying asset. Denote by g (pj1 ) the value of the claim as a function of the
price pj1 of asset j in period 1. The price of the claim in 0 is denoted by v (pj0 ) :
When we add an additional asset, i.e. the contingent claim under consideration,
to the set of assets considered in the previous section, then optimal behaviour of
households implies
1 u0 (c1 )
v (pj0 ) = E 0 g (pj1 ) : (7.47)
1 + r Eu (c1 )
If all agents were risk neutral, we would know that the price of such a claim would
be given by
1
v (pj0 ) = Eg (pj1 ) (7.48)
1+r
Both equations directly follow from the …rst-order condition for assets. Under risk
neutrality, a household’s utility function is linear in consumption and the …rst deriv-
ative is therefore a constant. Including the contingent claim into the set of assets
available to households, exactly the same …rst-order condition for their pricing would
hold.

139
7.7.3 Risk-neutral valuation
There is a strand in the …nance literature, see e.g. Brennan (1979), that asks under
what conditions a risk neutral valuation of contingent claims holds (risk neutral val-
uation relationship, RNVR) even when households are risk averse. This is identical
to asking under which conditions marginal utilities in (7.46) do not show up. We will
now brie‡y illustrate this approach and drop the index j.
Assume the distribution of the price p1 can be characterized by a density f (p1 ; ) ;
where Ep1 . Then, if a risk neutral valuation relationship exists, the price of the
contingent claim in zero is given by
Z
1
v (p0 ) = g (p1 ) f (p1 ; ) dp1 ; with = (1 + r) p0 :
1+r
This is (7.48) with the expectations operator being replaced by the integral over
realizations g (p1 ) times the density f (p1 ). With risk averse households, the pricing
relationship would read under this distribution
Z
1 u0 (c1 )
v (p0 ) = g (p1 ) f (p1 ; ) dp1 :
1+r Eu0 (c1 )

This is (7.47) expressed without the expectations operator. The expected value
is left unspeci…ed here as it is not a priori clear whether this expected value equals
(1 + r) p0 also under risk aversion. It is then easy to see that a RNVR holds if
u0 (c1 ) =Eu0 (c1 ) = f (p1 ) =f (p1 ; ) : Similar conditions are derived in that paper for
other distributions and for longer time horizons.

7.8 Natural volatility I (*)

Natural volatility is a view about why growing economies experience phases of high
and phases of low growth. The central belief is that both long-run growth and short-
run ‡uctuations are jointly determined by economic forces that are inherent to any real
world economy. Long-run growth and short-run ‡uctuations are both endogenous and
two sides of the same coin: They both stem from the introduction of new technologies.
It is important to note that no exogenous shocks occur according to this approach.
In this sense, it di¤ers from RBC and sunspot models and also from endogenous
growth models with exogenous disturbances (endogenous RBC models).
There are various models that analyse this view in more details and an overview
is provided at http://www.wifak.uni-wuerzburg.de/vwl2/nv. This section will look
at a simple model that provides the basic intuition. More on natural volatility will
follow in ch. ?? and ch. 10.5.

7.8.1 The basic idea

The basic mechanism of the natural volatility literature (and this is probably a nec-
essary property of any model that wants to explain both short-run ‡uctuations and
long-run growth) is that some measure of productivity (this could be labour or total

140
 factor productivity) does not grow smoothly over time as in most models of exogenous
or endogenous long-run growth but that productivity follows a step function.
N
log of
productivity
x
smooth productivity growth
x h
x h
x h
x h productivity step function
x h
h

N
time

Figure 36 Smooth productivity growth in balanced growth models (dashed line) and
step-wise productivity growth in models of natural volatility

With time on the horizontal and the log of productivity on the vertical axis, this
…gure shows as the dashed line a smooth productivity growth path. This is the smooth
growth path that induces balanced growth. In models of natural volatility, the growth
path of productivity has periods of no change at all and points in time of discrete
jumps. After a discrete jump, returns to investment go up and an upward jump in
growth rates results. Growth rates gradually fall over time as long as productivity
remains constant. With the next jump, growth rates jump up again. While this step
function implies long-run growth as productivity on average grows over time, it also
implies short-run ‡uctuations.
The precise economic reasons given for this step function - which is not simply
imposed but always follows from some deeper mechanisms - di¤er from one approach
to the other. A crucial implication of this step-function is the implicit belief that
economically relevant technological jumps take place once every 4-5 years. Each
cycle of an economy and thereby also long-run growth go back to relatively rare
events. Fluctuations in time series that are of higher frequency than these 4-5 years
either go back to exogenous shocks, to measurement error or other disturbances of
the economy.
The step function sometimes captures jumps in total factor productivity, some-
times only in labour productivity of a most recent vintage of a technology. This di¤er-
ence is important for the economic plausibility of the models. Clearly, one should not
build a theory on large aggregate shocks to TFP as those are not easily observable.
Some papers in the literature show indeed how small changes in technology can have
large e¤ects. See Francois and Lloyd-Ellis (2003) or Wälde (2002, 2005). See also
Matsuyama (1999).
This section presents the simples stochastic natural volatility model which allows
to show most easily the di¤erence to exogenous shock models of the business cycle.

141
7.8.2 A simple stochastic model
This section presents the simplest possible model that allows to understand the dif-
ference between the stochastic natural volatility and the RBC approach.
Technologies
Let the technology be described by a Cobb-Douglas speci…cation,
Yt = At Kt Lt1 ;
where At represents total factor productivity, Kt is the capital stock and Lt are hours
worked. Capital can be accumulated according to
Kt+1 = (1 ) Kt + It
The technological level follows
At+1 = (1 + qt ) At
where
0 pt
qt = with probability : (7.49)
1 1 pt
The probability depends on resources Rt invested into R&D,
pt = p (Rt ) :
Clearly, the function p (Rt ) in this discrete time setup must be such that 0 p (Rt )
1:
The speci…cation of technological progress in (7.49) is probably best suited to point
out the di¤erences to RBC type approaches: The probability that a new technology
occurs is endogenous. This shows both the “new growth literature” tradition and
the di¤erences to endogenous growth type RBC models. In the latter approach, the
growth rate is endogenous but shocks are still exogenously imposed. Here, the source
of growth and of ‡uctuations all stem from one and the same source, the jumps in qt
in (7.49).
Optimal behaviour of households
The resource constraint the economy needs to obey in each period is given by
Ct + It + Rt = Y;
where Ct ; It and Rt are aggregate consumption, investment and R&D expenditure,
respectively. Assume that optimal behaviour of households implies consumption and
investment into R&D amounting to
Ct = sK Yt ; Rt = s R Y t ;
where sK and sR are two constant saving rates. This would be the outcome of a two-
period maximization problem or an in…nite horizon maximization problem with some
parameter restriction similar to e.g. Benhabib and Rustichini (1994) or, in continuous
time, Xie (1991,) or Wälde (2005). As the natural volatility literature has various
papers where the saving rate is not constant (Matsuyama, 1999, assumes a constant
saving rate, but see Matsuyama, 2001), it seems reasonable not to fully develop an
optimal saving approach here as it is not central to the natural volatility view.

142
7.8.3 Equilibrium
Equilibrium is determined by

Kt+1 = (1 ) Kt + Yt Rt Ct
= (1 ) Kt + (1 sK s R ) Yt :

plus the random realization of technology jumps, where the probability of a jump
depends on investment in R&D,

pt = p (sR Yt ) :

When we use as auxiliary variable capital per e¤ective labour, kt Kt = (At L) ;

we obtain with KAt+1
tL
= kt+1 AAt+1
t
= kt+1 (1 + qt )

(1 ) kt + (1 sK sR ) k t
kt+1 = : (7.50)
1 + qt
The convergence behaviour is illustrated in the following …gure.

Figure 37 The Sisyphus economy - convergence to a temporary steady state (phasendi-

agram.nb)

The …gure plots the current capital stock per e¤ective labour kt on the horizontal
and kt+1 on the vertical axis. As long as there is no technology jump, i.e. as long
as qt = 0; the capital stock kt converges to its temporary steady state kq just as in
deterministic models. It follows by setting kt+1 = kt = kq in (7.50) and is given by

(1 ) kq + (1 sK sR ) kq
kq = , (qt + ) kq = (1 sK sR ) kq
1 + qt
1=(1 )
1 sK sR
, kq = : (7.51)
qt +

143
 With a new technology, kt = Kt = (At L) decreases and the kt+1 line increases, as
shown by the dashed line in …g. 37. As a consequence, as can also be seen in (7.51),
the steady state increases. Subsequently, the economy approaches the steady state
again. As growth is higher immediately after a new technology is introduced (growth
is high when the economy is far away from its steady state), growth rates after the
introduction of a new technology is high and then gradually falls. Cyclical growth
is therefore characterized by a Sisyphus-type behaviour: kt permanently approaches
the steady state but eventually is thrown back due to the arrival of a new technology.
t.b.c.

7.9 Exercises chapter 7 and further reading

More on basic concepts of random variables than in ch. 7.1 and 7.2 can be found
in Evans, Hastings and Peacock (2000). For a more advanced treatment, see John-
son, Kotz and Balakrishnan (1995). The results on distributions here are used and
extended in Kleiber, Sexauer and Wälde (2006).
Some references on stochastic di¤erence equations include Pötzelberger, Babillot
et al, Vervaat (1979) and Wolfe (1982).

144
 Exercises Chapter 7
Applied Intertemporal Optimization
Stochastic di¤erence equations and
applications

1. Properties of random variables

Is the following function a density function? Draw this function.
j xj
f (x) = a e ; > 0: (7.52)

State possible assumptions about the range of x and values for a:

2. The properties of uncertain technological change

Assume that total factor productivity in (7.14) is given by

(a)
A with p
At = ;
A with 1-p
¯
(b) the density function
f (At ); At 2 A,Ā ;
¯
(c) the probability function g(At ); At 2 fA1 ; A2 ; :::; An g ;
(d) At = Ai with probability pi and
(e) At+1 = At + "t+1 : Make assumptions for and "t+1 and discuss them.

What is the expected total factor productivity and what is its variance? What
is the expected output level and what is its variance, given a technology as in
(7.13)?

3. Stochastic di¤erence equations

Consider the following stochastic di¤erence equation
2
yt+1 = byt + m + vt+1 ; vt ~N (0; v)

(a) Describe the limiting distribution of yt .

(b) Does the expected value of yt converge to its …xpoint monotonically? How
does the variance of yt evolve over time?

145
4. Saving under uncertainty
Consider an individual’s maximization problem

max Et fu (ct ) + u (ct+1 )g

subject to wt = ct + st ; (1 + rt+1 ) st = ct+1

(a) Solve this problem by replacing her second period consumption by an ex-
pression that depends on …rst period consumption.
(b) Consider now the individual decision problem given the utility function
u (ct ) = ct : Should you assume a parameter restriction on ?
(c) What is your implicit assumption about ? Can it be negative or larger
than one? Can the time preference rate ; where = (1 + ) 1 ; be nega-
tive?

5. Closed-form solution for CES utility function

Let households maximize a CES-utility function

Ut = Et c1t + (1 1
) ct+1 = c1t + (1 ) Et c1t+1

subject to budget constraints (7.18) and (7.19).

(a) Show that an optimal consumption-saving decision given budget constraints

(7.18) and (7.19) implies savings of
wt
st = n o" :
1+ (1 )

Show further that consumption when old is

wt
ct+1 = (1 + rt+1 ) n o"
1+ (1 )

and that consumption of the young is

n o"
(1 )
ct = n o" wt : (7.53)
1+ (1 )

(b) Discuss the link between the savings expression st and the one for the loga-
rithmic case in (7.24). Point out why ct+1 is uncertain from the perspective
of t: Is ct uncertain from the perspective of t - and of t 1?

6. OLG in general equilibrium

Build an OLG model in general equilibrium with capital accumulation and auto-
regressive total factor productivity, ln At+1 = ln At + "t : What is the reduced
form? Can a phase-diagram be drawn?

146
7. Asset pricing
Under what conditions is there a risk neutral valuation formula for assets? In
other words, under which conditions does the following equation hold?
1
pjo = Epj1
1+r

147
8 Multi-period models
Uncertainty in dynamic models is probably most often used in discrete time models.
Looking at time with a discrete perspective has the advantage that timing issues
are very intuitive: Something happens today, something tomorrow, something the
day before. Time in the real world, however, is continuous. Take any two points
in time, you will always …nd a point in time in between. What is more, working
with continuous time models under uncertainty has quite some analytical advantages
which make certain insights - after an initial more heavy investment into techniques
- much simpler.
We will nevertheless follow the previous structure of this book and …rst present
models in discrete time. They are also, as mentioned, probably the more widely used
models in the literature.

8.1 Dynamic programming

Maximization problems in discrete times can be solved by using many methods. One
particularly useful one is - again - dynamic programming. We therefore start with
this method and consider the stochastic sibling to the deterministic case in ch. 3.4.
Due to uncertainty, our objective function is slightly changed. In contrast to (3.1),
it now reads
t
Ut = Et 1=t u (c ) ; (8.1)
where the only di¤erence consists in the expectations operator Et : As we are in
an environment with uncertainty, we do not know all consumption levels c with
certainty. We therefore need to form expectations about the implied instantaneous
utility from consumption, u (c ). The budget constraint is given by

xt+1 = f (xt ; ct ; "t ) : (8.2)

This constraint shows why we need to form expectations about future utility: The
value of the state variable xt+1 depends on some random source, denoted by "t : Think
of this "t as uncertain TFP or uncertain returns to investment.
The optimal program is de…ned in analogy to (3.9) by

V (xt ; "t ) max Ut subject to xt+1 = f (xt; ct ; "t ) :

fc g

The additional term in the value function is "t : It is useful to treat "t explicitly as a
state variable for reasons we will soon see.
In order to solve the maximization problem, we again follow the three step scheme.

DP1: Bellman equation and …rst-order conditions

The Bellman equation is

V (xt ; "t ) = max fu (ct ) + Et V (xt+1 ; "t+1 )g :

ct

148
It again exploits the fact that the objective function (8.1) is additively separable,
despite the expectations operator, assumes optimal behaviour as of tommorrow and
shifts the expectations operator behind instantaneous utility of today as u (ct ) is
certain given that ct is a control variable. The …rst-order condition is

@xt+1 @f (:)
u0 (ct ) + Et Vxt+1 (xt+1 ; "t+1 ) = u0 (ct ) + Et Vxt+1 (xt+1 ; "t+1 ) = 0: (8.3)
@ct @ct
It corresponds to the …rst-order condition (3.12) in a deterministic setup. The “only”
di¤erence consists in the expectations operator Et : Equation (8.3) provides again
an (implicit) functional relationship between consumption and the state variable,
ct = ct (xt ; "t ) :

DP2: Evolution of the costate variable

The derivative of the maximized Bellman equation with respect to the state vari-
able is (using the envelope theorem)

@xt+1
Vxt (xt ; "t ) = Et Vxt+1 (xt+1 ; "t+1 ) :
@xt
Observe that xt+1 by the constraint (8.2) is given by f (xt ; ct ; "t ) ; i.e. by quantities
that are known in t: Hence, the derivative @xt+1 =@xt = fxt is non-stochastic and we
can write this expression as (note the similarity to (3.13))

Vxt (xt ; "t ) = fxt Et Vxt+1 (xt+1 ; "t+1 ) : (8.4)

This step makes clear why it is useful to include "t as a state variable into the ar-
guments of the value function. If we had not done so, one could get the impression,
for the same argument that xt+1 is known in t due to (8.2), that the shadow price
Vxt (xt+1 ; "t+1 ) is non-stochastic as well and the expectations operator would not be
needed. Given that the value of the optimal program depends on "t ; however, it is
clear that V 0 (xt+1 ; "t+1 ) is random in t indeed. (Some of the subsequent applications
will treat "t as an implicit state variable and not always be as explicit as here.)

DP3: Inserting …rst-order conditions

Using that @f (:) =@ct = fct is non-random in t; we can rewrite the …rst-order con-
dition as u0 (ct ) + fct Et Vxt+1 ((xt+1 ; "t+1 )) = 0: Inserting it in (8.4) gives Vxt (xt ; "t ) =
fxt 0 fx
f ct
u (ct ) : Shifting this expression by one period yields Vxt+1 (xt+1 ; "t+1 ) = fc t+1 u0 (ct+1 ) :
t+1
Inserting this into the costate equation (8.4) again, we obtain

fct
u0 (ct ) = Et fx u0 (ct+1 ) :
fct+1 t+1

149
8.2 Household utility maximization
Let us now look at a …rst example - a household that maximizes utility. The objective
function is given by (8.1),
t
Ut = Et 1=t u (c ) :
It is maximized subject to a budget constraint of which we know from (??) or (3.33)
that it …ts well into a general equilibrium setup,

at+1 = (1 + rt ) at + wt pt ct :

Note that the budget constraint must hold after realization of random variables, not
in expected terms. From the perspective of t; all prices (rt , wt , pt ) in t are known,
prices in t + 1 are uncertain.

DP1: Bellman equation and …rst-order conditions

Having understood in the previous general chapter that uncertainty can explicitly
be treated in the form of a state variable, we limit our attention to the endogenous
state variable at here. It will turn out that this keeps notation simpler. The value of
optimal behaviour is therefore expressed by V (at ) and the Bellman equation can be
written as
V (at ) = max fu (ct ) + Et V (at+1 )g :
ct

The …rst-order condition for consumption is

dat+1
u0 (ct ) + Et V 0 (at+1 ) = u0 (ct ) Et V 0 (at+1 ) pt = 0:
dct
It can be written as
u0 (ct ) = pt Et V 0 (at+1 ) (8.5)
as the price pt is known in t:

DP2: Evolution of the costate variable

Di¤erentiating the maximized Bellman equation gives (using the envelope theo-
rem)
dat+1
V 0 (at ) = Et V 0 (at+1 ) , V 0 (at ) = [1 + rt ] Et V 0 (at+1 ) : (8.6)
dat
Again, the term [1 + rt ] was put in front of the expectations operator as rt is
known in t: This di¤erence equation describes the evolution of the shadow price of
wealth in the case of optimal consumption choices.

DP3: Inserting …rst-order conditions

150
 0
Inserting the …rst-order condition (8.5) gives V 0 (at ) = [1 + rt ] u p(ct t ) : Inserting this
expression twice into the di¤erentiated maximized Bellman equation (8.6) gives a nice
Euler equation,
u0 (ct ) u0 (ct+1 ) u0 (ct ) u0 (ct+1 )
[1 + rt ] = [1 + rt ] Et [1 + rt+1 ] , = Et :
pt pt+1 pt (1 + rt+1 ) 1 pt+1
(8.7)
Rewriting it as we did before with (7.40), we get
( )
u0 cit+1 pt
Et =1
u0 (cit ) (1 + rt+1 ) 1 pt+1
which allows to given the same interpretation as the …rst-order condition in a two-
period model, both deterministic (as in eq. (2.9) in ch. 2.2.2) and stochastic (as in
eq. (7.38) in ch. 7.6) and as in deterministic in…nite horizon models (as in eq. (3.5) in
ch. 3.2): Relative marginal utility must be equal to relative marginal prices - taking
into account that marginal utility in t + 1 is discounted at and the price in t + 1 is
discounted by using the interest rate.
Using two further assumption, the expression in (8.7) can be rewritten such that
we come even closer to the deterministic Euler equations: First, let us choose the
output good as numeraire and thereby set prices pt = pt+1 = p constant. This allows
us to remove pt and pt+1 from (8.7). Second, we assume that the interest rate is
known (say we are in a small open economy with international capital ‡ows). Hence,
expectations are formed only with respect to consumption in t + 1: Taking these two
aspects into account, we can write (8.7) as
u0 (ct ) 1
0
= : (8.8)
Et u (ct+1 ) 1= (1 + rt+1 )
This is now as close as possible to (2.9) and (3.5). The ratio of (expected discounted)
marginal utilities is identical to the ratio of relative prices.

8.3 A central planner in an oil-importing economy

Consider another economy where output is produced with oil,
Yt = At Kt Ot L1t
Again, total factor productivity At is stochastic. Now, the price of oil, qt ; is stochastic
as well. Let capital evolve according to
Kt+1 = (1 ) Kt + Yt qt O t Ct
which is a trade balance and good market clearing condition, all in one. The central
planner maximizes
t
max Et 1=t u (C )
fC ;O g

by choosing a path of aggregate consumption ‡ows C :and oil consumption Ot : At

t, all variables indexed t are known. The only uncertainty consists about At+1 and
about the price of oil in future periods.

151
 DP1: Bellman equation and …rst-order conditions

The Bellman equation reads V (Kt ) = maxCt ;Ot fu (Ct ) + Et V (Kt+1 )g : The …rst-
order condition for consumption is

dV (Kt+1 )
u0 (Ct ) + Et [ 1] = 0 , u0 (Ct ) = Et V 0 (Kt+1 ) (8.9)
dKt+1
For oil it reads
@ @Yt
Et V 0 (Kt+1 ) [Yt qt O t ] = 0 , = qt :
@Ot @Ot

DP2: Evolution of the costate variable

The derivative of the Bellman equation with respect to the capital stock Kt gives
(using the envelope theorem)

@ @Yt
V 0 (Kt ) = Et V (Kt+1 ) = 1 + Et V 0 (Kt+1 ) (8.10)
@Kt @Kt
h i
@Yt
just as in the economy without oil. The term 1 + @Kt
can be pulled out of the
@Yt
expectation operator as At and thereby @Kt
is known at the moment of the savings
decision.

DP3: Inserting …rst-order conditions

Following the same steps as above without oil, we again end up with

@Yt+1
u0 (Ct ) = Et u0 (Ct+1 ) 1 +
@Kt+1

The crucial di¤erence is that now expectations are formed with respect to technolog-
ical uncertainty and with respect to uncertainty about the price of oil.

8.4 Asset pricing in a one-asset economy

This section takes a simple stochastic model with one asset, physical capital. It then
derives an equation that expresses the price of capital in terms of income streams
from holding capital. In order to be as explicit as possible about the nature of this
(real) capital price, we do not choose a numeriare good in this section.

152
8.4.1 The model
Technologies
The technology used to produce a homogenous output good is of the simple Cobb-
Douglas form
Y t = A t Kt L 1 ; (8.11)
where TFP At is stochastic. Labour supply L is exogenous and …x, the capital stock
is denoted by Kt :
Households
Preferences of households are standard and given by
1 t
Ut = Et =t u (c ) :
We start from a budget constraint that can be derived as the budget constraint
(2.42) in a deterministic world. Wealth is held in units of capital Kt where the price
of one unit is vt : When we de…ne the interest rate as
wtK
rt (8.12)
vt
the budget constraint (2.42) reads
wt pt
kt+1 = (1 + rt ) kt + ct : (8.13)
vt vt
Goods market
Investment and consumption goods are traded on the same goods market. Total
supply is given by Yt ; demand is given by gross investment Kt+1 Kt + Kt and
consumption Ct . Expressed in a well-known way, goods market equilibrium yields
the resource constraint of the economy,
Kt+1 = Kt + Yt Ct Kt : (8.14)

8.4.2 Optimal behaviour

Firms maximize instantaneous pro…ts which implies …rst-order conditions
wt = pt @Yt =@L; wtK = pt @Yt =@Kt : (8.15)
Factor rewards are give by their value marginal products.
Given the preferences of households and the contraint in (8.13), optimal behaviour
of households is described by (this follows identical steps as e.g. in ch. 8.2 and is
treated in ex. 2),
u0 (Ct ) u0 (Ct+1 )
= Et : (8.16)
pt =vt (1 + rt+1 ) 1 pt+1 =vt+1
This is the standard Euler equation extended for prices, given that we have not chosen
a numeraire. We replaced ct by Ct to indicate that this is the evolution of aggregate
(and not individual) consumption.

153
8.4.3 The pricing relationship
Let us now turn to the main objective of this section and derive an expression for the
real price of one unit of capital, i.e. the price of capital in units of the consumption
good. Starting from the Euler equation (8.16), we insert the interest rate (8.12) in
its general formulation, i.e. including all prices, and rearrange to …nd

vt wK vt+1 vt+1 @Yt+1

u0 (Ct ) = Et u0 (Ct+1 ) 1 + t+1 = Et u0 (Ct+1 ) (1 + ):
pt vt+1 pt+1 pt+1 @Kt+1
(8.17)
Now de…ne a discount factor t+1 u0 (Ct+1 ) =u0 (Ct ) and dt as the “net dividend
payments”, i.e. payments to the owner of one unit of capital. Net dividend payments
per unit of capital amount to the marginal product of capital @Yt =@Kt minus the
share of capital that depreciates - that goes “kaputt” - each period times the real
@Yt vt
price vt =pt of one unit of capital, dt @K t pt
: Inserting this yields

vt vt+1
= Et t+1 dt+1 + : (8.18)
pt pt+1

Note that all variables uncertain from the perspective of today in t appear behind
the expectations operator.
Now assume for a second we were in a deterministic world and the economy is in
a steady state. Equation (8.18) hcould then bei written with t+1 = 1 and without the
vt vt+1
expectations operator as pt = dt+1 + pt+1 : Solving this linear di¤erential equation
forward, starting in vt and inserting repeatedly gives

vt vt+4
= dt+1 + dt+2 + dt+3 + dt+4 +
pt pt+4
2 3 4 4 vt+4
= dt+1 + dt+2 + dt+3 + dt+4 + :
pt+4
Continuing to insert, one eventually and obviously ends up with
vt T s T vt+T
= s=1 dt+s + :
pt pt+T
The price vt of a unit of capital is equal to the discounted sum of future dividend pay-
ments plus its discounted price (once sold) in t+T . In an in…nite horizon perspective,
this becomes
vt s vt+T
= 1 s=1 dt+s + lim T :
pt T !1 pt+T
In our stochastic setup, we can proceed according to the same principles as in
the deterministic world but need to take the expectations operator
h and thei discount
vt+1 vt+2
factor t into account. We replace pt+1 in (8.18) by Et+1 t+2 dt+2 + pt+2 and then

154
vt+2 =pt+2 and so on to …nd

vt vt+3
= Et t+1 dt+1 + Et+1 t+2 dt+2 + Et+2 t+3 dt+3 +
pt pt+3
2 3 3 vt+3
= Et t+1 dt+1 + t+1 t+2 dt+2 + t+1 t+2 t+3 dt+3 + t+1 t+2 t+3
pt+3
3 vt+3
= Et s=1 s dt+s + Et 3 ; (8.19)
pt+3
where we de…ned the discount factor to be

s s s s u0 (Ct+n ) su
0
(Ct+s )
s n=1 t+n = n=1 = :
u0 (Ct+n 1 ) u0 (Ct )

The discount factor adjusts discounting by the preference parameter ; by relative

marginal consumption and by prices. Obviously, (8.19) implies for larger time hori-
zons pvtt = Et Ts=1 s dt+s + Et T vt+T : Again, with an in…nite horizon, this reads
vt 1
= Et s=1 s dt+s + lim Et T vt+T : (8.20)
pt T !1

The real price vt =pt amounts to the discounted sum of future dividend payments dt+s :
The discount factor is s which contains marginal utilities, relative prices and the
individual’s discount factor : The term limT !1 Et T vt+T is a “bubble term”for the
price of capital and can usually be set equal to zero. As the derivation has shown, the
expression for the price vt =pt is “simply”a rewritten version of the Euler equation.

8.4.4 More real results

The price of capital again

The result on the determinants of the price of capital is useful for economic in-
tuition and received a lot of attention in the literature. But can we say more about
the real price of capital? The answer is yes and it comes from the resource constraint
(8.14). This constraint can be understood as a goods market clearing condition. The
supply of goods Yt equals demand resulting from gross investment Kt+1 Kt + Kt
and consumption. The price of one unit of the capital good therefore equals the price
of one unit of the consumption and the output good, provided that investment takes
place, i.e. It > 0: Hence,
vt = pt: : (8.21)
The real price of capital vt =pt is just equal to one. Not surprisingly, after all, capital
goods and consumption goods are traded on the same market.

The evolution of consumption (and capital)

155
 When we want to understand what this model tells us about the evolution of
consumption, we can look at a modi…ed version of (8.16) by inserting the interest
rate (8.12) with the marginal product of capital from (8.15) and the price expression
(8.21),
@Yt+1
u0 (Ct ) = Et u0 (Ct+1 ) 1 + :
@Kt+1
This is the standard Euler equation (see e.g. (8.8)) that predicts how real consumption
evolves over time, given the real interest rate and the discount factor :
Together with (8.14), we have a system in two equations that determine Ct and
Kt (given appropriate boundary conditions). The price pt and thereby the value vt
can not be determined (which is of course a consequence of Walras’law). The relative
price is trivially unity from (8.21), vt =pt = 1: Hence, the predictions concerning real
variables do not change when a numeraire good is not chosen.9

An endowment economy

Consider an individual that can save in one asset and whose budget constraint is
given by (8.13). Let this household behave optimally such that optimal consumption
follows (8.17).
Now change the capital accumulation equation (8.14) such that - for whatever
reasons - K is constant and let also, for simplicity, depreciation be zero, = 0. Then,
output is given according to (8.11) by Yt = At K L1 ; i.e. it follows some exogenous
stochastic process, depending on the realization of At . This is the exogenous endow-
ment of the economy for each period t: Further, consumption equals output in each
period, Ct = Yt :
Inserting output into the Euler equation (8.17) gives

vt vt+1 @Yt+1
u0 (Yt ) = Et u0 (Yt+1 ) (1 ) +
pt pt+1 @Kt+1

The equation shows that in an endowment economy where consumption is exoge-

nously given at each point in time and households save by holding capital (which
is constant on the aggregate level), the price vt =pt of the asset changes over time
such that households want to consume optimally the exogenously given amount Yt :
This equation provides a description of the evolution of the price of the asset in an
endowment economy. These aspects were analysed e.g. by Lucas (1978) and many
others.
9
This statement is not as obvious as it might appear from remembering Walras’law from under-
graduate micro courses. There is a literature that analyses the e¤ects of the choice of numeraire for
real outcomes of the economy when there is imperfect competition. See e.g. Gabszewicz and Vial
(1972) or Dierker and Grodahl (1995).

156
8.5 Asset pricing with many assets (*)
8.5.1 A many-sector economy
Consider an economy consisiting of many sectors i. Each sector produces a good Y (i)
using capital and labour,
Yt (i) = A (i) K (i) L (i)1 a
:
Let TFP A (i) be uncertain. Capital is sector speci…c and pays a sector speci…c return
@Yt (i)
vt+1 (i) vt (i) + @Kt (i)
rt (i) = : (8.22)
vt (i)
This return takes physical depreciation into account.

8.5.2 The household´s maximization problem (*)

Consider a household that has an investment and a consumption problem. This is a
typical CAP setup. She maximizes
1 t
max Et t=0 u (Ct )
fCt ; t (i)g

subject to
at+1 = (1 + rt ) at + wt pt ct (8.23)
n
where wealth is given by at = i=1 vit kit ; the interest rate is is the weighted sum of
asset i speci…c interest rates,
n
rt = i=1 it rit : (8.24)
The weights it vit kit =at are given by the share of wealth held in asset i at t: They
sum to unity,
n n
i=1 it =1, t (1) = 1 i=2 it :

In order to solve this maximization problem, we follow the three dynamic program-
ming steps.
DP1: Bellman equation and …rst-order conditions
The Bellman equation reads
V (at ) = fu (ct ) + Et V (at+1 )g (8.25)
and …rst-order conditions are for consumption
u0 (ct ) = Et V 0 (at+1 ) pt (8.26)
and for shares t (i)
drt
Et V 0 (at+1 ) at = 0 , Et V 0 (at+1 ) [rt (1) rt (i)] = 0 (8.27)
d t (i)
In order to get an expression that allows us to link the value of an asset to future
dividend payments as in (8.16), we rewrite the foc as follows.

157
 DP2: Evolution of the costate variable

First compute the derivative of the maximized Bellman equation with respect to
assets at : With the envelope theorem, this gives

V 0 (at ) = Et V 0 (at+1 ) (1 + rt ) = Et V 0 (at+1 ) + Et V 0 (at+1 ) rt

P
= Et V 0 (at+1 ) + 0
t (i) Et V (at+1 ) rt (i) (8.28)

As (8.27) is identical to

Et V 0 (at+1 ) rt (1) = Et V 0 (at+1 ) rt (i) (8.29)

we can write (8.28) as

P
n
V 0 (at ) = Et V 0 (at+1 ) + t (i) Et V 0 (at+1 ) rt (1)
i=1
= Et V 0 (at+1 ) + Et V 0 (at+1 ) rt (1) (8.30)
P
where the last step used the fact that only t (i) depends on i and t (i) = 1: As
this last step can be undertaken with any sector i, we have

V 0 (at ) = Et V 0 (at+1 ) (1 + rt (i))

and also
Et V 0 (at+1 ) (1 + rt (i)) = Et V 0 (at+1 ) (1 + rt (1)) : (8.31)
Hence, with the derivative of Bellman equation we obtained this alternative version
(8.31) for the …rst-order condition (8.27) for shares.
Now let´s assume, there is one riskless asset, and call this asset ”asset number 1”.
Then (8.30) reads
V 0 (at ) = (1 + rt (1)) Et V 0 (at+1 ) (8.32)
where the interest rate was pulled in front of the expectation operator as rt (1) is
certain. This is the evolution of the shadow price v 0 (at ) which we need (in addition
to (8.31)) in what follows.

DP3: Inserting …rst-order conditions

With (8.26) and (8.32) we then get a relationship between marginal utility from
consumption and the marginal value of wealth,10

u0 (ct )
V 0 (at ) = (1 + rt (1)) : (8.33)
pt
10
Without this relationship, a ”nice” Euler equation as (8.34) could not be derived. Hence, the
assumption of a risk less asset is crucial.

158
Inserting this twice into (8.32) yields
u0 (ct ) u0 (ct+1 )
(1 + rt (1)) = (1 + rt (1)) Et (1 + rt+1 (1)) ,
pt pt+1
u0 (ct ) u0 (ct+1 )
= Et (8.34)
pt (1 + rt (1)) 1 pt+1
With the usual interpretation (compare (8.7) in ch. 8.2), this is the Euler equation
for savings in the risky asset.
The …rst-order condition (8.31) for assets becomes with (8.33)
u0 (ct+1 ) u0 (ct+1 )
Et (1 + rt (i)) = Et (1 + rt (1)) (8.35)
pt+1 pt+1
where we used again that rt (1) is certain t. This expression can also be written as
u0 (ct+1 ) u0 (ct+1 )
Et = Et :
(1 + rt+1 (1)) 1 pt+1 (1 + rt+1 (i)) 1 pt+1
The Euler equation (8.34) can therefore alternatively be written as
u0 (ct ) u0 (ct+1 )
= Et (8.36)
pt (1 + rt+1 (i)) 1 pt+1
This is the Euler equation expressing returns in terms of the risky asset i: Comparing
(8.34) and (8.36) shows the indi¤erence between marginally saving in asset i or the
risken asset.

8.5.3 The fundamental pricing rule

Remember the de…nition of the interest rate in (8.22). Insert in (8.36) and …nd
u0 (ct ) u0 (ct+1 ) wtK + vt+1 u0 (ct+1 )
= Et (1 + rt+1 (i)) Et
pt pt+1 vt pt+1
@Yt
where wtK = @K t
vt are de…ned as divided payments. Multiply by vt (which is
known at t), divide by u0 (ct ) and …nd
u0 (ct+1 ) =pt+1
v t = Et wtK + vt+1 :
u0 (ct ) =pt
Now express this equation for the next period, i.e. replace t by t + 1 and get vt+1 =
0
K
Et+1 wt+1 + vt+2 uu0 (c t+2 )=pt+2
(ct+1 )=pt+1
: Inserting this yields

u0 (ct+2 ) =pt+2 u0 (ct+1 ) =pt+1

vt = Et wtK + Et+1 wt+1 K
+ vt+2 0
u (ct+1 ) =pt+1 u0 (ct ) =pt
( 0 )
2 K u0 (ct+2 )=pt+2 u0 (ct+1 )=pt+1
wtK u (cu0t+1 )=pt+1
(ct )=pt
+ w t+1 0
u (ct+1 )=pt+1 0
u (ct )=pt
= Et Et+1 0 (c 0 :
+ 2 vt+2 uu0 (c t+2 )=pt+2 u (ct+1 )=pt+1
t+1 )=pt+1 u0 (ct )=pt

159
Noting that Et Et+1 = Et and inserting very often, one obtains

t+n s u0 (cj+1 ) =pj+1 u0 (cj+1 ) =pj+1

vt = Et s=t j=t wsK + vt+n s
j=t :
u0 (cj ) =pj u0 (cj ) =pj

Hence, the value of a …rm is the discounted sum of future divided payments plus
u0 (cj+1 )=pj+1
a bubble term. The interesting aspect is that the discount factor sj=t u0 (cj )=pj
depends on preferences of households.

8.5.4 The CAP formula

The well-known CAP formula relates excess returns to volatility measures. This
relationship can be obtained here as well by starting from the …rst-order condition
(8.27) and following steps similar to those undertaken further below in the continuous
time model in ch. 10.4.3.

8.6 Matching and unemployment

Consider a …rm that can change the number of employees only slowly due to searching
and matching on the labour market.
- coming soon -

8.7 How to do without dynamic programming

8.7.1 Individual utility maximization
We will now get to know a method that allows us to solve stochastic intertemporal
problems in discrete time without dynamic programming. One could ask oneself after
this chapter why dynamic programming exists at all ...
t
The objective is again maxfct g E0 1 t=0 u (ct ) subject to the constraint at+1 =
(1 + rt ) at + wt ct : Again, the control variables of the household are fct g ; the state
variable is at ; the interest rate rt and the wage wt are exogenously given. Now rewrite
the objective function and insert the constraint twice,
s 1 t
E0 t=0 u (ct ) + s u (cs ) + s+1 u (cs+1 ) + 1 t
t=s+2 u (ct )
s 1 t
= E0 t=0 u (ct ) + E0 s u ((1 + rs )as + ws as+1 )
s+1 1 t
+ E0 ((1 + rs+1 )as+1 + ws+1 as+2 ) + E0 t=s+2 u (ct ) :

Note that the expectations operator always refers to knowledge available at the be-
ginning of the planning horizon, i.e. to t = 0:
Now compute the …rst-order condition with respect to as+1 : This is unusual as we
directly choose the state variable which is usually understood to be indirectly chosen
by the control variable. Clearly, however, this is just a convenient trick and we really,
by choosing as+1 ; a state variable, we choose cs . The derivative with respect to as+1
yields
E0 s u0 (cs ) = E0 s+1 u0 (cs+1 ) (1 + rs+1 ):

160
This almost looks like the standard optimal consumption rule. The di¤erence consists
in the expectations operator being present on both sides. This is not surprising as we
optimally chose as+1 (i.e. cs ), knowing only the state of the system in t = 0: If we now
assume we are in s; our expectations would be based on knowledge in s and we could
replace E0 by Es : We would then obtain Es s u0 (cs ) = s u0 (cs ) for the left-hand side
and our optimality rule reads
u0 (cs ) = Es u0 (cs+1 ) (1 + rs+1 ):
This is the rule we know from Bellman approaches, provided e.g. in (8.7).

8.7.2 Contingent claims pricing

Let us now ask how assets would be priced that pay a return only in T periods.
Consider an economy with an asset that pays a return r in each period and one long-
term asset which can be sold only after T periods. Assuming that investors behave
rationally, i.e. they maximize an intertemporal utility function subject to constraints,
the price of the long-term asset can be most easily found by using a Lagrange approach
or by straightforward inserting.
We assume that an investor maximizes her expected utility E0 Tt=0 t u (ct ) subject
to the constraints
c0 + m0 p0 + a0 = w0
ct + at = (1 + r)at 1 + wt ; 1 t T 1;
cT = m0 pT + (1 + r) aT 1 + wT :

In period 0; the individual uses labour income w0 to pay for consumption goods c0 ; to
buy m0 units of the long-term asset and for “normal”assets a0 . In periods 1 to T 1;
the individual uses her assets at 1 plus returns r on assets and her wage income wt to
…nance consumption and buy again assets at (or keep those of the previous period).
In the …nal period T; the long-term asset has a price of pT and is sold. Wealth of the
previous period plus interest plus labour income wT are further sources of income to
pay for consumption cT .
This maximization problem can most easily be solved by inserting consumption
levels for each period into the objective function. The objective function then reads
T 1 t
u (w0 m0 p0 a0 ) + E0 t=1 u (wt + (1 + r)at 1 at )
T
+ E0 u(m0 pT + (1 + r)aT 1 + wT )
and the control variables are wealth holdings at in periods t = 0; :::; T 1 and the
number of assets m0 bought in period zero.
Let us now look at the …rst-order conditions. The …rst-order condition for wealth
in period zero is
u0 (c0 ) = (1 + r) E0 u0 (c1 ) : (8.37)
Wealth holdings in any period t > 0 are optimally chosen according to
E0 t u0 (ct ) = E0 t+1
(1 + r)u0 (ct+1 ) , E0 u0 (ct ) = (1 + r) E0 u0 (ct+1 ) : (8.38)

161
We can insert (8.38) into the …rst-period condition (8.37) su¢ ciently often and …nd

u0 (c0 ) = (1 + r)2 2
E0 u0 (c2 ) = ::: = (1 + r)T T
E0 u0 (cT ) (8.39)

The …rst-order condition for the number of assets is

T
p0 u0 (c0 ) = E0 u0 (cT )pT : (8.40)

When we insert combined …rst-order conditions (8.39) for wealth holdings into the
…rst-order condition (8.40) for assets, we obtain
T T
p0 (1 + r)T E0 u0 (cT ) = E0 u0 (cT )pT ,
u0 (cT )
p0 = (1 + r) T E0 pT :
E0 u0 (cT )

which is an equation where we nicely see the analogy to the two period example in
1 u0 (c1 )
ch. 7.7.1. Instead of pj0 = 1+r E Eu 0 (c ) j1 in (7.46) where we discount by one period
1
p
only and evaluate returns at expected marginal utility in period 1, we discount by T
periods and evaluate returns at marginal utility in period T:
This equation also o¤ers a lesson for life when we assume risk-neutrality for sim-
plicity: if the payo¤ pT of a long-term asset is not high enough such that the current
price is higher than the present value of the payo¤, p0 > (1 + r) T pT , then the long-
term asset is simply dominated by short-term investments that pay a return of r
per period. Optimal behaviour would imply not to buy the long-term asset and just
put wealth in “normal” assets. This should be kept in mind when talking next time
to your insurance agent who tries to sell you life-insurance or private pension plans.
Just ask for the present value of the payo¤s and compare them to the present value
of what you pay into the savings plan.

8.7.3 Sticky prices (*)

The setup

Sticky prices are a fact of life. In macro economic models, they are either assumed
or result from some adjustment-cost setup. Here is a simpli…ed way to derive sluggish
price adjustment. This example is inspired by Ireland (2004 NBER WP), going back
to Rotemberg (1982 JPE).
The objective function of the …rm is to maximize its present value de…ned by the
sum of discounted expected pro…ts,
t
1 1
Vt = Et =t :
1+r

Pro…ts at a point t in time are given by

=p x w l (p ; p 1)

162
where (p ; p 1 ) are price adjustment costs. These are similar in spirit to the
adjustment costs presented in ch. 5.5.1. We will later use a speci…cation given by
2
p p 1
(p ; p 1) = : (8.41)
2 p 1

This speci…cation only captures the essential mechanism that is required to make
prices sticky, i.e. the fact that the price change is squared (in fact, as with all
adjustment cost mechanisms, any power larger than 1 should do the job). More care
about economic implications needs to be taken when a reasonable model is to be
speci…ed.
The …rm uses a technology
x =A l :
We assume that there is a certain demand elasticity " for the …rm’s output. This
can re‡ect a monopolistic competition setup. The …rm can choose its output x at
each point in time freely by hiring the corresponding amount of labour l : Labour
productivity A or other quantities can be uncertain.

A solution

Inserting everything into the objective function yields

t
1 1 w
Vt = Et =t p x x (p ; p 1)
1+r A
wt wt+1
= Et pt xt xt (pt ; pt 1 ) + Et pt+1 xt+1 xt+1 (pt+1 ; pt )
At At+1
t
1 1 w
+ Et =t+2 p x x (p ; p 1) :
1+r A

The second and third line present a rewritten way that allows to better see the in-
tertemporal structure of the maximization problem. We now maximize this objective
function by choosing output xt for today. (Output levels in the future are chosen at
a later stage.)
The …rst-order condition is
d [pt xt ] wt d (pt ; pt 1 ) d (pt+1 ; pt )
Et Et =0,
dxt At dxt dxt
d [pt xt ] wt d (pt ; pt 1 ) d (pt+1 ; pt )
= + + Et :
dxt At dxt dxt
It has certain well-known components and some new ones. If there were no adjustment
costs, i.e. (:) = 0; the intertemporal problem would become a static one and the
usual condition would equate marginal revenue d [pt xt ] =dxt with marginal cost wt =At .
With adjustment cost, however, a change in output today not only a¤ects adjustment
cost today d (pdx
t ;pt 1 )
t
but also (expected) adjustment cost Et d (pdx
t+1 ;pt )
t
tomorrow. As

163
all variables with index t are assumed to be known in t; expectations are formed only
with respect to adjustment cost tomorrow in t + 1:
Specifying the adjustment cost function as in (8.41) and computing marginal rev-
enue using the demand elasticity "t gives
" # " #
2 2
d [pt xt ] wt d pt pt 1 d pt+1 pt
= + + Et ,
dxt At dxt 2 pt 1 dxt 2 pt
dpt 1 wt pt pt 1 1 dpt pt+1 pt d pt+1
xt + pt = 1 + t pt = + + Et
dxt At pt 1 pt 1 dxt pt dxt pt
wt pt pt 1 pt pt+1 pt pt+1 dpt
= + "t 1 Et ,
At pt 1 xt pt 1 pt p2t dxt
wt pt pt 1 pt pt+1 pt pt+1 pt 1
1 + t 1 pt = + "t 1 Et " ,
At pt 1 xt pt 1 pt p2t xt t
wt pt pt 1 pt 1 pt+1 pt
1 + t 1 pt = + "t 1 "t 1 Et pt+1
At pt 1 xt pt 1 pt xt pt
dxt pt
where "t dpt xt
is the demand elasticity for the …rm’s good. Again, = 0 would
give the standard static optimality condition 1 + t 1 pt = wt =At where the price is
a markup over marginal cost. With adjustment cost, prices change only slowly.

8.7.4 Optimal employment with adjustment cost (*)

The setup

Consider a …rm that maximizes pro…ts as in ch. 5.5.1,

1 1
t = Et =t t : (8.42)
(1 + r)

We are today in t; time extends until in…nity and the time between today and in…nity
is denoted by : There is no particular reason why the planning horizon is in…nity in
contrast to ch. 5.5.1. The optimality condition for employment we will stress here is
identical to a …nite horizon problem.
Instantaneous pro…ts are given by the di¤erence between revenue in t; which is
identical to output F (Lt ) with an output price normalized to unity, labour cost wt Lt
and adjustment cost (Lt Lt 1 ) ;

t = F (Lt ) wt Lt (Lt Lt 1 ) : (8.43)

Costs induced by the adjustment of the number of employees between the previous
period and today are captured by (:). Usually, one assumes costs both from hiring
and from …ring individuals, i.e. both for an increase in the labour force, Lt Lt 1 >
0; and from a decrease, Lt Lt 1 < 0: A simple functional form for (:) which
captures this idea is a quadratic form, i.e. (Lt Lt 1 ) = 2 (Lt Lt 1 )2 , where is
a constant.

164
 Uncertainty for a …rm can come from many sources: Uncertain demand, uncer-
tainty about the production process, uncertainty in labour cost or other sources. As
we express pro…ts in units of the output good and assume that the real wage wt ,
i.e. the amount of output goods to be paid to labour, is uncertain. Adjustment cost
(Lt Lt 1 ) are certain, i.e. the …rm knows by how many units of output pro…ts
reduce when emplyoment changes by Lt Lt 1 :
As in static models of the …rm, the control variable of the …rm is employment Lt :
In contrast to static models, however, employment decisions today in t have not only
e¤ects on employment today but also on employment tommorrow as the employment
decision in t a¤ects adjustment costs in t + 1: There is therefore an intertemporal link
the …rm needs to take into account which is not present in static models of the …rm.

Solution

This maximization problem can be solved directly by inserting (8.43) into the
objective function (8.42). One can then choose optimal employment for some point
in time t s < 1 after having split the objective function into several subperiods -
as e.g. in the previous chapter 8.7.1. The solution reads (to be shown in exercise 6)
0
0 0 (Lt+1 Lt )
F (Lt ) = wt + (Lt Lt 1 ) Et :
1+r
When employment Lt is chosen in t; there is only uncertainty about Lt+1 : The current
wage wt (and all other deterministic quantities as well) are known with certainty. Lt+1
is uncertain, however, from the perspective of today as the wage in + 1 is unknown
and L +1 will be have to be adjusted accordingly in + 1: Hence, expectations ap-
ply only to the adjustment-cost term which refers to adjustment cost which occur in
period t + 1: Economically speaking, given employment Lt 1 in the previous period,
employment in t is chosen such that marginal productivity of labour equals labour
cost adjusted for current and expected future adjustment cost. Expected future ad-
justment cost are discounted by the interest rate r to obtain its present value.
When we specify the adjustment cost function as a quadratic function, (Lt Lt 1 ) =
2
[Lt Lt 1 ]2 ; we obtain

[Lt+1 Lt ]
F 0 (Lt ) = wt + [Lt Lt 1 ] Et :
1+r
If there were no adjustment cost, i.e. = 0; we would have F 0 (Lt ) = wt : Employment
would be chosen such that marginal productivity equals the real wage. This con…rms
the initial statement that the intertemporal problem of the …rm arises purely from
the adjustment costs. Without adjustment cost, i.e. with = 0; the …rm has the
“standard”instantaneous, period-speci…c optimality condition.

8.8 An explicit time path for a boundary condition

Sometimes, an explicit time path for optimal behaviour is required. The transversality
condition is then usually not very useful. A more pragmatic approach sets assets at

165
some future point in time at some exogenous level. This allows then to easily (at least
numerically) compute the optimal path for all points in time before this …nal point.
Let T be the …nal period of life in our model, i.e. set aT +1 = 0 (or some other
level e.g. the deterministic steady state level). Then, from the budget constraint, we
can deduce consumption in T;

aT +1 = (1 + rT ) aT + wT cT , cT = (1 + rT ) aT + wT :

Optimal consumption in T 1 still needs to obey the Keynes-Ramsey rule, i.e.

u0 (cT 1) = ET 1 [1 + rT ] u0 (cT ) :

As the budget constraint requires

aT = (1 + rT 1) aT 1 + wT 1 cT 1;

optimal consumption in T 1 is determined by

0
u0 (cT 1) = ET 1 [1 + rT ] u ((1 + rT ) [(1 + rT 1 ) aT 1 + wT 1 cT 1] + wT )

This is one equation in one unknown, cT 1 , where expectations need to be formed

about rT and wT and wT 1 are unknown. When we assume a probability distribu-
tion for rT and wT , we can replace ET 1 by a summation over states and solve this
expression numerically in a straightforward way.

8.9 Exercises chapter 8 and further reading

A recent introduction and detailed analysis of discrete time models with uncertainty
in the RBC tradition with homogenous and heterogenous agents is by Heer and
Mausner (2005). A more rigorous approach than here is taken by Stokey and Lucas
(1989). An almost comprehensive in-depth presentation of macroeconomic aspects
under uncertainty is provided by Ljungqvist and Sargent (2004).
On capital asset pricing in one-sector economies, references include Jermann (1998),
Danthine, Donaldson and Mehra (1992), Abel (1990), Rouwenhorst (1995), Stokey
and Lucas (1989, ch. 16.2) and Lucas (1978). An overview is in ch. 13 of Ljungqvist
and Sargent (2004).

166
 Exercises Chapter 8
Applied Intertemporal Optimization
Discrete time in…nite horizon models
under uncertainty

1. Central planner
Consider an economy where output is produced by

Yt = At Kt Lt1

Again, as in the OLG example in equation (7.13), total factor productivity At

is stochastic. Let capital evolve according to

Kt+1 = (1 ) Kt + Y t Ct

The central planner maximizes

1 t
max Et =t u (C )
fC g

by again choosing a path of aggregate consumption ‡ows C : At t, all variables

indexed t are known. The only uncertainty consists about At+1 : What are
optimality conditions?

2. A household maximization problem

Consider the optimal saving problem of the household in ch. 8.4. Derive Euler
equation (8.16).

3. Closed-form solution
1
Solve this model for the utility function u (C) = C 1 1 and for = 1: Solve it
for a more general case (Benhabib and Rustichini, 1994).

4. Habit formation
Assume instantaneous utility depends not only on current consumption but also
on habits (see e.g. Abel, 1990). Let the utility function therefore look like
1 t
Ut = Et =t u (c ; v ) ;

where v stands for habits like e.g. past consumption, v = v (c 1 ; ct 2 ; :::) :

Let such an individual maximize utility subject to the budget constraint

at+1 = (1 + rt ) at + wt pt ct

167
 (a) Assume the individual lives in a deterministic world and derive a rule for
an optimal consumption path where the e¤ect of habits are explicitly taken
into account. Specify habits by v = c 1 :
(b) Let there be uncertainty with respect to future prices. At a point in time
t, all variables indexed by t are known. What is the optimal consumption
rule when habits are treated in a parametric way?
(c) Choose a plausible instantaneous utility function and discuss the implica-
tions for optimal consumption given habits v = c 1 .

5. Risk-neutral valuation
Under which conditions is there a risk neutral valuation relationship for contin-
gent claims in models with many periods?

6. Labour demand under adjustment cost

Solve the maximization problem of the …rm in ch. 8.7.4 by directly inserting
pro…ts (8.43) into the objective function (8.42) and then choosing Lt .

7. Solving by inserting
Solve the problem from ch. 8.7 in a slightly extended version, i.e. with prices pt .
t
Maximize E0 1 t=0 u (ct ) by choosing a time path fct g for consumption subject
to at+1 = (1 + rt ) at + wt pt ct :

8. Matching on labour markets (exam)

Let employment Lt in a …rm follow

Lt+1 = (1 s) Lt + Vt ;

where s is a constant separation rate, is a constant matching rate and Vt

denotes the number of jobs a …rm currently o¤ers. The …rm’s pro…ts in period
are given by the di¤erence between revenue p Y (L ) and cost, where cost stem
from wage payments and costs for vacancies V captured by a parameter ;

= p Y (L ) w L V :

The objective function of the …rm is given by

1 t
t = Et =t ;

where is a discount factor and Et is the expectations operator.

(a) Assume a deterministic world. Let the …rm choose the number of vacancies
optimally. Use a Lagrangian to derive the optimality condition. Assume
that there is an interior solution. Why is this an assumption that might
not always be satis…ed from the perspective of a single …rm?

168
 (b) Let us now assume that there is uncertain demand which translates into
uncertain prices p which are exogenous to the …rm. Solve the optimal
choice of the …rm by inserting all equations into the objective function.
Maximize by choosing the state variable and explain also in words what
you do. Give an interpretation of the optimality condition. What does it
imply for the optimal choice of V ?

9. Optimal training for a marathon (exam)

Imagine you want to participate in a marathon or any other sports event. It
will take place in m days, i.e. in t + m where t is today. You know that
taking part in this event requires training e ; 2 [t; t + m] : Unfortunately,
you dislike training, i.e. your instantaneous utility u (e ) decreases in e¤ort,
u0 (e ) < 0. On the other hand, training allows you to be successful in the
marathon: more e¤ort increases your personal …tness F . Assume that …tness
follows F +1 = (1 ) F + e , with 0 < < 1; and …tness at t + m is good for
you yielding happiness of h (Ft+m ) :

(a) Formulate formally an objective function which captures the trade-o¤s in

such a training program.
(b) Assume that everything is deterministic. How would your training schedule
look like (the optimal path of e )?
(c) In the real world, normal night life reduces …tness in a random way, i.e.
is stochastic. How does now your training schedule look like?

169
Part IV
Stochastic models in continuous
time
The choice between working in discrete or continuous time is partly driven by previ-
ous choices: If the literature is mainly in discrete time, students will …nd it helpful
to work in discrete time as well. This dominance of discrete time seems to hold
for macroeconomics. On the other hand, there are very good books that can help
as introductions (and go well beyond an introduction) to work with uncertainty in
continuous time. An example is Turnovsky (1997, 2000).
Whatever the tradition in the literature, continuous time models have the huge
advantage that they are analytically generally more tractable, once some initial invest-
ment into new methods is digested. One example is, as some papers in the literature
have shown (e.g. Wälde, 2005 or Posch and Wälde, 2005), that continuous time mod-
els with uncertainty can be analyzed in simple phase diagrams as in deterministic
continuous time setups. In other …elds, continuous time uncertainty is dominating,
as e.g. in …nance. See ch. 9.5 and 10.6 on further reading for references from many
…elds.
This …nal part of the book presents tools that allow to work with uncertainty in
continuous time models. It is probably the most innovative part of this book as many
results from recent research directly ‡ow into it.

9 Stochastic di¤erential equations and rules for

di¤erentials
When working in continuous time, uncertainty enters the economy usually in the
form of Brownian motion, Poisson processes or Levy processes. This uncertainty
is represented in economic models by stochastic di¤erential equations (SDEs) which
describe e.g. the evolution of prices or technology frontiers. This section will cover a
wide range of di¤erential equations (and show how to work with them) that appear
in economics and …nance. It will also show how to work with functions of stochastic
variables, e.g. how output evolves given that TFP is stochastic or how wealth of a
household grows over time, given that the price of the asset held by the houshold is
random. The entire treatment here, as before in this book, will be non-rigorous and
focused on “how to compute things”.

9.1 Some basics

9.1.1 Stochastic processes
We got to know random variables in ch. 7.1. A random variable relates in some
loose sense to a stochastic process how (deterministic) static models relate to (deter-

170
ministic) dynamic models: Static models describe one equilibrium, dynamic models
describe a sequence of equilibria. A random variable has “when looked at once”(e.g.
when throwing a die once) one realization. A stochastic process describes a sequence
of random variables and therefore “when looked at once”, describes a sequence of
realizations. More formally, we have the following

De…nition 8 (Ross, 1996) A stochastic process is a parameterized collection of ran-

dom variables
fX (t)gt2[t0 ;t1 ] :

Let us look at an example for a stochastic process. We start from the normal
distribution of ch. 7.2.2 whose mean and variance are given by and 2 and its
p 1 1 z 2
density function is f (z) = 2 2 e 2 ( ) : Now de…ne a normally distributed
random variable Z (t) that has a mean and a variance that is a function of some t,
i.e. instead of 2 , write 2 t: Hence, the random variables we just de…ned has as density
p 1 1 zp
2

function f (z) = 2 2t e 2 t : By having done so and by interpreting t as

time, Z (t) is in fact a stochastic process: We have a collection of random variables,
all normally distributed, they are parameterized by time t:
Stochastic processes can be stationary, weakly stationary or non-stationary. They
are non-stationary if they are not weakly stationary. Stationarity is a more restrictive
concept than weak stationarity.

De…nition 9 (Ross, 1996, ch. 8.8): A process X (t) is stationary if X (t1 ) ; :::; X (tn )
and X (t1 + s) ; :::; X (tn + s) have the same joint distribution for all n and s:

An implication of this de…nition, which might help to get some “feeling” for this
de…nition, is that a stationary process X (t) implies that, being in t = 0, X (t1 ) and
X (t2 ) have the same distribution for all t2 > t1 > 0: A weaker concept of stationarity
only requires the …rst two moments of X (t1 ) and X (t2 ) (and a condition on the
covariance) to be satis…ed.

De…nition 10 (Ross, 1996) A process X (t) is weakly stationary if the …rst two mo-
ments are the same for all t and the covariance between X (t2 ) and X (t1 ) depends
only on t2 t1 ;
2
E0 X (t) = ; V arX (t) = ; Cov (X (t2 ) ; X (t1 )) = f (t2 t1 ) ;
2
where and are constants and f (:) is some function.

The probably best known stochastic process, especially from …nance, is Brownian
motion. It is sometimes called Wiener process after the mathematician Wiener who
provided the following de…nition.

De…nition 11 (Ross, 1996) Brownian motion

A stochastic process z (t) is a Brownian motion process if (i) z (0) = 0; (ii) the process
has stationary independent increments and (iii) for every t > 0; z (t) is normally
distributed with mean 0 and variance 2 t:

171
 The …rst condition z (0) = 0 is a normalization. Any z (t) that starts at, say, z0
can be rede…ned as z (t) z0 : The second condition says that for t4 > t3 t2 > t1
the increment z (t4 ) z (t3 ) ; which is a random variable, is independent of previous
increments, say z (t2 ) z (t1 ). ’Independent increments’implies that Brownian motion
is a Markov process: Assuming that we are in t3 today, the distribution of z (t4 )
depends only on z (t3 ) ; i.e. on the current state, and not on previous states like
z (t1 ). Increments are said to be stationary if, according to the above de…nition of
stationarity, the stochastic process X (t) z (t) z (t s) where s is a constant
has the same distribution for any t. The third condition is …nally the heart of the
de…nition - z (t) is normally distributed. The variance increases linearly in time; the
Wiener process is therefore nonstationary.
Let us now de…ne a stochastic process which plays also a major role in economics.
De…nition 12 Poisson process (adapted following Ross 1993, p. 210)
A stochastic process q (t) is a Poisson process with arrival rate if (i) q (0) = 0; (ii)
the process has independent increments and (iii) the increment q ( ) q (t) in any
interval of length t (the number of “jumps”) is Poisson distributed with mean
[ t] ; i.e. q ( ) q (t) ~Poisson( [ t]) :
A Poisson process (and other related processes) are also sometimes called “count-
ing processes”as q (t) counts how often a jump has occured, i.e. how often something
has happend.
There is a close similarity in the …rst two points of this de…nition with the de…nition
of Brownian motion. The third point here means more precisely that the probability
that the process increases n times between t and > t is given by

[ t] ( [ t])n
P [q ( ) q (t) = n] = e ; n = 0; 1; ::: (9.1)
n!
We know this probability from the de…nition of the Poisson distribution in ch. 7.2.1.
This is probably where the Poisson process got its name from. Hence, one could think
of as many stochastic processes as there are distributions, de…ning each process by
the distribution of its increments.
The most common way how Poisson processes are presented is by looking at the
distribution of the increment q ( ) q (t) over a very small time intervall [t; ] : The
increment q ( ) q (t) for very close to t is usually expressed by dq (t) : A stochastic
process q (t) is then a Poisson process if its increment dq (t) is driven by
n
dq (t) = 01 with prob. 1 dt
with prob. dt ; (9.2)

where the parameter is again called the arrival rate. A high then means that the
process jumps more often than with a low .11
These stochastic processes (and other processes) can now be combined in various
ways to construct more complex processes. These more complex processes can nicely
be represented by stochastic di¤erential equations (SDEs).
11
Note that the probabilities given in (9.2) are an approximation to the ones in (9.1) for t = dt:
We will return to this in ch. 9.4.2, see Poisson process II.

172
9.1.2 Stochastic di¤erential equations
The most frequently used SDEs include Brownian motion as the source of uncertainty.
These SDEs are used to model e.g. the evolution of asset prices or budget constraints
of households. Other examples include SDEs with Poisson uncertainty used explicitly
e.g. in the natural volatility literature, in …nance, international macro or in other
contexts mentioned above. Finally and more recently, Levy processes are used in
…nance as they allow for a much wider choice of properties of distributions of asset
returns than e.g. Brownian motion. We will now get to know examples for each type.
For all Brownian motions that will follow, we will assume, unless explicitly stated
otherwise, that increments have a standard normal distribution, i.e. Et [z ( ) z (t)] =
0 and vart [z ( ) z (t)] = t: It is therefore su¢ cient, consistent with most papers in
the literature and many mathematical textbooks, to work with a normalization of
in de…nition 11 of Brownian motion to 1:
Brownian motion with drift
This is one of the simplest SDEs. It reads
dx (t) = adt + bdz (t) : (9.3)
2
The constant a can be called drift rate, b is sometimes refered to as the variance rate
of x (t) : In fact, ch. 9.4.4 shows that the expected increase of x (t) is determined by
a only (and not by b). In contrast, the variance of x ( ) for some future > t is only
determined by b. The drift rate a is multiplied by dt; a “short” time intervall, the
variance parameter b is multiplied by dz (t) ; the increment of the Brownian motion
process z (t) over a small time intervall. This SDE (and all the others following later)
therefore consists of a deterministic part (the dt-term) and a stochastic part (the
dz-term).
An intuition for this di¤erential equation can be most easily gained by undertaking
a comparison with a deterministic di¤erential equation. If we neglected the Wiener
process for a moment (set b = 0), divide by dt and rename the variable to y; we obtain
the simple ordinary di¤erential equation
y_ (t) = a (9.4)
whose solution is y (t) = y0 + at: When we draw this solution and also the above SDE
for three di¤erent realizations of z (t), we obtain the following …gure.

173
Figure 38 The solution of the deterministic di¤erential equation (9.4) and three re-
alizations of the related stochastic di¤erential equation (9.3) Realization B M _ m ulti.n b

Hence, intuitively speaking, adding a stochastic component to the di¤erential

equation leads to ‡uctuations around the deterministic path. Clearly, how much
the solution of the SDE di¤ers from the deterministic one is random, i.e. unknown.
Further below in ch. 9.4.4, we will understand that the solution of the deterministic
di¤erential equation (9.4) is identical to the evolution of the expected value of x (t) ;
i.e. y (t) = E0 x (t) for t > 0:

Generalized Brownian motion (Ito processes)

A more general way to describe stochastic processes is the following SDE

dx (t) = a (x (t) ; z (t) ; t) dt + b (x (t) ; z (t) ; t) dz (t) : (9.5)

Here, one also refers to a (:) as the drift rate and to b2 (:) as the instantaneous variance
rate. Note that these functions can be stochastic themselves. In addition to arguments
x (t) and time, Brownian motion z (t) can be included in these arguments. Thinking
of (9.5) as a budget constraint of a household, an example could be that wage income
or the interest rate depend on the current realization of the economy’s fundamental
source of uncertainty, which is z (t) :

Stochastic di¤erential equations with Poisson processes

Di¤erential equations can of course also be constructed that are driven by a Pois-
son process. A very simple example is

dx (t) = adt + bdq (t) : (9.6)

A realization of this path for x (0) = x0 is in the following …gure and can be understood
very easily. As long as no jump occurs, i.e. as long as dq = 0; the variable x (t) follows
dx (t) = adt which means linear growth, x (t) = x0 + at: This is plotted as the thin
line. When q jumps, i.e. dq = 1; x (t) increases by b : Writing dx (t) = x~ (t) x (t) ;
where x~ (t) is the level of x immediately after the jump, and letting the jump be “very
fast”such that dt = 0 during the jump, we have x~ (t) x (t) = b 1; where the 1 stems
from dq (t) = 1: Hence,
x~ (t) = x (t) + b: (9.7)
Clearly, the point in times when a jump occurs are random. A tilde (~) will always
denote in what follows (and in various papers in the literature) the value of a quantity
immediately after a jump.

174
Figure 39 An example of a Poisson process with drift (thick line) and a deterministic
di¤erential equation (thin line)

In contrast to Brownian motion, a Poisson process contributes to the increase of

the variable of interest: Without the dq (t) term (i.e. for b = 0), x (t) would follow the
thin line. With occasional jumps, x (t) jumps faster. In the Brownian motion case of
the …gure before, realizations of x (t) remained “close to” the deterministic solution.
This is simply due to the fact that the expected increment of Brownian motion is
zero while the expected increment of a Poisson process is positive.
Note that in the more formal literature, the tilde is not used but a di¤erence is
made between x (t) and x (t ) where t stands for the point in time an “instant”
before t. (This is probably easy to understand on an intuitive level, thinking about
it for too long might not be a good idea as time is continuous ...) One would then
express the change in x due to a jump by x (t) = x (t ) + b as the value of x to which
b is added is the value of x before the jump. As the tilde-notation turned out to be
relatively intuitive, we will follow it in what follows.12

A geometric Poisson process

An further example would be the geometric Poisson process

dx (t) = a (q (t) ; t) x (t) dt + b (q (t) ; t) x (t) dq (t) : (9.8)

Processes are usually called geometric when they describe the rate of change of some
RV x (t) ; i.e. dx (t) =x (t) is not a function of x (t) : In this example, the deterministic
part shows that x (t) grows at the rate of a (:) in a deterministic way and jumps by
b (:) percent, when q (t) jumps. Note that in contrast to a Brownian motion SDE,
a (:) here is not the average growth rate of x (t) (see below on expectations).
Geometric Poisson processes as here are sometimes used to describe the evolution
of asset prices in a simple way. There is some deterministic growth component a (:)
12
The technical background for the t notation is the fact that the process x (t) is a so called
cádlág process ("continu a droite, limites a gauche"). That is, the paths of x (t) are continuous from
the right with left limits. The left limit is denoted by x (t ) lims"t x (s). See Sennewald (2006) or
Sennewald and Wälde (2006) for further details and references to the mathematical literature.

175
and some stochastic component b (:) : When the latter is positive, this could re‡ect
new technologies in the economy. When the latter is negative, this equation could be
used to model negative shocks like oil-price shocks or natural disasters.
Aggregate uncertainty and random jumps
An interesting extension of a Poisson di¤erential equation consists in making the
amplitude of the jump random. Taking a simple di¤erential equation with Poisson
uncertainty as starting point, dA (t) = bA (t) dq (t) ; where b is a constant, we can
now assume that b (t) is governed by some distribution, i.e.
2
dA (t) = b (t) A (t) dq (t) ; where b (t) ~ ; : (9.9)
Assume that A (t) is total factor productivity in an economy. Then, A (t) does not
change as long as dq (t) = 0: When q (t) jumps, A (t) changes by b (t) ; i.e. dA (t)
A~ (t) A (t) = b (t) A (t) ; which we can rewrite as
A~ (t) = (1 + b (t)) A (t) ; 8t where q (t) jumps.
This equation says that whenever a jump occurs, A (t) increases by b (t) percent,
i.e. by the realization of the random variable b (t) : Obviously, the realization of b (t)
matters only for points in time where q (t) jumps.
Note that (9.9) is the stochastic di¤erential equation representation of the evolu-
tion of the states of the economy in the Pissarides-type matching model of Shimer
(2005), where aggregate uncertainty, A (t) here, follows from a Poisson process. The
presentation in Shimer’s paper is “A shock hits the economy according to a Pois-
son process with arrival rate , at which point a new pair (p0 ; s0 ) is drawn from a
state dependent distribution.”(p. 34). Note also that using (9.9) and assuming large
families such that there is no uncertainty from labour income left on the household
level would allow to analyze the e¤ects of saving and thereby capital accumulation
over the business cycle in a closed-economy model with risk-averse households. The
background for the saving decision would be ch. 10.1.

9.1.3 The integral representation of stochastic di¤erential equations

Stochastic di¤erential equations as presented here can also be represented by integral
versions. This is identical to the integral representations for deterministic di¤erential
equations in ch. 4.3.3. The integral representation will be used frequently when
computing moments of x ( ): As an example, think of the expected value of x for
some future point in time ; when expectations are formed today in t; i.e. information
until t is available, Et x ( ) : See ch. 9.4.4 or 10.1.6.
Brownian motion
Consider a di¤erential equation as (9.5). It can more rigorously be represented by
its integral version,
Z Z
x ( ) x (t) = a(x; s)ds + b(x; s)dz (s) : (9.10)
t t

176
This version is obtain by …rst rewriting (9.5) as dx (s) = a (x; s) ds +b (x; s) dz (s) ;
i.e by simply changing the time index from t to s (and dropping zR(s) and writting
x instead of x (s) to shorten notation). Applying then the integral t on both sides
gives (9.10).
This implies, inter alia, a “di¤erentiation rule”
Z Z
d a(x; s)ds + b(x; s)dz (s) = d [x ( ) x (t)] = dx ( )
t t
= a(x; )d + b(x; )dz ( ) :

Poisson processes

Now consider a generalized version of the SDE in (9.6), with again replacing t by
s, dx (s) = a (xR(s) ; q (s)) ds +b (x (s) ; q (s)) dq (s) : The integral representation reads,
after applying t to both sides,
Z Z
x ( ) x (t) = a (x (s) ; q (s)) ds + b (x (s) ; q (s)) dq (s) :
t t

This can be checked by computing the di¤erential with respect to time :

9.2 Functions of stochastic processes

Possibly the most important aspect when working with stochastic processes in contin-
uous time is that rules for computing di¤erentials of functions of stochastic processes
are di¤erent from standard rules. These rules are provided by various forms of Ito’s
Lemma or change of variable formulas (CVF). Ito’s Lemma is a “rule” how to com-
pute di¤erentials when the basic source of uncertainty is Brownian motion. The CVF
provides corresponding rules when uncertainty stems from Poisson processes or Levy
processes.

9.2.1 Why all this?

“Computing di¤erentials of functions of stochastic processes”sounds pretty abstract.
Let us start with an example from deterministic continuous time setups which gives
an idea what the economic background for such di¤erentials are.
Imagine the capital stock of an economy follows K_ (t) = I (t) K (t) ; an ordinary
di¤erential equation (ODE) known from ch. 4. Assume further that total factor
productivity grows at an exogenous rate of g; A_ (t) =A (t) = g: Let output be given
by Y (A (t) ; K (t) ; L) and let us ask how output grows over time. The reply would
be provided by looking at the derivative of Y (:) with respect to time,
d dA (t) dK (t) dL
Y (A (t) ; K (t) ; L) = YA + YK + YL :
dt dt dt dt
Alternatively, written as a di¤erential, we would have

dY (A (t) ; K (t) ; L) = YA dA (t) + YK dK (t) + YL dL:

177
We can now insert equations describing the evolution of TFP and capital, dA (t) and
dK (t) ; and take into account that employment L is constant. This gives

dY (A (t) ; K (t) ; L) = YA gA (t) dt + YK [I (t) K (t)] dt + 0:

Dividing by dt would give a di¤erential equation that describes the growth of Y; i.e.
Y_ (t) :
The objective of the subsequent sections is to provide rules on how to compute
di¤erentials, of which dY (A (t) ; K (t) ; L) is an example, in setups where K (t) or
A (t) are described by stochastic DEs and not ordinary DEs as just used in this
example.

9.2.2 Computing di¤erentials for Brownian motion

One stochastic process

Lemma 3 (Øksendal, 2000, Theorem 4.1.2.) Consider a function F (t; x) of the

di¤usion process x 2 R that is at least twice di¤erentiable in x and once in t. Ito’s
Lemma gives the di¤erential dF as
1
dF = Ft dt + Fx dx + Fxx (dx)2 (9.11)
2
where (dx)2 is computed by using

dtdt = dtdz = dzdt = 0; dzdz = dt: (9.12)

Let us look at an example. Assume that x (t) is described by a generalized Brown-

ian motion as in (9.5). The square of dx is then given by

(dx)2 = a2 (:) (dt)2 + 2a (:) b (:) dtdz + b2 (:) (dz)2 = b2 (:) dt;

where the last equality used the “rules”from (9.12). Unfortunately, these rules have
a pretty complex stochastic background and an explanation would go far beyond the
objective of these notes. For those interested in the details, see the further-reading
chapter 9.5 on “general background”. The di¤erential of F (t; x) then reads
1
dF = Ft dt + Fx a (:) dt + Fx b (:) dz + Fxx b2 (:) dt
2
1
= Ft + Fx a (:) + Fxx b2 (:) dt + Fx b (:) dz: (9.13)
2
When we compare this di¤erential with the “normal”one, we recognize familiar terms:
The partial derivatives times deterministic changes, Ft +Fx a (:) ; would appear also in
circumstances where x follows a deterministic evolution. Put di¤erently, for b (:) = 0
in (9.5), the di¤erential dF reduces to fFt + Fx a (:)g dt: Brownian motion therefore
a¤ects the di¤erential dF in two ways: First, the stochastic term dz is added and sec-
ond, maybe more “surprisingly”, the deterministic part of dF is also a¤ected through
the quadratic term containing the second derivative Fxx :

178
 The lemma for many stochastic processes

This was the simple case of one stochastic process. Now consider the case of many
stochastic processes. Think of the price of many stocks traded on the stock market.
We then have the following

Lemma 4 (slightly simpli…ed version of Øksendal, 2000, Theorem 4.2.1.) Consider

dx1 = u1 dt + v11 dz1 + ::: + v1m dzm

..
.
dxn = un dt + vn1 dz1 + ::: + vnm dzm

or in matrix notation
dx = udt + vdz(t)
where
0 1 0 1 0 1 0 1
x1 u1 v11 ::: v1m dz1
B C B .. C ; v = B .. C ; dz = B .. C :
x = @ ... A ; u = @ . A @
..
. . A @ . A
xn un vn1 ::: vnm dzm

Consider further a function F (t; x) from [0; 1[ Rn to R with time t and the n
processes in x as arguments. Then

n 1 n n
dF (t; x) = Ft dt + i=1 Fxi dxi + i=1 j=1 Fxi xj [dxi dxj ] (9.14)
2
where, as an extension to (9.12),

dtdt = dtdzi = dzi dt = 0 and dzi dzj = ij dt: (9.15)

When all zi are mutually independent then ij = 0 for i 6= j and ij = 1 for i = j:

When two Brownian motions zi and zj are correlated, ij is the correlation coe¢ cient
between their increments dzi and dzj :

An example with two stochastic processes

Let us now consider an example for a function F (t; x; y) of two stochastic processes.
As an example, assume that x is described by a generalized Brownian motion similar
to (9.5), dx = a (x; y; t) dt + b (x; y; t) dzx and the stochastic process y is described by
dy = c (x; y; t) dt + g (x; y; t) dzy : Ito’s Lemma (9.14) gives the di¤erential dF as
1
dF = Ft dt + Fx dx + Fy dy + Fxx (dx)2 + 2Fxy dxdy + Fyy (dy)2 (9.16)
2
Given the rule in (9.15), the squares and the product in (9.16) are

(dx)2 = b2 (t; x; y) dt; (dy)2 = g 2 (t; x; y) dt; dxdy = xy b (:) g (:) dt;

179
where xy is the correlation coe¢ cient of the two processes. More precisely, it is the
correlation coe¢ cient of the two normally distributed random variables that underlie
the Wiener processes. The di¤erential (9.16) therefore reads

dF = Ft dt + a (:) Fx dt + b (:) Fx dzx + c (:) Fy dt + g (:) Fy dzy

1
+ Fxx b2 (:) dt + 2 xy Fxy b (:) g (:) dt + Fyy g 2 (:) dt
2
1
= Ft + a (:) Fx + c (:) Fy + b2 (:) Fxx + 2 xy b (:) g (:) Fxy + g 2 (:) Fyy dt
2
+ b (:) Fx dzx + g (:) Fy dzy (9.17)

Note that this di¤erential is almost simply the sum of the di¤erentials of each
stochastic process independently. The only term that is added is the term that
contains the correlation coe¢ cient. In other words, if the two stochastic processes
were independent, the di¤erential of a function of several stochastic processes equals
the sum of the di¤erential of each stochastic process individually.

An example with one stochastic process and many Brownian motions

A second example stipulates a stochastic process x (t) governed by dx = u1 dt +

m
When we compute the square of dx; we obtain
i=1 vi dzi :

(dx)2 = (u1 dt)2 + 2u1 dt [ m

i=1 vi dzi ] +( m
i=1 vi dzi )
2
=0+0+( m
i=1 vi dzi )
2
;

where the second equality used (9.12). The di¤erential of F (t; x) therefore reads from
(9.14)

m 1 m 2
dF (t; x) = Ft dt + Fx [u1 dt + i=1 vi dzi ] + Fxx [ i=1 vi dzi ]
2
1 m 2 m
= fFt + Fx u1 g dt + Fxx [ i=1 vi dzi ] + Fx i=1 vi dzi :
2
2
Computing the [ m
i=1 vi dzi ] term requires to take potential correlations into account.
For any two uncorrelated increments dzi and dzj ; dzi dzj would from (9.15) be zero.
When they are correlated, dzi dzj = ij dt which includes the case of dzi dzi = dt:

9.2.3 Computing di¤erentials for Poisson processes

When we consider the di¤erential of a function of the variable that is driven by the
Poisson process, we need to take the following CVFs into consideration.

One stochastic process

Lemma 5 Let there be a stochastic process x (t) driven by Poisson uncertainty q (t)
described by
dx (t) = a (:) dt + b (:) dq (t) :

180
What was stressed before for Brownian motion is valid here as well: The functions
a (:) and b (:) in the deterministic and stochastic part of this SDE can have as argu-
ments any combinations of q (t) ; x (t) and t or can be simple constants. Consider the
function F (t; x) : The di¤erential of this function is

dF (t; x) = Ft dt + Fx a (:) dt + fF (t; x + b (:)) F (t; x)g dq: (9.18)

This rule is very intuitive: The di¤erential of a function is given by the “normal
terms” and by a “jump term”. The “normal terms” include the partial derivatives
with respect to time t and x times changes per unit of time (1 for the …rst argument
and a (:) for x) times dt. Whenever the process q increases, x increases by the b (:) :
The “jump term” therefore captures that the function F (:) jumps from F (t; x) to
F (t; x~) = F (t; x + b (:)).

Two stochastic processes

Lemma 6 Let there be two independent Poisson processes qx and qy driving two
stochastic processes x (t) and y (t) ;

dx = a (:) dt + b (:) dqx ; dy = c (:) dt + g (:) dqy

and consider the function F (x; y) : The di¤erential of this function is

dF (x; y) = fFx a (:) + Fy c (:)g dt + fF (x + b (:) ; y) F (x; y)g dqx

+ fF (x; y + g (:)) F (x; y)g dqy : (9.19)

Again, this “di¤erentiation rule” consists of the “normal” terms and the “jump
terms”. As the function F (:) depends on two arguments, the normal term contains
two drift components, Fx a (:) and Fy c (:) and the jump term contains the e¤ect of
jumps in qx and in qy : Note that the dt term does not contain the time derivative
Ft (x; y) as in this example F (x; y) is assumed not to be a function of time and
therefore Ft (x; y) = 0: In applications where F (:) is a function of time, the Ft (:)
would of course have to be taken into consideration.
Let us now consider a case that is frequently encountered in economic models
when there is one economy-wide source of uncertainty, say new technologies arrive or
commodity price shocks occur according to some Poisson process, and many variables
in this economy (e.g. all relative prices) are a¤ected simultaneously by this one shock.
The CVF in situations of this type reads

Lemma 7 Let there be two variables x and y following

dx = a (:) dt + b (:) dq; dy = c (:) dt + g (:) dq;

where uncertainty stems from the same q for both variables. Consider the function
F (x; y) : The di¤erential of this function is

dF (x; y) = fFx a (:) + Fy c (:)g dt + fF (x + b (:) ; y + g (:)) F (x; y)g dq:

181
 One nice feature about di¤erentiation rules for Poisson processes is their very
intuitive structure. When there are two independent Poisson processes as in (9.19),
the change in F is given by either F (x + b (:) ; y) F (x; y) or by F (x; y + g (:))
F (x; y) ; depending on whether one or the other Poisson process jumps. When both
arguments x and y are a¤ected by the same Poisson process, the change in F is given
by F (x + b (:) ; y + g (:)) F (x; y) ; i.e. the level of F after a simultaneous change of
both x and y minus the pre-jump level F (x; y).

Many stochastic processes

We now present the most general case. Let there be n stochastic processes xi (t)
and de…ne the vector x (t) = (x1 (t) ; :::; xn (t))T : Let stochastic processes be described
by n SDEs

dxi (t) = i (:) dt + i1 (:) dq1 + ::: + im (:) dqm ; i = 1; : : : ; n; (9.20)

where ij (:) stands for ij (t; x (t)) : Each stochastic process xi (t) is driven by the
same m Poisson processes. The impact of Poisson process qj on xi (t) is captured by
ij (:) : Note the similarity to the setup for the Brownian motion case in (9.14).

Proposition 8 (Sennewald 2006 and Sennewald and Wälde 2006) Let there be n sto-
chastic processes described by (9.20). For a once continuously di¤erentiable function
F (t; x), the process F (t; x) obeys
n
dF (t; x (t)) = fFt (:) + i=1 Fxi (:) i (:)g dt
m
+ j=1 F t; x (t) + j (:) F (t; x (t)) dqj , (9.21)

where ft and fxi , i = 1; : : : ; n, denote the partial derivatives of f with respect to t and
T
xi , respectively, and j stands for the n-dimensional vector function 1j ; : : : ; nj .

The intuitive understanding is again simpli…ed by focusing on “normal”continuous

terms and on “jump terms”. The continuous terms are as before and simply describe
the impact of the i (:) in (9.20) on F (:) : The jump terms show how F (:) changes
from F (t; x (t)) to F t; x (t) + j (:) when Poisson process j jumps. The argument
x (t) + j (:) after the jump of qj is obtained by adding ij to component xi in x; i.e.
x (t) + j (:) = x1 + 1j ; x2 + 2j ; :::; xn + nj :

9.2.4 Brownian motion and a Poisson process

There are much more general stochastic processes in the literature than just Brownian
motion or Poisson processes. This section provides a CVF for a function of a vari-
able which is driven by both Brownian motion and a Poisson process. More general
processes than just additive combinations are so-called Levy processes which will be
analyzed in future editions of these notes.

182
Lemma 8 Let there be a variable x which is described by

dx = a (:) dt + b (:) dz + g (:) dq (9.22)

and where uncertainty stems from Brownian motion z and a Poisson process q. Con-
sider the function F (t; x) : The di¤erential of this function is

1
dF (t; x) = Ft + Fx a (:) + Fxx b2 (:) dt+Fx b (:) dz +fF (t; x + g (:)) F (t; x)g dq:
2
(9.23)

Note that this lemma is just a “combination”of Ito’s Lemma (9.13) and the CVF
for a Poisson process from (9.18). For an arrival rate of zero, i.e. for dq = 0 at all
times, (9.23) is identical to (9.13). For b (:) = 0; (9.23) is identical to (9.18).

9.2.5 Application - Option pricing

One of the most celebrated papers in economics is the paper by Black and Scholes
(1973) in which they derived a pricing formula for options. This section presents the
…rst steps towards obtaining this pricing equation. The subsequent chapter 9.3.1 will
complete the analysis. This section presents a simpli…ed version (by neglecting jumps
in the asset price) of the derivation of Merton (1976). The basic question is: what is
the price of an option on an asset if there is absence of arbitrage on capital markets?

The asset and option price

The starting point is the price S of an asset which evolves according to a geometric
process
dS
= dt + dz: (9.24)
S
Uncertainty is modelled by the increment dz of Brownian motion. We assume that
the economic environment is such (inter alia short selling is possible, there are no
transaction cost) that the price of the option is given by a function F (:) having as
arguments only the price of the asset and time, F (t; S (t)): The di¤erential of the
price of the option is then given from (9.11) by
1
dF = Ft dt + FS dS + FSS (dS)2 : (9.25)
2
As the square of dS is given by (dS)2 = 2
S 2 dt; the di¤erential reads

1 2 dF
dF = Ft + SFS + S 2 FSS dt + SFS dz , F dt + F dz (9.26)
2 F
where the last step de…ned
1 2
Ft + SFS + 2
S 2 FSS SFS
F = ; F = : (9.27)
F F

183
 Absence of arbitrage

Now comes the trick - the no-arbitrage consideration. Consider a portfolio that
consists of N1 units of the asset itself, N2 options and N3 units of some riskless assets,
say wealth in a savings account. The price of such a portfolio is then given by

P = N1 S + N2 F + N3 M;

where M is the price of one unit of the riskless asset. The proportional change of the
price of this portfolio can be expressed as (holding the Ni s constant, otherwise Ito’s
Lemma would have to be used)
dP N1 S dS N2 F dF N3 M dM
dP = N1 dS + N2 dF + N3 dM , = + + :
P P S P F P M
De…ning shares of the portfolio held in these three assets by 1 N1 S=P and 2
N2 S=P; inserting option and stock price evolutions from (9.24) and (9.25) and letting
the riskless asset M pay a constant return of r, we obtain

dP=P 1dt + 1 dz + 2 F dt+ 2 F dz + (1 1 2 ) rdt

= f 1[ r] + 2 [ F r] + rg dt + f 1 + 2 F g dz: (9.28)

Now assume someone chooses weights such that the portfolio no longer bears any risk

1 + 2 F = 0: (9.29)

The return of such a portfolio with these weights must then of course be identical to
the return of the riskless interest asset, i.e. identical to r;
dP=dt r F r
= 1 [ r] + 2 [ F r] + rj = =r, = :
P riskless
1 2 F
F

If the return of the riskless portfolio did not equal the return of the riskless interest
rates, there would be arbitrage possibilities. This approach is therefore called no-
arbitrage pricing.

The Black-Scholes formula

Finally, inserting F and F from (9.27) yields the celebrated di¤erential equation
that determines the evolution of the price of the option,
1 2
r Ft + SFS + S 2 FSS rF 1 2
= 2
, S 2 FSS + rSFS rF + Ft = 0: (9.30)
SFS 2
Clearly, this equation does not to say what the price F of the option actually is. It
only says how it changes over time and in reaction to S. But as we will see in ch.
9.3.1, this equation can actually be solved explicitly for the price of the option. Note
also that we did not make any assumption so far about what type of option we talk
about.

184
9.2.6 Application - Deriving a budget constraint
Most maximization problems require a constraint. For a household, this is usually
the budget constraint. It is shown here how the structure of the budget constraint
depends on the economic environment the household …nds itself in and how the CVF
needs to be applied here.
Let wealth at time t be given by the number n (t) of stocks a household owns times
their price v (t), a (t) = n (t) v (t). Let the price follow a process that is exogenous to
the household (but potentially endogenous in general equilibrium),

dv (t) = v (t) dt + v (t) dq (t) ; (9.31)

where and are constants. For we require > 1 to avoid that the price can
become zero or negative. Hence, the price grows with the continuous rate and at
discrete random times it jumps by percent. The random times are modeled by the
jump times of a Poisson process q (t) with arrival rate , which is the “probability”
that in the current period a price jump occurs. The expected (or average) growth
rate is then given by + (see ch. 9.4.4).
Let the household earn dividend payments, (t) per unit of asset it owns, and
labour income, w (t). Assume furthermore that it spends p (t) c (t) on consumption,
where c (t) denotes the consumption quantity and p (t) the price of one unit of the
consumption good. When buying stocks is the only way of saving, the number of
stocks held by the household changes in a deterministic way according to

n (t) (t) + w (t) p (t) c (t)

dn (t) = dt:
v (t)

When savings n (t) (t) + w (t) p (t) c (t) are positive, the number of stocks held by
the household increases by savings divided by the price of one stock. When savings
are negative, the number of stocks decreases.
The change of the household’s wealth, i.e. the household’s budget constraint, is
then given by applying the CVF to a (t) = n (t) v (t). The appropriate CVF comes
from (9.19) where only one of the two di¤erential equations shows the increment of
the Poisson process explicitly. With F (x; y) = xy, we obtain

n (t) (t) + w (t) p (t) c (t)

da (t) = v (t) + n (t) v (t) dt
v (t)
+ fn (t) [v (t) + v (t)] n (t) v (t)g dq (t)
= fr (t) a (t) + w (t) p (t) c (t)g dt + a (t) dq (t) ; (9.32)

where the interest-rate is de…ned as

(t)
r (t) + :
v (t)

This is a very intuitive budget constraint: As long as the asset price does not jump,
i.e., dq (t) = 0, the household’s wealth increases by current savings, r (t) a (t) + w (t)

185
p (t) c (t), where the interest rate, r (t), consists of dividend payments in terms of the
asset price plus the deterministic growth rate of the asset price. If a price jump occurs,
i.e., dq (t) = 1, wealth jumps, as the price, by percent, which is the stochastic part
of the overall interest-rate. Altogether, the average interest rate amounts to r (t) +
(see ch. 9.4.4).

9.3 Solving stochastic di¤erential equations

Just as there are theorems on uniqueness and existence for deterministic di¤erential
equations, there are theorems for SDEs on these issues. There are also solution
methods for SDEs. We will consider here some examples for solutions of SDEs.
Just as for ordinary deterministic di¤erential equations in ch. 4.3.2, we will simply
present solutions and not show how they can be derived. Solutions of stochastic
di¤erential equations d (x (t)) are, in analogy to the de…nition for ODE, again time
paths x (t) that satisfy the di¤erential equation. Hence, by applying Ito’s Lemma or
the CVF, one can verify whether the solutions presented here are indeed solutions.

9.3.1 Some examples for Brownian motion

This section …rst looks at SDEs with Brownian motion which are similar to the ones
that were presented when introducing SDEs in ch. 9.1.2: We start with Brownian
motion with drift as in (9.3) and then look at an example for generalized Brownian
motion in (9.5). In both cases, we work with SDEs which have an economic interpre-
tation and are not just SDEs. Finally, we complete the analysis of the Black-Scholes
option pricing approach.

Brownian motion with drift 1

As an example for Brownian motion with drift, consider a representation of a

production technology which could be called a “di¤erential-representation” for the
technology. This type of presenting technologies was dominant in early contributions
that used continuous time methods under uncertainty but is sometimes still used
today. A simple example is

dY (t) = AKdt + Kdz (t) ; (9.33)

where Y (t) is output in t; A is a (constant) measure of total factor productivity, K is

capital, is some variance measure of output and z is Brownian motion. The change
of output at each instant is then given by dY (t). See “Further reading”on references
to the literature.
What does such a representation of output imply? To see this, look at (9.33) as
Brownian motion with drift, i.e. consider A; K; and to be a constant (which is
relaxed in ex. 12). The solution to this di¤erential equation starting in t = 0 with Y0
and z (0) is
Y (t) = Y0 + AKt + Kz (t) :

186
To simplify an economic interpretation set Y0 = z (0) = 0: Output is then given by
Y (t) = (At + z (t)) K: This says that with a constant factor input K; output in t
is determined by a deterministic and a stochastic part. The deterministic part At
implies linear (i.e. not exponential as is usually assumed) growth, the stochastic part
z (t) implies deviations from the trend. As z (t) is Brownian motion, the sum of
the deterministic and stochastic part can become negative. This is an undesirable
property of this approach.
To see that Y (t) is in fact a solution of the above di¤erential-representation, just
apply Ito’s Lemma and recover (9.33).

Brownian motion with drift 2

As a second example and in an attempt to better understand why output can

become negative, consider a standard representation of a technology Y (t) = A (t) K
and let TFP A follow Brownian motion with drift,

dA (t) = gdt + dz (t) ;

where g and are constants. What does this alternative speci…cation imply?
Solving the SDE yields A (t) = A0 + gt + z (t) (which can again be checked by
applying Ito’s lemma). Output is therefore given by

Y (t) = (A0 + gt + z (t)) K = A0 K + gKt + Kz (t)

and can again become negative.

Geometric Brownian motion

Let us now assume that TFP follows geometric Brownian motion process,

dA (t) =A (t) = gdt + dz (t) ; (9.34)

where again g and are constants. Let output continue to be given by Y (t) = A (t) K:
The solution for TFP, provided an initial condition A (0) = A0 ; is given by
1
A (t) = A0 e(g )t+
2 z(t)
2 : (9.35)

At any point in time t; the TFP level depends on time t and the current level of the
stochastic process z (t) : This shows that TFP at each point t in time is random and
thereby unkown from the perspective of t = 0: Hence, a SDE and its solution describe
the deterministic evolution of a distribution over time. One could therefore plot a
picture of A (t) which in principle would look like the evolution of the distribution in
ch. 7.4.1.
Interestingly, and this is due to the geometric speci…cation in (9.34) and impor-
tant for representing technologies in general, TFP can not become negative. While
Brownian motion z (t) can take any value between minus and plus in…nity, the term
1 2
e(g 2 )t+ z(t) is always positive. With an AK speci…cation for output, output is

187
 1 2
always positive, Y (t) = A0 e(g 2 )t+ z(t) K. In fact, it can be shown that output and
TFP are lognormally distributed. Hence, the speci…cation of TFP with geometric
Brownian motion provides an alternative to the di¤erential-representation in (9.33)
which avoids the possibility of negative output.
The level of TFP at some future point in time t is determined by a deterministic
part, g 12 2 t; and by a stochastic part, z (t) : Apparently, the stochastic nature
of TFP has an e¤ect on the deterministic term. The structure (the factor 1=2 and the
quadratic term 2 ) reminds of the role the stochastic disturbance plays in Ito’s lemma.
There as well (see e.g. (9.13)), the stochastic disturbance a¤ects the deterministic
component of the di¤erential. As we will see later, however, this does not a¤ect
expected growth. As we will see in (9.53), expected output grows at the rate g (and
is thereby independent of the variance parameter ).
One can verify that (9.35) is a solution to (9.34) by using Ito’s lemma. To do so, we
bring (9.34) into a form which allows us to apply the formulas which we got to know
in ch. 9.2.2. De…ne x (t) g 21 2 t + z (t) and A (t) F (x (t)) = A0 ex(t) : As a
consequence, the di¤erential for x (t) is a nice SDE, dx (t) = g 21 2 dt + dz (t) :
As this SDE is of the form as in (9.5), we can use Ito’s lemma from (9.13) and …nd

1 2 1 2
dA (t) = dF (x (t)) = Fx (x (t)) g + Fxx (x (t)) dt + Fx (x (t)) dz:
2 2

Inserting the …rst and second derivatives of F (x (t)) yields

1 1
dA (t) = A0 ex(t) g 2
+ A0 ex(t) 2
dt + A0 ex(t) dz ,
2 2
1 2 1 2
dA (t) =A (t) = g + dt + dz = gdt + dz;
2 2

where the “i¤” reinserted A (t) = A0 ex(t) and divided by A (t) : As A (t) satis…es the
orignianl SDE (9.34), A (t) is a solution of (9.34).

Option pricing

Let us come back to the Black-Scholes formula for option pricing. The SDE
derived above in (9.30) describes the evolution of the price F (t; S (t)) ; where t is
time and S (t) the price of the underlying asset at t. We now look at a European
call option, i.e. an option which gives the right to buy an asset at some …xed point
in time T; the maturity date of the option. The …xed exercise or strike price of the
option, i.e. the price at which the asset can be bought is denoted by E:
Clearly, at any point in time t when the price of the asset is zero, the value of
the option is zero as well. This is the …rst boundary condition for our SDE (9.30).
When the option can be exercised at T and the price of the asset is S; the value of
the option is zero if the exercise price E exceeds the price of the asset and S E if
not. This is the second boundary condition.

F (t; 0) = 0 F (T; S) = max f0; S Eg ;

188
In the latter case where S E > 0, the owner of the option would in fact buy the
asset.
Given these two boundary conditions, the SDE has the solution (originating from
physics, see Black and Scholes, 1973, p. 644)
r[T t]
F (t; S (t)) = S (t) (d1 ) Ee (d2 )

where
Z y
1 u2
(y) = p e 2 du;
2 1
r2
ln S(t)
E
+ r 2
(T t) p
d1 = p ; d2 = d1 T t:
T t
The expression F (t; S (t)) gives the price of an option at a point in time t T where
the price of the asset is S (t) : It is a function of the cumulative standard normal
distribution (y) : For any path of S (t) ; time up to maturity T a¤ects the option
price through d1 ; d2 and directly in the second term of the above di¤erence. More
interpretation is o¤ered by many …nance textbooks.

9.3.2 A general solution for Brownian motions

Consider the di¤erential equation
m
dx (t) = fA (t) x (t) + a (t)g dt + i=1 fBi (t) x (t) + bi (t)g dW i (t) (9.36)

with boundary value x (0) = x0 : W i (t) is a Brownian motion. The correlation coe¢ -
cient of its increments with the increments of W j (t) is ij : Its solution is
0 1
Rt 1
B x0 + (s) fa (s) Bb (s)g ds C
x (t) = B
(t) @ 0 C (9.37)
Pm Rt
1
A
+ (s) bi (s) dW i (s)
i=1 0

where
Rt 1
P
m t
R
(A(s) 2 BB (s) )ds+ Bi (s)dW i (s)
(t) = e 0 i=1 0 ;
m m
BB (s) = i=1 j=1 ij Bi (s) Bj (s) ;
m m
Bb (s) = i=1 j=1 ij Bi (s) bi (s) :

Arnold (1974, ch. 8.4) provides the solution for independent Brownian motions.
This solution is an “educated guess”. To obtain some intuition for (9.37), we can
consider the case of certainty. For Bi (t) = bi (t) = 0; this is an linear ODE and the
solution should correspond to the results we know from ch. 4.3.2. For the general
case, we now show that (9.37) indeed satis…es (9.36).

189
 Proof. In order to use Ito’s Lemma, de…ne
Z t Z t
1 Pm
y (t) A (s) BB (s) ds + Bi (s) dW i (s)
0 2 i=1 0

such that
(t) = ey(t) (9.38)
and de…ne
Zt Zt
1 P
m
1
z(t) x0 + (s) fa (s) Bb (s)g ds + (s) bi (s) dW i (s)
i=1
0 0

Then the solution (9.37) reads

x(t) = ey(t) z(t) f (y(t); z(t)) (9.39)

where
1 P
m
dy(t) = A (t) BB (t) dt + Bi (t) dW i (t)
2 i=1
1 P
m
1
dz(t) = (t) fa (t) Bb (t)g dt + (t) bi (t) dW i (t)
i=1

In order to compute the di¤erential dx(t), we have to apply the multidimensional

Ito-Formula (9.16) where time is not an argument of f (:): This gives
1
dx(t) = ey(t) z(t)dy(t) + ey(t) dz(t) + [ey(t) z(t)[dy]2 + 2ey(t) [dydz] + 0 [dz]2 ] (9.40)
2
2
2
P
m
i
As [dy] = Bi (t) dW (t) by (9.15) - all terms multiplied by dt equal zero - we
i=1
obtain
P
m P
m
[dy]2 = ij Bi (t) Bj (t) dt (9.41)
i=1 j=1

Further, again by (9.15),

m i m 1
dydz = i=1 Bi (t) dW (t) i=1 (t) bi (t) dW i (t)
= (t) 1 m m
i=1 j=1 ij Bi (t) bi (t) dt: (9.42)

Hence, reinserting (9.38) and (9.39) and inserting in (9.40) gives

1 m i
dx(t) = x(t) A (t) BB (t) dt + i=1 Bi (t) dW (t)
2
m i
+ fa (t) Bb (t)g dt + i=1 bi (t) dW (t)
1 m m m m
+ x(t) i=1 j=1 ij Bi (t) Bj (t) dt + 2 i=1 j=1 ij Bi (t) bi (t) dt
2

190
Rearranging gives

dx(t) = fx(t)A (t) + a (t)g dt + m i

i=1 fx(t)Bi (t) + bi (t)g dW (t)
1
x(t) BB (t) + Bb (t) dt
2
1
+ x(t) m m m m
i=1 j=1 ij Bi (t) Bj (t) + i=1 j=1 ij Bi (t) bi (t) dt
2
= fx(t)A (t) + a (t)g dt + m i
i=1 fx(t)Bi (t) + bi (t)g dW (t) :

9.3.3 Di¤erential equations with Poisson processes

The presentation of solutions and their verifycation for Poisson processes follows a
similar structure as for Brownian motion. We start here with a geometric Poisson
process and compare properties of the solution with the TFP Brownian motion case.
We then look at a more general process - the description of a budget constraint -
which extends the geometric Poisson process. One should keep in mind, as stressed
already in ch. 4.3.3, that solutions to di¤erential equations are di¤erent from the
integral representation of e.g. ch. 9.1.3.

A geometric Poisson process

Imagine that TFP follows a deterministic trend and occasionally makes a discrete
jump. This is capture by a geometric description as in (9.34) only that Brownian
motion is replaced by a Poisson process,

dA (t) =A (t) = gdt + dq (t) : (9.43)

Again, g and are constant.

The solution to this SDE is given by

A (t) = A0 egt+[q(t) q(0)] ln(1+ )

: (9.44)

In contrast to the solution (9.35) for the Brownian motion case, uncertainty does not
a¤ect the deterministic part here. As before, TFP follows a deterministic growth
component and a stochastic component, [q (t) q (0)] ln (1 + ). The latter makes
future TFP uncertaint from the perspective of today.
The claim that A (t) is a solution can be proven by applying the appropriate CVF.
This will be done for the next, more general, example.

A budget constraint

As a second example, we look at a dynamic budget constraint,

da (t) = fr (t) a (t) + w (t) c (t)g dt + a (t) dq: (9.45)

191
De…ning (s) w (s) c (s) ; the backward solution of (9.45) with initial condition
a (0) = a0 reads Z t
y(t) y(s)
a (t) = e a0 + e (s) ds (9.46)
0

where y (t) is Z t
y (t) = r (u) du + [q (t) q (0)] ln (1 + ) : (9.47)
0

Note that the solution in (9.46) has the same structure as the solution to a determin-
istic version of the di¤erential equation (9.45) (which we would obtain for = 0).
Put di¤erently, the stochastic component in (9.45) only a¤ects the discount factor
y (t) : This is not surprising - in a way - as uncertainty is proportional to a (t) and the
factor can be seen as the stochastic component of the interest payments on a (t) :
We have just stated that (9.46) is a solution. This should therefore be verifyed.
To this end, de…ne Z t
y(s)
z(t) a0 + e (s)ds; (9.48)
0

and write the solution (9.46) as a(t) = ey(t) z(t) where from (9.47) and (9.48),
y(t)
dy(t) = r(t)dt + ln(1 + )dq(t); dz(t) = e (t)dt: (9.49)

We have thereby de…ned a function a (t) = F (y (t) ; z (t)) where the SDEs describing
the evolution of y (t) and z (t) are given in (9.49). The CVF (9.19) then says

da(t) = ey(t) z(t)r(t)dt + ey(t) e y(t) (t)dt + ey(t)+ln(1+ ) z(t) ey(t) z(t) dq
= fr (t) a (t) + (t)g dt + a (t) dq:

This is the original di¤erential equation. Hence, a (t) is a solution for (9.45).

The intertemporal budget constraint

In stochastic worlds, there is also a link between dynamic and intertemporal bud-
get constraints, just as in deterministic setups as e.g. in ch. 4.3.4. We …rst present
here a budget constraint for a …nite planning horizon and then generalize the result.
For the …nite horizon case, we can rewrite (9.46) as
Z t Z t
y(s) y(t)
e c (s) ds + e a (t) = a0 + e y(s) w (s) ds:
0 0

This formulation suggests a standard economic interpretation. Total expenditure over

the planning horizon from 0 to t on the left-hand side must equal total wealth on the
right-hand side. Total expenditure consists of the present value of the consumption
expenditure path c (s) and the present value of assets a (t) the household wants to
hold at the end of the planning period, i.e. at t: Total wealth is given by initial
…nancial wealth a0 and the present value of current and future wage income w (s) : All

192
discounting takes place at the realized stochastic interest rate y (s) - no expectations
are formed.
In order to obtain an in…nite horizon intertemporal budget constraint, the solution
(9.46) should be written more generally - after replacing t by and 0 by t - as
Z
y( )
a ( ) = e at + ey( ) y(s) (w (s) c (s)) ds (9.50)
t

where y( ) is Z
y( ) = r (u) du + ln (1 + ) [q ( ) q (t)] : (9.51)
t
y( )
Multiplying (9.50) by e and rearranging gives
Z
y( )
a( )e + e y(s) (c (s) w (s))ds = a(t):
t

y( )
Letting go to in…nity, assuming a no-Ponzi game condition lim !1 a( )e =0
and rearranging again yields
Z 1 Z 1
y(s)
e c (s) ds = a(t) + e y(s) w (s) ds:
t t

The present value of consumption expenditure needs to equal current …nancial

wealth a (t) plus “human wealth”, i.e. the present value of current and future labour
income. Here as well, discounting takes place at the “risk-corrected” interest rate
as captured by y ( ) in (9.51). Note also that this intertemporal budget constraint
requires equality in realizations, not in expected terms.

A switch process

Here are two funny processes which have an interesting simple solution. Consider
the initial value problem,

dx (t) = 2x (t) dq (t) ; x (0) = x0 :

The solution is x (t) = ( 1)q(t) x0 ; i.e. x (t) oscillates between x0 and x0 :

Now consider the transformation y = y + x: It evolves according to

dy = 2 (y y) dq; y 0 = y + x0 :

Its solution is y (t) = y + ( 1)q(t) x0 and y (t) oscillates now between x (t) oscillates
between y x0 and y + x0 :
This could be nicely used for models with wage uncertainty, where wages are
sometimes high and sometimes low. An example would be a matching model where
labour income is w (i.e. y + x0 ) when employed and b (i.e. y x0 ) when unemployed.
The di¤erence between labour income levels is then 2x0 . The drawback is that the
probability of …nding a job is identical to the probability of losing it.

193
9.4 Expectation values
9.4.1 The idea
What is the idea behind expectations of stochastic processes? When thinking of a
stochastic process X (t), either a simple one like Brownian motion or a Poisson process
or more complex ones described by SDEs, it is useful to keep in mind that one can
think of the value X ( ) the stochastic process takes at some future point in time as
a “normal” random variable. At each future point in time X ( ) is characterized by
some mean, some variance, by a range etc. This is illustrated in the following …gure.

Figure 40 The distribution of X ( ) at = :4 > t = 0

The …gure shows four realizations of the stochastic process X (t) : The starting
point is today in t = 0 and the mean growth of this process is illustrated by the
dotted line. When we think of possible realizations of X ( ) for = :4 from the
perspective of t = 0; we can imagine a vertical line that cuts through possible paths
at = :4: With su¢ ciently many realizations of these paths, we would be able to
make inferences about the distributional properties of X (:4) : As the process used
for this simulation is a geometric Brownian motion process as in (9.34), we would
…nd that X (:4) is lognormally distributed as depicted above. Clearly, given that we
have a precise mathematical description of our process, we do not need to estimate
distributional properties for X ( ) ; as suggested by this …gure, but we can explicitly
compute them.
What this section therefore does is to provide tools that allow to determine proper-
ties of the distribution of a stochastic process for future points in time. Put di¤erently,
understanding a stochastic process means understanding the evolution of its distri-
butional properties. We will …rst start by looking at relatively straightforward ways
to compute means. Subsequently, we provide some martingale results which allow
us to then understand a more general approach to computing means and also higher
moments.

194
9.4.2 Simple results
Brownian motion I
Consider a Brownian motion process Z ( ) ; > 0: Let Z (0) = 0. From the
de…nition of Brownian motion in ch. 9.1.1, we know that Z ( ) is normally distributed
with mean 0 and variance 2 : Let us assume we are in t today and Z (t) 6= 0. What
is the mean and variance of Z ( ) for > t?
We construct a stochastic process Z ( ) Z (t) which at = t is equal to zero. Now
imagining that t is equal to zero, Z ( ) Z (t) is a Brownian motion process as de…ned
in ch. 9.1.1. Therefore, Et [Z ( ) Z (t)] = 0 and vart [Z ( ) Z (t)] = 2 [ t] :
This says that the increments of Brownian motion are normally distributed with mean
zero and variance 2 [ t] : Hence, the reply to our question is
2
Et Z ( ) = Z (t) ; vart Z ( ) = [ t] :
For our convention 1 which we follow here as stated at the beginning of ch. 9.1.2,
the variance would equal vart Z ( ) = t:
Brownian motion II
Consider again the geometric Brownian process dA (t) =A (t) = gdt + dz (t) de-
1 2
scribing the growth of TFP in (9.34). Given the solution A (t) = A0 e(g 2 )t+ z(t)
from (9.35), we would now like to know what the expected TFP level A (t) in t from
the perspective of t = 0 is. To this end, apply the expectations operator E0 and …nd
1 1
E0 A (t) = A0 E0 e(g )t+ = A0 e(g )t E e
2 z(t) 2
z(t)
2 2
0 ; (9.52)
1 2
where the second equality exploited the fact that e(g 2 )t is non-random. As z (t)
is standard normally distributed, z (t) ~N (0; t), z (t) is normally distributed with
N (0; 2 t) : As a consequence (compare ch. 7.3.3, especially eq. (7.6)), e z(t) is log-
1 2
normally distributed with mean e 2 t : Hence,
1
E0 A (t) = A0 e(g )t e 21
2 2t
2 = A0 egt : (9.53)
The expected level of TFP growing according to a geometric Brownian motion process
grows at the drift rate g of the stochastic process. The variance parameter does
not a¤ect expected growth.
Poisson processes I
The expected value in t of q ( ) ; where > t; and its variance are (cf. e.g. Ross,
1993, p. 210 and 241)
Et q ( ) = q (t) + [ t] ; >t (9.54)
vart q ( ) = [ t] :
The expected value Et q ( ) directly follows from the de…nition of a Poisson process
in ch. 9.1.1, see (9.1). There are simple generalizations of the Poisson process where
the variance di¤eres from the mean (see ch. 9.4.4).

195
 Poisson processes II

Let q (t) be a Poisson process with arrival rate : What is the probability that it
jumped exactly once between t and ?
[ t] ( [ t])n
Given that by def. 12, e n!
is the probability that q jumped n times by
; i.e. q ( ) = n; this probability is given by P (q ( ) = q (t) + 1) = e [ t] [ t] :
Note that this is a function which is non-monotonic in time :

Figure 41 The probability of one jump by time

How can the expression P (q ( ) = q (t) + 1) = e [ t] [ t] be reconciled with

the usual description of a Poisson process, as e.g. in (9.2), where it says that the
probability of a jump is given by dt? When we look at a small time intervall dt;
we can neglect e dt as it is "very close to one" and we get P (q ( ) q (t) = 1) =
P (dq (t) = 1) = dt: As over a very small instant dt a Poisson process can either jump
or not jump (it can not jump more than once in a small instant dt), the probability
of no jump is therefore P (dq (t) = 0) = 1 dt:
What is the length of time between two jumps? Clearly, we do not know as jumps
are random. What is the probability that the length of time between two jumps is
smaller or equal to ? The probability that there is no jump by is given, following
(9.1), by P [q ( ) = q (t)] = e [ t] : The probability that there is at least one jump
by is therefore P [q ( ) > q (t)] = 1 e [ t] : The latter is the cumulative density
function of the exponential distribution. Hence, the length t between two jumps
[ t]
is exponentially distributed with density e :

Poisson process III

Following the same structure as with Brownian motion, we now look at the geo-
metric Poisson process describing TFP growth (9.43). The solution was given in
(9.44) which reads, slightly generalized with the perspective of t and initial condition
At ; A ( ) = At eg[ t]+[q( ) q(t)] ln(1+ ) : What is the expected TFP level in ?

196
 Applying the expectation operator gives

Et A ( ) = At eg[ t] q(t) ln(1+ )

Et eq( ) ln(1+ )
(9.55)

where, as in (9.52), we split the exponential growth term into its deterministic and
stochastic part. To proceed, we nee the following

Lemma 9 (taken from Posch and Wälde, 2006) Assume that we are in t and form
expectations about future arrivals of the Poisson process. The expected value of ckq( ) ,
conditional on t where q (t) is known, is
k
Et (ckq( ) ) = ckq(t) e(c 1) ( t)
; > t; c; k = const:

Note that for integer k, these are the raw moments of cq( ) :
Proof. We can trivially rewrite ckq( ) = ckq(t) ck[q( ) q(t)] : At time t; we know
the realization of q(t) and therefore Et ckq( ) = ckq(t) Et ck[q( ) q(t)] : Computing this
expectation requires the probability that a Poisson process jumps n times between t
and : Formally,
( t)
kn e ( ( t))n 1 e (ck (
( t)
t))n
Et ck[q( ) q(t)]
= 1
n=0 c = n=0
n! n!
( t) (ck 1) ( t) k
(ck 1) ( t) 1 e (c ( t))n
=e n=0
n!
ck ( t) k
k 1 e (c ( t))n k
= e(c 1) ( t)
n=0 = e(c 1) ( t)
;
n!
n ck ( t) k n
where e n!( ) is the probability of q ( ) = n and 1 n=0
e (c ( t))
n!
= 1 denotes
the summation of the probability function over the whole support of the Poisson
distribution which was used in the last step. For a generalization of this lemma, see
ex. 7.
To apply this lemma to our case Et eq( ) ln(1+ ) ; we set c = e and k = ln (1 + ) :
ln(1+ ) 1) [
Hence, Et eq( ) ln(1+ ) = eln(1+ )q(t) e(e t]
= eq(t) ln(1+ ) e [ t] = eq(t) ln(1+ )+ [ t]

and, inserted in (9.55), the expected TFP level is

Et A ( ) = At eg[ t] q(t) ln(1+ ) q(t) ln(1+ )+

e [ t]
= At e(g+ )[ t]
:

This is an intuitive result: The growth rate of the expected TFP level is driven both by
the deterministic growth component of the SDE (9.43) for TFP and by the stochastic
part. The growth rate of expected TFP is higher, the higher the determinist part,
the g; the more often the Poisson process jumps on average (a higher arrival rate )
and the higher the jump (the higher ).
This con…rms formally what was already visible in and informally discussed after
…g. 39 of ch. 9.1.2: A Poisson process as a source of uncertainty in a SDE implies
that average growth is not just determined by the deterministic part of the SDE (as
is the case when Brownian motion constitutes the disturbance term) but also by the
Poisson process itself. For a positive ; average growth is higher, for a negative ; it
is lower.

197
9.4.3 Martingales
Martingale is an impressive word for a simple concept. Here is a simpli…ed de…nition
which is su¢ cient for our purposes. More complete de…nitions (in a technical sense)
are in Øksendal (1998) or Ross (1996).

De…nition 13 A stochastic process x (t) is a martingale if, being in t today, the

expected value of x at some future point in time equals the current value of x;

Et x ( ) = x (t) ; t: (9.56)

As the expected value of x (t) is x (t) ; Et x (t) = x (t) ; this can easily be rewritten
as Et [x ( ) x (t)] = 0: This is identical to saying that x (t) is a martingale if the
expected value of its increments between now t and somewhere at in the future
is zero. This de…nition and the de…nition of Brownian motion imply that Brownian
motion is a martingale.
In what follows, we will use the martingale properties of certain processes rela-
tively frequently, e.g. when computing expected growth rates. Here are now some
fundamental examples for martingales.

Brownian motion

First, look at Brownian motion where we have a central result useful for many
applications
R where expectations and other moments are computed. It states that
t
f (z (s) ; s) dz (s), where f (:) is some function and z (s) is Brownian motion, is a
martingale (Corollary 3.2.6, Oksendal, 1998, p. 33),
Z
Et f (z (s) ; s) dz (s) = 0: (9.57)
t

Poisson uncertainty

A similar fundamental result for Poisson processes exists. We will use in what
follows the martingale property of various expressions containing R Poisson uncer-
tainty.
R These expressions are identical to or special cases of t f (q (s) ; s) dq (s)
t
f (q (s) ; s) ds; of which Garcia and Griego (1994, theorem 5.3) have shown that
it is a martingale indeed,
Z Z
Et f (q (s) ; s) dq (s) f (q (s) ; s) ds = 0: (9.58)
t t

As always, is the (constant) arrival rate of q (s).

198
9.4.4 A more general approach to computing moments
When we want to understand moments of some stochastic process, we can proceed in
two ways. Either, a SDE is expressed in its integral version, expectations operators
are applied and the resulting deterministic di¤erential equation is solved. Or, the
SDE is solved directly and then expectation operators are applied. We already saw
examples for the second approach in ch. 9.4.2. We will now follow the …rst way as
this is generally the more ‡exible one. We …rst start with examples for Brownian
motion processes and then look at cases with Poisson uncertainty.

The drift and variance rates for Brownian motion

We will now return to our …rst SDE in (9.3), dx (t) = adt + bdz (t) ; and want to
understand why a is called the drift term and b2 the variance term. To this end, we
compute the mean and variance of x ( ) for > t:
We start by expressing (9.3) in its integral representation as in ch. 9.1.3. This
gives
Z Z
x ( ) x(t) = ads + bdz(s) = a [ t] + b [z( ) z (t)] : (9.59)
t t

The expected value Et x ( ) is then simply

Et x( ) = x (t) + a [ t] + bEt [z( ) z (t)] = x (t) + a [ t] ; (9.60)

where the second step used the fact that the expected increment of Brownian motion
is zero. As the expected value Et x( ) is a deterministic variable, we can compute
the usual time derivative and …nd how the expected value of x ( ), being today in t,
changes the further the point lies in the future, dEt x( )=d = a: This makes clear
why a is referred to as the drift rate of the random variable x (:) :
Let us now analyse the variance of x ( ) where we also start from (9.59). The
variance can from (7.4) be written as vart x( ) = Et x2 ( ) [Et x( )]2 : In contrast
to the term in (7.4) we need to condition the variance on t: If we computed the
variance of x ( ) from the perspective of some earlier t; the variance would di¤er -
as will become very clear from the expression we will see in a second. Applying the
expression from (7.4) also shows that we can look at any x ( ) as a “normal”random
variable: Whether x ( ) is described by a stochastic process or by some standard
description of a random variable, an x ( ) for any …x future point in time has some
distribution with corresponding moments. This allows us to use standard rules for
computing moments. Computing …rst Et x2 ( ) gives by inserting (9.59)

Et x2 ( ) = Et [x(t) + a [ t] + b [z( ) z (t)]]2

= Et [x(t) + a [ t]]2 + 2 [x(t) + a ( t)] b [z( ) z (t)] + [b (z( ) z (t))]2
= [x(t) + a [ t]]2 + 2 [x(t) + a ( t)] bEt [z( ) z (t)] + Et [b (z( ) z (t))]2
= [x(t) + a [ t]]2 + b2 Et [z( ) z (t)]2 ; (9.61)

199
where the last equality used again that the expected increment of Brownian motion is
zero. As [Et x( )]2 = [x (t) + a [ t]]2 from (9.60), inserting (9.61) into the variance
expression gives vart x( ) = b2 Et [z( ) z (t)]2 :
Computing the mean of the second moment gives

Et [z( ) z (t)]2 = Et z 2 ( ) 2z( )z (t) + z 2 (t) = Et z 2 ( ) z 2 (t) = vart z ( ) ;

where we used that z (t) is known in t and therefore non-stochastic, that Et z( ) =

z(t) and equation (7.4) again. We therefore found that vart x( ) = b2 vart z ( ) ; the
variance of x ( ) is b2 times the variance of z ( ) : As the latter variance is given by
vart z ( ) = t; given that we focus on Brownian motion with standard normally
distributed increments, we found

vart x( ) = b2 [ t] :

This equation shows why b is the variance parameter for x (:) and why b2 is called
its variance rate. The expression also makes clear why it is so important to state the
current point in time, t in our case. If we were further in the past or is further in
the future, the variance would be larger.

Expected returns - Brownian motion

Imagine an individual owns wealth a that is allocated to N assets such that shares
in wealth are given by i ai =a: The price of each asset follows a certain pricing rule,
say geometric Brownian motion, and let’s assume that total wealth of the household,
neglecting labour income and consumption expenditure, evolves according to
N
da (t) = a (t) rdt + i=1 i i dzi (t) ;
N
where r i=1 i ri : This is in fact the budget constraint (with w c = 0) which
will be derived and used in ch. 10.4 on capital asset pricing. Note that Brownian
motions zi are correlated, i.e. dzi dzj = ij dt as in (9.15). What is the expected return
and the variance of holding such a portfolio, taking i and interest rates and variance
parameters as given?
Using the same approach as in the previous example we …nd that the expected
return is simply given by r: This is due to the fact that the mean of the BMs zi (t)
are zero. The variance of a (t) is - coming soon -

Expected returns - Poisson

Let us now compute the expected return of wealth when the evolution of wealth is
described by the budget constraint in (9.32), da (t) = fr (t) a (t) + w (t) p (t) c (t)g dt+
a (t) dq (t). When we want to do so, we …rst need to be precise about what we mean
by the expected return. We de…ne it as the growth rate of the mean of wealth when
consumption expenditure and labour income are identical, i.e. when total wealth
changes only due to capital income.

200
 Using this de…nition, we …rst compute the expected wealth level at some future
point in time : Expressing this equation in its integral representation as in ch. 9.1.3
gives
Z Z
a ( ) a (t) = fr (s) a (s) + w (s) p (s) c (s)g ds + a (s) dq (s) :
t t

Applying the expectations operator yields

Z Z
Et a ( ) a (t) = Et fr (s) a (s) + w (s) p (s) c (s)g ds + Et a (s) dq (s)
Z t Z t

= Et r (s) Et a (s) ds + Et a (s) ds:

t t

The second equality used an ingredient of the de…nition of the expected return, i.e.
w (s) = p (s) c (s) ; that r (s) is independent of a (s) and the martingale result of (9.58).
When we de…ne the mean of a (s) from the perspective of t by (s) Et a (s) ; this
equation reads
Z Z
( ) a (t) = Et r (s) (s) ds + (s) ds:
t t

Computing the derivative with respect to time gives

_( )
_ ( ) = Et r ( ) ( ) + ( ), = Et r ( ) + : (9.62)
( )
Hence, the expected return is given by Et r ( ) + :
Expected growth rate
Finally, we ask what the expected growth rate and the variance of the price v ( ) is
when it follows the geometric Poisson process known from (9.31), dv ( ) = v ( ) d +
v ( ) dq ( ) :
The expected growth rate can be de…ned as Et v( v(t)
) v(t)
: Given that v (t) is known in
t; we can write this expected growth rate as Et v(v(t)
) v(t)
where expectations are formed
only with respect to the future price v ( ) : Given this expression, we can follow the
usual approach. The integral version of the SDE, applying the expectations operator,
is Z Z
Et v ( ) v (t) = Et v (s) ds + Et v (s) dq (s) :
t t
Pulling the expectations operator inside the integral, using the martingale result from
(9.58) and de…ning (s) Et v (s) gives
Z Z
( ) v (t) = (s) ds + (s) ds:
t t

The time- derivative gives _ ( ) = ( )+ ( ) which shows that the growth

rate of the mean of the price is given by + : This is, as just discussed, also the
expected growth rate of the price v ( ) :
We now turn to the variance of v ( ) : - coming soon -

201
 Disentangling the mean and variance of a Poisson process

Consider the SDE dv (t) =v (t) = dt + dq (t) where the arrival rate of q (t) is
given by = : The mean of v ( ) is Et v ( ) = v (t) exp( + )( t) and hthereby indepen-
i
2 2
[ t]
dent of : The variance, however, is given by vart v1 ( ) = [Et v ( )] exp 1 .
A mean preserving spread can thus be achieved by an increase of : This increases
the randomly occurring jump and reduces the arrival rate = - the mean remains
unchanged, the variance increases.13

Computing the mean step-by-step for a Poisson example

Let us now consider some stochastic process X (t) described by a di¤erential equa-
tion and ask what we know about expected values of X ( ) ; where lies in the future,
i.e. > t: We take as …rst example a stochastic process similar to (9.8). We take as
an example the speci…cation for total factor productivity A,

dA (t)
= dt + 1 dq1 (t) 2 dq2 (t) ; (9.63)
A (t)

where and i are positive constants and the arrival rates of the processes are given
by some constant i > 0. This equation says that TFP increases in a deterministic
way at the constant rate (note that the left hand side of this di¤erential equation
gives the growth rate of A (t)) and jumps at random points in time. Jumps can be
positive when dq1 = 1 and TFP increases by the factor 1 ; i.e. it increases by 1 %;
or negative when dq2 = 1; i.e. TFP decreases by 2 %:
The integral version of (9.63) reads (see ch. 9.1.3)
Z Z Z
A ( ) A (t) = A (s) ds + 1 A (s) dq1 (s) 2 A (s) dq2 (s)
t t t
Z Z Z
= A (s) ds + 1 A (s) dq1 (s) 2 A (s) dq2 (s) : (9.64)
t t t

When we form expectations, we obtain

Z Z Z
Et A ( ) A (t) = Et A (s) ds + 1 Et A (s) dq1 (s) 2 Et A (s) dq2 (s)
t t t
Z Z Z
= Et A (s) ds + 1 1 Et A (s) ds 2 2 Et A (s) ds:
t t t
(9.65)
13
This is taken from Sennewald and Wälde (2006). An alternative is provided by Steger (2005)
who uses two symmetric Poisson processes instead of one here. He obtains higher risk at an invariant
mean by increasing the symmetric jump size.

202
where the second equality used the martingale result from ch. 9.4.3, i.e. the expression
in (9.58). Pulling the expectations operator into the integral gives14
Z Z Z
Et A ( ) A (t) = Et A (s) ds + 1 1 Et A (s) ds 2 2 Et A (s) ds:
t t t

When we …nally de…ne m1 ( ) Et A ( ) ; we obtain a deterministic di¤erential equa-

tion that describes the evolution of the expected value of A ( ) from the perspective
of t: We …rst have the integral equation
Z Z Z
m1 ( ) A (t) = m1 (s) ds + 1 1 m1 (s) ds 2 2 m1 (s) ds
t t t

which we can then di¤erentiate with respect to time by applying the rule for di¤er-
entiating integrals from (4.9),

m
_ 1 ( ) = m1 ( ) + 1 1 m1 ( ) 2 2 m1 ( )
=( + 1 1 2 2 ) m1 ( ) : (9.66)

We now immediately see that TFP does not increase in expected terms, more
precisely Et A ( ) = A (t) ; if + 1 1 = 2 2 : Economically speaking, if the increase
in TFP through the deterministic component and the stochastic component 1
is “destroyed” on average by the second stochastic component 2 ; TFP does not
increase.

9.5 Exercise to chapter 9 and further reading

Mathematical background

There are many textbooks on di¤erential equations, Ito calculus, change of vari-
able formulas and related aspects in mathematics. One that is widely used is Øk-
sendal (1998). A more technical approach is presented in Protter (1995). A classic
mathematical reference is Gihman and Skorohod (1972). A special focus with a de-
tailed formal analysis of SDEs with Poisson processes can be found in Garcia and
Griego (1994). They also provide solutions of SDEs and the background for com-
puting moments of stochastic processes. Further solutions, applied to option pricing,
are provided by Das and Foresi (1996). The CVF for the combined Poisson-di¤usion
setup in lemma 8 is a special case of the expression in Sennewald (2006) which in
turn is based on Øksendal and Sulem (2005). Øksendal and Sulem present CVFs for
more general Levyprocesses of which the SDE (9.22) is a very simple special case.

A less technical background

14
The integral and the expectations operator can be exchanged when, under a technical condition,
both the expected integral and the integral of the expected expression exist. The expected integral
exists by assumption as otherwise our di¤erential equation would be speci…ed in a meaningless way.
The existence of the integral of the expected expression will be shown by computing it. The existence
proof is therefore an ex-post proof. We are grateful to Ken Sennewald for discussions of this issue.

203
 A very readable introduction to stochastic processes is by Ross (1993, 1996). He
writes in the introduction to the …rst edition of his 1996 book that his text is “a
nonmeasure theoretic introduction to stochastic processes”. This makes this book
highly accessible for economists.
An introduction with many examples from economics is Dixit and Pindyck (1994).
See also Turnovsky (1997, 2000) for many applications. Brownian motion is treated
extensively in Chang (2004).
The CVF for Poisson processes is most easily accessible in Sennewald (2006) or
Sennewald and Wälde (2006). Sennewald (2006) provides the mathematical proofs,
Sennewald and Wälde (2006) has a focus on applications.

How to present technologies in continuous time

There is a tradition in economics starting with Eaton (1981) where output is rep-
resented by a stochastic di¤erential equation as presented in ch. 9.3.1. This and
similar representations of technologies are used by Epaulart and Pommeret (2003),
Pommeret and Smith (2005), Turnovsky and Smith (2006), Turnovsky (2000), Chat-
terjee, Giuliano and Turnovsky (2004), and many others. It is well known (see e.g.
footnote 4 in Grinols and Turnovsky (1998)) that this implies the possibility of nega-
tive Y . An alternative where standard Cobb-Douglas technologies are used and TFP
is described by a SDE to this approach is presented in Wälde (2005) or Nolte and
Wälde (2007).

Application of Poisson processes in economics

The Poisson process is widely used in …nance (early references are Merton, 1969,
1971) and labour economics (in matching and search models, see e.g. Pissarides,
2000, Burdett and Mortenson, 1998 or Moscarini, 2005). See also the literature
on the real options approach to investment (McDonald and Siegel, 1986, Dixit and
Pindyck, 1994, Chen and Funke 2005 or Guo et al., 2005). It is also used in growth
models (e.g. quality ladder models à la Aghion and Howitt, 1992 or Grossman and
Helpman, 1991), in analyzes of business cycles in the natural volatility tradition (e.g.
Wälde, 2005), contract theory (e.g. Guriev and Kvasov, 2005), in the search approach
to monetary economics (e.g. Kiyotaki and Wright, 1993 and subsequent work) and
many other areas. Further examples include Toche (2005), Steger (2005), Laitner and
Stolyarov (2004), Farzin et al. (1998), Hassett and Metcalf (1999), Thompson and
Waldo (1994), Palokangas (2003, 2005) and Venegas-Martínez (2001).
One Poisson shock a¤ecting many variables (as stressed in lemma 7) was used by
Aghion and Howitt (1992) in their famous growth model. When deriving the budget
constraint in Appendix 1 of Wälde (1999a), it is taken into consideration that a jump
in the technology level a¤ects both the capital stock directly as well as its price. Other
examples include the natural volatility papers by Wälde (2002, 2005) and Posch and
Wälde (2006).

204
 Exercises Chapter 9
Applied Intertemporal Optimization
Stochastic di¤erential equations and rules
for di¤erentiating

1. Di¤erentials of functions of stochastic processes I

Assume x (t) and y (t) are two correlated Wiener processes. Compute d [x (t) y (t)],
d [x (t) =y (t)] and d ln x (t) :

2. Di¤erentials of functions of stochastic processes II

(a) Show that with F = F (x; y) = xy and dx = f x (x; y) dt + g x (x; y) dq x

and dy = f y (x; y) dt + g y (x; y) dq y , where q x and q y are two independent
Poisson processes, dF = xdy + ydx.
(b) Does this also hold for two Wiener processes q x and q y ?

3. Correlated jump processes (exam)

Let qi (t) for i = 1; 2; 3 be three independent Poisson processes. De…ne two
jump processes by qx (t) q1 (t) + q2 (t) and qy (t) q1 (t) + q3 (t) : Let labour
productivity in sector X and Y be driven by

dA (t) = dt + dqx (t) ;

dB (t) = dt + dqy (t) :

Let GDP in a small open economy (with internationally given constant prices
px and py ) be given by

Y (t) px A (t) Lx + py B (t) Ly :

(a) Given an economic interpretation to equations dA (t) and dB (t), based on

qi (t) ; i = 1; 2; 3:
(b) How does GDP evolve over time in this economy if labour allocation Lx
and Ly is invariant? Express the di¤erential dY (t) by using dqx and dqy
if possible.

4. Deriving a budget constraint

Consider a household who owns some wealth a = n1 v1 + n2 v2 , where ni denotes

205
 the number of shares held by the household and vi is the value of the share.
Assume that the value of the share evolves according to

dvi = i vi dt + i vi dqi :

Assume further that the number of shares held by the household changes ac-
cording to
( 1 n1 + 2 n2 + w e)
dni = i dt;
vi
where i is the share of savings used for buying stock i:

(a) Give an interpretation (in words) to the last equation.

(b) Derive the household’s budget constraint.

5. Option pricing
Assume the price of an asset follows dS=S = dt + dz + dq (as in Merton,
1976), where z is Brownian motion and q is a Poisson process. This is a gen-
eralization of (9.24) where = 0: How does the di¤erential equation look like
that determines the price of an option on this asset?
6. Martingales

(a) The weather tommorow will be just the same as today. Is this a martin-
gale?
(b) Let z (s) be Brownian motion. Show that Y (t) de…ned by
Z t Z
1 t 2
Y (t) exp f (s) dz (s) f (s) ds (9.67)
0 2 0
is a martingale.
(c) Show that X (t) de…ned by
Z t Z t
1
X (t) exp f (s) dz (s) f 2 (s) ds
0 2 0

is also a martingale.

7. Expectations - Poisson
Assume that you are in t and form expectations about future arrivals of the
Poisson process q (t). Prove the following statement by using lemma 9: The
number of expected arrivals in the time interval [ ; s] equals the number of
expected arrivals in a time interval of the length s for any s > t,
k
Et (ck[q( ) q(s)]
) = E(ck[q( ) q(s)]
) = e(c 1) ( s)
; > s > t; c; k = const:

Hint: If you want to cheat, look at the appendix to Posch and Wälde (2006). It
is available e.g. at www.waelde.com/publications.

206
 8. Expected values
Show that the growth rate of the mean of x (t) described by the geometric
Brownian motion
dx (t) = ax (t) dt + bx (t) dz (t)
is given by a:

(a) Do so by using the integral version of this SDE and compute the increase
of the expected value of x (t).
(b) Do the same as in (a) but solve …rst the SDE and compute expectations
by using this solution.
(c) Compute the covariance of x ( ) and x (s) for >s t:

9. Expected returns
Consider the budget constraint

da (t) = fra (t) + w c (t)g dt + a (t) dz (t) :

(a) What is the expected return for wealth? Why does this expression di¤er
from (9.62)?
(b) What is the variance of wealth?

10. “Di¤erential-representation”of a technology

(a) Consider the “di¤erential-representation”of a technology, dY (t) = AKdt+

Kdz (t) ; as presented in (9.33). Compute expected output of Y ( ) for
> t and the variance of Y ( ) :
(b) Consider a more standard representation by Y (t) = A (t) K and let TFP
follow dA (t) =A (t) = gdt + dz (t) as in (9.34). What is the expected
output level Et Y ( ) and what is its variance?

11. Solving stochastic di¤erential equations I

Consider the di¤erential equation

dx (t) = [a (t) x (t)] dt + B1 (t) x (t) dW1 (t) + b2 (t) dW2 (t)

where W1 and W2 are two correlated Brownian motions.

(a) What is the solution of this di¤erential equation?

(b) Use Ito’s Lemma to show that your solution is a solution.

12. Solving stochastic di¤erential equations II

Look at the generalization of (9.33) where now the capital stock can change
over time, i.e. dY (t) = AK (t) dt + K (t) dz (t) : Proof that Y (t) = ::: coming
soon ::: is a solution of this SDE.

207
13. Dynamic and intertemporal budget constraints - Brownian motion
Consider the dynamic budget constraint

da ( ) = (r ( ) a ( ) + w ( ) p ( ) c ( )) dt + a ( ) dz ( ) ;

where z ( ) is Brownian motion.

(a) Show that the intertemporal version of this budget constraint, after making
a standard economic assumption, can be written as
Z 1 Z 1
'( )
e p ( ) c ( ) d = at + e '( ) w ( ) d
t t

where the discount factor ' ( ) is given by

Z Z
1 2
'( ) = r (s) ds + dz ( ) :
t 2 t

(b) Now assume we are willing to assume that intertemporal budget constraints
need to hold in expectations only and not in realizations: we require only
that agents balance at each instant expected consumption expenditure to
current …nancial wealth plus expected labour income. Show that the in-
tertemporal budget constraint then simpli…es to
Z 1 R Z 1 R
r(s)ds
Et e t p ( ) c ( ) d = at + Et e t r(s)ds w ( ) d ;
t t

i.e. all variance terms drop out of the discount factor.

208
10 Dynamic programming
We now return to our main concern: How to solve maximization problems. We
…rst look at optimal behaviour under Poisson uncertainty where we analyse cases for
uncertain asset prices and labour income. We then switch to Brownian motion and
look at capital asset pricing as an application.

10.1 Optimal saving under Poisson uncertainty

10.1.1 The setup
Let us consider an individual that tries to maximize her objective function that is
given by Z 1
U (t) = Et e [ t] u (c ( )) d : (10.1)
t
The structure of this objective function is identical to the one we know from de-
terministic continuous time models in e.g. (5.1) or (5.28): We are in t today, the
time preference rate is > 0; instantaneous utility is given by u (c ( )) : Given the
uncertain environment the individual lives in, we now need to form expectations as
consumption in future points in time is unknown.
When formulating the budget constraint of a household, we have now seen at
various occasions that it is a good idea to derive it from the de…nition of wealth
of a household. We did so in discrete time models in ch. 2.5.5 and in continuous
time models in ch. 4.3.4. Deriving the budget constraint in stochastic continuous
time models is especially important as a budget constraint in an economy where the
fundamental source of uncertainty is Brownian motion looks very di¤erent from one
where uncertainty stems from Poisson or Levy processes. For this …rst example, we
use the budget constraint (9.32) derived in ch. 9.2.6,

da (t) = fr (t) a (t) + w (t) pc (t)g dt + a (t) dq (t) ; (10.2)

where the interest rate r (t) + (t) =v (t) was de…ned as the deterministic rate of
change of the price of the asset (compare the equation for the evolution of assets
in (9.31)) plus dividend payments (t) =v (t) : We treat the price p here as a constant
(see the exercises for an extension). Following the tilde notation from (9.7), we can
express wealth a~ (t) after a jump by

a
~ (t) = (1 + ) a (t) : (10.3)

The budget constraint of this individual re‡ects standard economic ideas about
budget constraints under uncertainty. As visible in the derivation in ch. 9.2.6, the
uncertainty for this households stems from uncertainty about the evolution of the
price (re‡ected in ) of the asset she saves in. No statement was made about the
evolution of the wage w (t) : Hence, we take w (t) here as parametric, i.e. if there are
stochastic changes, they all come as a surprise and are therefore not anticipated. The
household does take into consideration, however, the uncertainty resulting from the

209
evolution of the price v (t) of the asset. In addition to the deterministic growth rate
of v (t) ; v (t) changes in a stochastic way by jumping occasional by percent (again,
see (9.31)). The returns to wealth a (t) are therefore uncertain and are composed of
the “usual”r (t) a (t) term and the stochastic a (t) term.

10.1.2 Three dynamic programming steps

We will solve this problem as before by dynamic programming methods. Note, how-
ever, that it is not obvious whether the above problem can be solved by dynamic
programming methods. In principle, a proof is required that dynamic programming
indeed yields necessary and su¢ cient conditions for an optimum. While proofs exist
for bounded instantaneous utility function u (t) ; such a proof did not exist until re-
cently for unbounded utility functions. Sennewald (2006) extends the standard proofs
and shows that dynamic programming can also be used for unbounded utility func-
tions. We can therefore follow the usual three-step approach to dynamic programming
here as well. The result of this section combine

DP1: Bellman equation and …rst-order conditions

The …rst tool we need to derive rules for optimal behavior is the Bellman equation.
De…ning the optimal program as V (a) maxfc( )g U (t) subject to a constraint, this
equation is given by (see Sennewald and Wälde, 2006, or Sennewald, 2006)
1
V (a (t)) = max u (c (t)) + Et dV (a (t)) : (10.4)
c(t) dt
The Bellman equation has this basic form for “most”maximization problems in con-
tinuous time. It can therefore be taken as the starting point for other maximization
problems as well, independently e.g. of whether uncertainty is driven by Poisson
processes, Brownian motion or Levy processes. We will see examples for related
problems later (see ch. 10.1.4, ch. 10.2.2 or ch. 10.3.2) and discuss then how to ad-
just certain features of this “general”Bellman equation. In this equation, the variable
a (t) represents the state variable, in our case wealth of the individual.
Given the general form of the Bellman equation in (10.4), we need to compute the
di¤erential dV (a (t)) : Given the evolution of a (t) in (10.2) and the CVF from (9.18),
we …nd

dV (a (t)) = V 0 (a) fra + w pcg dt + fV (a + a) V (a)g dq:

In contrast to the CVF notation in e.g. (9.18), we use here and in what follows
simple derivative signs like V 0 (a) as often as possible in contrast to e.g. Va (a) :
This is possible as long as the functions, like the value function V (a) here, have one
argument only. Forming expectations about dV (a (t)) is easy and they are given by

Et dV (a (t)) = V 0 (a) fra + w pcg dt + fV (~

a) V (a)g Et dq:

The …rst term, the “dt-term” is known in t: The current state a (t) and all prices
are known and the shadow price V 0 (a) is therefore also known. As a consequence,

210
expectations need to be applied only to the “dq-term”. The …rst part of the “dq-term”,
the expression V ((1 + ) a) V (a) is also known in t as again a (t), parameters and
the function V are all non-stochastic. We therefore only have to compute expectations
about dq: From (9.54), we know that Et [q ( ) q (t)] = [ t] : Now replace q ( )
q (t) by dq and t by dt and …nd Et dq = dt: The Bellman equation therefore reads

V (a) = max fu (c (t)) + V 0 (a) [ra + w pc] + [V ((1 + ) a) V (a)]g : (10.5)

c(t)

Note that forming expectations the way just used is somewhat “‡oppy”. Doing it
in the more stringent way introduced in ch. 9.4.4 would, however, lead to identical
results.
The …rst-order condition is

u0 (c) = V 0 (a) p: (10.6)

As always, (current) utility from an additional unit of consumption u0 (c) must equal
(future) utility from an additional unit of wealth V 0 (a), multiplied by the price p of
the consumption good, i.e. by the number of units of wealth for which one can buy
one unit of the consumption good.

DP2: Evolution of the costate variable

In order to understand the evolution of the marginal value V 0 (a) of the optimal
program, i.e. the evolution of the costate variable, we need to (i) compute the partial
derivative of the maximized Bellman equation with respect to assets and (ii) compute
the di¤erential dV 0 (a) by using the CVF and insert the partial derivative into this
expression. These two steps corresponds to the two steps in DP2 in the deterministic
continuous time setup of ch. 6.1.
(i) In the …rst step, we state the maximized Bellman equation from (10.5) as the
Bellman equation where controls are replaced by their optimal values,

V (a) = u (c (a)) + V 0 (a) [ra + w pc (a)] + [V ((1 + ) a) V (a)] :

We then compute again the derivative with respect to the state - as in discrete time
and deterministic continuous time setups - as this gives us an expression for the
shadow price V 0 (a). In contrast to the previous emphasis on Ito’s Lemmas and CVFs,
we can use for this step standard rules from algebra as we compute the derivative for
a given state a - the state variable is held constant and we want to understand the
derivative of the function V (a) with respect to a: We do not compute the di¤erential
of V (a) and ask how the value function changes as a function of a change in a.
Therefore, using the envelope theorem,

V 0 (a) = V 0 (a) r + V 00 (a) [ra + w pc] + [V 0 (~

a) [1 + ] V 0 (a)] : (10.7)

We used here the de…nition of a

~ given in (10.3).

211
 (ii) In the second step, the di¤erential of the shadow price V 0 (a) is computed,
given again the evolution of a (t) in (10.2),

dV 0 (a) = V 00 (a) [ra + w pc] dt + [V 0 (~

a) V 0 (a)] dq: (10.8)

Finally, replacing V 00 (a) [ra + w pc] in (10.8) by the same expression from (10.7)
gives

dV 0 (a) = f( r) V 0 (a) [V 0 (~
a) [1 + ] V 0 (a)]g dt + fV 0 (~
a) V 0 (a)g dq:

DP3: Inserting …rst-order conditions

Finally, we can replace the marginal value by marginal utility from the …rst-
order condition (10.6). In this step, we employ that p is constant and therefore
dV 0 (a) = p 1 du0 (c). Hence, the Keynes-Ramsey rule describing the evolution of
marginal utility reads

du0 (c) = f( r) u0 (c) [u0 (~

c) [1 + ] u0 (c)]g dt + fu0 (~
c) u0 (c)g dq: (10.9)

Note that the constant price p dropped out. This rule shows how marginal utility
changes in a deterministic and stochastic way.

10.1.3 The solution

The dynamic programming approach provided us with an expression in (10.9) which
describes the evolution of marginal utility from consumption. While there is a one-to-
one mapping from marginal utility to consumption which would allow some inferences
about consumption from (10.9), it would nevertheless be more useful to have a Keynes-
Ramsey rule for optimal consumption itself.

The evolution of consumption

If we want to know more about the evolution of consumption, we can use the CVF
formula as follows. Let f (:) be the inverse function for u0 ; i.e. f (u0 (c)) = c; and
apply the CVF to f (u0 (c)) : This gives

df (u0 (c)) = f 0 (u0 (c)) f( r) u0 (c) [u0 (~

c) [1 + ] u0 (c)]g dt+ff (u0 (~
c)) f (u0 (c))g dq:
df (u0 (c)) dc 1
As f (u0 (c)) = c; we know that f (u0 (~
c)) = c~ and f 0 (u0 (~
c)) du0 (c)
= du0 (c)
= u00 (c)
:
Hence,
1
dc = f( r) u0 (c) [u0 (~
c) [1 + ] u0 (c)]g dt + f~
c cg dq ,
u00
(c)
u00 (c) u0 (~
c) u00 (c)
dc = r + [1 + ] 1 dt f~
c cg dq: (10.10)
u0 (c) 0
u (c) u0 (c)
This is the Keynes-Ramsey rule that describes the evolution of consumption under
optimal behaviour for a household that faces stochastic interest rate uncertainty re-
sulting from Poisson processes. This equation is useful to understand e.g. economic

212
‡uctuations in a natural volatility setup (see e.g. Matsuyama, 1999, Francois and
Lloyd-Ellis, 2003 or Wälde, 1999, 2005). It corresponds to its deterministic pendant
in (5.30) in ch. 5.5.3: By setting = 0 here (implying dq = 0), noting that we treated
the price as a constant and dividing by dt, we obtain (5.30).

A speci…c utility function

Let us now assume for the instantaneous utility function the widely used constant
relative risk aversion (CRRA) instantaneous utility function,

c ( )1 1
u (c ( )) = ; > 0: (10.11)
1
Then, the Keynes-Ramsey rule becomes

dc n h c io c~
= r + (1 + ) 1 dt + 1 dq: (10.12)
c c~ c
The left-hand side gives the proportional change of consumption times ; the inverse
1
of the intertemporal elasticity of substitution : This corresponds to c=c
_ in the
deterministic Keynes-Ramsey rule in e.g. (5.31). Growth of consumption depends
on the right-hand side in a deterministic way on the usual di¤erence between the
interest rate and time preference rate plus the “ term”which captures the impact
of uncertainty. When we want to understand the meaning of this term, we need to
…nd out whether consumption jumps up or down, following a jump of the Poisson
process. When is positive, the household holds more wealth and consumption
should increase. Hence, the ratio c=~ c is smaller than unity and the sign of the bracket
term (1 + ) cc~ 1 is qualitatively unclear. If it is positive, consumption growth is
faster in a world where wealth occasionally jumps by percent.
The dq-term gives discrete changes in the case of a jump in q: It is tautological,
however: When q jumps and dq = 1 and dt = 0 for this small instant of the jump,
(10.12) says that dc=c on the left-hand side is given by f~ c=c 1g on the right hand
side. As the left hand side is by de…nition of dc given by [~ c c] =c; both sides are
identical. Hence, the level of c~ after the jump needs to be determined in an alternative
way.

10.1.4 Optimal consumption and portfolio choice

This section analyses a more complex maximization problem than the one presented
in ch. 10.1.1. In addition to the consumption-saving trade-o¤, it includes a portfolio
choice problem. Interestingly, the solution is much simpler to work with as c~ can
explicitly be computed and closed form solutions can easily be found.

The maximization problem

Consider a household that is endowed with some initial wealth a0 > 0. At each
instant, the household can invest its wealth a (t) in both a risky and a safe asset. The

213
share of wealth the household holds in the risky asset is denoted by (t). The price
v1 (t) of one unit of the risky asset obeys the SDE

dv1 (t) = r1 v1 (t) dt + v1 (t) dq (t) ; (10.13)

where r1 2 R and > 0. That is, the price of the risky asset grows at each instant with
a …xed rate r1 and at random points in time it jumps by percent. The randomness
comes from the well-known Poisson process q (t) with arrival rate . The price v2 (t)
of one unit of the safe asset is assumed to follow

dv2 (t) = r2 v2 (t) dt; (10.14)

where r2 0. Let the household receive a …xed wage income w and spend c (t) 0
on consumption. Then, in analogy to ch. 9.2.6, the household’s budget constraint
reads

da (t) = f[ (t) r1 + (1 (t)) r2 ] a (t) + w c (t)g dt + (t) a (t) dq (t) : (10.15)

We allow wealth to become negative but we could assume that debt is always covered
by the household’s lifetime labour income discounted with the safe interest rate r2 ,
i.e. a (t) > w=r2 .
Let intertemporal preferences of households be identical to the previous maximiza-
tion problem - see (10.1). The instantaneous utility function is again characterized
by CRRA as in (10.11), u(c( )) = (c1 1) = (1 ) : The control variables of the
household are the nonnegative consumption stream fc (t)g and the share f (t)g held
in the risky asset. To avoid a trivial investment problem, we assume

r1 < r 2 < r 1 + : (10.16)

That is, the guaranteed return of the risky asset, r1 , is lower than the return of the
riskless asset, r2 , whereas, on the other hand, the expected return of the risky asset,
r1 + , shall be greater than r2 . If r1 was larger than r2 , the risky asset would
dominate the riskless one and noone would want to hold positive amounts of the
riskless asset. If r2 exceeded r1 + ; the riskless asset would dominate.

DP1: Bellman equation and …rst-order conditions

Again, the …rst step of the solution of this maximization problem requires a Bell-
man equation. De…ne the value function V again as15 V (a (t)) maxfc( )g U (t) :
The basic Bellman equation is taken from (10.4). When computing the di¤eren-
tial dV (a (t)) and taking into account that there are now two control variables, the
Bellman equation reads

V (a) = max fu(c) + [( r1 + (1 ) r2 ) a + w c] V 0 (a) + [V (~

a) V (a)]g ;
c(t); (t)
(10.17)
15
One can show that the value function in this example does not depend on initial time but on
initial wealth only. An “ex-post proof” is given in Sennewald and Wälde (2006) where an explicit
expression for the value function is provided.

214
where a~ (1 + ) a denotes the post-jump wealth if at wealth a a jump in the risky
asset price occurs. The …rst-order conditions which any optimal paths must satisfy
are given by
u0 (c) = V 0 (a) (10.18)
and
V 0 (a) (r1 r2 ) a + V 0 (~
a) a = 0: (10.19)
While the …rst …rst-order condition equates as always marginal utility with the shadow
price, the second …rst-order condition determines optimal investment of wealth into
assets 1 and 2; i.e. the optimal share : The latter …rst-order condition contains a
deterministic and a stochastic term and households hold their optimal share if these
two components just add up to zero. Assume, consistent with (10.16), that r1 < r2 : If
we were in a deterministic world, i.e. = 0; households would then only hold asset 2
as its return is higher. In a stochastic world, the lower instantaneous return on asset
1 is compensated by the fact that, as (10.13) shows, the price of this asset jumps up
occasionally by the percentage : Lower instantaneous returns r1 paid at each instant
are therefore compensated for by large occasional positive jumps.
As this …rst-order condition also shows, returns and jumps per se do not matter:
The di¤erence r1 r2 in returns is multiplied by the shadow price V 0 (a) of capital and
the e¤ect of the jump size times its frequence, ; is multiplied by the shadow price
0
V (~a) of capital after a jump. What matters for the household decision is therefore
the impact of holding wealth in one or the other asset on the overall value from
behaving optimally, i.e. on the value function V (a). The channels through which
asset returns a¤ect the value function is …rst the impact on wealth and second the
impact of wealth on the value function i.e. the shadow price of wealth.
We can now immediately see why this more complex maximization problem yields
simpler solutions: Replacing in equation (10.19) V 0 (a) with u0 (c) according to (10.18)
yields for a 6= 0
u0 (~
c) r2 r 1
0
= ; (10.20)
u (c)
where c~ denotes the optimal consumption choice for a
~. Hence, the ratio for optimal
consumption after and before a jump is constant. If we assume e.g. a CES utility
function as in (10.11), this jump is given by
1=
c~
= : (10.21)
c r2 r1

No such result on relative consumption before and after the jump is available for the
maximization problem without a choice between a risky and a riskless asset.
Since by assumption (10.16) the term on the right-hand side is greater than 1, this
equation shows that consumption jumps upwards if a jump in the risky asset price
occurs. This result is not surprising since, if the risky asset price jumps upwards, so
does the household’s wealth.

DP2: Evolution of the costate variable

215
 In the next step, we compute the evolution of V 0 (a (t)), the shadow price of wealth.
Assume that V is twice continuously di¤erentiable. Then, due to budget constraint
(10.15), the CVF from (9.18) yields

dV 0 (a) = f[ r1 + (1 ) r2 ] a + w cg V 00 (a) dt
0 0
+ fV (~a) V (a)g dq (t) : (10.22)

Di¤erentiating the maximized Bellman equation yields under application of the en-
velope theorem

V 0 (a) = f[ r1 + (1 ) r2 ] a + w cg V 00 (a)
+ f r1 + [1 ] r2 g V 0 (a) + fV 0 (~
a) [1 + ] V 0 (a)g :

Rearranging gives

f[ r1 + (1 ) r2 ] a + w cg V 00 (a)
=f [ r1 + (1 ) r2 ]g V 0 (a) fV 0 (~
a) [1 + ] V 0 (a)g :

Inserting this into (10.22) yields

f [ r1 + (1 ) r2 ]g V 0 (a)
dV 0 (a) = 0 dt + fV 0 (~
a) V 0 (a)g dq (t) :
f[1 + ] V (~ a) V 0 (a)g

DP3: Inserting …rst-order conditions

Replacing V 0 (a) by u0 (c) following the …rst-order condition (10.18) for optimal
consumption, we obtain

f [ r1 + (1 ) r2 ]g u0 (c)
du0 (c) = 0 dt + fu0 (~
c) u0 (c)g dq (t) :
f[1 + ] u (~ c) u0 (c)g

Now applying the CVF again to f (x) = (u0 ) 1 (x) and using (10.20) leads to the
Keynes-Ramsey rule for general utility functions u,

u00 (c) r2 r1 u00 (c)

dc = r1 + [1 ] r2 + [1 + ] 1 dt f~
c cg dq (t) :
u0 (c) u0 (c)

As c~ is also implicity determined by (10.20), this Keynes-Ramsey rule describes the

evolution of consumption under Poisson uncertainty without the c~ term. This is the
crucial modelling advantage from introducing an additional asset into a standard
consumption-saving problem. Apart from this simpli…cation, the structure of this
Keynes-Ramsey rule is identical to the one in (10.10) without a second asset.

A speci…c utility function

216
 For the CRRA utility function as in (10.11), the elimination of c~ becomes even
simpler and we obtain with (10.21)
( )
1=
dc (t) 1 r2 r1
= r2 1 dt + 1 dq (t) :
c (t) r 2 r1
The optimal change in consumption can thus be expressed in terms of well-known pa-
rameters. As long as the price hof the risky asset does not
i jump, optimal consumption
r2 r1
grows constantly by the rate r2 1 = . The higher the risk-free
interest rate, r2 , and the lower the guaranteed interest rate of the risky asset, r1 , the
discrete growth rate, , the probability of a price jump, , the time preference rate,
, and the risk aversion parameter, , the higher becomes the consumption growth
rate. If the risky asset price jumps, consumption jumps as well to its new higher level
c (t) = [( ) = (r2 r1 )]1= c (t). Here the growth rate depends positively on , , and
r1 , whereas r2 and have negative in‡uence.

10.1.5 Other ways to determine c~

The question of how to determine c~ without an additional asset is in principle identical
to determining the initial level of consumption, given a deterministic Keynes-Ramsey
rule as e.g. (5.5). Whenever a jump in c following (10.12) occurs, the household
faces the issue of how to choose the initial level of consumption after the jump. In
principle, the level c~ is therefore pinned down by some transversality condition. In
practice, the literature o¤ers two ways, how c~ can be determined.
One asset and idiosyncratic risk
When households determine optimal savings only, as in our setup where the only
…rst-order condition is (10.6), c~ can be determined (in principle) if we assume that
the value function is a function of wealth only - which would be the case in our
household example if the interest rate and wages did not depend on q. This would
naturally be the case in idiosyncratic risk models where aggregate variables do not
depend on individual uncertainty resulting from q: The …rst-order condition (10.6)
then reads u0 (c) = V 0 (a) (with p = 1). This is equivalent to saying that consumption
is a function of the only state variable, i.e. wealth a, c = c (a) : An example for c (a)
is plotted in the following …gure.
N
c

s
dq = 1 c(a)
N

e
c

s
c
N

a0 a1 a2 a
217
Figure 42 Consumption c as a function of wealth a

As consumption c does not depend on q directly, we must be at the same con-

sumption level c (a) ; no matter “how we got there”, i.e. no matter how many jumps
took place before. Hence, if we jump from some a1 to a2 because dq = 1; we are at
the same consumption level c (a2 ) as if we had reached a2 smoothly without jump.
The consumption level c~ after a jump is therefore the consumption level that belongs
to the asset level after the jump according to the policy function c (a) plotted in the
above …gure, c~ = c (a2 ). See Schlegel (2004) for a numerical implementation of this
approach.
Finding value functions
A very long tradition exists in economics where value functions are found by an
“educated guess”. Experience tells us - based on …rst examples by Merton (1969,
1971) - how value functions generally look like. It is then possible to …nd, after some
attempts, the value function for some speci…c problem. This then implies explicit -
so called closed-form solutions - for consumption (or any other control variable). For
the saving and investment problem in ch. 10.1.4, Sennewald and Wälde (2006, sect.
3.4) presented a value function and closed-form solutions for a 6= 0 of the form
h i1 " #
w 1 1
a + r2 w a + rw2
V (a) = ; c= a+ ; = 1 :
1 r2 r2 r 1 a

Consumption is a constant share ( is a collection of parameters) out of the total

wealth, i.e. …nancial wealth a plus “human wealth”w=r2 (the present value of current
and future labour income). The optimal share depends on total wealth as well, but
also on interest rates, the degree of risk-aversion and the level of the jump of the
risky price in (10.13). Hence, it is possible to work with complex stochastic models
that allow to analyse many interesting real-world features and nevertheless end up
with explicit closed-form solutions. Many further examples exist - see ch. 10.6 on
further reading.
Finding value functions for special cases
As we have just seen, value functions and closed-form solutions can be found
for some models which have “nice features”. For a much larger class of models -
which are then standard models - closed form solutions can not be found for general
parameter sets. Economists then either go for numerical solutions, which preserves a
certain generality as in principle the properties of the model can be analyzed for all
parameter values, or they restrict the parameter set in a useful way. Useful means that
with some parameter restriction, value functions can be found again and closed-form
solutions are again possible.
Doing more serious maths
- coming soon -

218
10.1.6 Expected growth
Let us now try to understand the impact of uncertainty on expected growth. In order
to compute expected growth of consumption from realized growth rates (10.12), we
rewrite this equation as
c (t)
dc (t) = r (t) + (1 + ) 1 c (t) dt + f~
c (t) c (t)g dq:
c~ (t)
Expressing it in its integral version as in (9.64), we obtain for > t;
Z
c (s)
[c ( ) c (t)] = r (s) + (1 + ) 1 c (s) ds
t c~ (s)
Z
+ f~
c (s) c (s)g dq (s) :
t

Applying the expectations operator, given knowledge in t; yields

Z
1 c (s)
Et c ( ) c (t) = Et r (s) + (1 + ) 1 c (s) ds
t c~ (s)
Z
+ Et f~
c (s) c (s)g dq (s) :
t

Using again the martingale resultR from ch. 9.4.3 as already in R(9.65), i.e. the ex-
pression in (9.58), we replace Et t f~ c (s) c (s)g dq (s) by Et t f~
c (s) c (s)g ds;
i.e.
Z
1 c (s)
Et c ( ) c (t) = Et r (s) + (1 + ) 1 c (s) ds
t c~ (s)
Z
+ Et f~
c (s) c (s)g ds:
t

Di¤erentiating with respect to time yields

1 c( )
dEt c ( ) =d = Et r( ) + (1 + ) 1 c ( ) + Et f~
c( ) c ( )g :
c~ ( )
Let us now put into the perspective of time ; i.e. lets move time from t to and
let’s ask what expected growth of consumption is. This shift in time means formally
that our expectations operator becomes E and we obtain
dE c ( ) =d 1 c( ) c~ ( )
= E r( ) + (1 + ) 1 + E 1 :
c( ) c~ ( ) c( )
In this step we used the fact that, due to this shift in time, c ( ) is now known and
we can pull it out of the expectations operator and divide by it. Combining brackets
yields
dE c ( ) =d 1 c( ) c~ ( )
= E r( ) + (1 + ) 1 + 1
c( ) c~ ( ) c( )
1 c( ) c~ ( )
= E r( ) + (1 + ) + 1 :
c~ ( ) c( )

219
10.2 Matching on the labour market: where value functions
come from
Value functions are widely used in matching models. Examples are unemployment
with frictions models of the Mortensen-Pissarides type or shirking models of unem-
ployment (Shapiro and Stiglitz, 1984). These value functions can be understood very
easily on an intuitive level, but they really come from a maximization problem of
households. In order to understand when value functions as the ones used in the just
mentioned examples can be used (e.g. under the assumption of no saving, or being
in a steady state), we now derive value functions in a general way and then derive
special cases used in the literature.

10.2.1 A household
Let wealth a of a household evolve according to
da = fra + z cg dt:
Wealth increases per unit of time dt by the amount da which depends on current
savings ra + z c: Labour income is denoted by z which includes income w when
employed and unemployment bene…ts b when unemployed, z = w; b: Labour income
follows a stochastic Poisson di¤erential equation as there is job creation and job
destruction. In addition, we assume technological progress that implies a constant
growth rate g of labour income. Hence, we can write
dz = gzdt dqw + dqb ;
where w b: Job destruction takes place at an exogenous state-dependent
arrival rate s (z) : The corresponding Poisson process counts how often our household
moved from employment into unemployment is qw : Job creation takes place at an
exogenous rate (z) which is related to the matching function presented in (5.42).
The Poisson process related to the matching process is denoted by qb : It counts how
often a household leaves her “b-status”, i.e. how often she found a job. As an
individual cannot loose her job when she does not have one and as …nding a job
makes (in this setup) no sense for someone who has a job, both arrival rates are state
dependent. As an example, when an individual is employed, (w) = 0; when she is
unemployed, s (b) = 0:

z w b
(z) 0
s (z) s 0

Table State dependend arrival rates

R1 [ t]
Let the individual maximize expected utility Et t e u (c ( )) d ; where in-
1
stantaneous utility is of the CES type, u (c) = c 1 1 with > 0:

220
10.2.2 The Bellman equation and value functions
The state space is described by a and z: The Bellman equation has the same structure
as in (10.4). The adjustments that need to be made here follow from the fact that we
have two state variables instead of one. Hence, the basic structure from (10.4) adopted
to our problem reads V (a; z) = maxfc(t)g u (c) + dt1 Et dV (a; z) : The change of
V (a; z) is, given the evolution of a and z from above and the CVF from (9.21),
dV (a; z) = fVa [ra + z c] + Vz gzg dt
+ fV (a; z ) V (a; z)g dqw + fV (a; z + ) V (a; z)g dqb :
Forming expectations, remembering that Et dqw = s (z) dt and Et dqb = (z) dt; and
“dividing”by dt gives the Bellman equation
u (c) + Va [ra + z c] + Vz gz
V (a; z) = max :
c +s (z) [V (a; z ) V (a; z)] + (z) [V (a; z + )
V (a; z)]
(10.23)
The value functions in the literature are all special cases of this general Bellman
equation.
Denote by U V (a; b) the expected present value of (optimal behaviour of a
worker) being unemployed (as in Pissarides, 2000, ch. 1.3) and by W V (a; w) the
expected present value of being employed. As the probability of loosing a job for an
unemployed worker is zero, s (b) = 0; and (b) = ; the Bellman equation (10.23)
reads
U = max fu (c) + Ua [ra + b c] + Uz gz + [W U ]g ;
c
where we used that W = V (a; b + ) : When we assume that agents behave optimally,
i.e. we replace control variables by their optimal values, we obtain the maximized
Bellman equation,
U = u (c) + Ua [ra + b c] + Uz gb + [W U] :
When we now assume that households can not save, i.e. c = ra + b; and that there
is no technological progress, g = 0; we obtain
U = u (ra + b) + [W U] :
Assuming further that households are risk-neutral, i.e. u (c) = c; and that they have
no capital income, i.e. a = 0, consumption is identical to unemployment bene…ts
c = b: If the interest rate equals the time preference rate, we obtain eq. (1.10) in
Pissarides (2000),
rU = b + [W U ] :

10.3 Optimal saving under Brownian motion

10.3.1 The setup
Consider an individual whose budget constraint is given by
da = fra + w pcg dt + adz:

221
The notation is as always, uncertainty stems from Brownian R 1motion z: The individual
maximizes a utility function as given in (10.1), U (t) = Et t e [ t] u (c ( )) d . The
value function is de…ned by V (a) = maxfc( )g U (t) : We again follow the three step
scheme for dynamic programming.

10.3.2 Three dynamic programming steps

DP1: Bellman equation and …rst-order conditions

The Bellman equation is given for Brownian motion by (10.4) as well. When a
maximization problem other than one where (10.4) is suitable is to be formulated and
solved, …ve adjustments are in principle possible for the Bellman equation. First, the
discount factor might be given by some other factor - e.g. the interest rate r when
the present value of some …rm is maximized. Second, the number of arguments of
the value function needs to be adjusted to the number of state variables. Third, the
number of control variables depends on the problem that is to be solved and, fourth,
the instantaneous utility function is replaced by what is found in the objective function
- which might be e.g. instantaneous pro…ts. Finally and obviously, the di¤erential
dV (:) needs to be computed according to the rules that are appropriate for the
stochastic processes which drive the state variables.
As the di¤erential of the value function, following Ito’s Lemma in (9.13), is given
by
1
dV (a) = V 0 (a) [ra + w pc] + V 00 (a) 2 a2 dt + V 0 (a) adz;
2
forming expectations Et and dividing by dt yields the Bellman equation for our speci…c
problem

1 2 2
V (a) = max u (c (t)) + V 0 (a) [ra + w pc] + V 00 (a) a
c(t) 2

and the …rst-order condition is

u0 (c (t)) = V 0 (a) p (t) : (10.24)

DP2: Evolution of the costate variable

(i) The derivative of the maximized Bellman equation with respect to the state
variable is (using the envelope theorem)
1
V 0 = V 00 [ra + w pc] + V 0 r + V 000 2 a2 + V 00 2
a,
2
1 000 2 2
( r) V 0 = V 00 [ra + w pc] + V a + V 00 2 a: (10.25)
2
Not surprisingly, given that the Bellman equation already contains the second deriva-
tive of the value function, the derivative of the maximized Bellman equation contains
its third derivative V 000 .

222
 (ii) Computing the di¤erential of the shadow price of wealth V 0 (a) gives, using
Ito’s Lemma (9.13),
1 2 2
dV 0 = V 00 da + V 000 a dt
2
1
pc] dt + V 000 2 a2 dt + V 00 adz;
= V 00 [ra + w
2
and inserting into the partial derivative (10.25) of the maximized Bellman equation
yields
2
dV 0 = ( r) V 0 dt V 00 adt + V 00 adz
2
= ( r) V 0 V 00 a dt + V 00 adz: (10.26)
As always at the end of DP2, we have a di¤erential equation (or di¤erence equation in
discrete time) which determines the evolution of V 0 (a) ; the shadow price of wealth.
DP3: Inserting …rst-order conditions
Assuming that the evolution of aggregate prices is independent of the evolution of
the marginal value of wealth, we can write the …rst-order condition for consumption
in (10.24) as du0 (c) = pdV 0 + V 0 dp: This follows e.g. from Ito’s Lemma (9.16) with
0
pV 0 = 0: Using (10.26) to replace dV ; we obtain

du0 (c) = p ( r) V 0 V 00 2 a dt + V 00 adz + V 0 dp

u0 (c)
= ( r) u00 (c) c0 (a) 2 a dt + u00 (c) c0 (a) adz + u0 (c) dp=p;
p
(10.27)
where the second equality used the …rst order condition u0 (c (t)) = V 0 p (t) to re-
place V 0 and the partial derivative of this …rst-order condition with respect to assets,
u00 (c) c0 (a) = V 00 p; to replace V 00 :
When comparing this with the expression in (10.9) where uncertainty stems from
a Poisson process, we see two common features: First, both KRRs have a stochastic
term, the dz-term here and the dq-term in the Poisson case. Second, uncertainty
a¤ects the trend term for consumption in both terms. Here, this term contains the
second derivative of the instantaneous utility function and c0 (a) ; in the Poisson case,
we have the c~ terms. The additional dp-term here stems from the assumption that
prices are not constant. Such a term would also be visible in the Poisson case with
‡exible prices.

10.3.3 The solution

Just as in the Poisson case, we want a rule for the evolution of consumption here as
well. We again de…ne an inverse function and end up in the general case with
u00 u00 (c) 1 u000 (c)
dc = r + ca 2
a+ [ca a]2 dt (10.28)
u0 u0 (c) 2 u0 (c)
u00 (c) dp
c a adz + :
u0 (c) p

223
 With a CRRA utility function, we can replace the …rst, second and third derivative
of u (c) and …nd the corresponding rule

dc ca
1 hc i2
2 a
= r a+ [ + 1] a dt (10.29)
c c
2 c
ca dp
+ adz + :
c p

10.4 Capital asset pricing

Let us again consider a typical CAP problem. This follows and extends Merton (1990,
ch. 15; 1973). The presentation is in a simpli…ed way.

10.4.1 The setup

The basic structure of the setup is identical to before. There is an objective function
and a constraint. The objective function captures the preferences of our agent and
is described by a utility function as in (10.1). The constraint is given by a budget
constraint which will now be derived, following the principles of ch. 9.2.6.
Wealth a of households consist of a portfolio of assets i
N
a= i=1 pi ni ;

where the price of an asset is denoted by pi and the number of shares held by ni . The
total number of assets is given by N . Let us assume that the price of an asset follows
geometric Brownian motion,
dpi
= i dt + i dzi ; (10.30)
pi
where each price is driven by its own drift parameter i and its own variance para-
meter i : Uncertainty results from Brownian motion zi which is also speci…c for each
asset. These parameters are exogenously given to the household but would in a gen-
eral equilibrium setting be determined by properties of e.g. technologies, preferences
and other parameters of the economy.
Households can buy or sell assets by using a share i of their savings,
1 N
dni = i i=1 i ni +w c dt: (10.31)
pi
When savings are positive and a share i is used for asset i; the number of stocks held
in i increases. When savings are negative and i is positive, the number of stocks i
decreases.
The change in the households wealth is given by da = N i=1 d (pi ni ) : The wealth
held in one asset changes according to
N
d (pi ni ) = pi dni + ni dpi = i i=1 i ni +w c dt + ni dpi :

224
The …rst equality uses Ito’s Lemma from (9.16), taking into account that second
derivatives of F (:) = pi ni are zero and that dni in (10.31) is deterministic and there-
fore dpi dni = 0: Using the pricing rule (10.30) and the fact that shares add to unity,
N
i=1 i = 1, the budget constraint of a household therefore reads

N N
da = i=1 i ni +w c dt + i=1 ni pi [ i dt + i dzi ]

N i N
= i=1 ni pi + ni pi i +w c dt + i=1 ni pi i dzi
pi
N i N
= i=1 ai + i +w c dt + i=1 ai i dzi :
pi

Now de…ne i as always as the share of wealth held in asset i; i ai =a: Then, by
de…nition, a = N i=1 i a and shares add up to unity,
N
i=1 i = 1: We rewrite this for
later purposes as
N 1
N = 1 i=1 i (10.32)
De…ne further the interest rate for asset i and the interest rate of the market portfolio
by
i N
ri + i; r i=1 i ri : (10.33)
pi
This gives us the budget constraint,
N N
da = a i=1 i ri +w c dt + a i=1 i i dzi
N
= fra + w cg dt + a i=1 i i dzi : (10.34)

10.4.2 Optimal behaviour

Let us now consider an agent who behaves optimally when choosing her portfolio
and in making her consumption-saving decision. We will not go through all steps to
derive a Keynes-Ramsey rule as asset pricing requires only the Bellman equation and
…rst-order conditions.

The Bellman equation

The Bellman equation is given by (10.4), i.e. V (a) = maxc(t); i (t) u (c (t)) + dt1 Et dV (a) .
Hence, we need again the expected change of the value of one unit of wealth. With
one state variable, we simply apply Ito’s Lemma from (9.11) and …nd

1 1 1
Et dV (a) = Et V 0 (a) da + V 00 (a) [da]2 : (10.35)
dt dt 2

In a …rst step required to obtain the explicit version of the Bellman equation,
we compute the square of da: It is given, taking (9.12) into account, by [da]2 =
2
a2 N i=1 i i dzi : When we compute the square of the sum, the expression for the
product of Brownian motions in (9.15) becomes important as correlation coe¢ cients

225
need to be taken into consideration. Denoting the covariances by ij i j ij , we
get

[da]2 = a2 [ 1 1 dz1 + 2 2 dz2 + ::: + n n dzn ]

2

2 2 2 2
= a2 1 1 dt + 1 1 2 2 12 dt + ::: + 2 2 1 1 12 dt + 2 2 dt + ::: + :::
2 N N
=a j=1 i=1 i j ij dt: (10.36)

Now rewrite the sum in (10.36) as follows

N N N 1 N N
j=1 i=1 j i ij = j=1 i=1 j i ij + i=1 i N iN
N 1 N 1 N N 1
= j=1 i=1 j i ij + i=1 i N iN + j=1 N j N j
N 1 N 1 N 1 N 1 2 2
= j=1 i=1 j i ij + i=1 i N iN + j=1 N j N j + N N
N 1 N 1 2 2
= j=1 i=1 j i ij + 2 N 1
i=1 i N iN + N N

As the second term, using (10.32), can be written as N i=1

1
i N iN = 1 N 1
j=1 i
N 1
i=1 i iN ;
our (da)2 reads
n o
2 2 N 1 N 1 N 1 N 1 N 1 2 2
(da) = a j=1 i=1 j i ij + 2 1 j=1 i i=1 i iN + 1 j=1 i N dt
n o
N 1 N 1 2
= a2 j=1 i=1 j i ( ij 2 iN ) + 2 N 1
i=1 i iN + 1
N 1
i=1 i
2
N dt: (10.37)

The second preliminary step for obtaining the Bellman equation uses (10.32) again
and expresses the interest rate from (10.33) as a sum of the interest rate of asset N
(which could but does not need to be a riskless asset) and weighted excess returns
ri rN ;
N 1 N 1 N 1
r= i=1 i ri + 1 i=1 i rN = rN + i=1 i [ri rN ] : (10.38)

The Bellman equation with (10.37) and (10.38) now …nally reads

1
V (a) = max u (c) + V 0 (a) (rN + N 1
i=1 i [ri rN ])a + w c + V 00 (a) [da]2 ;
c(t); i (t) 2

where (da)2 should be thought of representing (10.37).

First-order conditions

The …rst-order conditions are the …rst-order condition for consumption,

u0 (c) = V 0 (a) ;

and the …rst-order condition for assets. The …rst-order condition for consumption has
the well-known form.

226
 To compute …rst-order conditions for shares i, we compute d f:g =d i for (10.37),
d f:g
= N 1
j=1 j ( ij 2 iN ) + N 1
i=1 i ( ij 2 iN ) +2 iN 2 1 N 1
j=1 i
2
N
d i
N 1 N 1 2
= 2 j=1 j( ij 2 iN ) + iN 1 j=1 i N
N 1 N 1 2
=2 j=1 j ( ij iN ) + 1 j=1 i iN N
N
=2 j=1 i [ ij iN ] : (10.39)

Hence, the derivative of the Bellman equation with respect to i is with (10.38) and
(10.39)
1
V 0 a[ri rN ] + V 00 2a2 N
j=1 i [ ij iN ] = 0 ()
2
V 00 N
ri rN = a j=1 i [ ij iN ] : (10.40)
V0
The interpretation of this optimality rule should take into account that we assumed
that an interior solution exists. This condition therefore says that agents are indi¤er-
ent between the current portfolio and a marginal increase in a share i if the di¤erence
in instantaneous returns, ri rN , is compensated by the covariances of of assets i and
N: Remember that from (10.33), instantaneous returns are certain at each instant.

10.4.3 Capital asset pricing

Given optimal behaviour of agents, we now derive the well-known capital asset pricing
equation. Start by assuming that asset N is riskless. This implies that it has a
variance of zero and therefore a covariance N j with any other asset of zero as well,
00
N j = 0 8j: Denote the return for asset N by r: De…ne further a VV 0 ; the
N
covariance of asset i with the market portfolio as iM j=1 j ij ; the variance of
2 N
the market portfolio as M j=1 i iM and the return of the market portfolio as
N
r r
i=1 i i as in (10.33).
We are now only few steps away from the CAP equation. Using the de…nition of
r; and iM allows to rewrite the …rst-order condition for shares (10.40) as

ri r= iM : (10.41)

Multiplying this …rst-order condition by the share i gives i [ri r] = i iM : Sum-

ming over all assets, i.e. applying N
j=1 to both sides, and using the above de…nitions
yields
r r = 2M :
Dividing this expression by version (10.41) of the …rst-order condition yields the
capital asset pricing equation,
iM
ri r= 2
(r r) :
M

2
The ratio iM = M is what is usually called the -factor.

227
10.5 Natural volatility II (*)
Before this book comes to an end, the discussion of natural volatility models in ch.
7.8 is completed in this section. We will present a simpli…ed version of those models
that appear in the literature which are presented in stochastic continuous time setups.
The usefulness of Poisson processes will become clear here. Again, more background
is available at www.wifak.uni-wuerzburg.de/vwl2/nv.

10.5.1 An RBC model

This section presents the simplest general equilibrium setup that allows to study
‡uctuations stemming from occasional jumps in technology. The basic belief is that
economically relevant changes in technologies are rare and occur every 5-8 years.
Jumps in technology means that the technological level, as captured by the TFP
level, increases. Growth cycles therefore result without any negative TFP shocks.

Technologies

The economy produces a …nal good by using a Cobb-Douglas technology

Y = AK L1 : (10.42)

Total factor productivitiy follows a geometric Poisson process with drift

dA=A = dt + dq; (10.43)

where g and b are positive constants and is the exogenous arrival rate of the Poisson
process q. We know from (9.66) in ch. 9.4.4 that the growth rate of the expected
value of A is given by + :
The …nal good can be used for consumption and investment, Y = C + I; which
implies that the prices of all these goods are identical. Choosing Y as the numeraire
good, the price is one for all these goods. Investment increases the stock of production
units K if investment is larger than depreciation, captured by a constant depreciation
rate ;
dK = (Y C K) dt: (10.44)
There are …rms who maximize instantaneous pro…ts. They do not bear any risk
and pay factors r and w; marginal productivities of capital and labour.

Households

Housholds maximize their objective function

Z 1
U (t) = Et e [ t] u (c ( )) d
t

by choosing the consumption path fc ( )g : Instantaneous utility can be speci…ed by

c1 1
u (c) = : (10.45)
1

228
Wealth of households consists in shares in …rms which are denoted by k: This wealth
changes in a deterministic way (we do not derive it here but it could be done following
the steps in ch. 9.2.6), despite the presence of TFP uncertainty. This is due to two
facts: First, wealth is measured in units of physical capital, i.e. summing k over all
households gives K: As the price of one unit of K equals the price of one unit of the
output good and the latter was chosen as numeraire, the price of one unit of wealth is
non-stochastic. This di¤ers from (9.31) where the price jumps when q jumps. Second,
a jump in q does not a¤ect k directly. This could be the case when new technologies
make part of the old capital stock obsolete. Hence, the constraint of housholds is a
budget constraint which reads
dk = (rk + w c) dt: (10.46)

Optimal behaviour

When computing optimal consumption levels, households take the capital stock
k and the TFP level A as their state variables into account. This setup is there-
fore similar to the deterministic two-state maximization problem in ch. 6.3. Going
through similar steps (concerning e.g. the substituting out of cross derivatives in step
DP2) and taking the speci…c aspects of this stochastic framework into account, yields
optimal consumption following (see exercise 8)
u00 (c) u0 (~
c) u00 (c)
dc = r + 1 dt f~
c cg dq: (10.47)
u0 (c) 0
u (c) u0 (c)
Despite the deterministic constraint (10.46) and due to TFP jumps in (10.43), con-
sumption jumps as well: a dq term shows up in this expression and marginal utility
levels before (u0 (c)) and after the jump (~
uc ), using the notation from (9.7) appear
as well. The reason is straightforward: Whenever there is a discrete increase in the
TFP level, the interest rate and wages jump. Hence, returns for savings or households
change and the household adjusts her consumption level. This is in principle iden-
tical to the behaviour in the deterministic case as illustrated in …g. 27 in ch. 5.5.3.
(Undertaking this here for this stochastic case would be very useful.)

General equilibrium

We are now in the position to start thinking about the evolution of the economy
as a whole. It is described by a system in three equations. TFP follows (10.43). The
capital stock follows
dK = AK L1 C K dt
from (10.42) and (10.44). Aggregate consumption follows
dC C C
= r 1 dt + 1 dq
C C~ C~
from (10.47) and (10.45) and from aggregating over households. This system looks
fairly familiar from deterministic models, the only substantial di¤erence lies in the dq
term and the post-jump consumption levels C: ~

229
 The simplest way to get an intuition about how the economy evolves consists in
looking at an example, i.e. by looking at a solution of the above system that holds
for a certain parameter set. We choose as example the parameter set for which the
saving rate is constant, C = (1 s) Y:
t.b.c.
... , we can draw a phase diagram and illustrate cyclical growth.

Figure 43 Cyclical growth

10.5.2 A natural volatility model

t.b.c.

10.5.3 Summary

10.6 Exercises chapter 10 and further reading

Mathematical background on dynamic programming

There are many, and in most cases much more technical, presentations of dynamic
programming in continuous time under uncertainty. A classic mathematical reference
is Gihman and Skorohod (1972(Gihman and Skorohod 1972)) and a widely used
mathematical textbook is Øksendal (1998); see also Protter (1995). These books are
probably useful only for those wishing to work on the theory of optimization and
not on applications of optimization methods. Optimization with unbounded utility
functions by dynamic programming was studied by Sennewald (2006).
With SDEs we need boundary conditions as well. In the in…nite horizon case, we
would need a transversality condition (TVC). See Smith (1996) for a discussion of
a TVC in a setup with Epstein-Zinn preferences. Sennewald (2006) has TVCs for
Poisson uncertainty.

Applications

Books in …nance that use dynamic programming methods include Du¢ e (1988,
2001) and Björk (2004). Stochastic optimization for Brownian motion is also covered
nicely by Chang (2004).
A maximization problem of the type presented in ch. 10.1 was …rst analyzed in
Wälde (1999, 2005b). This chapter combines these two papers. These two papers
were also jointly used in the Keynes-Ramsey rule appendix to Wälde (2005). It is
also used in Posch and Wälde (2006), Sennewald and Wälde (2006) and elsewhere.
Optimal control in stochastic continuous time setups is used in many applications.
Examples include issues in international macro (Obstfeld, 1994, Turnovsky, 1997,
2000), international …nance and debt crises (Stein, 2006) and many others (coming
soon). A …rm maximization problem with risk-neutrality where R&D increases quality

230
of goods, modelled by a stochastic di¤erential equation with Poisson uncertainty, is
presented and solved by Dinopoulos and Thompson (1998, sect. 2.3).
The Keynes-Ramsey rule in (10.27) was derived in a more or less complex frame-
work by Breeden (1986) in a synthesis of his consumption based capital asset pricing
model, Cox, Ingersoll and Ross (1985) in their continuous time capital asset pricing
model, or Turnovsky (2000) in his textbook. Cf. also Obstfeld (1994).

Closed form solutions

Closed-form solutions and analytical expressions for value functions are derived
by many authors. This approach was pioneered by Merton (1969, 1971) for Brownian
motion. Chang (2004) devotes an entire chapter (ch. 5) on closed-form solutions for
Brownian motion. For setups with Poisson-uncertainty, Dinopoulos and Thompson
(1998, sect. 2.3), Wälde (1999b) or Sennewald and Wälde (2006, ch. 3.4) derive
closed-form solutions.
Closed-form solutions for Levy processes are available e.g. from Aase (1984),
Framstad, Øksendal and Sulem (2001) and in the textbook by Øksendal and Sulem
(2005).

Closed-form solutions with parameter restrictions

Sometimes, restricting the parameter set of the economy in some intelligent way al-
lows to provide closed-form solutions for very general models. These solutions provide
insights which can not that easily be obtained by numerical analysis. Early examples
are Long and Plosser (1983) and Benhabib and Rustichini (1994) who obtain closed-
form solutions for a discrete-time stochastic setup. In deterministic, continuous time,
Xie (1991, 1994) and Barro, Mankiw and Sala-i-Martin (1995) use this approach as
well. Wälde (2005) and Wälde and Posch (2006) derive closed form solutions for
business cycle models with Poisson uncertainty.

Continuous time methods

There are various recent papers which use continuous time methods under un-
certainty. For examples from …nance and monetary economics, see DeMarzo and
Urošević (2006), Gabaix et al. (2006), Maenhout (2006), Piazzesi (2005), examples
from risk theory and learning include Kyle et al. (2006) and Keller and Rady (1999),
for industrial organization see e.g. Murto (2004), in spatial economics there is e.g.
Gabaix (1999), the behaviour of households in the presence of durable consumption
goods is analyzed by Bertola et al. (2005). The real options approch to investment
under uncertainty is another larger area. A recent contribution is by Guo et al.
(2005). Further references to papers that use Poisson processes can be found in ch.
9.5.

231
 Exercises Chapter 10
Applied Intertemporal Optimization
Dynamic Programming in continuous
time under uncertainty

1. Optimal saving under Poisson uncertainty with two state variables (exam)
Consider the objective function
Z 1
U (t) = Et e [ t] u (c ( )) d
t

and the budget constraint

da (t) = fra (t) + w p (t) c (t)g dt + a (t) dq (t) ;

where r and w are constant interest and wage rates, q (t) is a Poisson process
with an exogenous arrival rate and is a constant as well. Letting g and
denote constants, assume that the price p (t) of the consumption good follows

dp (t) = p (t) [gdt + dq (t)] :

(a) Derive a rule which optimally describes the evolution of consumption. De-
riving this rule in the form of marginal utility, i.e. du0 (c (t)) is su¢ cient.
(b) Derive a rule for consumption, i.e. dc (t) = :::
(c) Derive a rule for optimal consumption for = 0 or = 0:

2. Optimal saving under Brownian motion

Derive the Keynes-Ramsey rules in (10.28) and (10.29), starting from the rule
for marginal utility in (10.27).

3. Adjustment cost R1
Consider a …rm that maximizes its present value de…ned by = Et t e r[ t] ( ) d :
The …rm’s pro…t is given by the di¤erence between revenues and costs,
2
= px ci ; where output is assumed to be a function of the current cap-
ital stock, x = k : The …rm’s control variable is investment i that determines
its capital stock,
dk = (i k) dt:

232
 The …rm operates in an uncertain environment. Output prices and costs for
investment evolve according to
dp=p = p dt + p dzp ; dc=c = c dt + c dzc ;

where zp and zc are two independent stochastic processes.

(a) Assume zp and zc are two independent Brownian motions. Set p = p = 0;

such that the price p is constant. What is the optimal investment behaviour
of the …rm?
(b) Consider the alternative case where cost c are constant but prices p follow
the above SDE. How would much would the …rm now invest?
(c) Provide an answer to the question in a) when zc is a Poisson process.
(d) Provide an answer to the question in b) when zp is a Poisson process.

4. Firm speci…c technological progress (exam)

Consider a …rm facing a demand function with price elasticity "; x = p " ;
where p is the price and is a constant. Let the …rm’s technology be given
q
by x = a l where a > 1: The …rm can improve its technology by investing
into R&D. R&D is modelled by the Poisson process q which jumps with arrival
rate (lq ) where lq is employment in the research department of the …rm. The
exogenous wage rate the …rm faces amounts to w:

(a) What is the optimal static employment level l of this …rm for a given
technological level q?
(b) Formulate an intertemporal objective function given by the present value of
the …rms pro…t ‡ows over an in…nite time horizon. Continue to assume that
q is constant and let the …rm choose l optimally from this intertemporal
perspective. Does the result change with respect to (a)?
(c) Using the same objective function as in (b), let the …rm now determine
both l and lq optimally. What are the …rst order conditions? Give an
interpretation in words.
(d) Compute the expected output level for > t, given the optimal employ-
ment levels l and lq : In other words, compute Et x ( ) : Hint: Derive …rst
a stochastic di¤erential equation for output x (t) :

5. Budget constraints and optimal saving and …nance decisions (exam)

Imagine an economy with two assets, physical capital K (t) and government
bonds B (t). Let wealth a (t) of households be given by a (t) = v (t) k (t) + b (t)
where v (t) is the price of one unit of capital, k (t) is the number of stocks and
b (t) is the nominal value of government bonds held by the household. Assume
the price of stocks follows
dv (t) = v (t) dt + v (t) dq (t) ;
where and are constants and q (t) is a Poisson process with arrival rate :

233
 (a) Derive the budget constraint of the household. Use (t) v (t) k (t) =a (t)
as the share of wealth held in stocks.
(b) Derive the budget constraint of the household by assuming that q (t) is
Brownian motion.
(c) Now let the household live in a world with three assets (in addition to the
two above, there are assets available on f oreign markets). Assume that
the budget constraint of the household is given by

da (t) = fr (t) a (t) + w (t) p (t) c (t)g dt+ k k (t) a (t) dq (t)+ f f (t) a (t) dqf (t) ;

where
r (t) = k (t) rk + f (t) rf + (1 k (t) f (t)) rb
is the interest rate depending on weights i (t) and constant instantaneous
interest rates rk ; rf and rb : Let q (t) and qf (t) be two Poisson processes.
Given the usual objective function
Z 1
U (t) = Et e [ t] u (c ( )) d ;
t

what is the optimal consumption rule? What can be said about optimal
shares k and f ?

6. Capital asset pricing in ch. 10.4.3

The covariance of asset i with the market portfolio is denoted by iM ; the
variance of the market portfolio is 2M and the return of the market portfolio is
rM i i ri :

(a) Show that the covariance of asset i with the market portfolio iM is given
by Nj=1 j ij :
N 2
(b) Show that j=1 i iM is the variance of the market portfolio M.

234
7. Standard and non-standard technologies (exam)
Let the social welfare function of a central planner be given by
Z 1
U (t) = Et e [ t] u (C ( )) d :
t

(a) Consider an economy where the capital stock follows dK = AK L1 [ dt + dz]

( K + C) dt where dz is the increment of Brownian motion and and
are constants. Derive the Keynes-Ramsey rule for this economy.
(b) Assume that dY = [ dt + dz] and that is constant. What is the
expected level of Y ( ) for > t; i.e. Et Y ( )?
(c) Consider an economy where the technology is given by Y = AK L1
with dA = Adt + Adz; where z is Brownian motion. Let the capital
stock follow dK = (Y K C) dt: Derive the Keynes-Ramsey rule for
this economy as well.
(d) Is there a parameter constellation under which the Keynes-Ramsey rules
are identical?

8. Standard and non-standard technologies II (exam)

Provide answers to the same questions as in "Standard and non-standard tech-
nologies" but assume that z is a Poisson process with arrival rate : Compare
your result to (10.47).

235
11 Notation and variables, references and index
Notation and variables

The notation is as homogenous as possible. Here are the general rules but some
exceptions are possible.

Variables in capital, like capital K or consumption C denote aggregate quanti-

ties, lower-case letters pertain to the household level

A function f (:) is always presented by using parentheses (:) ; where the paren-
theses give the arguments of functions. Brackets [:] always denote a multiplica-
tion operation

A variable xt denotes the value of x in period t in a discrete time setup. A

variable x (t) denotes its value at a point in time t in a continuous time setup. We
will stick to this distinction in this textbook both in deterministic and stochastic
setups. (Note that the mathematical literature on stochastic continuous time
processes, i.e. what is treated here in part IV, uses xt to express the value of x
at a point in time t:)

A dot indicates a time derivative, x_ (t) dx=dt:

A derivative of a function f (x) where x is a scalar and not a vector is abreviated

by f 0 (x) df (x) =dx: Partial derivatives of a function f (x; y) are denoted by
e.g. fx (x; y) = fx (:) = fx @f (x; y) =@x:

Here is a list of variables and abbreviations which shows that some variables
have multi-tasking abilities, i.e. they have multiple meanings

Greek letters

output elasticity of capital

= 1= (1 + ) discount factor in discrete time
preference parameter on …rst period utility in two-period models
depreciation rate
share of wealth held in the risky asset
i share of wealth held in asset i
instantaneous pro…ts, i.e. pro…ts t in period t or (t) at point in time t
time preference rate, correlation coe¢ cient between random variables
ij correlation coe¢ cient between two random variables i and j
some future point in time, > t
i the share of savings used for buying stock i

adjustment cost function

236
 Latin letters

CVF change of variable formula

e expenditure e = pc; e¤ort, exponential function
f (x + y) a function f (:) with argument x + y
f [x + y] some variable f times x + y
q (t) Poisson process
SDE stochastic di¤erential equation
t time t standing for today (see also )
TVC transversality condition
u instantaneous utility (see also )
x …xpoint of a di¤erence or di¤erential equation (system)
z (t) Brownian motion, Wiener process

237
References
Aase, K. K. (1984): “Optimum Portfolio Diversi…cation in a General Continuous-
Time Model,”Stochastic Processes and their Applications, 18, 81–98.

Abel, A. B. (1990): “Aggregate Asset Pricing; Asset Prices under Habit Formation
and Catching up the Joneses,”American Economic Review; AEA Papers and Pro-
ceedings, 80(2), 38–42.

Aghion, P., and P. Howitt (1992): “A Model of Growth Through Creative Destruc-
tion,”Econometrica, 60, 323–351.

Araujo, A., and J. A. Scheinkman (1983): “Maximum principle and transversality con-
dition for concave in…nite horizon economic models,”Journal of Economic Theory,
30, 1 –16.

Arrow, K. J.and Kurz, M. (1970): Public investment, the rate of return and optimal
…scal policy. Johns Hopkins University Press, Baltimore.

Azariadis, C. (1993): Intertemporal Macroeconomics. Basil Blackwell, Oxford.

Barro, R. J., N. G. Mankiw, and X. Sala-i Martin (1995): “Capital Mobility in

Neoclassical Models of Growth,”American Economic Review, 85, 103–15.

Bellman, R. (1957): Dynamic Programming. Princeton University Press, Princeton.

Benhabib, J., and A. Rustichini (1994): “A note on a new class of solutions to dy-
namic programming problems arising in economic growth,” Journal of Economic
Dynamics and Control, 18, 807–813.

Bertola, G., L. Guiso, and L. Pistaferri (2005): “Uncertainty and Consumer Durables
Adjustment,”Review of Economic Studies, 72, 973U1007.½

Björk, T. (2004): Arbitrage Theory in Continuous Time, 2nd ed. Oxford University
Press.

Black, F., and M. Scholes (1973): “The Pricing of Options and Corporate Liabilites,”
Journal of Political Economy, 81, 637–659.

Blanchard, O., and S. Fischer (1989): Lectures on Macroeconomics. MIT Press.

Blanchard, O. J. (1985): “Debt, De…cits, and Finite Horizons,” Journal of Political

Economy, 93, 223–47.

Bossmann, M., C. Kleiber, and K. Wälde (2007): “Bequests, taxation and the distri-
bution of wealth in a general equilibrium model,” Forthcoming Journal of Public
Economics.

Braun, M. (1975): Di¤erential equations and their applications (short version).

Springer, Berlin.

238
Brock, W. A., and A. G. Malliaris (1989): “Di¤erential Equations, Stability and
Chaos in Dynamic Economics,”Advanced Textbooks in Economics.

Buiter, W. H. (1981): “Time Preference and International Lending and Borrowing in

an Overlappig-Generations Model,”Journal of Political Economy, 89, 769–797.

Burdett, K., and D. T. Mortensen (1998): “Wage Di¤erentials, Employer Size, and
Unemployment,”International Economic Review, 39(2), 257–73.

Chang, F.-R. (2004): Stochastic Optimization in Continuous Time. Cambridge Uni-

versity Press, Cambridge, UK.

Chen, Y.-F., and M. Funke (2005): “Non-Wage Labour Costs, Policy Uncertainty
And Labour Demand - A Theoretical Assessment,” Scottish Journal of Political
Economy, 52, 687–709.

Chiang, A. C. (1984): Fundamental Methods of Mathematical Economics. McGraw-

Hill, Third Edition, New York.

(1992): Elements of dynamic optimization. McGraw-Hill, New York.

Chow, G. C. (1997): Dynamic Economics. Optimization by the Lagrange Method.

Oxford University Press, Oxford.

Cox, J. C., J. E. Ingersoll, and S. A. Ross (1985): “An intertmporal general Equilib-
rium Model of asset prices,”Econometria, 53(2), 363–.

Danthine, Jean Pierre Donaldson, J. B., and R. Mehra (1992): “the equity premium
and the allocation of income risk,”Journal fo Economics and Control, 16, 509 532.

Das, S. R., and S. Foresi (1996): “Exact Solutions for Bond and Option Prices with
Systematic Jump Risk,”Review of Derivatives Research, 1, 7–24.

de la Croix, D., and P. Michel (2002): A theory of economic growth: dynamics and
policy in overlapping generations. Cambridge University Press.

Dierker, E., and B. Grodahl (1995): “Pro…t maximization mitigates competition,”

Economic Theory, 7, 139–160.

Dinopoulos, E., and P. Thompson (1998): “Schumpeterian Growth Without Scale

E¤ects,”Journal of Economic Growth, 3, 313–335.

Dixit, A. K. (1990, 2nd ed.): Optimization in economic theory. Oxford University

Press.

Dixit, A. K., and R. S. Pindyck (1994): Investment Under Uncertainty. Princeton

University Press.

Du¢ e, D. (1988): Security Markets - Stochastic Models. Academic Press.

239
 (2001): Dynamic Asset Pricing Theory, 3rd ed. Princeton University Press.

Eaton, J. (1981): “Fiscal Policy, In‡ation and the Accumulation of Risky Capital,”
Review of Economic Studies, 48(153), 435–445.

Epaulard, A., and A. Pommeret (2003): “Recursive Utility, Endogenous Growth, and
the Welfare Cost of Volatility,”Review of Economic Dynamics, 6(2), 672–684.

Evans, Hastings, and Peacock (2000): Statistical Distributions, 3rd edition. Wiley.

Farzin, Y. H., K. Huisman, and P. Kort (1998): “Optimal Timing of Technology

Adoption,”Journal of Economic Dynamics and Control, 22, 779–799.

Feichtinger, G., and R. F. Hartl (1986): Optimale Kontrolle Ökonomischer Prozesse:

Anwendungen des Maximumprinzips in den Wirtschaftswissenschaften. de Gruyter,
Berlin.

Fichtenholz (1997): Di¤erential und Integralrechnung, 14. Au‡age. Harri Deutsch.

Framstad, N. C., B. Øksendal, and A. Sulem (2001): “Optimal Consumption and

Portfolio in a Jump Di¤usion Market with Proportional Transaction Costs,”Jour-
nal of Mathematical Economics, 35, 233–257.

Francois, P., and H. Lloyd-Ellis (2003): “Animal Spirits Trough Creative Destruc-
tion,”American Economic Review, 93, 530–550.

Gabaix, X. (1999): “Zipf’s Law For Cities: An Explanation,”The Quarterly Journal

of Economics, 114(3), 739–767.

Gabaix, X., P. Gopikrishnan, V. Plerou, and H. E. Stanley (2006): “Institutional

Investors and Stock Market Volatility,”Quarterly Journal of Economics, 121, 461–
504.

Gabszewicz, J., and J. P. Vial (1972): “Oligopoly "à la Cournot" in a general equi-
librium analysis,”Journal of Economic Theory, 4, 381–400.

Gandolfo, G. (1996): Economic Dynamics, 3rd edition. Springer, Berlin.

García, M. A., and R. J. Griego (1994): “An Elementary Theory of Stochastic Dif-
ferential Equations Driven by A Poisson Process,” Commun. Statist. - Stochastic
Models, 10(2), 335–363.

Gihman, I. I., and A. V. Skorohod (1972): Stochastic Di¤erential Equations. Springer-

Verlag, New York.

Grinols, E. L., and S. J. Turnovsky (1998): “Consequences of debt policy in a sto-

chastically growing monetary economy,” International Economic Review, 39(2),
495–521.

240
Grossman, G. M., and E. Helpman (1991): Innovation and Growth in the Global
Economy. The MIT Press, Cambridge, Massachusetts.

Grossman, G. M., and C. Shapiro (1986): “Optimal Dynamic R&D Programs,”

RAND Journal of Economics, 17, 581–593.

Guo, X., J. Miao, and E. Morelle (2005): “Irreversible investment with regime shifts,”
Journal of Economic Theory, 122, 37U59.½

Guriev, S., and D. Kvasov (2005): “Contracting on Time,” American Economic Re-
view, 95(5), 1369–1385.

Hassett, K. A., and G. E. Metcalf (1999): “Investment with Uncertain Tax Policy:
Does Random Tax Discourage Investment?,”Economic Journal, 109, 372–393.

Heer, B., and A. Maußner (2005): Dynamic General Equilibrium Modelling, Compu-
tational Methods and Applications. Springer, Berlin.

Intriligator, M. D. (1971): Mathematical optimization and economic theory. Engle-

wood Cli¤s, New Jersey.

Jermann, U. J. (1998): “Asset pricing in production economies,”Journal of Monetary

Economics, 41, 257–275.

Johnson, N. L., S. Kotz, and N. Balakrishnan (1995): Continuous Univariate Distri-

butions, Vol. 2, 2nd edition. Wiley.

Judd, K. L. (1985): “The Law of Large Numbers with a Continuum of I.I.D. Random
Variables,”Journal of Economic Theory, 35, 19–25.

Kamien, M. I., and N. L. Schwartz (1991): Dynamic optimization. The calculus of

variations and optimal control in economics and management. 2nd edition. Ams-
terdam, North-Holland.

Kamihigashi, T. (2001): “Necessity of Transversality Conditions for In…nite Horizon

Problems,”Econometrica, 69, 995–1012.

Keller, G., and S. Rady (1999): “Optimal Experimentation in a Changing Environ-

ment,”Review of Economic Studies, 66, 475–507.

Kiyotaki, N., and R. Wright (1993): “A Search-Theoretic Approach to Monetary

Economics,”American Economic Review, 83, 63–77.

Kleiber, C., and S. Kotz (2003): Statistical Size Distributions in Economics and
Actuarial Sciences. Wiley.

Kyle, A. S., H. Ou-Yang, and W. Xiong (2006): “Prospect theory and liquidation
½
decisions,”Journal of Economic Theory, 129, 273U288.

241
Laitner, J., and D. Stolyarov (2004): “Aggregate returns to scale and embodied tech-
nical change: theory and measurement using stock market data,”Journal of Mon-
etary Economics, 51, 191–233, pdf.

Leonard, D., and N. V. Long (1992): Optimal Control Theory and Static Optimization
in Economics. Cambridge University Press, New York.

Ljungqvist, L., and T. J. Sargent (2004): Recursive Macroeconomic Theory 2nd ed.
MIT Press.

Long, J. B. Jr.., and C. I. Plosser (1983): “Real Business Cycles,”Journal of Political

Economy, 91, 39–69.

Lucas, R. E. Jr.. (1978): “Asset Prices in an Exchange Economy,”Econometrica, 46,

1429–1445.

(1988): “On the Mechanics of Economic Development,”Journal of Monetary

Economics, 22, 3–42.

Maenhout, P. J. (2006): “Robust portfolio rules and detection-error probabilities for

a mean-reverting risk premium,”Journal of Economic Theory, 128, 136U163.½

Matsuyama, K. (1999): “Growing Through Cycles,”Econometrica, 67, 335–347.

(2001): “Growing through Cycles in an In…nitely Lived Agent Economy,”

Journal of Economic Theory, 100, 220–234.

McDonald, R., and D. Siegel (1986): “The Value of Waiting to Invest,” Quarterly
Journal of Economics, 101, 707–27.

Merton, R. C. (1969): “Lifetime Portfolio Selection Under Uncertainty: The

Continuous-Time Case,”Review of Economics and Statistics, 51, 247–257.

(1971): “Optimum Consumption and Portfolio Rules in a Continuous-Time

Model,”Journal of Economic Theory, 3, 373–413.

(1976): “Option Pricing When Underlying Stock Returns are Discontinuous,”

Journal of Financial Economics, 3, 125–144.

Michel, P. (1982): “On the Transversality Condition in In…nite Horizon Optimal

Problems,”Econometrica, 50, 975–985.

Mitrinovic, D. (1970): Analytic Inequalities. Springer, Berlin.

Moscarini, G. (2005): “Job Matching and The Wage Distribution,” Econometrica,

½
73(2), 481U516.

Murto, P. (2004): “Exit in Duopoly under Uncertainty,”RAND Journal of Economics,

35, 111–127.

242
Obstfeld, M. (1994): “Risk-Taking, Global Diversi…cation, and Growth,” American
Economic Review, 84, 1310–29.

Øksendal, B. (1998): Stochastic Di¤erential Equations. Springer, Fifth Edition,

Berlin.

Øksendal, B., and A. Sulem (2005): Applied Stochastic Control of Jump Di¤usions.
Springer-Verlag, Berlin.

Palokangas, T. (2003): “Common Markets, Economic Growth, and Creative Destruc-

tion,”Journal of Economics, 10, 57–76.

(2005): “International Labor Union policy and Growth with Creative De-
struction,”Review of International Economics, 13, 90–105.

Piazzesi, M. (2005): “Bond Yields and the Federal Reserve,” Journal of Political
Economy, 113(2), 311–344.

Pissarides, Christopher, A. (2000): Equilibrium Unemployment Theory. MIT.

Pommeret, A., and W. T. Smith (2005): “Fertility, volatility, and growth,”Economics

Letters, 87(3), 347–353.

Protter, P. (1995): Stochastic Integration and Di¤erential Equations. Springer-Verlag,

Berlin.

Rebelo, S. (1991): “Long-Run Policy Analysis and Long-Run Growth,” Journal of

Political Economy, 99(3), 500–521.

Ross, S. M. (1993): Introduction to Probability Models, 5th edition. Academic Press,

San Diego.

(1996): Stochastic processes, 2nd edition. Academic Press, San Diego.

Rouwenhorst, K. G. (1995): Asset Pricing Implications fo Equilibrium Business Cycle

Models. Princeton University press, in Frontiers of Business Cycles, ed. Thomas F.
Cooley, 294-330, New Jersey.

Santanu Chatterjee, P. G., and S. J. Turnovsky (2004): “Capital Income Taxes and
Growth in a Stochastic Economy: A Numerical Analysis of the Role of Risk Aver-
sion and Intertemporal Substitution,” Journal of Public Economic Theory, 6(2),
277–310.

Schlegel, C. (2004): “Analytical and Numerical Solution of a Poisson RBC model,”

Dresden Discussion Paper Series in Economics, 5/04.

Sennewald, K. (2006): “Controlled Stochastic Di¤erential Equations under Poisson

Uncertainty and with Unbounded Utility,” Journal of Economic Dynamics and
Control, forthcoming.

243
Sennewald, K., and K. Wälde (2005): “"Ito’s Lemma" and the Bellman equation for
Poisson processes: An applied view,”Journal of Economics, forthcoming.
Shapiro, C., and J. E. Stiglitz (1984): “Equilibrium Unemployment as a Worker
Discipline Device,”American Economic Review, 74, 433–44.
Shell, K. (1969): Applications of Pontryagin’s Maximum Principle to Economics.
Springer, Berlin.
Shimer, R. (2005): “The Cyclical Behavior of Equilibrium Unemployment and Va-
cancies,”American Economic Review, 95, 25–49.
Smith, W. T. (1996): “Feasibility and Transversality Conditions for Models of Port-
folio Choice with Non-Expected Utility in Continuous Time,” Economics Letters,
53, 123–31.
Steger, T. M. (2005): “Stochastic growth under Wiener and Poisson uncertainty,”
Economics Letters, 86(3), 311–316.
Stein, J. L. (2006): Stochastic Optimal Control, International Finance, and Debt
Crises. Oxford University Press.
Stokey, N. L., R. E. Lucas Jr., and E. C. Prescott (1989): Recursive Methods in
Economics Dynamics. Harvard University Press, Cambridge, MA.
Thompson, P., and D. Waldo (1994): “Growth and trusti…ed capitalism,”Journal of
Monetary Economics, 34, 445–462.
Toche, P. (2005): “A tractable model of precautionary saving in continuous time,”
Economics Letters, 87, 267–272.
Turnovsky, S. J. (1997): International macroeconomic dynamics. MIT Press.
(2000): International macroeconomic dynamics Methods of Macroeconomic
Dynamics. MIT Press.
Turnovsky, S. J., and W. T. Smith (2006): “Equilibrium consumption and precaution-
ary savings in a stochastically growing economy,” Journal of Economic Dynamics
and Control, 30(2), 243–278.
Varian, H. (1992): Microeconomic Analysis. Norton, Third Edition.
Venegaz-Martinez, F. (2001): “Temporary Stabilization: A Stochastic Analysis,”
Journal of Economic Dynamics and Control, 25, 1429–1449.
Vissing-Jorgensen, A. (2002): “Limited Asset Market Participation and the Elasticity
of Intertemporal Substittuion,”Journal of Political Economy, 110, 825–853.
Wälde, K. (1996): “Proof of Global Stability, Transitional Dynamics, and Interna-
tional Capital Flows in a Two-Country Model of Innovation and Growth,”Journal
of Economics, 64, 53–84.

244
 (1999a): “A Model of Creative Destruction with Undiversi…able Risk and
Optimising Households,”Economic Journal, 109, C156–C171.

(1999b): “Optimal Saving under Poisson Uncertainty,”Journal of Economic

Theory, 87, 194–217.

(2002): “The Economic Determinants of Technology Shocks in a Real Busi-

ness Cycle Model,”Journal of Economic Dynamics & Control, 27, 1–28.

(2005): “Endogenous Growth Cycles,” International Economic Review, 46,

867–894.

(2005b): “Corrigendum to "Optimal Saving under Poisson Un-

certainty",” Journal of Economic Theory, forthcoming, available at
www.waelde.com/publications.

Walsh, C. E. (2003): Monetary Theory and Policy. MIT Press.

Xie, D. (1991): “Increasing Returns and Increasing Rates of Growth,” Journal of

Political Economy, 99, 429 –435.

(1994): “Divergence in Economic Performance: Transitional Dynamics with

Multiple Equilibria,”Journal of Economic Theory, 63, 97 –112.

245
Index
adjustment cost, 164, 165 Hamiltonian, present value, 95, 97
adjustment costs, 84, 163 heterogeneous agents, 134

Bellman equation, 35, 38, 40, 104, 148, identically and independently distributed
150, 152, 157, 210, 221, 222, 226 (iid), 123, 126, 128
adjusting the basic form, 222 integral representation of di¤erential equa-
basic form, 210 tions, 68, 176, 199, 202, 219
Bellman equation, maximized, 41, 105, 149, interest rate, de…nition, 18
150, 158, 221 investment, gross, 17
De…nition, 36, 211 investment, net, 17
Brownian motion, 170, 171, 173, 178, 186, Ito’s Lemma, 177, 190, 223
195, 198, 221, 237
budget constraint, 25, 31, 37, 49, 69, 74, Keynes-Ramsey rule, 106, 166
76, 86, 148, 150, 157, 191, 192, Leibniz rule, 65
209, 221, 225 level e¤ect
dynamic, 87 dynamic, 41, 88
capital asset pricing (CAP), 139, 157, 224 martingale, 198, 219
central planner, 14, 33, 50, 100, 151, 167
CES utility, 146 natural volatility, 140, 213, 228
closed-form solution, 8, 167, 213, 231 numeraire good, 7, 25, 43, 152, 156, 228
2 period model, 130, 131, 136 numerical solution, 166
Cobb-Douglas utility, 7, 131
consumption level and interest rate, 88 Poisson process, 170, 172, 180, 203, 205,
control variable, 35, 78, 95, 105, 129, 160, 209, 237
161, 221
representative agent assumption, 134
CVF (change of variable formula), 177,
resource constraint, 17
181, 185, 211, 212
risk aversion
depreciation rate, 17 absolute, 87
relative, 90
Euler’s theorem, 15, 16
expectations operator, 124, 129, 131, 150, shadow price, 14, 15, 36, 79, 80, 94, 158
160, 199, 203, 219 state variable, 35, 78, 79, 95, 107, 149,
expected value, 195, 202 160, 210, 222
sticky prices, 162
…rms, intertemporal optimization, 83, 162 switch process, 193
…rst-order condition with economic inter-
pretation, 135, 215 Wiener process, 171, 237

goods market equilibrium, 17 zero-motion line, 57, 60, 91

Hamiltonian, 76, 79, 81

Hamiltonian, current value, 84, 93, 106

246
12 Exams
sorry - all future exams deleted ... :-( ;-)

247