DP Noas
DP Noas
DP Noas
Tristany Armangue-Jubert
Now, let us name this maximized value function, which tells us the “value” of having
k0 :
X∞
V (k0 ) = max∞ β t u (F (kt , 1) + kt (1 − δ) − kt+1 )
{kt+1 }t=0
t=0
u (F (kt , 1) + kt (1 − δ) − kt+1 ) + βV (k ′ )
V (k) = max ′k
The reason this holds is because of consistency: if V (k) is the maximized value function
when the state is k, it must also be the maximized value function when the state is k ′ .
More formally, let us assume u is bounded. Then a problem of the form:
∞
X
v(x̃) = β t u(xt , xt+1 ),
t=0
1
In the context of the NGM, this is just saying that:
∞
X
β t u (F (kt , 1) + kt (1 − δ) − kt+1 ) = u(F (k0 , 1) + k0 (1 − δ) − k1 )
t=0
∞
X
+β β t u (F (kt+1 , 1) + kt+1 (1 − δ) − kt+2 )
|t=0 {z }
PV of sequence of k starting from k1
Where:
1. xt ∈ X for all t
2. Γ : X → X is a map from xt to a subset of X (feasible actions).
3. R : X × X → R is a return function that maps any two elements from X to the
real line (generally a utility function in our case)
The above problem can be restated recursively as:
Where x is the state variable and x′ is the choice variable. From the above, it follows
that the object of interest is now V . That is, we are interested in solving the above
functional equation (FE) to find the V .
Note that in the above representation, V (x) represents the value of entering a given
period with state x. Thus, solving for V is solving for the value of entering a period
with any state x ∈ X.
2
Where T V is a new function given V .
The method that we will use to solve the functional equation is to iterate on the operator.
As we will show in what follows:
1. {V n (x)} → V
2. T V = V
That is, iterating the operator will lead to convergence to a limit function V and that
V is a fixed point of operator T . Finally, we will also show that V is a solution of the
problem specified in equation 1.
The above, as related to our problem of interest, says that T has exactly one fixed point
V and that we can converge to V by iterating the operator regardless of our initial guess.
However, we require that:
(S, ρ) be a complete metric space. In our case S is a set of functions and ρ a metric
such as the supremum metric.
T must map a complete metric space to itself.
T must be a contraction mapping.
A metric space is a set S together with a metric ρ : S × S → R such that for all
x, y, z ∈ S:
ρ(x, y) ≥ 0 with equality only if x = y.
ρ(x, y) = ρ(y, x).
3
ρ(x, z) ≤ ρ(x, y) + ρ(y, z)
In what follows, we will just accept that that (S, ρ), where S is the set of continuous
and bounded functions from A to R and ρ(x, y) = supt∈A |x(t) − y(t)| (the sup-metric),
is a metric space.
Now, a complete metric space is a metric space (S, ρ) where all Cauchy sequences have
a limit in (S, ρ).
Theorem 2.2: C(X) with the sup-metric is a complete metric space
4
Figure 1: T is a contraction
Let X ⊆ Rn and let B(X) be the set of bounded functions f : X → R with the
metric ρ(x, y) = maxt∈X |f (t) − g(t)|.
Let T : B(X) → B(X) satisfy:
∀f, g ∈ B(X) such that f (x) ≤ g(x) for all x ∈ X implies:
(T f )(x) ≤ (T g)(x) ∀x ∈ X
2.2.3 Summarizing
Consider once again the DP problem:
Let us introduce some requirements that guarantee that the solution to the FE exists,
5
is unique and has some desirable properties.
R1: X is a convex subset of Rn and Γ : X → X is a non-empty, compact-valued and
continuous correspondence.
R2: R is bounded and continuous and β ∈ (0, 1). This in turn ensures the value function
V will also be bounded.
Under requirements R1 and R2:
T : C(X) → C(X)
T has exactly one fixed point V in C(X).
For any V0 ∈ C(X), ρ(T n V0 , V ) ≤ β n ρ(V0 , V ) for n = 0, 1, 2, ...
G(x) = {x′ ∈ Γ(x) : V (x) = R(x, x′ ) + βV (x′ )} is compact valued and upper
hemicontinuous.
R3: ∀x′ ∈ X, R(,̇x′ ) is strictly increasing. (R is strictly increasing in the current
state).
R4: Γ is monotone: x ≤ x′ ⇒ Γ(x) ⊆ Γ(x′ ). (Feasibility set is expanding in the current
state).
Under R1-R4 V is strictly increasing.
R5: R is strictly concave.
R6: Γ is convex.
Under R1-R2 and R5-R6, V is strictly concave and G is continuous and single-valued.
R7: R is continuously differentiable in the interior of A = {(x, x′ ) ∈ X × X : x′ ∈
Γ(x)}.
Under R1-R2 and R5-R7, for any x0 ∈ X o such that g(x0 ) ∈ Γ(x0 )o , V is continuously
differentiable at x0 .
3 Computation
Recall that we are interested in solving problems of the form:
We know that under some requirements on X, R, Γ and β, we can exploit the contraction
mapping theorem and iterate on the operator implied by the above representation to find
V ∗ , the function that satisfies the functional equation.
Before starting, let us look at a simpler application of the CMT. Suppose we are inter-
ested in finding the fixed point of f (x) = x2 on R with the Euclidean distance. Since we
6
have a contraction map on a complete metric space, we know there will be exactly one
fixed point and that we can get to it by starting from any initial guess x0 .
Computationally, this is simple:
# Dependencies
using CSV, DataFrames
-14
124
100 69
2
50
0 1 2 3 4 5 6 7 8 9 10
In the previous example, I show 4 starting guesses only to highlight the fact that the
only effect a starting guess has is on speed of convergence. Generally, since we know
that any starting guess will suffice, we will only make one.
7
3.1 Simple VFI
Let us now return to our object of interest, the functional equation. Now we have a
map from a set of functions onto itself. Note that we cannot apply our operator on a
continuous function on the computer to obtain a new function.
Consider the neoclassical growth model:
where f (k) = k α + (1 − δ)k. To solve the above, we must discretize the state space such
that we can operate on our function V .
To that end, we typically select N points around k ∗ . Naturally, the larger the N the
higher the computational burden.
Given this discrete set K, we can define the following iteration rule:
Thus, for every possible k, we check every possible k ′ and record which one attains
the maximum. After that, we check the distance between T V and V to check for
convergence.
Computationally, this may be done as:
# Dependencies
using CSV, DataFrames
# Settings
max_iter = 2000
tolerance = 0.00001
# Parameters
const beta = 0.9
const alpha = 0.75
const delta = 0.3
8
# Inner loop - state space
for (idx_k, k) in enumerate(grid_K)
# Empty array for all guesses for this k
U = zeros(length(grid_K))
# Compute deviation
dev = maximum(abs.(TV-V))
# Check convergence
if dev <= tolerance
break
end
end
9
13
Policy Function
45-degree line
12
11
k′
10
9
9 9.5 10 10.5 11 11.5 12 12.5 13
k
Figure 3: g(k)
10