Dynamic Programming

Chapter 7: Valuation

Thomas J. Sargent and John Stachurski

2024

1/78
 Topics

• The Knaster–Tarski fixed point theorem

• Du’s theorem

• Risk-sensitive preferences

• Epstein-Zin preferences

• Koopmans operators

2/78
 Nonlinear valuation

In DP, the objective to be optimized is always lifetime value

Lifetime value is an aggregation of a flow of rewards (Rt )t⩾0

Key issue: how to combine (Rt )t⩾0 into lifetime value?

What we have seen so far:

P
• Standard approach: E t⩾0 β
tR
t
P  Qt 
• State-dependent discounting: E t⩾0 i=0 βi Rt

Now we go further

3/78
In particular, we drop the linearity assumption

• lifetime value computed from a (nonlinear) recursion

• closely related to “recursive preferences”

To solve these problems we need more fixed point theory

We focus on fixed point results for order-preserving maps

• existence results
• global stability results

4/78
 Knaster–Tarski Fixed Point Theorem

We start with a simple version of the famous KT theorem, where

• X is a finite set
• I = [v1 , v2 ] is a nonempty order interval in (RX , ⩽)
• T is a self-map on I
• fix(T ) = the set of fixed points of T in I

Theorem (Knaster–Tarski). If T is order-preserving on I, then

fix(T ) is nonempty and contains least and greatest elements a ⩽ b
Moreover,
T k v1 ⩽ a ⩽ b ⩽ T k v2 for all k ⩾ 0

5/78
Ex. Sketch the one-dimensional case I = [0, 1] and convince
yourself that a fixed point must exist
The KT fixed point theorem has a huge range of applications

However, it only yields existence

• Uniqueness and stability can fail

Ex. Continuing your sketch, provide a counterexample to

uniqueness when I = [0, 1]

Next we seek additional conditions that guarantee uniqueness and

stability

6/78
 Concavity, Convexity and Stability

A self-map T on a convex subset D of Rn is called convex if

T (λu + (1 − λ)v) ⩽ λT u + (1 − λ)T v

whenever u, v ∈ D and λ ∈ [0, 1]

T is called concave if

λT u + (1 − λ)T v ⩽ T (λu + (1 − λ)v)

whenever u, v ∈ D and λ ∈ [0, 1]

• Here ⩽ is, as usual, the pointwise order

• Note that the definitions look identical to the scalar case

7/78
 Du’s Theorem

Theorem. Let
• X be a finite set
• I := [v1 , v2 ] be a nonempty order interval in (RX , ⩽)
• T be a self-map on I

If T is order-preserving, then T is globally stable on I under any

one of
1. T is concave and T v1  v1
2. T is concave and ∃ a δ > 0 such that T v1 ⩾ v1 + δ(v2 − v1 )
3. T is convex and T v2  v2
4. T is convex and ∃ a δ > 0 such that T v2 ⩽ v2 − δ(v2 − v1 )

8/78
 45

v2

Tv

v1

v
v1 v2

Figure: Concave case (one-dimensional)

9/78
 45

v2
Tv

v1

v
v1 v2

Figure: Convex case (one-dimensional)

10/78
 Power-transformed affine equations

Now let’s add more structure

• extends global stability to unbounded domains

• necessary and sufficient conditions

To begin, let X be finite and consider the vector equation

v = [h + (Av)1/θ ]θ (v ∈ V )

where

• θ is a nonzero parameter
• A ∈ L(RX )
• V = (0, ∞)X and h ∈ V

11/78
To analyze the equation we introduce the self-map

Gv = [h + (Av)1/θ ]θ (v ∈ V )

By extending Du’s theorem, we can obtain

Theorem. If A is irreducible, then the following statements are

equivalent

1. ρ(A)1/θ < 1

2. G is globally stable on V

In the case ρ(A)1/θ ⩾ 1, the map G has no fixed point in V

12/78
The one-dimensional case (so ρ(A) = A)

• globally stable iff A1/θ < 1

2
θ = −1.5 θ = −0.5
45◦ 45◦

0
0 2 0 2

2
θ = 0.5 θ = 1.5
45◦ 45◦

0
0 2 0 2

Figure: Shape properties of Gv = [h + (Av)1/θ ]θ on (0, ∞)

13/78
Example. Kleinman et al. (ECMA 2023) study a model of
migration with capital accumulation

Optimal consumption for landlords is ct = σt Rt kt where

• kt is capital and Rt is the gross rate of return

• σt is a state-dependent process obeying
h iψ
(ψ−1)/ψ −1/ψ
σt−1 = 1 + β ψ Et Rt+1 σt+1 (1)

Assume Rt = f (Xt ) where

• X is finite
• (Xt ) is P -Markov

14/78
 −1/ψ
Setting vt = σt , we can write (1) as
 h iψ 1/ψ
vt = 1 + β Et Rt+1
ψ (ψ−1)/ψ
vt+1 (2)

Define A ∈ L(RX ) by
X
(Av)(x) = β f (x′ )(ψ−1)/ψ v(x′ )P (x, x′ )
x′

Ex. Show the following are equivalent

1. ∃ a unique v ∈ V := (0, ∞)X s.t. vt = v(Xt ) solves (2)

2. ρ(A)ψ < 1

15/78
Proof: Consider a v ∈ V such that vt = v(Xt ) solves (2)

This v must satisfy

 " #ψ 1/ψ
 X 
v(x) = 1 + βψ f (x′ )(ψ−1)/ψ v(x′ )P (x, x′ )
 
x′

Equivalently,
v = [1 + (Av)ψ ]1/ψ

By the result on slide 12,

a unique solution exists in V ⇐⇒ ρ(A)ψ < 1

16/78
Motivation: Limitations of time additive preferences

Why go beyond standard specifications such as

X
v(x) = Ex β t u(Ct ) ?
t⩾0

Well documented problems

• failure of discounted expected utility model in experiments

• β constant and < 1 fails in surveys / experiments
• does not accommodate ambiguity
• cannot separate risk aversion and elasticity of intertemporal
substitution

17/78
Motivation: Limitations of time additive preferences

Why go beyond standard specifications such as

X
v(x) = Ex β t u(Ct ) ?
t⩾0

Well documented problems

• failure of discounted expected utility model in experiments

• β constant and < 1 fails in surveys / experiments
• does not accommodate ambiguity
• cannot separate risk aversion and elasticity of intertemporal
substitution

17/78
Another issue: agent is indifferent to variations in the joint
distribution of rewards that leaves marginal distributions unchanged
Example. Suppose you accept a new job

• duration is for life

• daily consumption = daily wage

Your boss offers you two options:

(A) Your boss will flip a coin on day 1 only and set

daily wage = 1{heads} × 10000 + 1{tails} × 1

(B) Your boss will flip a coin every day and set

daily wage = 1{heads} × 10000 + 1{tails} × 1

18/78
Strict preference between A and B implies choice cannot be
rationalized with time additive preferences

To see why, let φ(10000) = φ(1) = 1/2

• Option A =⇒ C1 ∼ φ and Ct = C1 for t ⩾ 2

• Option B =⇒ (Ct )t⩾1 iid

∼ φ

Either way,
X X  
β u(1) + u(10000)
E t
β u(Ct ) = β Eu(Ct ) =
t
1−β 2
t⩾1 t⩾1

19/78
 A recursive view of time additive preferences

Warm up 1: Study standard preferences from a recursive point of

view

Consider the recursion

Vt = u(Ct ) + β Et Vt+1

where

• (Vt )t⩾0 is to be determined

• Et is expectation conditional on X0 , . . . , Xt
• (Xt ) is P -Markov on X
• Ct = c(Xt ) is a consumption path and u ∈ RX

20/78
How to solve
Vt = u(Ct ) + β Et Vt+1 (3)

Since

• consumption is a function of (Xt )t⩾0

• knowledge of Xt is sufficient to forecast (Ct+j )j⩾1

we guess that Vt = v(Xt ) for some v ∈ RX

• Called a stationary Markov solution to (3)

Under this conjecture, (3) can be rewritten as

v(Xt ) = u(c(Xt )) + β Et v(Xt+1 )

21/78
Conditioning on Xt = x and setting r := u ◦ c, this becomes

v(x) = r(x) + β Ex v(Xt+1 )

X
= r(x) + β v(x′ )P (x, x′ )
x′

In vector form, v = r + βP v
From the NSL, the unique solution is
X
v ∗ = (I − βP )−1 r = βtP tr
t⩾0

As proved earlier, the r.h.s. is equal to

X
v ∗ (x) = Ex β t r(Xt ) when (Xt ) is P -Markov
t⩾0

22/78
In summary:

the recursive representation

Vt = u(Ct ) + β Et Vt+1

and the sequential representation

X
V 0 = E0 β t r(Xt )
t⩾0

specify the same lifetime value

So is the recursive formulation redundant?

No because it gives us a formula to build on!

23/78
In summary:

the recursive representation

Vt = u(Ct ) + β Et Vt+1

and the sequential representation

X
V 0 = E0 β t r(Xt )
t⩾0

specify the same lifetime value

So is the recursive formulation redundant?

No because it gives us a formula to build on!

23/78
Pursuing this idea leads to lifetime valuations without any
sequential representations

• Occurs when Vt is nonlinear in current rewards and

continuation values
• Such specifications are called recursive preferences

Remark.
The term “recursive preferences” is confusing, since traditional
time additive preferences also admit a recursive specification

Economists use the term “recursive preferences” when lifetime

utility can only be expressed recursively

We follow this convention

24/78
Warm up 1: Nonlinear “expectations”

Suppose

• Y is a random variable taking values in interval I ⊂ R

• φ : I → I is strictly increasing and E|φ(Y )| < ∞

Ex. Prove:

1. If φ is convex, then

EY ⩽ φ−1 (Eφ(Y ))
2. If φ is concave, then

φ−1 (Eφ(Y )) ⩽ EY

25/78
Proof: We prove case 1, where φ is convex

By Jensen’s inequality, we have

φ(EY ) ⩽ Eφ(Y )

Since φ is strictly increasing on R

• φ−1 exists on R and
• φ−1 is increasing on R

∴ EY ⩽ φ−1 (Eφ(Y ))

26/78
 Risk-Sensitive Preferences

Our first nonlinear example of recursive preferences

We replace the time additive recusion

X
v(x) = r(x) + β v(x′ )P (x, x′ )
x′

with ( )
1 X
v(x) = r(x) + β ln exp(θv(x′ ))P (x, x′ )
θ ′ x

• θ is a nonzero constant in R

27/78
We understand
( )
1 X
v(x) = r(x) + β ln exp(θv(x′ ))P (x, x′ )
θ ′ x

as “defining” lifetime utility under risk-sensitive preferences

• solution v ∈ RX gives lifetime value from any state

But we can’t be sure we have a definition at this point

• What if there is no v that solves this recursion?

• What if there are many?

Key task: find conditions under which a unique solution exists

28/78
As a preliminary step, let’s try to understand the nonlinear
expectation term

In general, the entropic risk-adjusted expectation of random

variable ξ is
1
Eθ [ξ] = ln E[exp(θξ)]
θ

• θ is a nonzero parameter

Ex. Prove that, if ξ is normally distributed, then

Var[ξ]
Eθ [ξ] = E[ξ] + θ
2

Notice that θ ↓ lowers appetite for risk and θ ↑ does the opposite

29/78
More generally,

For any random variable ξ taking values in X,

1. Eθ [ξ] ⩽ E[ξ] for all θ < 0
2. Eθ [ξ] ⩾ E[ξ] for all θ > 0

Moreover, these inequalities are strict if and only if Var[ξ] > 0

Proof: Apply Jensen’s inequality

(Strict case uses a strict version of Jensen’s inequality)

30/78
 Existence and uniqueness

Consider the risk-sensitive Koopmans operator Kθ : RX → RX

( )
1 X
′ ′
(Kθ v)(x) = r(x) + β ln exp(θv(x ))P (x, x )
θ ′x ∈X

Equivalent:

1. v ∈ RX solves the risk-sensitive preference specification

2. v is a fixed point of Kθ

31/78
We continue to assume that

• P ∈ M(RX ) and r ∈ RX
• θ is nonzero and β ⩾ 0

Proposition. If β < 1, then Kθ is globally stable on RX

We prove a more general result below

Implications

1. For all nonzero θ, lifetime utility is uniquely defined

2. The unique solution v ∗ can be computed by successive
approximation using Kθ

32/78
We continue to assume that

• P ∈ M(RX ) and r ∈ RX
• θ is nonzero and β ⩾ 0

Proposition. If β < 1, then Kθ is globally stable on RX

We prove a more general result below

Implications

1. For all nonzero θ, lifetime utility is uniquely defined

2. The unique solution v ∗ can be computed by successive
approximation using Kθ

32/78
Example. Suppose r(x) = x and Xt+1 = ρXt + σWt+1 where

• (Wt ) iid
∼ N (0, 1)
• |ρ| < 1

In this setting, we need to solve for v satisfying

v(x) = x + βEθ [v(ρx + σW1 )] (4)

Ex. Verify that v(x) = ax + b solves (4) when

1 β (aσ)2
a := and b := θ
1 − ρβ 1−β 2

We can also compute an approximation to v by discretizing the

state process via Tauchen’s method and iterating on Kθ
33/78
using LinearAlgebra, QuantEcon

function create_rs_utility_model(;
n=180, # size of state space
β=0.95, # time discount factor
ρ=0.96, # correlation coef in AR(1)
σ=0.1, # volatility
θ=-1.0) # risk aversion
mc = tauchen(n, ρ, σ, 0, 10) # n_std = 10
x_vals, P = mc.state_values, mc.p
r = x_vals # special case u(c(x)) = x
return (; β, θ, ρ, σ, r, x_vals, P)
end

Listing 1: Risk sensitive utility model parameters (rs_utility.jl)

34/78
 30
approximate fixed point
20 v(x) = ax + b
10

−10

−20

−30

−40

−50

−3 −2 −1 0 1 2 3
x

Figure: Approximate and true solutions in the Gaussian case

Ex. Replicate the figure

35/78
 Epstein–Zin preferences

One of the most popular recursion preferences specifications

Has been used to study

• asset pricing
• business cycles
• monetary policy
• fiscal policy
• optimal taxation
• climate policy, etc., etc.

36/78
With Epstein–Zin preferences, lifetime value obeys
n o1/α
Vt = (1 − β)Ctα + β[Et Vt+1
γ α/γ
]

where

• γ, α are nonzero parameters

• β ∈ (0, 1)

Assume

• Ct = c(Xt ) where c ∈ RX
+

• (Xt )t⩾0 is P -Markov on finite set X

37/78
We conjecture a solution of the form Vt = v(Xt )

Now condition on x to get

 " #α/γ 1/α
 X 
v(x) = (1 − β)c(x)α + β v(x′ )γ P (x, x′ )
 
x′

Equivalently, v is a fixed point of the Epstein–Zin Koopmans

operator
 " #α/γ 1/α
 X 
(Kv)(x) = (1 − β)c(x)α + β v(x′ )γ P (x, x′ )
 
x′

38/78
Let’s rewrite more generally as
 " #α/γ 1/α
 X 
(Kv)(x) = h(x) + β v(x′ )γ P (x, x′ )
 
x′

Pointwise on X this is
n o1/α
Kv = h + β[P v γ ]α/γ

• all operations are pointwise on vectors/functions

• we assume h  0 to make sure Kv is real-valued

Ex. Prove that K is a self-map on V := (0, ∞)X

39/78
 Epstein–Zin stability

Proposition. If P is irreducible and h  0, then K is globally

stable on V

A proof is provided below

40/78
Ex. Compute the fixed point of K in V starting with this code

include("s_approx.jl")
using LinearAlgebra, QuantEcon

function create_ez_utility_model(;
n=200, # size of state space
ρ=0.96, # correlation coef in AR(1)
σ=0.1, # volatility
β=0.99, # time discount factor
α=0.75, # EIS parameter
γ=-2.0) # risk aversion parameter

mc = tauchen(n, ρ, σ, 0, 5)
x_vals, P = mc.state_values, mc.p
c = exp.(x_vals)

return (; β, ρ, σ, α, γ, c, x_vals, P)
end

41/78
The figure plots every 10th iterate, repeated 100 times.

1.6
v0
1.4 v∗

1.2

1.0

0.8

0.6

0.4

0.2

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

x

Figure: Convergence of Koopmans iterates for Epstein–Zin utility

42/78
 Proof of global stability

K is difficult to work with because

• linear and nonlinear transformations are intertwined

• there are several cases for the parameters that we need to
handle

To tackle these issues we

1. introduce an operator K̂ obtained from K via a smooth

transformation
2. prove that (V̂ , K̂) and (V, K) are topologically conjugate
3. obtain conditions under which K̂ is globally stable on V

43/78
We define K̂ via
n oθ γ
K̂v = h + β(P v)1/θ where θ :=
α

The operator K̂ is simpler than K because

• one parameter θ rather than α, γ

• separates linear and nonlinear parts of the mapping

We continue to assume that P is irreducible and h  0

Ex. Prove that K̂ is a self-map on V := (0, ∞)X

44/78
Let Φ be defined by Φv = v γ

Ex. Show that

1. Φ is a homeomorphism from V to itself and

2. (V, K) and (V, K̂) are topologically conjugate under Φ

Proof: Evidently Φ is a homeomorphism from V to itself

In addition, for v ∈ V ,
n oθ n oγ/α
K̂Φv = h + β(P Φv)1/θ = h + β(P v γ )α/γ = ΦKv

Hence (V, K) and (V, K̂) are topologically conjugate, as claimed

45/78
Now set A = β θ P , so that

K̂v = [h + (Av)1/θ ]θ

By the result on slide 12,

K̂ is globally stable on V ⇐⇒ ρ(A)1/θ < 1

In our case
ρ(A)1/θ = ρ(β θ P )1/θ = β

It follows that K̂ is globally stable on V whenever β < 1

Since (V, K) and (V, K̂) are topologically conjugate, K has the
same properties on V

46/78
 A general representations

We have discussed two examples of recursive preferences

Now let’s build a general representation

While various constructions can be found in the decision theory

literature, many are challenging for applied researchers hoping to
do quantitative work

Here we give a relatively parsimonious definition based

• a Markov environment
• fixed point theory

47/78
We will build our theory from two components

1. an “aggregation function” and

2. a “certainty equivalent operator”

Below we define these objects and give examples

Then we put them together to get recursive preferences

48/78
 Certainty equivalent operators

Given V ⊂ RX , a certainty equivalent operator on V is a

self-map R on V such that

1. R is order-preserving on V and
2. R (λ1) = λ1 for all λ ∈ R with λ1 ∈ V

Example. Conditional expectations is a certainty equivalent

operator

To see this, set V = RX , fix P ∈ M(RX )

• u ⩽ v implies P u ⩽ P v, so P preserves order on V and

• P (λ1) = λP 1 = λ1

49/78
 Certainty equivalent operators

Given V ⊂ RX , a certainty equivalent operator on V is a

self-map R on V such that

1. R is order-preserving on V and
2. R (λ1) = λ1 for all λ ∈ R with λ1 ∈ V

Example. Conditional expectations is a certainty equivalent

operator

To see this, set V = RX , fix P ∈ M(RX )

• u ⩽ v implies P u ⩽ P v, so P preserves order on V and

• P (λ1) = λP 1 = λ1

49/78
Example. Let V = RX and fix

• P ∈ M(RX ) and
• nonzero θ

The operator
( )
1 X
′ ′
(Rθ v)(x) = ln exp(θv(x ))P (x, x )
θ ′ x

is called the entropic certainty equivalent operator

Ex. Prove that Rθ is a certainty equivalent operator on V

50/78
Example. Let V = (0, ∞)X and fix

• P ∈ M(RX ) and
• nonzero γ

The operator
( )1/γ
X
′ γ ′
(Rγ v)(x) = v(x ) P (x, x )
x′

is called the Kreps-Porteus certainty equivalent operator

Ex. Confirm that Rγ is a certainty equivalent operator on V

51/78
Let V = RX and fix P ∈ M(RX ) and τ ∈ [0, 1]
Let Rτ be defined by (Rτ v)(x) = Qτ v(X) where

• X ∼ P (x, ·)
• Qτ is the quantile functional

More specifically,
( )
X
(Rτ v)(x) = min y ∈ R 1{v(x ) ⩽ y}P (x, x ) ⩾ τ
′ ′

x′

Rτ is called the quantile certainty equivalent

Confirm that Rτ defines a certainty equivalent operator on V

52/78
A certainty equivalent operator R on V is called

• positive homogeneous on V if

R(λv) = λRv for all v ∈ V and λ ⩾ 0 with λv ∈ V

• superadditive on V if

R(v + w) ⩾ Rv + Rw for all v, w ∈ V with v + w ∈ V

• subadditive on V if

R(v + w) ⩽ Rv + Rw for all v, w ∈ V with v + w ∈ V

• constant-subadditive on V if

R(v + λ1) ⩽ Rv + λ1 for all v ∈ V , λ ⩾ 0 with v + λ1 ∈ V

53/78
Example. Let V = (0, ∞)X and fix P ∈ M(RX )

In this setting, the Kreps–Porteus certainty equivalent operator

( )1/γ
X
′ γ ′
(Rγ v)(x) = v(x ) P (x, x )
x′

is

1. subadditive on V when γ ⩾ 1
2. superadditive on V when γ ⩽ 1

Proof: See the book

Ex. Prove that the quantile certainty equivalent operator Rτ is

constant-subadditive

54/78
Ex. Show that the entropic certainty equivalent operator Rθ is
constant-subadditive
Proof: Fix v ∈ V , P ∈ M(RX ) and λ ∈ R+
Let X be a draw from P (x, ·)
We have
1
(Rθ (v + λ))(x) = ln {E exp[θ(v(X) + λ)]}
θ
1
= ln {E exp[θv(X)] · exp(θλ)}
θ
1
= ln {E exp[θv(X)]} + λ
θ

Hence constant-subadditivity holds

55/78
Let V be convex and let R be a positive homogeneous certainty
equivalent operator on V

Lemma
1. R is subadditive on V =⇒ R is convex on V
2. R is superadditive on V =⇒ R is concave on V

Proof: Regarding (i), fix λ ∈ [0, 1] and v, w ∈ V

Using subadditivity and positive homogeneity, we have

R(λv + (1 − λ)w) ⩽ R(λv) + R((1 − λ)w) = λRv + (1 − λ)Rw

This proves that R is convex on V . The proof of (ii) is similar.

56/78
Lemma The Kreps–Porteus certainty equivalent operator Rγ is
1. convex on V when γ ⩾ 1 and
2. concave on V when γ ⩽ 1

Ex. Check it

This result will be useful later, when we introduce ambiguity

• Convexity / concavity connect to Du’s theorem

57/78
 Monotonicity

Let X be partially ordered and let iRX be the set of increasing

functions in RX

Let V be such that iRX ⊂ V ⊂ RX and let R be a certainty

equivalent on V

We call R monotone increasing if R is invariant on iRX

Ex. Prove: If P ∈ M(RX ) is monotone increasing, then

1. the entropic certainty equivalent operator Rθ is monotone

increasing on V = RX
2. the Kreps–Porteus certainty equivalent operator Rγ is
monotone increasing on V = (0, ∞)X

58/78
 Aggregation

Koopmans operators are typically constructed by combining

1. a certainty equivalent operator and

2. an aggregation function

Given V ⊂ RX , an aggregator on V is a map A : X × R → R

such that

1. w(x) = A(x, v(x)) is in V whenever v ∈ V and

2. y 7→ A(x, y) is increasing for all x ∈ X

Intuitively, an aggregator combines current state and continuation

values to measure lifetime value

59/78
Common types of aggregators include the

• Leontief aggregator

Amin (x, y) = min{r(x), βy} with r ∈ RX and β ⩾ 0

• Uzawa aggregator

Auzawa (x, y) = r(x) + b(x)y with r ∈ RX and b ∈ RX

+

• CES aggregator

Aces (x, y) = {r(x)α + βy α }1/α with r ∈ RX

+ , β ⩾ 0, α 6= 0

An important special case of the CES & Uzawa aggregators is the

• additive aggregator

Aadd (x, y) = r(x) + βy with r ∈ RX and β ⩾ 0

60/78
Given V ⊂ RX , we call K a Koopmans operator on V if

(Kv)(x) = A(x, (Rv)(x)) (5)

for

1. some aggregator A on V and

2. a certainty equivalent operator R on V

Ex. Show that every Koopmans operator on V is an

order-preserving self-map on V

In what follows we write (5) as

K =A◦R

61/78
We have already met the following special cases

1. The risk-sensitive Koopmans operator

Kθ = Aadd ◦ Rθ

where Rθ is the entropic certainty equivalent operator

2. The Epstein–Zin Koopmans operator

K = Aces ◦ Rγ

where Rγ is the Kreps–Porteus expectations operator

62/78
Another special case is

K = Aadd ◦ P

where P ∈ M(RX ) is ordinary conditional expectations

In this case

• K is called time additive

• corresponds to traditional discounted expectations model

63/78
The lifetime value generated by Koopmans operator K on
V ⊂ RX is the unique fixed point v of K in V , whenever it exists

• v(x) = lifetime value given initial state x

Example. In the case of time additive preferences

• Kv = r + βP v with β ∈ (0, 1)
• unique fixed point is v ∗ = (I − βP )−1 r

We know that
X
v ∗ (x) = E β t r(Xt ) when (Xt ) is P -Markov and X0 = x
t⩾0

= lifetime value

64/78
Example. Let Kθ be the risk sensitive Koopmans operator

Kθ is globally stable on V = RX when β ∈ (0, 1)

• see the result on slide 32

The unique fixed point of K in V is interpreted as lifetime value

under risk-sensitive preferences

• for one instance, see the fig on slide 35

65/78
 Finite horizons
Given arbitrary Koopmans operator K, we identify K m w with total
m-period utility given terminal condition w

Example. For Kv = r + βP v, we have

X
m−1
m
K w= (βP )t r + (βP )m w
t=0

If (Xt ) is P -Markov, we can write this as

"m−1 #
X
(K m w)(x) = Ex β t r(Xt ) + β m w(Xm )
t=0

= lifetime utility up to date m

66/78
Let

• X = (X, ) be partially ordered

• iRX be the set of increasing functions in RX
• V be such that iRX ⊂ V ⊂ RX

Let K = A ◦ R be a globally stable Koopmans operator on V

Ex. Show that the unique fixed point v ∗ is increasing on X

whenever

1. A(x, v) ⩽ A(x′ , v) whenever v ∈ V and x  x′ , and

2. R is monotone increasing on V

67/78
We call aggregator A on V a Blackwell aggregator if ∃ a
β ∈ (0, 1) such that

A(x, y + λ) ⩽ A(x, y) + βλ for all x ∈ X, y ∈ R and λ ∈ R+

Ex. Show that Amin (x, y) = min{r(x), βy} is a Blackwell

aggregator when β < 1

Proof: Fixing x ∈ X, y ∈ R and λ ∈ R+ , we have

min{r(x), βy + βλ} ⩽ min{r(x) + βλ, βy + βλ}

= min{r(x), βy} + βλ

That is, A(x, y + λ) ⩽ A(x, y) + βλ

68/78
We call aggregator A on V a Blackwell aggregator if ∃ a
β ∈ (0, 1) such that

A(x, y + λ) ⩽ A(x, y) + βλ for all x ∈ X, y ∈ R and λ ∈ R+

Ex. Show that Amin (x, y) = min{r(x), βy} is a Blackwell

aggregator when β < 1

Proof: Fixing x ∈ X, y ∈ R and λ ∈ R+ , we have

min{r(x), βy + βλ} ⩽ min{r(x) + βλ, βy + βλ}

= min{r(x), βy} + βλ

That is, A(x, y + λ) ⩽ A(x, y) + βλ

68/78
Proposition. If
1. A is a Blackwell aggregator and
2. R is constant-subadditive
then K = A ◦ R is a contraction on V with respect to k · k∞

Proof: Fixing v ∈ V and λ ∈ R+ , we have

K(v + λ) = A(·, R(v + λ)) ⩽ A(·, Rv + λ)

Since A is a Blackwell aggregator, the last bound yields

K(v + λ) ⩽ A(·, Rv) + βλ = Kv + βλ

Hence K satisfies Blackwell’s condition for a contraction

69/78
Let’s now return to the global stability claim for the risk-sensitive
Koopmans operator Kθ on slide 32

Pick any nonzero θ

Recall that Kθ = Aadd ◦ Rθ when

• Aadd is the additive aggregator and

• Rθ is the entropic certainty equivalent

Note that

1. Aadd is a Blackwell aggregator and

2. Rθ is constant-subadditive

Hence Kθ is globally stable on RX

70/78
Suppose V = RX and

(Kτ v)(x) = r(x) + β(Rτ v)(x) (x ∈ X) (6)

for β ∈ (0, 1), r ∈ RX and quantile certainty equivalent Rτ

• represents quantile preferences (de Castro and Galvao 2019)

• τ parameterizes attitude to risk

Since Rτ is constant-subadditive and Aadd is Blackwell, Kτ is

globally stable

Ex. Suppose Kτ in (6) is replaced by with K = Amin ◦ Rτ . Under

analogous assumptions, prove that K is globally stable.

71/78
 Uzawa aggregation and stability

Consider the Koopmans operator K = Auzawa ◦ R

• V is some subset of RX and

• R is a certainty equivalent operator on V

Example. Suppose V = RX and R = P ∈ M(RX )

Then
Kv = r + Lv where L(x, x′ ) = b(x)P (x, x′ )

Ex. Show that K is globally stable on V whenever ρ(L) < 1

72/78
Example. In Krussell and Smith 1998, Toda 2019, Cao 2020, etc.,
∞ Y
" t #
X
v(x) = Ex βi u(Ct ) with β0 := 1 (7)
t=0 i=0

Suppose Ct = c(Xt ) and βt = b(Xt ) where

• b ⩾ 0 and
• (Xt ) is P -Markov for some P ∈ M(RX )

Set L(x, x′ ) := b(x)P (x, x′ ) and suppose ρ(L) < 1

Then v in (7) is the unique f.p. of

Kv = u ◦ c + bP v = Auzawa P v

73/78
 Stability via concavity

Now consider Kv = r + bRv when R is not in M(RX )

When does global stability hold (allowing b > 1 in some states)?

Suppose

(a) bRv ⩽ c + Lv for c ∈ RX and L ∈ L(RX ) with ρ(L) < 1

(b) r  0 and R is concave on RX+

Let V = [0, v̄] where v̄ := (I − L)−1 (r + c)

Proposition. If (a)–(b) hold, then K is globally stable on V

74/78
Proof: Under (a)–(b), K is a self-map on V

Indeed, we have

0  r = r + 0 = r + bR0 = K0

and

K v̄ = r + bRv̄ ⩽ r + c + Lv̄ = r + c − (I − L)v̄ + v̄ = v̄

In addition, K is concave and order-preserving on V

The claim now follows from Du’s theorem

75/78
 EZ preferences with state-dependent discounting

CES-Uzawa aggregator + Kreps–Porteus certainty equivalent

operator leads to
n o1/α
Kv = h + b [P v γ ]α/γ

In what follows we

• take V = (0, ∞)X

• assume h, b ∈ V and P ∈ M(RX ) is irreducible

Ex. Show that K is self-map on V

76/78
To discuss stability of K we introduce the operator A ∈ L(RX )
defined by
X γ
(Av)(x) := b(x)θ v(x′ )P (x, x′ ) where θ :=
′
α
x

Note that we can now write K as follows

n o1/α
Kv = h + [Av γ ]α/γ

Proposition. K is globally stable on V if and only if ρ(A)α/γ < 1

77/78
Proof: Let
n oθ
K̂v = h + (Av)1/θ

Let Φ be defined by Φv = v γ

Ex. Show that K̂ is globally stable on V if and only if ρ(A)α/γ < 1

• Hint: See the result on slide 12

Ex. Show that (V, K) and (V, K̂) are topologically conjugate
under Φ

The claim in the proposition follows

78/78