Pontryagin’s maximum principle: relevance of

some economic applications

Federico De Angelis

May 17, 2021

Contents

1 The maximum principle: history and applications 1

1.1 Dynamic models . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Continous time dynamic models . . . . . . . . . . . . . . . . . . 2
1.3 The two main results . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Optimal control theory . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4.1 Closed/open loop and stochasticity . . . . . . . . . . . . . 6
1.4.2 Mathemathic writing . . . . . . . . . . . . . . . . . . . . . 7
1.5 Optimal control theory: from the origins, to nowdays . . . . . . . 8
1.6 The birth of Calculus of variations . . . . . . . . . . . . . . . . . 10
1.7 Hamilton, Jacobi and the HJ equation . . . . . . . . . . . . . . . 11
1.8 the birth of optimal control theory . . . . . . . . . . . . . . . . . 12
1.9 Applications of control theroy in economics . . . . . . . . . . . . 12

i
Chapter 1

The maximum principle:

history and applications

In this thesis, I show the application of a theorem, known as Pontryagin’s Max-

imum Principle, to some economic problems;

1.1 Dynamic models

A dynamic model is a mathamthical model whose porpuse is to describe the
evolution variables in time. Now, these variable can take the form of any quan-
tity we are in terested in : it may be the countof red cells in the blood, the
number of the exemplars of some endangered specie in a parituclar contest or
the number of dishes a restaurant serves in an hour; t it may also be the GDP
of a country in a given year, tha bakn account of a person in a given moment....

The characteristic all vvariables, in any model, sahre, are:

1. They are real numbers, that is, they are numbers like 12.24, 381, π etc...
2. They are located some where in time.
Time also is a number, belongin to the real set. So, for exampe, the variabel x
representing GDP will be a fuccntion x : T → R of time, like any other varaible
representing any other thigs. SoA dynamic system s ocnstituted of real
functions of the real line (which constitutes time)
Now, examine real model: Suppoose we want to model the evolution in time
of the GDP of a Country. Consider the variables

Y (t) = GDP G(t) = publicexpenseC(t) = populationconsumption

(1.1)
D(t) = publicdebt; r(t) = rateof interest (1.2)
T (t) = taxes (1.3)

1
Now, we have said the variable describe the evoultions of this quantities (GDP,
interest rate...) in time. Now, we have to decide how they do this.
If a variable is a funciton of time, it will have a form like: GDP (t) = t2 ,
r(t) = t − 2. So, we either alredy know these varaibles. In these case, we are
speaking of exogenous varaibles; or, they may be to be determined, in which case
we speak of endogenous variables. In the last case, the functions that describe
the motion of the variables are unknown. How to know un unknown function?
As in highschool we learnt to solve equations, lik 2x + 3 = x2 , here we have to
solve for functions, that is, like f (x) = f (x)2 + 1. MOre precisely, in dynamic
models, we usually know
1. The form of the derivatives of our variables.
2. The value of the variables in a specific point in time.
For example, we may know that GDP in 0 equals 1000, and that its derivative,
GDP 0 , equals to publicexpenditure − interestpaid. More specifically, we are
know talking about dynamic models in continous time

1.2 Continous time dynamic models

Dynamic models in continous time are constituted by a system of differential
equations, like

ẋ1 = f1 (t, x1 , x2 , . . . , xn−1 , xn )

ẋ2 = f2 (t, x1 , x2 , . . . , xn−1 , xn )
..
. (1.4)
ẋn−1 = fn−1 (t, x1 , x2 , . . . , xn−1 , xn )
ẋn = fn (t, x1 , x2 , . . . , xn−1 , xn )

For example, going back to our dynamic model before, it could have this
form:

Ẏ (t) = 0.8G(t) − r(t)D(t) (1.5)

Ċ(t) = 0.9Y (t) + 3t − r(t) (1.6)
Ġ(t) = 0.5T (t) − 0.6Y (t)C(t) (1.7)
Ḋ(t) = G(t) − T (t) (1.8)
Ṫ (t) = 0.85D(t) − 0.6Y (t) (1.9)

Here, r(t) is inxerted as an exogenous variable.

To solve a dynamic system, one needs the theory of differential equations
(Appendix, section ref). Now, in this thesis, we are not only interested in how
some variables evolve, given their system of differential equations; we may be
interested in how the evolutions of this variables vchanges, by changing their

2
dynamics f1 , f2 , . . . , fn . To do this, we need to split the variables in two kinds:
state variables and control variables. State variables will still have the form like
in system (1.10), this time, their system of equations will change into:

ẋ1 = f1 (t, x, u)
ẋ2 = f2 (t, x, u)
..
. (1.10)
ẋn−1 = fn−1 (t, x, u)
ẋn = fn (t, x, u)

where x = (x1 , x2 . . . , xn−1 , xn ) and u = (u1 , u2 , . . . , uk−1 , uk ). What is this new

vector of variables u? They are called control variables, and they are functions
of t as well (u1 (t), u2 (t), . . . ). Their carachterestich is that they are supposed
to be decided by the modeler; that is, the model can be studied by observing
how the state varaibles (x1 , x2 , . . . ) are determined by the control varaible. Lets
make an exampl.Consider a state varaible whose derivative is

ẋ = 3t + u (1.11)

If we set u = 4t, we will obtain

ẋ = 3t + at = 7t (1.12)

The idea behind optimal control is to use the controls to drive the state
varaibles, thorught their dynamics, towards some objective. For example, sup-
pose I have, again
ẋ = 3t + u (1.13)
What if, I want to minimize the value x takes at a certain time (for examle, at
t = 10), while, at the same time, minimizing the minimum value u takes during
this “journey” from, for example, t = 0, to t = 10? An infitie tyrpe of problems
can be formulazed in this way.

In this thesis, I willl focus on the problems where the objective can be ex-
pressed as an integral, over an interval of time, of some funciton fo time, x and
u. That is, an objective function in the form
Z t1


f t, x(t), u(t) dt (1.14)
t0

In the last case, this can be seen as: minimize the integral of x over 0 - 10 with
|u| ≤ α for all t ∈ [0, 10].

1.3 The two main results

In optimal control theory, the two most knownresults for optimization problems.

3
 Integrals are the way we sum infinite quantities: for example, on a time
period 0 - 100, if my derivative is x0 (t) = t, and I start in 0 with a quantity of
100, I will get, on day 10, a quantity of:
Z 1
x(10) = 100
|{z} + 0t dt (1.15)
0
initial quantity

and we have Z 1
0t dt = 50 (1.16)
0
so, I shall have, in 10, 100 + 50 = 150.
So, te resume, you can immagine the state variable x (in the last case, my
wallet) as a quantity that changes in time; its instantaneous change is given by
its derviative, x0 (t), which tell’s us how much x is changing, given that we can
compute this integrating x0 over the time period of interest. You could think of
x(t) as a liquid, and of x0 (t) as the water flowing out of a tube, that can either
fill up the container if positive, or drown it: In the next paragraph, I give the

Figure 1.1:

general structure of an optimal control problem

1.4 Optimal control theory

Immagine a system, made of the follwing components.
1. State variables: the variables that we want to “control”.
2. Control variables: the variables which we control directy, and allow us to
drive the state variables in some specified direction.

4
 3. State equations: the equations that link the state variables and the con-
trol variables, thus determine the direction taken by the state varaibles,
once we have chosen the control variables. In a continous time1 frame-
work, these will be differential equation while, in discrete time, we’ll
havedifference equations.

4. Initial condition: the starting point, given exogenously, of the state vari-
ables.
5. Objective functional: it is a functional2 , depending on both the state and
the control variables, that we want to maximize/minimize.

The scheme below shows the relationship standing between control variables
and state variables: given the value of the state variable and the control vari-

Figure 1.2:

able at a certain time t, that s x(t) (state variable) and u(t) (control variable),
the next value of the state variable is determined by its state equation. If we
are in a continous time framework, the next value of the state variable is to be
intended as the value of x(t) at a time very near to t, that is t + δt; in a discrete
time framework, this value is what x takes one period ahead, so in t + 1. The
equations will represent the derivative of the state-variable (in continous time),
and the change (or “jump”) of the state variable in discrete time.
Control variables are used to steer state variables into some direction, with the
aim of maximizing some functional; for example, we may want x(t) reach a
certain state X, starting at X0 at time 0, as quickly as possible.

1 to see what continous time/discrete time mean, see Appendix section ??

2 It is a functional since it depends on a function, the control variable u(t)

5
1.4.1 Closed/open loop and stochasticity
A fundamental important distinction made in the context of optimal control
theroy is weather the control function, u(t), is choosen entirely at the beginning
of the process (in t0 ), or is determined step-by step as the process goes on. In
the first case, the control is said to be in open-loop form: this means that the
control is entirely predetermined for the entire period. In the second case, we
have a closed-loop control.

If we are in a deterministic context (that is, whithout stochastic events) this

makes no difference; otherwise, the closed-loop control can reach objectives that
the open-loop can’t when there is uncertainty/casuality. For example, suppose
a person has to choose, day by day, if going out for a walk, with the “objective”
of avoiding rain.
A control in open-loop form would be to decide, starting at day 0, on which days
to go out, and on which not, for the entire period. This would clearly imply
that the decision of going out or not would not depend on the conditions of the
day/recent weather forecasts, since the entire control strategy u((t) of going out
or not has already been decided at day 0 for the entire time period.
In a closed-loop stragegy, instead, the person makes the decision every day, de-
pending on the most recent weather forecasts, or maybe just looking out of the
window. It is obvious that this would allow this person to stick to its objective
(avoiding rain) much more efficently.
As shown in the figure below, an open-loop control works like this: the control
strategy u(t) is chosen for the whole time period t0 - t1 in t0 ; the state vari-
able x(t) evolves depending on its initial state x0 and its state equation.In the

Figure 1.3:

closed-loop form, instead, the control works as a feed-back from the actual state,
x(t), and the control it-self: at each time, u(t) takes the best value depending
on the value of the state variable observed at that time. This is way closed-loop
control are also called “feed-back” controls.

This was a very brief introduction to the subject of optimal control theory, in

6
 Figure 1.4:

next section, I shortly explain what a mathemathical model is; in the following
section, I report some examples of old, and recent, optimal control problems.

1.4.2 Mathemathic writing

In this thesis, every optimal control problem will have the same structure. x(t)
and u(t) will be the control variables. Since all problems will be defined in
continous time, x(t) will be linked to u(t) thought its derivative

dx
= ẋ = g(t, x, u) (1.17)
dt
The objective of the optimal control problem will be always to maximize (or
minimize) an integral like
Z t1
f (t, x, u) dt (1.18)
t0

with f (t, x, u) is a function that depends on t, x(t) and u(t), as g(t, x, u) in the
derivative of x(t) ((1.8)).

The Value Function:

Z t1

V (t, x) = max f (t, x, u) dt + φ x(t1 ) (1.19)
u∈C(t,x) t

The Pontyagin’s Maximum Principle

Pontryagin’s Maximum Principle is a theorem that gives (if some assumption
are respected) necessary conditions for a control strategy u(t) to be optimal.
Define the Hamiltonian function as

H(t, x, u, p0 , p) = p0 f (t, x, u) + pg(t, x, u) (1.20)

7
Then, if u(t) is an optimal control for problem P , there must be a continous
function p : [t0 , t1 ] → R such that:
1. u(t) ∈ arg maxH(t, x, u, p) ∀ t ∈ [t0 , t1 ].
∂
2. ṗ = − ∂x H(t, x, u, p) for a.e. t ∈ [t0 , t1 ].

3. p(t1 ) = 0.

A special case: the Calculus of Variations

The Calculus of variations can be considered the antecessor of Optimal control
theory. In general, a calculus of variation problem can be written as
Z t1
max f (t, x, ẋ) dt (1.21)
x t0
x(t0 ) = x0 (1.22)

As you can see, a calculus of variation problem is a special case of an optimal

control problem: in fact, if we substitute u to ẋ in (1.8), we obtain an optimal
control problem with dxdt = u (that is with g(t, x, u) = u). In the next section, I
start making some example of ancient problems, that could be know be written
in the form of Calculus of Variation (and therefore of optimal control) problems;
then I show, step by stape, how the Calculus of variations, in the form of (1.8)
and (1.8) was born, and how it then brought to optimal control theory.

1.5 Optimal control theory: from the origins, to

nowdays
Optimal Control theory is only the last step, of a series of events that have
seen the work of a number of great minds. But to start, lets go back to 300
B:C.: at that time, Eulcid of Alexandria was examinating a problem to find the
shortest curve which connects two points (A and B) on the plane. The solution,
now, would seem obvious: the straight line is the shortest. Anyway prooving
this mathematically is not so obvious; Euclid, in fact, did not do this, and let
others, some centuries/millenials later to find how to do it. The Pontryagin’s
amximimum principle, and the Euler equation, allow us to mathematically prove
the problem (see Appendix, section 1.8).
Some time after, in a period estimated to between 150 B.C and 100 A.C., it
was Heron of Alexandria to formulate a principle for light behaviour in space:
he asserted that a ray of light, reflected by a mirror (Figure 1.8)) toward hu-
man eye, will always choose the path which minimizes the time it takes to go
from the mirror to the eye (as he wrote in his book,Catoprics); he anticipated
a principle, known as Fermat’s principle, that will be formulated by the french
mathematician Pierre Fermat many centuries later. This problem also, can be
stated, and proved, throught the Calculus of Variations form.

8
 Figure 1.5:

Figure 1.6:

Another example of an old problem, which goes back to the Eineid, of Virgil,is
the story of Queen Dido; basing on the legend, she was the daughter of the
Phoenician king of 9th century B.C., and was forced to a long exile by her
brother, Pigmalion, who assassinated her husband; after a long journey in the
Mediterrean see, Queen Dido ended up in Tunisis. There, king Iarba allowed
her to construct her own city. The perimeter of the city would be delined by
the area she would be able to sorround the skin of a bull.
This problem is now called Isoperimetric problem; its solution is shown in
Appendix, section 1.8; it is collected in the mathemathical book of Pappus
of Alexandria, (Mathamatical Collection, Book 5), where he inserted, several
mathematical problems such as from Eulcid, Archimedes and Zeonodrus.

A lot of time later, in the 17th centurys, Pierre De Fermat addressed the prob-
lem of finding the path the lights follows whene reflected/refracted throught a

9
 Figure 1.7:

medium. He affirmed that “nature works in those ways which are easiet and
fastest”. This equation: Z x1
ds
δ =0 (1.23)
x0 v
where x0 and x1 are two point travelled by light, v is the instant veloicty and
ds the travelled spece, expresses mathamatically this finding on the behaviour
of nature; in fact, δ is the change of the integral around an optimal trajectory,
meaning that the light follows a path that minimizes time (which is given by
the integral of space over velocity).
Some time later Isaac newton, in his Philosophiae Naturalis Principia Matem-
atica studied the problem of a cone, travelling throught a medium; trying to
find the shape of the cone that would provide minimum dragging resistance.
But the first problem who really set the start, for the mathematic commu-
nity as a whole, of an era of intense and fruitous research that would have set
the promis for the brith of Calculus of variations, happened in 1696 when the
youngest member of teh most notorious mathematical family at the time (and
nowdays), Jhanne Bernoulli, posed a problem known as the Brachistochrone to
the scientific community.

1.6 The birth of Calculus of variations

The Brachistochrone problem, treated about finding the trajectory a falling
object (with no attricion) has to follow to reach, starting at point A, point B.
Actually, this question had been already posed some time before (1638) by
Galileo Galielei, who tried to solve the problem, finding out that the optimal
trajectory was a circle; he was indeet wrong, and didn’t provide a proof. The
Brachistochrone problem was solved by Bernoulli’s younger brother, Jakcob
(with whom he hade at a comptetive relationship), Leibniz, de L’Hopital, and
Newton.
As said before, this problem set for a period of intense mathemathical research;

10
 Figure 1.8:

the next big step was made by Leonard Euler, who developed a riguourous,
complete theory that would become known officially as Calculus of variations.
While his father, who was a Protestant minister, wished him to follow his path,
Euler was observed by Johann bernoulli while studying at the university of
Basel, and was taken by him as an apprentice. Under the advice of Bernoulli,
Euler started studying advanced math textbook sand, in1744, published his
cornerwork, the Methodus Inveniendi Lineas Curva [...] that is today considerd
the first formalization start of Calculus of Variations; in the book the Euler
equation made its first appearance:
d
Lẋ (t, x, ẋ) = Lx (t, x, ẋ) (1.24)
dt
A few years later, in 1755, a young teenager of Turin, Lagrange, wrote a letter
to Euler, in which he reported some of the results present in Methodus Inve-
niendi Lineas Curva [...], re-derived purely analitically, contrary to the geomet-
ric passages Euler had followed in his work. This constituted a great progress
in the formalization of Calculus of Variations; Euler did not fail to publicily
recognize to Lagrange his merit; morover, his equation become known as the
Euler-Lagrange equation.

1.7 Hamilton, Jacobi and the HJ equation

Some time later, in 1830s’, a phisician, Hamilton, while formulating what is
now known as Hamiltonian mechaniqs; he derived a system, now known as the
Hamiltonian system, which described the law of motions of particles in space;
he derived the trajectory of a moving object as a solution of a Calculus of Vari-
ations problem, which coincides with the Euler Lagrange equation.
A few years later, Jacobi, wo would become known as one of the main contribu-
tors to optimal contro theory, reviewed Hamilton’s work; he noticed that some

11
results could be better be expressed with a partial differential equation (now
known as Hamilton-Jacobi equation)

Vt + H(t, p, q) (1.25)

1.8 the birth of optimal control theory

During 1960’s the advent of computers allowed to process mathamthical oper-
ations that was not possible before; this, toghether with the post-war contest,
was the engine for the use of theory of the Calculus of Variations and differ-
netnail equaions to develop a new branch of mathamthics, kmown as Optimal
Control; nowdays, the two main approached used in the application of Optimal
Control theory to real worlds problems, are the Bellman’s dynami programming
(thorught the Hamilton-Jacobi,-bellman equations) and POntyagyn’s maximum
principle.

1.9 Applications of control theroy in economics

Optimal control theory has seen many resarhcch. IN econimcs; some examples,
eare. What I report is some application, an what I found out.
FOr example

12