Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Tutorial on Control and State Constrained Optimal

Control Problems and Applications –
Part 3 : Pure State Constraints

Helmut Maurer

University of Münster
Institute of Computational and Applied Mathematics

SADCO Summer School

Imperial College, London, September 5, 2011

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Outline

1 Theory of Optimal Control Problems with Pure State

Constraints

2 Academic Example: order q = 1 of the state constraint

3 Van der Pol Oscillator : order q = 1 of the state constraint

4 Example: Immune Response

5 Example: Optimal Control of a Model of Climate Change

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Optimal Control Problem with Pure State Constraints

State x(t) ∈ Rn , Control u(t) ∈ R (for simplicity).
All functions are assumed to be sufficiently smooth.
Dynamics and Boundary Conditions
ẋ(t) = f (x(t), u(t)), t ∈ [0, tf ],
x(0) = x0 ∈ Rn , ψ(x(tf )) = 0 ∈ Rk ,
(0 = ϕ(x(0), x(tf )) , mixed boundary conditions)

Pure State Constraints and Control Bounds

s(x(t)) ≤ 0 , t ∈ [0, tf ], ( s : Rn → R )
α ≤ u(t) ≤ β , t ∈ [0, tf ].

Minimize
Z tf
J(u, x) = g (x(tf )) + f0 (x(t), u(t)) dt
0
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Hamiltonian and Minimum Principle

Hamiltonian
H(x, λ, u) = λ0 f (x, u) + λ f (x, u) λ ∈ Rn (row vector)

Let (u, x) ∈ L∞ ([0, tf ], R) × W 1,∞ ([0, tf ], Rn ) be a locally

optimal pair of functions. Then there exist
an adjoint (costate) function λ ∈ W 1,∞ ([0, tf ], Rn ) and a
scalar λ0 ≥ 0 ,
a multiplier function of bounded variation
µ ∈ BV ∞ ([0, tf ], R),
a multiplier ρ ∈ R associated to the boundary condition
ψ(x(tf )) = 0,
that satisfy the following conditions for a.a. t ∈ [0, tf ], where the
argument (t) denotes evaluations along the trajectory
(x(t), u(t), λ(t)) :

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Minimum Principle of Pontraygin et al. (Hestenes)

(i) Adjoint integral equation and transversality condition:

Rtf Rtf
λ(t) = Hx (s) ds + sx (x(s)) dµ(s)
t t
+ (λ0 g + ρψ)x (x(tf )) ( if s(x(tf )) < 0 ),

(iia) Minimum Condition for Hamiltonian:

H(x(t), λ(t), u(t)) = min { H(x(t), λ(t), u) | α ≤ u ≤ β }

(iii) positive measure dµ and complementarity condition:

Ztf
dµ(t) ≤ 0 and sx (t))dµ(t) = 0
0

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Order of a state constraint s(x(t)) ≤ 0

Define recursively functions s (k) (x, u) by

s (0) (x, u) = s(x) ,

∂s (k)
s (k+1) (x, u) = ∂x (x, u) f (x, u) , (k = 0, 1, ..)

Suppose there exist q ∈ N with

∂s (k)
∂u (x, u) ≡ 0 , i.e., s (k) = s (k) (x) , (k = 0, 1, .., q − 1),
∂s (q)
∂u (x, u) 6= 0 .

Then along a solution of ẋ(t) = f (x(t), u(t)) we have

dk
s (k) (x(t)) = dt k
s(x(t)) (k = 0, 1, .., q − 1),
dq
s (q (x(t), u(t)) = dt q s(x(t))

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Regularity conditions to ensure dµ(t) = η(t)dt

Regularity assumption on a boundary arc s(x(t)) = 0 , t1 ≤ t ≤ t2
.
∂s (q)
(x(t), u(t)) 6= 0 ∀ t1 ≤ t ≤ t2 .
∂u

Assumption on boundary control

There exists a sufficiently smooth boundary control u = ub (x))
with s(x, ub (x)) ≡ 0 .
Assumption: α < u(t) = ub (x(t)) < β ∀ t1 < t < t2 .

Regularity of multiplier (measure) µ

The regularity and assumption on boundary control imply that
there exist a smooth multiplier η(t) with
dµ(t) = η(t) dt t1 < t < t 2

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Minimum Principle under regularity

Augmented Hamiltonian

H(x, λ, η, u) = H(x, λ, u) + η · s(x)

Adjoint equation and jump conditions

λ̇(t) = −Hx (t) = −Hx (t) − η(t)sx (x(t))

λ(tk +) = λ(tk −) − νk sx (x(tk )) , νk ≥ 0

at each contact or junction time tk , νk = µ(tk +) − µ(tk −)

Minimum condition

H(x(t), λ(t), u(t)) = min { H(x(t), λ(t), u) | α ≤ u ≤ β }

Hu (t) = 0 on boundary arcs t1+ < t < t2−

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Academic Example
R2
Minimize J(x, u) = ( u 2 + x 2 ) dt
0
subject to ẋ = x 2 − u , x(0) = 1, x(2) = 1 ,
and the state constraint x(t) ≥ a , ∀ 0 ≤ t ≤ 2.

state trajectories x(t)

1
"x.dat"
0.95 "x-a=0.6.dat"
0.9 "x-a=0.7.dat"
"x-a=0.85.dat"
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0 0.5 1 1.5 2

Boundary arc x(t) ≡ a = 0.7 for t1 = 0.614 ≤ t ≤ t2 = 1.386 .

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Academic Example: state and control trajectories

state trajectories x(t)
1
"x.dat"
0.95 "x-a=0.6.dat"
0.9 "x-a=0.7.dat"
"x-a=0.85.dat"
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0 0.5 1 1.5 2

control u
2.5
"u.dat"
"u-a=0.6.dat"
2 "u-a=0.7.dat"
"u-a=0.85.dat"
1.5

0.5

-0.5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Academic Example: Computation of Multiplier η

State constraint: s(x) = a − x ≤ 0 .
Order of the state constraint is q = 1, since
s (1) (x, u) = −ẋ = −x 2 + u , (s (1) )u = 1 .
Boundary control u = ub (x) with s (1) (x, ub (x)) ≡ 0 is given by
ub (x) = x 2 = a2 .
Augmented Hamiltonian and adjoint equation:
H(x, λ, η, u) = u 2 + x 2 + λ(x 2 − u) + η(a − x) ,
λ̇ = −Hx = −2x − 2λx + η .
The minimum condition 0 = Hu = 2u − λ gives u = λ/2. Since
u = a2 holds on a boundary arc, we get λ̇ = 0 and hence the
multiplier η on the boundary
η(t) ≡ 2a(1 + 2a2 ) > 0 .
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Academic Example: state and control trajectories

state trajectories x(t)
1
"x.dat"
0.95 "x-a=0.6.dat"
0.9 "x-a=0.7.dat"
"x-a=0.85.dat"
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0 0.5 1 1.5 2

multiplier eta
5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Van der Pol Oscillator

Rtf
Minimize J(x, u) = ( u 2 + x12 + x22 ) dt (tf = 4) ,
0
subject to ẋ1 = x2 , ẋ2 = −x1 + x2 (1 − x12 ) + u ,
x1 (0) = x2 (0) = 1 , x1 (tf ) = x2 (tf ) = 0 ,
state constraint x2 (t) ≥ a ∀ 0 ≤ t ≤ tf .
state trajectories x2
1
"x2.dat"
0.8 "x2-a=-0.5.dat"
"x2-a=-0.4.dat"
0.6 "x2-a=-0.3.dat"
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
0 0.5 1 1.5 2 2.5 3 3.5 4

Boundary arc x2 (t) ≡ a = −0.4 for t1 = 0.887 ≤ t ≤ t2 = 2.62 .

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Van der Pol : Computation of Multiplier η

Order of the state constraint is q = 1, since s(x) = a − x and
s (1) (x, u) = −ẋ2 = x1 + x2 (x12 − 1) − u , (s (1) )u = −1 6= 0 .
Boundary control u = ub (x) with s (1) (x, ub (x)) ≡ 0 is given by
ub (x) = x1 + x2 (x12 − 1) = x1 + a(x12 − 1) .
Augmented Hamiltonian and adjoint equation:
H(x, λ, η, u) = u 2 + x12 + x22 + λ1 x2 + λ2 (−x1 + x2 (1 − x12 ) + u)
+η(a − x) ,
λ̇1 = −Hx1 = −2x1 + λ2 (1 + 2x1 x2 ) ,
λ̇2 = −Hx2 = −2x2 − λ1 − λ2 (1 − x12 ) + η .
The minimum condition 0 = Hu = 2u + λ2 gives u = −λ2 /2.
Hence, x1 − a + ax12 = −λ2 /2 holds on the boundary.
Differentiation yields
η = η(x, λ) = −4a2 x1 + λ1 + λ2 (1 − x12 ) .
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Van der Pol Oscillator : optimal control

state trajectories x2
1
"x2.dat"
0.8 "x2-a=-0.5.dat"
"x2-a=-0.4.dat"
0.6 "x2-a=-0.3.dat"
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
0 0.5 1 1.5 2 2.5 3 3.5 4

control u
3
2
1
0
-1
-2
-3
-4
-5
-6
0 0.5 1 1.5 2 2.5 3 3.5 4

Boundary arc x2 (t) ≡ a = −0.4 for t1 = 0.887 ≤ t ≤ t2 = 2.62 .

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Van der Pol Oscillator : multiplier η

state trajectories x2
1
"x2.dat"
0.8 "x2-a=-0.5.dat"
"x2-a=-0.4.dat"
0.6 "x2-a=-0.3.dat"
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
0 0.5 1 1.5 2 2.5 3 3.5 4

multiplier eta
16
14
12
10
8
6
4
2
0
0 0.5 1 1.5 2 2.5 3 3.5 4

Boundary arc x2 (t) ≡ a = −0.4 for t1 = 0.887 ≤ t ≤ t2 = 2.62 .

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

CASE I : Hamiltonian is regular

CASE I : control u appears ”nonlinearly” and U = R
Assume that
the Hamiltonian is regular, i.e., H(x, λ, u) admits a unique
minimum with respect to u ,
the strict Legendre condition Huu (t) > 0 holds.

Let q ≥ 1 be the order of the state constraint s(x) ≤ 0 and

consider a boundary arc with s(x(t)) = 0 ∀ t1 ≤ t ≤ t2 .
Junction conditions
q = 1 : The control u(t) and the adjoint variable λ(t) are
continuous at tk , k = 1, 2.
q = 2 : The control u(t) is continuous but the adjoint variable
may have jumps according to λ(tk +) = λ(tk −) − νk sx (tk )).
q ≥ 3 and q odd: If the control u(t) is piecewise analytic then
there are no boundary arcs but only contact points.
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Case II : control u appears linearly

Dynamics and Boundary Conditions

ẋ(t) = f1 (x(t)) + f2 (x(t)) · u(t), a.e. t ∈ [0, tf ],
x(0) = x0 ∈ Rn , ψ(x(tf )) = 0 ∈ Rk ,

Control and State Constraints

α ≤ u(t) ≤ β s(x(t) ≤ 0 ∀ t ∈ [0, tf ].

Minimize Z tf
J(u, x) = g (x(tf )) + ( f01 (x(t)) + f02 (x(t) ) · u(t) dt
0

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Case II : Hamiltonian and switching function

Normal Hamiltonian
H(x, λ, u) = f01 (x) + λf1 (x) + [ f02 (x) + λf2 (x) ] · u .

Augmented Hamiltonian H(x, λ, µ, u) = H(x, λ, u) + µ s(x)) .

Switching function
σ(x, λ) = Hu (x, λ, u) = f02 (x) + λf2 (x) , σ(t) = σ(x(t), λ(t)) .

On a boundary arc we have

s(x(t)) = 0 , t 1 ≤ t ≤ t2 ,
α < u(t) < β , t1 < t < t2 .

The minimum condition for the control implies

0 = Hu (x(t), λ(t), u(t)) = σ(t) , t1 + ≤ t ≤ t2 − .

Formally, the boundary control behaves like a singular control.

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Case II : Boundary Control and Junction Theorem

Let q be the order of the state constraint:

dq
s (q) (x, u) = q s(x) = s1 (x) + s2 (x) · u .
dt
The boundary control u = ub (x) is determined from
s (q) (x, u) = 0 as

u = ub (x) = −s1 (x)/s2 (x) .

Junction Theorem for q = 1

Let q = 1 and let a bang-bang arc be joined with a boundary
arc at t1 ∈ (0, T ).
Claim: If the control is discontinuous at t1 , then the adjoint
variable is continouus at t1 .

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Model of the immune response

Dynamic model of the immune response:

Asachenko A, Marchuk G, Mohler R, Zuev S,
Disease Dynamics, Birkhäuser, Boston, 1994.

Optimal control:
Stengel RF, Ghigliazza R, Kulkarni N, Laplace O,
Optimal control of innate immune response,
Optimal Control Applications and Methods 23, 91–104 (2002),
Lisa Poppe, Julia Meskauskas: Diploma theses, Universität
Münster (2006,2008).

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Innate Immune Response: state and control variables

State variables:
x1 (t) : concentration of pathogen
(=concentration of associated antigen)
x2 (t) : concentration of plasma cells,
which are carriers and producers of antibodies
x3 (t) : concentration of antibodies, which kill the pathogen
(=concentration of immunoglobulins)
x4 (t) : relative characteristic of a damaged organ
( 0 = healthy, 1 = dead )

Control variables:
u1 (t) : pathogen killer
u2 (t) : plasma cell enhancer
u3 (t) : antibody enhancer
u4 (t) : organ healing factor
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Generic dynamical model of the immune response

ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) .

Immune deficiency function triggered by target organ damage

 
cos(πx4 ) , 0 ≤ x4 ≤ 0.5
A(x4 ) = .
0 0.5 ≤ x4
For 0.5 ≤ x4 (t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2 (0) = 2, x3 (0) = 4/3, x4 (0) = 0
Case 1 : x1 (0) = 1.5, decay, requires no therapy (control)
Case 2 : x1 (0) = 2.0, slower decay, requires no therapy
Case 3 : x1 (0) = 3.0, diverges without control (lethal case)
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Optimal control model: cost functional

State x = (x1 , x2 , x3 , x4 ) ∈ R4 , Control u = (u1 , u2 , u3 , u4 ) ∈ R4

L2 -functional quadratic in control: Stengel et al.

Minimize J2 (x, u) = x1 (tf )2 + x4 (tf )2

Rtf
+ ( x12 + x42 + u12 + u22 + u32 + u42 ) dt
0

L1 -functional linear in control

Minimize J1 (x, u) = x1 (tf )2 + x4 (tf )2

Rtf
+ ( x12 + x42 + u1 + u2 + u3 + u4 ) dt
0
Control constraints: 0 ≤ ui (t) ≤ umax , i = 1, .., 4

Final time: tf = 10
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional, d = 0 : optimal state and control variables

State variables x1 , x2 , x3 , x4 and optimal controls u1 , u2 , u3 , u4 :

second-order sufficient conditions via matrix Riccati equation

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional, d = 0 : state constraint x4 (t) ≤ 0.2

State and control variables for state constraint x4 (t) ≤ 0.2 .

Boundary arc x4 (t) ≡ 0.2 for t1 = 0.398 ≤ t ≤ t2 = 1.35
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional, multiplier η(t) for constraint x4 (t) ≤ 0.2

Compute multiplier η as function of (x, λ) :

η(x, λ) = λ2 3π sin(πx4 )x1 x3 − λ1 + 2λ4 − 2x3 x1 + 2x4

Scaled multiplier 0.1 η(t) and boundary arc x4 (t) = 0.2

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional, delay d > 0 , constraint x4 (t) ≤ α
Dynamics with state delay d > 0

ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),

ẋ2 (t) = 3 cos(πx4 ) x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t)
x4 (t) ≤ α ≤ 0.5

Initial conditions
x1 (t) = 0 , −d ≤ t < 0, x1 (0) = 3,
x3 (t) = 4/3 , −d ≤ t ≤ 0,
x2 (0) = 2, x4 (0) = 0.

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional : delay d = 1 and x4 (t) ≤ 0.2

State variables for d = 0 and d = 1

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional : delay d = 1 and x4 (t) ≤ 0.2

Optimal controls for d = 0 and d = 1

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

2
L –functional, d = 1 : multiplier η(t) for x4 (t) ≤ 0.2

Compute multiplier η as function of (x, λ) :

η(x, y , λ) = λ2 3π sin(πx4 )y1 y3 − λ1 + 2λ4 − 2x3 x1 + 2x4

Scaled multiplier 0.1 η(t) and boundary arc x4 (t) = 0.2 ;

η(t) is discontinuous at t = d = 1

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

1
L –functional : no delays
Minimize
J1 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u1 + u2 + u3 + u4 ) dt
0

Dynamics with delay d and control constraints

ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t)x3 (t) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) ,

0 ≤ ui (t) ≤ umax , 0 ≤ t ≤ tf (i = 1, .., 4)

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

1
L –functional : umax = 2

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

1
L –functional : non-delayed, time–optimal control for
x1 (tf ) = x4 (tf ) = 0, x3 (tf ) = 4/3

umax = 1: minimal time tf = 2.2151, singular arc for u4 (t)

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Dynamical Model of Climate Change

A. Greiner, L. Grüne, and W. Semmler,

Growth and Climate Change: Threshhold and Multiple
Equilibria, Working Paper, Schwartz Center for Economic
Policy Studies, The New School, 2009, to appear.
State Variables:
K (t) : Capital (per capita)
M(t) : CO2 concentration in the atmosphere
T (t) : Temperature (Kelvin)
Control Variables
C (t) : Consumption
A(t) : Abatement per capita

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Dynamical Model of Climate Change

Production : Y = K 0.18 · D(T − To ), To = 288 (K )

Damage : D(T − To ) = (0.025 (T − To )2 + 1)−0.025
Dynamics of per-capita capital K

K̇ = Y − C − A − (δ + n)K , K (0) = K0 .
( δ = 0.075, n = 0.03 )

Emission : E = 3.5 · 10−4 · K /A

Dynamics of CO2 concentration M

Ṁ = 0.49 E − 0.1 M, M(0) = M0 .

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Dynamical Model of Climate Change (continued)

Albedo (non-reflected energy) at temperature T (Kelvin):
 
2 π(T −293)
1 − α1 (T ) = k1 π arctan 2
+ k2 ,
k1 = 5.6 · 10−3 , k2 = 0.1795 .
Radiative forcing : 5.35 ln (M) .
Outgoing radiative flux (Stefan-Boltzman-law) : ǫ σT T 4 .
Parameters : ǫ = 0.95 , σT = 5.67 · 10−8 ,
cth = 0.149707 , Q = 1367 .
Dynamics of temperature T with delay d ≥ 0
19
Ṫ (t) = cth · [ ( 1 − α1 (T (t)) · Q4 − 116 · ǫ · σT · T (t)4
+5.35 · ln (M(t − d)) ],
T (0) = T0 .
Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Optimal Control Model of Climate Change

Control constraints for 0 ≤ t ≤ tf = 200 :

0 < C (t) ≤ Cmax = 1 , 7 · 10−4 ≤ A(t) ≤ 3 · 10−3

State constraints of order 2 and 3 :

M(t) ≤ Mmax , 0 ≤ t ≤ tf = 200 ,

T (t) ≤ Tmax , te = 20 ≤ t ≤ tf = 200 .

Maximize consumption
Ztf
J(K , M, T , C , A) = e −(n−ρ)t ln (C (t)) dt
0

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

Stationary points of the canonical system

Constant Abatement A = 1.21 · 10−3 : 3 stationary points

Ts := 291.607, Ms = 2.05196, Ks = 1.44720,

Ts := 294.258, Ms = 1.90954, Ks = 1.34726,
Ts := 294.969, Ms = 2.07792, Ks = 1.46606.

Control variable abatement A(t) : Social Optimum

Ts := 288.286, Ms = 1.28500, Ks = 1.79647.

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

T (0) = 291, M(0) = 2.0, K (0) = 1.4

Concentration M , Emission E Temperature T (Kelvin)
2.5 292.2

2 292
291.8
1.5
291.6
1
291.4
0.5 291.2
0 291
0 50 100 150 200 0 50 100 150 200

Consumption C Capital K
1.6
1
1.4
0.98 1.2
0.96 1
0.94 0.8
0.6
0.92
0.4
0.9 0.2
0.88 0
0 50 100 150 200 0 50 100 150 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

T (0) = 291, M(0) = 2.0, K (0) = 1.4

T (tf ) = 291, M(tf ) = 1.8, K (tf ) = 1.4
Concentration M , Emission E Temperature T (Kelvin)
2.2 291.7
2
291.6
1.8
1.6 291.5
1.4 291.4
1.2
1 291.3
0.8 291.2
0.6
291.1
0.4
0.2 291
0 50 100 150 200 0 50 100 150 200

Consumption C Capital K
0.94 1.5
0.92 1.45
0.9 1.4
0.88 1.35
0.86
1.3
0.84
0.82 1.25
0.8 1.2
0.78 1.15
0.76 1.1
0 50 100 150 200 0 50 100 150 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

T (0) = 291, M(0) = 2.0, K (0) = 1.4

T (tf ) = 291, M(tf ) = 1.8, K (t) ≥ 1.2, 0.85 ≤ C (t) ≤ 1
Concentration M , Emission E Temperature T (Kelvin)
2.4 292.8
2.2 292.6
2 292.4
1.8
1.6 292.2
1.4 292
1.2 291.8
1 291.6
0.8
0.6 291.4
0.4 291.2
0.2 291
0 50 100 150 200 0 50 100 150 200

Consumption C Capital K
0.94 1.65
0.935 1.6
0.93
0.925 1.55
0.92 1.5
0.915
1.45
0.91
0.905 1.4
0.9 1.35
0.895
0.89 1.3
0.885 1.25
0 50 100 150 200 0 50 100 150 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

d = 10 : T (0) = 291, M(0) = 1.8, K (0) = 1.4

T (tf ) = 291, M(tf ) = 1.8, K (t) = 1.4
Concentration M , Emission E Temperature T (Kelvin)
2.2 291.8
2 291.6
1.8
291.4
1.6
1.4 291.2
1.2 291
1 290.8
0.8
290.6
0.6
0.4 290.4
0.2 290.2
0 50 100 150 200 0 50 100 150 200

Consumption C Capital K
1 1.5
1.45
0.95
1.4
0.9
1.35
0.85
1.3
0.8 1.25
0.75 1.2
0 50 100 150 200 0 50 100 150 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

T (0) = 294.5, M(0) = 2.2, K (0) = 1.4

T (tf ) = 291, M(tf ) = 1.8, K (t) = 1.4, 0.85 ≤ C (t) ≤ 1
Concentration M , Emission E Temperature T (Kelvin)
2.2 295
2 294.5
1.8
294
1.6
1.4 293.5
1.2 293
1 292.5
0.8
292
0.6
0.4 291.5
0.2 291
0 50 100 150 200 0 50 100 150 200

Consumption C Capital K
1.6
1
0.98 1.5
0.96 1.4
0.94 1.3
0.92 1.2
0.9
1.1
0.88
0.86 1
0.84 0.9
0 50 100 150 200 0 50 100 150 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

d = 10 : T (0) = 294.5, M(0) = 2.2, K (0) = 1.4

T (tf ) = 291, K (t) = 1.4, 0.85 ≤ C (t) ≤ 1
State constraint of order two : M(t) ≤ 1.9 for 30 ≤ t ≤ tf = 200
Concentration M , Emission E Temperature T (Kelvin)
2.2 295.5
2 295
1.8 294.5
1.6 294
1.4
293.5
1.2
293
1
0.8 292.5
0.6 292
0.4 291.5
0.2 291
0 50 100 150 200 0 50 100 150 200

Consumption C Capital K
1.4
1
0.98 1.35
0.96 1.3
0.94
0.92 1.25
0.9 1.2
0.88
1.15
0.86
0.84 1.1
0 50 100 150 200 0 50 100 150 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro
Theory of Optimal Control Problems with Pure State Constraints Academic Example: order q = 1 of the state constraint Va

d = 10 : T (0) = 294.5, M(0) = 2.2, K (0) = 1.4

T (tf ) = 291, K (t) = 1.4, 0.85 ≤ C (t) ≤ 1
State constraint of order two : M(t) ≤ 1.9 for 30 ≤ t ≤ tf = 200
Concentration M , Emission E adjoint variable lambdaK
2.2 1.1
2
1
1.8
1.6 0.9
1.4 0.8
1.2
1 0.7
0.8 0.6
0.6
0.5
0.4
0.2 0.4
0 50 100 150 200 0 20 40 60 80 100 120 140 160 180 200

adjoint variable lambdaM adjoint variable lambdaT

-0.05 1.2
-0.1 1
-0.15 0.8
-0.2 0.6
-0.25 0.4
-0.3 0.2
-0.35 0
-0.4 -0.2
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200

Helmut Maurer Tutorial on Control and State Constrained Optimal Control Pro