EE 223 Spring 2024

Homework 8
Due by 11 p.m. on Monday, 22 April 2024.
The homework should be submitted as a scanned pdf file to ananth at berkeley
dot edu
Please retain a copy of your submitted solution for self-grading.

1. The catenary problem is the problem of finding the equilibrium shape of a

chain of length L when it is suspended from its two end points. Physics
tells us that the chain will take a shape that minimizes its potential energy
under the influence of gravity. We will assume that the two end points are at
the same height, which we denote by x̄, and, more specifically, are pinned
down at the locations (t0 , x0 ) and (t1 , x1 ) where we have t0 = 0, t1 = 1, and
x0 = x1 = x̄. Here we use the notation t to coordinatize horizontal axis and
x to coordinatize the vertical axis, in keeping with our conventions. See the
figure.

(0, x) (1, x)

x(t)

Figure 1: A chain of length L suspended under the influence of gravity. The end
points of the chain are fixed at (0, x) and (1, x). The horizontal coordinate is
denoted t and the vertical coordinate is denoted x, so the graph of the curve is
(x(t), t ∈ [0, 1]), with x(0) = x(1) = x.

Let (x(t), ∈ [0, 1]), with x(0) = x(1) = x̄ denote the shape taken by the
chain. (We assume that the chain takes the shape of a single-valued function
of t ∈ [0, 1].) Suppose the chain has mass density ρ per unit length and
the g denotes the acceleration due to gravity. The potential energy of an

1
 element of arc length at location t (which is at height x(t)) is
infinitesimal√
then ρgx(t) 1 + ( dxdt (t)) dt. By choosing a unit system where ρg = 1 we
2

can therefore pose the catenary problem as one of minimizing the integral
√
1 dx
∫0 x(t) 1 + ( dt (t)) dt,
2

subject to the constraint

√
1 dx
∫0 1+( (t))2 dt = L.
dt
(a) Write the Euler-Lagrange equations for this problem, (as discussed
in class, these give first order conditions that must be satisfied by an
optimal solution to the problem under a condition about the existence
of one-parameter families of local perturbations around the optimal
solution under consideration).
(b) Show that for any η, κ, and t̃ the curve
t − t̃
x(t) = η + κ cosh , t ∈ [0, 1], (1)
κ
solves the Euler Lagrange equations for a suitable choice of the La-
grange multiplier appearing in that equation. Then, noting that there
are three parameters in the formula in equation (1), namely η, κ, and
t̃, and that there are three conditions that need to be satisfied (namely
x(0) = x̄, x(1) = x̄, and the length of the curve being L), find the
values of the parameters of the curve in equation (1) that are relevant
for the given problem.

2. This problem discusses the brachistochrone problem. See Fig. 1 for a de-
scription of the problem.
As discussed in class, the problem can be formulated as a variational prob-
lem, namely that of minimizing the integral
√
T 1 + ( dx
dt (t))
2

∫0 √ dt
x(t)

over (x(t), t ∈ [0, T ]), subject to x(0) = 0 and x(T ) = a.

2
Figure 2: The brachistochrone problem is to find the curve from (0, 0) to (T, a)
such that a particle sliding under gravity along this curve in a frictionless way,
starting at rest at (0, 0), arrives at (T, a) in the shortest possible time. We normal-
ize the acceleration due to gravity to 21 . Even though one of the axes is labelled t in
order to conform with our general notation later in the course, t does not represent
time.

It is known that the optimal paths are given by cycloids. A cycloid is the
path traced by a fixed point on the boundary of circle of a fixed radius that
rolls forward in a frictionless way on a line. See Fig. 2.
Show that the path traced by the cycloid defined by a circle of radius r which
goes through (0, 0) can be parametrized as
t(θ) = r(θ − sin θ)
x(θ) = r(1 − cos θ) (2)

In the rest of this problem, we are going to formulate the brachistochrone

problem as an optimal control problem (in contrast to a variational prob-
lem) and (informally) derive the form of the solution by working out the
following steps

dt (t) as a control, which we will call u(t), for t ∈

(a) Let us think of dx
[0, T ]. We can then pose the brachistochrone problem as the control

3
Figure 3: In this figure we consider a circle of radius r rolling to the right without
friction. We follow the trajectory of the thick point. It starts at (0, 0). Once
the circle has rolled by an angle of θ radians the thick point is at the location
(t(θ), x(θ)). For instance, (t( π2 , x( π2 )) = (( π2 − 1)r, r), (t(π), x(π)) = (πr, 2r)
and (t(2π), x(2π)) = (2πr, 0).

problem of minimizing:
√
T 1 + (u(t))2
∫0 √ dt,
x(t)

with the dynamics being given by

dx
(t) = u(t), t ∈ [0, T ]. (3)
dt
We introduce a Lagrange multiplier function to penalize deviations
of the dynamics from the prescribed equation (3). This leads us to

4
 consider the integral:
√
T 1 + (u(t))2 dx
∫0 √ + λ(t)(u(t) − (t))dt. (4)
x(t) dt

We then consider varying (x(t), t ∈ [0, T ]) to (x(t)+∆x (t), t ∈ [0, T ]),

subject to ∆x (0) = 0 and ∆x (T ) = 0, and of varying (u(t), t ∈ [0, T ])
to (u(t) + ∆u (t), t ∈ [0, T ]). At an optimum, the corresponding varia-
tion in the integral in (4) should be zero for all infinitesimal (∆x (t), t ≥
0) of the allowed type and for all infinitesimal (∆u (t), t ≥ 0). Show
that this leads to the equations
√
dλ 1 1 + (u(t))2
(t) = .
dt 2 (x(t)) 32

and
u(t)
λ(t) = − √ √ .
x(t) 1 + (u(t))2
(b) Conclude that √
1 + (u(t))2
√ + λ(t)u(t)
x(t))
is a constant along an optimal trajectory. Notice that this is the same
as saying that
dx∗
x∗ (t)(1 + ( (t))2 ) = C, t ∈ [0, T ], (5)
dt
for some constant C, whenever (x∗ (t), t ∈ [0, T ]) is an optimal trajec-
tory. The constant must be positive, since x∗ (T ) = a > 0.
(c) Show that for each positive constant C there is a corresponding pos-
itive radius r such that the cycloid parametrized as in (2) solves the
differential equation (5)

3. Consider the linear time-varying deterministic continuous-time scalar con-

trol system
dx T −t
(t) = x(t) + u(t).
dt 2

5
over the finite time interval [0, T ]. In this problem we study the optimal
control of this system to minimize the quadratic objective
T
∫0 (x (t) + u (t)) dt + αx (T ),
2 2 2

where α ≥ 0 is fixed.

(a) Our theory tells us that the optimal cost to go at time t ∈ [0, T ] is
quadratic in the state x at time t, given by s(t)x2 , where (s(t), t ∈
[0, T ]) solves the appropriate continuous time Riccati equation with
terminal condition s(T ) = α. Consider r(t) ∶= s(T − t) and show that
it satisfies the differential equation
dr
(t) = −r2 (t) + tr(t) + 1, t ∈ [0, T ], (6)
dt
with the initial condition r(0) = α.
(b) For the special case α = 0, show that r(t) = t solves the differential
equation (6).
(c) To study (6) for general α > 0, suppose we write r(t) as t + β(t)
and define z(t) ∶= β(t)
1
. Show that (z(t), t ∈ [0, T ]) must satisfy the
differential equation
dz
(t) = tz(t) + 1, t ∈ [0, T ], (7)
dt
with the appropriate initial condition, depending on α > 0, which you
should determine.
(d) Notice that (7) defines a time-varying linear system. Hence, we can
explicitly write down the solution for any initial condition, based on
the state transition function (ϕ(t, s), 0 ≤ s ≤ t) associated to the case
without exogeneous excitation, i.e. to
dz
(t) = tz(t), t ∈ [0, T ],
dt
Work this out and thereby determine the solution of (6) from the initial
condition r(0) = α for α > 0.

6
4. Consider the nonlinear controlled deterministic dynamical system in con-
tinuous time, given by
dx
(t) = −x(t)u(t),
dt
with the initial condition x(0) = 1.
We wish to control the system to minimize the objective
1
2
∫0 (x(t)u(t)) dt + x (1).
2

(a) Write down the appropriate Hamilton-Jacobi-Bellman (HJB) equa-

tion.
(b) Solve the HJB equation you found in the preceding part of the problem
to find the optimal control.
Hint: The solution of the HJB equation will be quadratic in the state
at each time, i.e. of the form a(t)x2 when in state x at time t ∈ [0, 1].

5. (Discrete time Pontryagin minimum principle)

Consider the deterministic dynamical system over a finite horizon given by

xk+1 = fk (xk , uk ), k = 0, 1, . . . , N − 1,

where xk ∈ Rnk for 0 ≤ k ≤ N and, for 0 ≤ k ≤ N − 1, the controls are

restricted to satisfy uk ∈ Uk where Uk is a convex subset of Rmk . Any
control sequence (u0 , . . . , uN −1 ) such that uk ∈ Uk for 0 ≤ k ≤ N − 1 is
called an admissible control sequence.
The objective to minimize the overall cost
N −1
∑ ck (xk , uk ) + cN (xN ),
k=0

over all choices of admissible control sequences (u0 , . . . , uN −1 ). Here the

initial state is thought of as being fixed at x0 .
To treat the dynamics as constraints, introduce the Lagrange multiplier se-
quence (λ1 , . . . , λN ), where λk ∈ Rnk for 1 ≤ k ≤ N , and consider the
Lagrangian
N −1
∑ (ck (xk , uk ) + λTk+1 (fk (xk , uk ) − xk+1 )) + cN (xN ),
k=0

7
whose domain is thought of as being
N N −1 N
D ∶= {(x0 , . . . , xN , u0 , . . . , uN −1 , λ1 , . . . , λN ) ∈ ∏ R
nk
× ∏ Uk × ∏ Rnk }.
k=0 k=0 k=1

Define, for each 0 ≤ k ≤ N − 1, the Hamiltonian

Hk (xk , uk , λk+1 ) ∶= ck (xk , uk ) + λTk+1 fk (xk , uk ).

(a) Show that the Lagrangian optimization problem can be written as one
of minimizing
N −1
∑ (Hk (xk , uk , λk+1 ) − λTk+1 xk+1 ) + cN (xN ),
k=0

over D.
(b) By considering infinitesimal perturbations (∆xk , 1 ≤ k ≤ N ) in the
state trajectory and infinitesimal perturbations (∆uk , 0 ≤ k ≤ N − 1) in
the control trajectory (subject to the constraint that the perturbed con-
trol trajectory is admissible) argue (informally) that if (u∗0 , . . . , u∗N −1 )
is an optimal admissible control strategy and (x∗0 , x∗1 , . . . , x∗N ), with
x∗0 = x0 , is the corresponding state trajectory, then, for the sequence
(λ∗1 , . . . , λ∗N ) given by

λ∗N = ∇xN cN (x∗N ),

k+1 ∇xk fk (xk , uk ), 1 ≤ k ≤ N − 1,
λ∗k = ∇xk ck (x∗k , u∗k ) + λ∗T ∗ ∗

(where, to make sense of the formula, recall that ∇xk fk (x∗k , u∗k ) is an
nk+1 × nk matrix), we must have

∇Tuk Hk (x∗k , u∗k , λ∗k+1 )(uk − u∗k ) ≥ 0,

for all uk ∈ Uk and 0 ≤ k ≤ N − 1.

This statement can be considered a version of the discrete time Pon-
tryagin minimum principle.