Optimal Control Problems

April 7, 2006

1
1.1 Max-perimeter rectangle inscribed in an ellipse
x2 y2
Find x; y to maximize J = 4(x + y) subject to the constraint a2 + b2 = 1. Evaluate for a = 1; b = 2.

1.2 Max-volume parallelepiped inscribed in an ellipsoid

x2 y2 z2
Find x; y; z to maximize J = 8(xyz) subject to the constraint a2 + b2 + c2 = 1. Evaluate for a = 1; b =
2; c = 3.

1.3 Min-distance between two lines

A straight line in 3-D is the intersection of two planes, represented by Ax = b where A is a 2 £ 3 matrix.
Use constrained optimization with Lagrange multipliers to show that the points on the lines closest to each
other are
x1 = D1¡1 (d1 + d2 ¡ C1 d2 ); x2 = D2¡1 (d1 + d2 ¡ C2 d1 )
where,
C1 = A> > ¡1
1 (A1 A1 ) A1 ; d1 = A> > ¡1
1 (A1 A1 ) b1 ; D1 = C1 + C2 ¡ C1 C2
C2 = A2 (A2 A>
>
2 )¡1
A2 ; d2 = A2 (A2 A>
>
2 ) ¡1
b2 ; D2 = C1 + C2 ¡ C2 C1

1.4 Min-time path

A lifeguard can run at velocity v1 and swim at velocity v2 . Let y > y1 be the water and y < y1 be the land.
The lifeguard is initially at the origin. Find the min-time path to (x2 ; y2 ). Assume that the optimal path
consists of two straight lines, so the problem can be stated as
y1 sec µ1 (y2 ¡ y1 ) sec µ2
min L = +
µ1 ;µ2 v1 v2
s.t. x2 ¡ y1 tan µ1 ¡ (y2 ¡ y1 ) tan µ2 = 0
a. Use a Lagrange multiplier to show that sinv1µ1 = sinv2µ2 , which is Snell's law.
b. Snell's law and the constraint are the two nonlinear equations in the unknowns. Evaluate the solution
numerically for y1 = 100, x2 = y2 = 300, v1 = 25, v2 = 6 (min-time = 45.1, straight-line-time = 52.8).

1.5 Quadratic index with linear equality constraints

>
Find the parameter vector x to minimize J = 1
2 (x Qx + u> Ru) subject to x + Gu = c. Using Lagrange
multipliers (¸) show that the solution is
· ¸
1 @Lmin
u = kC; Lmin = c> Sc; ¡¸ = Sc =
2 @c

1
where,
K = (R + G> QG)¡1 G> Q = R¡1 G> S
S = Q ¡ QG(R + G> QG)¡1 G> Q = (Q¡1 + GR¡1 G> )¡1
The alternative expressions are valid when R; Q are invertible and are a special case of the matrix inversion
lemma.

1.6 Max singular values of a matrix

Find the maximum and minimum values of the magnitude of the vector y = Ax subject to the constraint
that x has unit magnitude.

1.7 Min distance between ellipses

Develop an algorithm to ¯nd the minimum distance between two ellipses (x ¡ c1 )> P1 (x ¡ c1 ) = 1 and
(x ¡ c2 )> P2 (x ¡ c2 ) = 1. Evaluate for c1 = 0, P1 = I, c2 = [10; 10], P2 = diag(1; 5).

2
2.1 The brachistochrone problem
A bead slides on a wire without friction in a gravitational ¯eld. Find the shape of the wire to minimize the
time it takes to cover a given horizontal distance xf .
Posed and solved by Jakob Bernoulli in the 17th century.

2.2 *Zermelo's problem

A ship travels with constant velocity V with respect to the water through a region where the velocity of the
current is parallel to the x-axis and varies with y so that the ship movement satis¯es
x_ = V cos µ + uc (y); y_ = V sin µ
where µ is the ship's heading relative to the x-axis. Find µ(t) to minimize the time to move from the origin
to a given ¯nal position (xf ; yf ) (if feasible).

2.3 Leaky reservoir

The di®erential equation that describes a leaky reservoir is
x_ = ¡0:1x + u
where x is the height of water and u is the net in°ow rate of water. Assume that 0 · u(t) · M for all t.
1. Find the optimal control minimizing Z 100
J= ¡x(t)dt
0

2. Repeat part 1 with the additional constraint

Z 100
u(t)dt = K( a known constant)
0

3. Find the optimal control minimizing J = ¡x(100) subject to

Z 100
u(t)dt = K
0

2
2.4 Shape of a hanging chain
Given a chain of length 2L hanging from two points at the same level and at distance 2` apart, ¯nd the shape
of a hanging chain y(x) that minimizes the potential energy in the earth's gravitational ¯eld. That is, ¯nd
R` R`
u to minimize J = ¡g¾ ¡` yds subject to ¡` ds = 2L, y(¡`) = y(`) = 0, g is the gravitational constant, ¾
p
is the mass per unit length, ds = 1 + u2 dx and dy=dx = u. Show that the curve is the catenary
µ ¶
x `
y = H cosh ¡ cosh
H H
where H is determined by L=H = sinh `=H.
Posed by Galileo in 1638, solved by Jakob and Johannes Bernoulli and Leibnitz in 1690-1692.

2.5 Min surface area of soap ¯lm

Find the shape of a soap ¯lm stretched between two coaxial loops, of radius a and at distance 2` apart. The
surface is such that it has minimum area and, because of the problem symmetry, it is given by
Z ` p
A = 2¼ r 1 + u2 dx
¡`
dr
where, (r; x) are the cylindrical coordinates, dx = u, r(¡`) = r(`) = a.
x
The surface is the catenary r = H cosh H , where `=H is determined by a`(`=H) = cosh `=H. This
equation has two solutions for 0 · `=a · 0:663 and no solution for `=a > 0:663.
The minimum area is
½
2¼a2 [tanh(`=H) + (`=H)sech2 (`=H) if 0 · `=a · 0:528
Amin =
2¼a2 if `=a ¸ 0:528

3
3.1
For the optimal control problem
Z
1 1 2
min J(x; u) = (2x1 + x22 + u2 ) dt
x;u 2 0
s.t. x_ 1 = x2
x_ 2 = ¡x1 + [1 ¡ x21 ]x2 + u
with x(0) = x0 , x(tf ) = xf speci¯ed, determine
1. the costate equations;
2. the optimal control (minimizing the Hamiltonian) when u is not constrained.

3.2
The system
x_ = ¡x + u
is to be transferred to the origin from an arbitrary initial state in 1 unit of time. Determine the control that
achieves this objective and minimizes
Z
1 1 2
min J(x; u) = (3x + u2 ) dt
x;u 2 0
There are no constraints on u.

3
3.3 LQR Stabilization
Consider the LQR optimal control problem
Z
1 1 >
min J = (x Qx + u> Ru)dt
2 0
s.t. x_ = Ax + Bu

where R = R> > 0, Q = C > C, (A; B; C) minimal (c.c. and c.o.). The optimal control is u = Kx where
K = ¡R¡1 B > P is de¯ned in terms of the Riccati solution P .
1. Show that this controller stabilizes the system. (Use V = x> P x as a Lyapunov function and compute
its time derivative along the trajectories of x.)
2. Consider the controller u = ½Kx. With the same Lyapunov approach, ¯nd the minimum and maximum
values of ½ for which closed-loop stability is guaranteed. Interpret your results in terms of an upward and
downward gain margin.

3.4 Exponentially weighted LQR

Consider the LQR optimal control problem
Z
1 1 2±t >
min J = e (x Qx + u> Ru)dt
2 0
s.t. x_ = Ax + Bu

where R = R> > 0, Q = C > C, (A; B; C) minimal (c.c. and c.o.) and ± > 0. Compute the optimal control
in a feedback form (u = Kx) and show that this controller provides a stability margin of at least ±. That is,
Reeig(A + BK) < ¡± and the states converge to zero at least as fast as ce¡±t where c is a constant.

4
4.1 Time-optimal control
For the system

x_ 1 = x2
x_ 2 = u

determine the optimal control that transfers an initial state to the state xf = [1; 1]> in minimum time,
subject to the constraint ju(t)j · 1. In particular, develop expressions for the switching curve and give the
optimal control in a feedback form.

4.2 Weighted time-energy-optimal control

Determine the necessary conditions that should be satis¯ed by the extremals of
Z tf
min J(x; u) = (® + u2 ) dt
x;u 0
s.t. x_ 1 = x2
x_ 2 = u

where ® > 0 is a constant and x(0) = x0 , x(tf ) = xf are given.

4
5
5.1 Aircraft min-time to climb [Bryson et al. ASME J. Appl. Mech., 1962]
Find the minimum time °ight path for the F4 A/C to climb to 20km arriving with Mach 1 and level °ight.
Using a point-mass approximation (good for long-term performance problems) the equations of motion
are:

mV_ = T cos(® + ²) ¡ D ¡ mg sin °

mV °_ = T sin(® + ²) + L ¡ mg cos °
h_ = V sin °
T
m_ = ¡
cg
x_ = V cos °

where, V is velocity, ° is °ight path angle, h is altitude, m is mass, x is horizontal range, T is thrust, D
is drag, L is lift, ® is angle of attack, ² is thrust angle relative to the zero lift axis (constant, usually a few
degrees), g is the gravitational constant, c is a fuel consumption coe±cient equal to 1600 sec. The angle of
attack is used as the control variable.
The manufacturer's data for lift, drag, and max thrust were used after an analytical ¯t to allow the easy
computation of partial derivatives.

Tmax = [1M M 2 M 3 M 4 ]Q[1; h; h2 ; h3 ; h4 ]

2 3
30:21 ¡0:668 ¡6:877 1:951 ¡0:1512
6 ¡33:80 3:347 18:13 ¡5:865 0:4757 7
6 7
6
Q = 6 100:8 ¡77:56 5:441 2:864 ¡0:3355 7
7
4 ¡78:99 101:4 ¡30:28 3:236 ¡0:1089 5
18:74 ¡31:60 12:04 ¡1:785 0:09417

where, Tmax is in units of 1000lb, M is the Mach number, h is altitude in units of 10kft.
The aerodynamic data of the F4 A/C are:
Drag coe±cient: CD = CDo + ·CL® ®2
Lift Coe±cient: CL = CL® ®
The analytical ¯ts for the various parameters are given as follows:
8 £ ¡ M ¡0:98 ¢¤
>
< CDo = 0:013 + 0:0144 1 + tanh 0:06
M < 1:15 CL® = 3:44 + cosh2 1M ¡1
>
: £( 0:06 ) ¡ ¢¤
· = 0:54 + 0:15 1 + tanh M¡0:9 0:06
8 £ ¡ 0:17 ¢¤
>
< CDo = 0:013 + 0:0144 1 + tanh 0:06 ¡ 0:011(M ¡ 1:15)
M > 1:15 CL® = 3:44 + cosh2 1 0:15 ¡ 0:960:63 (M ¡ 1:15)
>
: £( 0:06 ) ¡ ¢¤
· = 0:54 + 0:15 1 + tanh 0:25 + 0:14(M ¡ 1:15)
0:06

5.2 Furnace temperature ramp-up problem [Tsakalis and Stoddard, 6th IEEE
Intl. Conf. ETFA, Los Angeles, 1997]
The optimal ramp-up problem arises frequently in applications where the system undergoes steady-state
transitions subject to bounded controls. The motivation of the particular problem studied here is from the
temperature control of di®usion furnaces used in semiconductor manufacturing.
The temperature control problem in di®usion furnaces is to follow a desired temperature trajectory
uniformly along the length of the tube. The control input is the power supplied to heating coils surrounding
the tube. Each coil represents a so-called heating zone; industrial di®usion furnaces may contain 3 to 9

5
heating zones. Temperature measurements are obtained by means of thermocouples at di®erent locations
along the tube, roughly corresponding to the center of each heating zone.
In a typical operation, the furnace is idling at a low temperature (300-600 deg.C). After loading the
wafers to be processed, the temperature is raised to the processing speci¯cations (500-1100 deg.C) and is
maintained constant throughout the wafer processing (e.g., oxidation, annealing, etc). After the end of the
process, the temperature is ramped down to the idling conditions until the next cycle.
The main objective of the controller is to maintain spatially uniform, accurate, and repeatable processing
temperature in the presence of perturbations, such as varying gas °ows, heat losses etc. However, it should
also provide good control during the ramp-up/ramp-down operations so that all wafers face similar conditions.
Furthermore, in order to reduce the cycle-time and increase productivity, the steady-state transitions should
be performed as quickly as possible but in a controlled manner.
While accurate furnace models are in¯nite dimensional and (mildly) nonlinear, linear low-order approx-
imations can be obtained via system identi¯cation techniques from data around an operating point. The
accuracy of such models is adequate in a 50-100 deg. neighborhood of the identi¯cation conditions and this
has proven su±cient for control design purposes.
With this background, the objective of this assignment is to investigate the properties of trajectories
that optimize the transition between two steady-states. Even though the system model represents an actual
furnace, several simpli¯cations are made in the interest of programming ease and computational speed. In
particular, the controller can use full-state information and the system is treated as linear in the region
of operation. The main objective is to minimize the time to raise the operating temperature from idling
to processing conditions while maintaining temperature uniformity across the furnace. In addition to that,
control input saturation presents an important constraint that must be taken into account.
Problem Statement: Consider a square linear system with problem data [A,B,C,D] supplied below.
Design a control input and ¯nd the optimal state trajectory that minimizes the time to transition from the
steady-state y = 0, x_ = 0, to the steady-state y = [100; 100; 100], x_ = 0, subject to the constraints:
1. ¡0:2 · ui · 0:8, for all i 2. jyi ¡ yj j · 0:5deg, for all i,j, where, ui is the i-th input (fraction of supplied
power) and yi is the i-th temperature.
To enforce the constraints on the input or the states, one approach that ¯ts the Mayer formulation is
to de¯ne new states (z_ = g 2 (u) or z_ = g 2 (x)) with boundary conditions z(0) = 0, z(tf ) = 0, where g(:) is
a function that is zero when the constraints are satis¯ed and nonzero when they are violated. The square
serves to ensure that z_ is di®erentiable, for the purposes of computing gradients.
Alternatively, a penalty approach may be used to enforce approximately the constraints. That is, the
integral cost function is augmented with a term Kg 2 (x; u), where K is a constant that determines the
penalty for violating the constraint. Here, the constraints can be considered as \soft." A violation of the
power fraction by 0.01, or the output matching by 0.1 are acceptable. Obviously, larger K will force the
solution to stay closer to the constraints, but they tend to make the problem ill-conditioned and slow down
the optimization.
After computing the optimal solution, it is interesting to compare it with the results of an ad-hoc solution
using an LQR tracking controller with a ramp reference input. For this, consider the i-th reference as ri (t) =
ci ramp(t¡ti ) and adjust the ramp rate ci and the time shift ti to avoid violation of the constraints, as much as
possible. For the simulations, the input constraint should be enforced using saturation functions, regardless
of the value of the computed input. For controllers that converge to the steady-state asymptotically, the
convergence time can be taken as the maximum time when all the outputs enter the envelope jyi ¡ 100j < 0:2
deg.
Furnace Problem Data (This model corresponds to a steady-state of all outputs being 900 deg C, and
the input powers being approximately 0.2 (20%). Time is measured in minutes.)
A1=[ -8.3342e+000 -1.3856e+001 2.9473e+000
8.8432e-001 -2.3043e-015 4.5864e+000
2.3438e-002 1.0633e-003 -8.2887e-003 ]

A2=[ -1.2461e+001 -1.3856e+001 4.4069e+000

1.8471e+000 -2.8167e-015 4.2459e+000

6
 4.2108e-002 1.0633e-003 -1.4891e-002 ]

A3=[ -8.1379e+000 -1.3856e+001 2.8779e+000

1.2471e+000 -2.4974e-015 4.4581e+000
4.8808e-002 1.0633e-003 -1.7260e-002 ]

B=[ -1.2818e+000 -9.0649e-002 -1.6503e-002

-1.8955e+000 -1.1612e-001 1.2613e-002
-2.1570e+000 -7.2264e-002 4.4509e-002
1.5346e-002 -8.4137e-001 -2.4555e-002
3.6399e-003 -5.0429e-001 -1.6513e-002
-1.8155e-002 -1.6804e+000 -2.2522e-002
6.6923e-003 -5.8721e-003 -1.4088e+000
8.8916e-003 -1.0094e-001 -1.5344e+000
2.4779e-002 -1.0204e-001 -2.2465e+000 ]

C1=[ 6.5318e+000 -3.4766e-015 -2.3099e+000 ]

A = [A1, 0*A1, 0*A1; 0*A1, A2, 0*A1; 0*A1, 0*A1, A3]

B = B

C = [C1, 0*C1, 0*C1; 0*C1, C1, 0*C1; 0*C1, 0*C1, C1]

D = zeros(3,3)

save isoc A B C D

Ref: A. Bryson Jr., Dynamic Optimization. Addison-Wesley, 1999.

7
 Solutions to Optimal Control Problems

April 7, 2006

1
1.1 Max-perimeter rectangle inscribed in an ellipse
Adjoin the constraint to the performance index with a Lagrange multiplier ¸:

P¹ = 4(x + y) + ¸(x2 =a2 + y 2 =b2 ¡ 1) : (1)

The stationarity conditions are then

P¹x = 4 + 2¸x=a2 = 0 ; (2)

P¹y = 4 + 2¸y=b2 = 0 : (3)

(2), (3), and the constraint are three equations for x; y; ¸. Eliminating ¸ between (2) and (3) gives

¸ = ¡2a2 =x = ¡2b2 =y ) x=a2 = y=b2 : (4)

Substituting (4) into the constraint gives

(a2 y=b2 )2 =a2 + y 2 =b2 = 1 ) y 2 = b4 =(a2 + b2 ) ) x2 = a4 =(a2 + b2 ) : (5)

1.2 Max-volume parallelepiped inscribed in an ellipsoid

Adjoin the constraint to the performance index with a Lagrange multiplier ¸:

V¹ = 8xyz + ¸(x2 =a2 + y 2 =b2 + z 2 =c2 ¡ 1) : (1)

The stationarity conditions are then

V¹x = 8yz + 2¸x=a2 = 0 ; (2)

V¹y = 8xz + 2¸y=b2 = 0 ; (3)
V¹z = 8xy + 2¸z=c2 = 0 : (4)

(2)-(4), and the constraint are four equations for x; y; ; z; ¸. Eliminating ¸ between (2) and (3) gives

¸ = ¡4a2 yz=x = ¡4b2 xz=y ; ) x2 =a2 = y 2 =b2 : (5)

Similarly, eliminating ¸ between (2) and (4) gives

x2 =a2 = z 2 =c2 : (6)

Substituting into the constraint gives

3x2 =a2 = 1 ) x2 =a2 = y 2 =b2 = z 2 =c2 = 1=3 : (7)

1
1.3 Min-distance between two lines
The problem may be stated as ¯nding (x1 ; x2 ) to min L = (x1 ¡ x2 )T (x1 ¡ x2 )=2 subject to A1 x1 = b1 and
A2 x2 = b2 where (A1 ; A2 ) each have two rows and three columns.
Adjoin the constraints with Lagrange multiplier vectors ¸1 and ¸2
¢
H = L + ¸T1 (A1 x1 ¡ b1 ) + ¸T2 (A2 x2 ¡ b2 ) : (1)

The stationarity conditions are

0 = Hx1 = (x1 ¡ x2 )T + ¸T1 A1 ; (2)

0 = Hx2 = ¡(x1 ¡ x2 )T + ¸T2 A2 : (3)

Equations (2), (3), and the constraints are 3+3+2+2=10 linear equations for the 3+3+2+2=10 unknowns
x1 ; x2 ; ¸1 ; ¸2 . Eliminating x1 using (2) and the ¯rst constraint gives

¸1 = (A1 AT1 )¡1 (A1 x2 ¡ b1 ) : (4)

Eliminating x2 using (3) and the second constraint set gives

¸2 = (A2 AT2 )¡1 (A2 x1 ¡ b2 ) : (5)

Substituting (4) and (4) into the transposes of (2) and (3) and solving for x1 and x2 gives the stated results.

1.4 Min-time path

a. Form the H¡ function

H = y1 sec µ1 =v1 + (y2 ¡ y1 ) sec µ2 =v2 + ¸[x2 ¡ y1 tan µ1 ¡ (y2 ¡ y1 ) tan µ2 ] : (1)

Necessary conditions for a stationary solution are

0 = Hµ1 = y1 sec µ1 tan µ1 =v1 ¡ ¸ y1 sec2 µ1 ; (2)

0 = Hµ2 = (y2 ¡ y1 ) sec µ2 tan µ2 =v2 ¡ ¸ (y2 ¡ y1 ) sec2 µ2 : (3)

Eliminating ¸ between (2) and (3) gives

¸ = sin µ1 =v1 = sin µ2 =v2 ; Q. E. D. (4)

b. Eqn. (4) and the constraint eqn. are two nonlinear eqns. for (µ1 ; µ2 ). They are solved using the MATLAB
Optimization Toolbox command FSOLVE.
An alternative computation is by implementing a Newton algorithm for the solution (µ(k + 1) = µ(k) ¡
[rF (µ(k))]¡1 F (µ(k))), where:
· ¸ · ¸
sin(µ1 )=v1 ¡ sin(µ2 )=v2 cos(µ1 )=v1 ¡ cos(µ2 )=v2
F (µ) = ; rF (µ) =
x2 ¡ y1 tan(µ1 ) ¡ (y2 ¡ y1 ) tan(µ2 ) ¡y1 = cos2 (µ1 ) (y1 ¡ y2 )= cos2 (µ2 )

The iteration converges from initial conditions [¼=4; ¼=4]T (the straight line path) or [0; 0]T in 20-30 itera-
tions. The resulting angles are correct modulo 2¼. (A quick ¯x of this is to use µ = tan¡1 (tan(µ)) after each
iteration.) However, checking the progress of the solution, we see a fairly \violent" behavior until the ¯nal
stages of the iteration. In general, such a generic algorithm need not converge when started far from the
solution.

2
1.5 Quadratic index with linear equality constraints
Adjoin the constraints with the Lagrange multiplier vector ¸
¢
H = (xT Qx + uT Ru)=2 + ¸T (x + Gu ¡ c) : (1)

It is assumed throughout that Q; R are symmetric. Also, for the existence of a minimizer, the second
order condition (hessian positive semi-de¯nite) yields R + GT QG ¸ 0 (its positive de¯niteness is a su±cient
condition). The stationarity conditions are

0 = Hx = xT Q + ¸T ; (2)
0 = Hu = uT R + ¸T G : (3)

Equations (2), (3), and the constraints are linear equations for the unknowns x; u; ¸. Solving (2) for ¸ gives

¸ = ¡Qx ´ ¡Q(c ¡ Gu) : (4)

Substituting (4) in (3) and solving for u gives

u = (R + GT QG)¡1 GT Qc = Kc ; (5)

Substituting (5) into the constraint gives

x = [I ¡ G(R + GT QG)¡1 GT Q]c : (6)

Substituting (6) into (4) gives

¡¸ = Q[I ¡ G(R + GT QG)¡1 GT Q]c = Sc (7)

Notice that, up to this point, the solution does not require Q; R to be invertible (only R + GT QG should be
invertible).
To derive the rest of the expressions, assume that R is invertible and combine (3) and (4) to get

u = R¡1 GT Qx (8)

Since Qx = ¡¸ = Sc, this establishes the second expression for K. Furthermore, substituting (8) in the
constraint and solving for x, we get

x = [I + GR¡1 GT Q]¡1 c = Q¡1 [Q¡1 + GR¡1 GT ]¡1 c (9)

where we assumed the invertibility of Q. Comparing (9) with (7), the second expression for S follows (the
equalities must hold for all c). These identities may also be derived by direct veri¯cation.
Finally, it remains to show that Lmin satis¯es the given expression. For this, at the optimum x = Q¡1 Sc
and u = Kc so,
Lmin = cT [S T Q¡1 QQ¡1 S + K T RK]c=2 (10)
Substituting the second expression for K and noting that S = S T , we have

Lmin = cT [SQ¡1 S + SGR¡1 RR¡1 GT S]c=2 = cT S[Q¡1 + GR¡1 RR¡1 GT ]Sc=2 = cT SS ¡1 Sc=2 = cT Sc=2
(11)
NOTE: The expressions for S are a special case of the matrix inversion lemma

(A + BCD)¡1 = A¡1 ¡ A¡1 B(C + DA¡1 B)¡1 DA¡1 : (12)

The signi¯cance of this lemma is the following: If A¡1 is already known, the inverse of (A + BCD) can be
calculated using the RHS of (15) which requires inverting a lower order matrix if the dimension of C is less
than the dimension of A.

3
1.6 Max singular values of a matrix
¢
Form H = xT AT Ax=2 + ¸(1 ¡ xT x). A necessary condition for a min is that

0 = Hx = xT AT A ¡ ¸x ) AT Ax = ¸x : (1)

This means that ¸ is an eigenvalue and x is an eigenvector of AT A. From the second order necessary
conditions for a min, Hx x ¸ 0, we have that AT A ¡ ¸I ¸ 0 so ¸ must be the smallest eigenvalue of AT A.
(Similarly for a max where ¸ is the largest eigenvalue of AT A.)
Multiplying (1) by xT gives
xT AT Ax = xT ¸x ´ ¸xT x ´ ¸ ; (2)
so the (smallest, largest) eigenvalue of AT A is the square of (min, max) value of jAxj, Q. E. D.
NOTE: The singular value decomposition (SVD) of a rectangular matrix A is
q
X
A= ®i uTi vi ; (3)
i=1

where vi ; ui = are unit column eigenvectors of (AT A; AAT ) and ®i is the square root of the ith singular
value of AT A, q is the min dimension of A. This decomposes the matrix into a sum of rank one matrices
(outer products of unit vectors). See `help SVD' in MATLAB.

1.7 Min distance between ellipses

This problem is formulated as

min (x1 ¡ x2 )T (x1 ¡ x2 )=2

x1 ;x2 2R2

s.t. xT1 P1 x1 = 1; xT2 P2 x2 = 1

where P1 = RT (µ)diag(1=4; 1)R(µ) and P2 = diag(1=16; 1=4). Then, the H-function becomes

H = (x1 ¡ x2 )T (x1 ¡ x2 )=2 + ¸1 (xT1 P1 x1 ¡ 1) + ¸2 (xT2 P2 x2 ¡ 1)

and the rest of the solution follows as usual (solve the 6 equations Hx1 = 0, Hx2 = 0 and the two constraints
for x1 ; x2 ; ¸1 ; ¸2 ).
An interesting related problem is to compute the minimizer when the point x2 is ¯xed. This problem
¯nds applications in orthogonal projections of an exterior point onto an ellipsoid. While this is a convex
problem and a local minimum is also a global minimum, the ¯rst order necessary conditions have more than
one solution where a Newton algorithm may get stuck. In this case the second order conditions can serve to
determine an interval for the Lagrange multiplier at the minimum and, thus, result in a very fast and reliable
algorithm. (See notes on adaptive algorithms, Projection on an ellipsoid for details.) The code implementing
this algorithm is given below:
function orppr1=orppr1(theta,Ra,cent,epsst);
%
% USAGE: orppr1(theta(n),Ra(n x n),cent(n),epsst)
%
% The function orppr1 computes the orthogonal projection of the parameter
% vector theta on the ellipsoid (x-cent)'Ra(x-cent)<=1 with
% tolerance epsst.
%

NN=length(theta);INN=eye(NN,NN);
epsst2=1+epsst;
xi=theta-cent; AA=(xi'*Ra*xi);
if AA < epsst2

4
 orppr1=theta;
else
b=sqrt(AA);
Z=(INN+b*Ra);
z1=Z\xi;r1=Ra*z1;z2=Z\r1;
f=xi'*z2-1;
while abs(f) >= epsst
r2=Ra*z2;
grad=-2*xi'*(Z\r2);
db=-f/grad;
while (b+db) <= 0
db=db/2;
end
b=b+db;
Z=(INN+b*Ra);
z1=Z\xi;r1=Ra*z1;z2=Z\r1;
f=xi'*z2-1;
end
orppr1=z1+cent;
end
Unfortunately, projections on more general convex sets are not as straightforward.

2
2.1 The brachistochrone problem
Let to ; Vo ; : : : denote the various quantities in the original coordinates and de¯ne the transformations:
q
p
t = to g=xf ; V = Vo = gxf ; x = xo =xf ; y = yo =xf

Then, the equations of motion become

p p
V_ = V_ o =g = sin °; x_ = x_ o = gxf = V cos °; x_ = y_ o = gxf = V sin °

We immediately observe that y_ = V V_ and, therefore, y = 12 V 2 where the integration constant is zero
since V (0) = y(0) = 0. This eliminates y from the rest of the problem.
The optimal control problem can now be written as
Z tf
min 1dt
°;V;x 0
s.t. V_ = sin °
x_ = V cos °
x(0) = 0 ; x(tf ) = 1 ; V (0) = 0

The Hamiltonian for this problem is

H = 1 + ¸1 sin ° + ¸2 V cos °

from which the necessary conditions for the optimal control are
@H
u: = 0 ) ¸1 cos ° = ¸2 V sin °
@°
¸_ 1 = ¡¸2 cos °
¸_ 2 = 0 ) ¸2 = const.

5
 At t = 0, and using the free ¯nal time condition H = 0, we have that ¸1 (0) = ¡1= sin °(0) 6
= 0. So, from
the minimization of the Hamiltonian w.r.t. ° it follows that cos °(0) = 0 ) °(0) = ¼=2. Eliminating ¸1
from H = 0 and ¸1 cos ° = ¸2 V sin ° we obtain

¡¸2 V cos2 ° ¡ cos ° = ¸2 V sin° ) cos ° = ¡¸2 V

Di®erentiation of the last equation yields

¸2 V_ = sin(°)°_ ) °_ = ¸2 ) °(t) = ¸2 t + ¼=2

where the integration constant is determined from the previously computed °(0).
With °(t) available, V and x can now be integrated directly yielding

sin ¸2 t
V (t) =
¸2
· ¸
1 sin 2¸2 t
x(t) = ¡ t¡
2¸2 2¸2

where we used the trigonometric identity sin2 a = (1¡cos 2a)=2 and the integration constants were determined
from V (0); x(0). Together with y = 12 V 2 and the de¯nition b = ¡¸2 , these equations describe the optimal
solution. The values of b and tf are determined indirectly by solving x(tf ) = 1; y(tf ) = yf .
With these expressions it is interesting to consider the problem of minimizing tf to reach x(tf ) = 1 with
y(tf ) being free. Letting a = 2btf , we have that the optimal solution at tf satis¯es

a2
= a ¡ sin a
t2f
a2
yf = 1 ¡ cos a
t2f

Minimizing t2f w.r.t. a from the ¯rst equation, we get that the minimizer satis¯es

aopt ¡ 2 sin aopt + aopt cos aopt = 0

whose solution is aopt = ¼. This, in turn, yields

µ p ¶
p 2 ¼
tf;opt = ¼ ; yf;opt = ; bopt =
¼ 2

2.2 *Zermelo's problem

2.3 Leaky Reservoir
Part 1: H = ¡x ¡ 0:1¸x + ¸u and the costate equation is

¸_ = 1 + 0:1¸ ; ¸(tf ) = 0

where tf = 100. Then, the optimal control is

½
M if ¸ < 0
u = arg min H =
u 0 if ¸ > 0

Solving the costate equations,

Z t
0:1t
¸(t) = e ¸(0) + e0:1(t¡¿ ) d¿
0
= ¡10 + (10 + ¸(0))e0:1t

6
With ¸(0) easily computed from the terminal condition, ¸(0) = ¡10 + 10=e10 , we have that ¸(t) < 0 for
t 2 [0; 100). Hence, u = M for all t, and no singular intervals exist.
Part 2: Adding the new constraint as a di®erential equation z_ = u with initial condition z(0) = 0 and
terminal condition z(tf ) = K, we have the following expression for the Hamiltonian

H = ¡x ¡ 0:1¸1 x + ¸1 u + ¸2 u

The costate equations become

¸_ 1 = 1 + 0:1¸1
¸_ 2 = 0

From the terminal conditions, x(tf ) is free but z(tf ) is ¯xed, so ¸1 (tf ) = 0 and ¸2 is free. ¸1 (t) can now be
determined as in Part 1, but ¸2 requires the solution of the TPBVP.
The optimal control is ½
M if ¸1 + ¸2 < 0
u = arg min H =
u 0 if ¸1 + ¸2 > 0
Without solving the complete system, it is obvious that ¸1 + ¸2 is an increasing exponential (no singular
intervals) and there are the following possibilities:
1. (¸1 + ¸2 )j0 < 0 and (¸1 + ¸2 )j100 · 0: u = M in [0; 100]. Apply the maximum input; the water supply
is never depleted or it is depleted at precisely the ¯nal time (K=M · 100). Notice that in the former
case, the water supply constraint is not achieved and, in a strict sense, the problem is infeasible.
2. (¸1 + ¸2 )j0 < 0 and (¸1 + ¸2 )j100 > 0: u = M in [0; t1 ] and u = 0 afterwards. The time t1 is determined
as the time where ¸1 (t1 ) + ¸2 (t1 ) = 0, which happens when the water source is depleted. So, the
optimal policy is to apply maximum °owrate until the supply is depleted.
3. (¸1 + ¸2 )j0 > 0. That means u = 0, and it can only happen if K = 0 (there is no water supply).
Part 3: Here we have the following expression for the Hamiltonian

H = ¡0:1¸1 x + ¸1 u + ¸2 u

The costate equations become

¸_ 1 = 0:1¸1
¸_ 2 = 0
@(¡x)
From the terminal conditions, x(tf ) is free but z(tf ) is ¯xed, so ¸1 (tf ) = @x jtf = ¡1 and ¸2 is free. The
optimal control is again ½
M if ¸1 + ¸2 < 0
u = arg min H =
u 0 if ¸1 + ¸2 > 0
but, in this case, ¸1 (0) = ¡e¡0:1tf and ¸1 (t)+¸2 = ¡e0:1(t¡tf ) +¸2 . This means that (¸1 +¸2 ) is a decreasing
function of time and the optimal control is to use our water supply at the end of the time interval.
As in Part 2, we can distinguish di®erent cases depending on the sign of (¸1 + ¸2 ) at 0 and 100. The
critical parameter here is the input switching time t1 = tf ¡ K=M and the optimal policy is u = M if t > t1
and u = 0 otherwise. (A similar switching time could have been de¯ned for Part 2, but the description of
the optimal policy was much easier there.)

2.4 Shape of a hanging chain

The optimal control problem can be written as
Z xf p
min y 1 + u2 dx
u;y;s x0

7
 dy
s.t. =u
dx
ds p
= 1 + u2
dx
x0 = ¡l ; xf = l
y(x0 ) = 0 ; y(xf ) = 0
s(x0 ) = 0 ; s(xf ) = 2L

The Hamiltonian for this problem is

p p
H =y 1 + u2 + ¸1 u + ¸2 1 + u2

from which the necessary conditions for the optimal control are
@H u ¸1
u: =0) p =¡
@u 1+u 2 y + ¸2
d¸1 p
= 1 + u2 ) ¸1 = ¡s + c1
dx
d¸2
= 0 ) ¸2 = const.
dx
Anticipating symmetry about the mid-point x = 0, we conjecture s(0) = L, u(0) = 0, and y(0) = ymin .
These also imply that ¸1 (0) = 0 (since @H
@u = 0) and c1 = L. Thus, we have

u s¡L
¸1 = L ¡ s ; p =
1 + u2 y + ¸2
Next, we eliminate x from the di®erential equations of y and s:
dy s¡L
= ) (y + ¸2 )2 = (s ¡ L)2 + c2
ds y + ¸2
where the last equation is derived after a straightforward separation of variables. To compute the constant
c2 we notice that at the end-points y = 0 so c2 = ¸22 ¡ L2 . At the mid-point s = L so c2 = (ymin + ¸2 )2 > 0.
Thus, ¸22 > L2 . At the mid-point the Hamiltonian (which is constant as a function of x) has the value ¸2 + y
and we can de¯ne
H 2 = c2 = ¸22 ¡ L2
Moreover, from the second order conditions for the minimum, the p minimizer of the Hamiltonian should
satisfy @ 2 H=@u2 ¸ 0 which, in turn, yields ¸2 + y ¸ 0 and H = + ¸22 ¡ L2 .
Substituting back in the equation for the optimal control
u ¸1 s¡L s=H ¡ L=H
p = =p =p
1+u2 y ¡ ¸2 2
H + (s ¡ L)2 1 + (s=H ¡ L=H)2

where the square root sign is taken to be consistent with u(x0 ) < 0; u(xf ) > 0. Thus,
s L
u= ¡
H H
which yields a di®erential equation for u that depends on no other variables:
du 1 ds 1p du p
= = 1 + u2 ) = 1 + u2
dx H dx H d(x=H)

The solution of this di®erential equation is1

u = sinh(x=H) + c3
1 Recall that cosh2 (x) ¡ sinh2 (x) = 1, d sinh x=dx = cosh x, d cosh x=dx = sinh x.

8
And c3 = 0 since u(0) = 0 at the mid-point. Furthermore, at the end-point xf , (s ¡ L)=H = L=H = u(xf ) =
sinh(l=H). So, the constant H should satsify
µ ¶
L l
= sinh
H H
dy
Finally, integrating dx = u we get that y(x) = H cosh(x=H) + c4 . Using the end-point condition y(l) =
y(¡l) = 0, it follows that c4 = ¡H cosh(l=H). (Notice that cosh is an even function.) Thus,
µ hxi · ¸¶
l
y(x) = H cosh ¡ cosh
H H

It is interesting to observe that the maximum de°ection of y, occuring at the mid-point, is ¡ym in =
H(cosh(l=H) ¡ 1). Normalizing L to 1 and letting l ! L we have that l=H ! 0 and, by a Taylor expansion,
s
1 l l3 l
' + 3
)H 'l
H H 6H 6(1 ¡ l)
r r
2
1 l 3 1¡l
¡ymin ' H 2
'
2H 2 l
2
Clearly, we must have ymin + l2 · 1 so this p
approximation can be meaningful only for (1 + l)l > 3=2 (i.e.,
l » 0:85L or more). Notice that ymin ! 0 as 1 ¡ l. So for a running length di®erence of only 1% (l = 0:99)
the maximum de°ection is 12:3% (ymin = 0:123)!
The \catenary" shape of the hanging chain is very close to a quadratic for medium-large l. But when l
becomes small the nature of cosh becomes dominant and y takes a U-shape. ¡ ¢
L
The analytic computation of y relies on the solution of H = sinh Hl for H. This is easily achieved by
re-writing the equation as sinh¡1 (ha) = h where h = l=H and a = L=l. In MATLAB, sinh¡1 is the built-in
function asinh. Now, in the vicinity of the solution, jrh sinh¡1 (ha)j = ja=rha sinh(h)j = j1= cosh(h)j < 1.
Since the map h = sinh¡1 (ha) is a local contraction, the iteration hk+1 = sinh¡1 (ahk ) converges to the
solution, locally. (In fact, the iteration converges to the positive root of the equation for any positive initial
condition. But, depending on a, the convergence can be very slow.)

2.5 Min surface area of soap ¯lm

a. The Hamiltonian in Bolza form and the augmented performance index are
p
H = r 1 + u2 + ¸u; © = º[r(`) ¡ a]: (1)

The optimality condition is

ru
0= p + ¸: (2)
1 + u2
Since H is not an explicit function of x; H = constant is an integral of the TPBVP. Use this integral in
place of the adjoint equation.
Substituting (2) into (1) and solving for u gives
p
r2 ¡ H 2
u= : (3)
H
Substituting (3) into dr=dx = u gives
dx dr
=p ;: (4)
H r ¡ H2
2

Integrating (4) gives

x x
r = H cosh ) u = H sinh : (5)
H H

9
Evaluating (5) at x = ` gives
a H `
= cosh ; (6)
` ` H
which is a transcendental equation that determines H=` given a=`.
b. The solution of (6) can be considered as the intersection of two functions of `=H, the straight line
y = (a=`)(`=H) and the curve y = cosh(`=H). There will be two solutions for `=a < :663, one solution for
`=a = :663 when the line is just tangent to the curve, and no solutions for `=a > :663.
c. Substituting (5) into the expression for the minimum area gives
Z ` · ¸
x 2 H 2`
A = 2¼ H cosh dx = 2¼H ` + sinh ; (7)
¡` H 2 H

which is a complicated function of `=a [determined from (6) given `=H]. Using (6) in (7) gives

Amin = 2¼a2 [tanh(`=H) + (`=H)sech2 (`=H)]: (8)

Amin is a monotonically increasing function of `=a in the range of interest, but it is greater than 2¼a2 =
area inside the two loops for `=H > :639 which corresponds to `=a > :528. Thus
½
2¼a2 [tanh(`=H) + (`=H)sech2 (`=H)]; 0 · `=a · :528;
Amin = (9)
2¼a2 ; `=a ¸ :528:

3
3.1
For this problem, the Hamiltonian is H = (2x21 + x22 + u2 )=2 + ¸1 x2 + ¸2 (¡x1 + (1 ¡ x21 )x2 + u). Then, the
costate equations are

¸_ 1 = ¡2x1 + ¸2 + 2x1 x2 ¸2
¸_ 2 = ¡x2 ¡ ¸1 ¡ ¸2 (1 ¡ x21 )

Minimizing H with respect to u we get

u = ¡¸2
and the boundary conditions are x(0) = x0 , x(1) = xf .

3.2
Here, H = (3x2 + u2 ) + ¸(¡x + u). Hence,

¸_ = ¸ ¡ 3x
u = ¡¸

Substituting back into x_ = ¡x + u we have

µ ¶ µ ¶µ ¶
d x ¡1 ¡1 x
=
dt ¸ ¡3 1 ¸
| {z }
A

which should be solved with boundary conditions x(0) = x0 and x(1) = 0.

The solution of the above problem is
µ ¶ µ ¶
x At x0
(t) = e
¸ ¸0

10
and the matrix exponential can be computed with a variety of methods (diagonalization, Laplace, Cayley-
Hamilton).
Let Áij (t) denote the entries of eAt . Then
0 = x(1) = Á11 (1)x0 + Á12 (1)¸0 ) ¸0 = ¡Á¡112 (1)Á11 (1)x0
) u(t) = ¡¸(t) = ¡Á21 (t)x0 + Á22 (t)Á¡1
12 (1)Á11 (1)x0

If desired, this optimal control can be expressed in feedback form by solving the state/costate equations
for x0 in terms of x(t) and substituting in the expression for u Notice that this control is only valid in the
interval [0; 1]. If applied beyond that point, it will cause the state to diverge from 0.
A variant of this problem is to include a terminal cost kx2 (tf )=2 and leave x(tf ) free. Then, the terminal
condition becomes ¸(tf ) = kx(tf ). Again, we can solve for ¸0 in terms of x0 :
¸0 = (Á22 (1) ¡ kÁ12 (1))¡1 (kÁ11 (1) ¡ Á21 (1))x0
which for large k approaches the previous expression. These results generalize in the multidimensional case
where the required inverses have been shown to exist under the assumptions of the LQR problem.

3.3 LQR Stabilization

Part 1. The solution of this LQR problem is u = Kx = ¡R¡1 B > P x where P > 0 satis¯es the Algebraic
Riccati equation
A> P + P A ¡ P BR¡1 B > P + Q = 0
Consider the function V = x> P x. Notice that V ¸ 0 for all x and V = 0 if and only if x = 0. Evaluating
the derivative of this function along the trajectories of the system,
V_ = x_ > P x + x> P x_ = x> (A> P + P A)x + 2x> P Bu
Observe that x> P B = ¡u> R. Then, using the Riccati,
V_ = x> (A> P + P A)x + x> P Bu + x> P Bu
= x> (A> P + P A)x ¡ x> P BR¡1 B > P x ¡ u> Ru
= ¡x> Qx ¡ u> Ru · 0
Since V is non-increasing, V (t) · V (0) implying that ¸min (P )kx(t)k2 · ¸max (P )kx0 k2 and, hence, kx(t)k ·
kkx0 k for some constant k and for any initial condition. This implies that the state-transition matrix of
A + BK is bounded and the zero-equilibrium is stable. In turn, the eigenvalues of A + BK have non-positive
real parts and any eigenvalues on the jw-axis are simple.

While the stability proof is fairly straightforward, asymptotic stability requires the use of a more elaborate
argument from the theory of Lyapunov functions. This is based on the so-called La-Salle's theorem which, in the
above setting, states that the system trajectories converge to a limit set characterized by V_ = 0. (Notice that V is
non-increasing and lower bounded, hence it reaches a limit.) The limit set contains trajectories of the system that
satisfy x> Qx = 0, u> Ru = 0 and x_ = Ax + Bu. Since R > 0, we have that u = 0. Furthermore, with Q = C > C,
the limit set contains state trajectories such that x_ = Ax and Cx = 0. But since (A; C) is completely observable,
the only trajectory contained in this set is x = 0. Hence x ! 0, implying that the zero-equilibrium is asymptotically
stable; in turn, the eigenvalues of A + BK have negative real parts.

Part 2. Repeating the stability argument with u = ½Kx we have

V_ = x> (A> P + P A)x + 2½x> P Bu
= x> (A> P + P A)x ¡ x> P BR¡1 B > P x + x> P BR¡1 B > P x ¡ 2½u> Ru
= ¡x> Qx ¡ (2½ ¡ 1)u> Ru
Thus, V_ · 0 provided that 2½ ¡ 1 > 0 or ½ > 1=2. This implies that stability is preserved if the control input
is multiplied by any constant in the interval (1=2; 1). Hence the upward gain margin of the LQR is in¯nity
and the downward gain margin is 1/2.

11
3.4 Exponentially weighted LQR
Consider the transformation of variables

~ = xe±t ; u
x ~ = ue±t

Then,
x~_ = ± x
~ + e±t (Ax + Bu) = (A + ±I)~x + Bu~
R1 > >
For this system we would like to minimize J = 0 (~ x Q~x+u ~ R~ u)dt. Using the standard LQR result, the
~ = ¡R¡1 B > P x
optimal control is u ~, where P > 0 is the solution of the Riccati

(A + ±I)> P + P (A + ±I) ¡ P BR¡1 B > P + Q = 0

Transforming the input back to the original coordinates,

u = ¡R¡1 B > P x

In other words, the solution to the exponentially weighted LQR problem involves a Riccati for the shifted
matrix A + ±I.
Furthermore, since the LQR is a stabilizing compensator, the eigenvalues of (A + ±I) + BK have non-
positive real parts, implying that the eigenvalues of A + BK have real parts less than ¡± (stability margin of
±) and the states converge to zero as e¡±t . Alternatively, since the LQR cost is ¯nite, e2±t x> Qx, e2±t u> Qu
are integrable, implying that x ! 0 as e¡±t . (The complete argument requires the use of a uniform continuity
condition and the observability of (A; C).)
It is worthwhile to point out that in this case the Lyapunov stability analysis results in V_ · ¡2±V . This
yields immediately that V (t) · e¡2±t V (0) and, thus, kx(t)k · ke¡±t kx0 k where k is a constant that depends
on the condition number (ratio of maximum/minimum eigenvalues) of P .
Finally, notice that the derivation of the above properties hinges on the controllability/observability of
the shifted system, i.e., (A + ±I; B; C). It is straightforward to verify that these properties hold if (A; B; C)
is c.c. and c.o. However, the weaker conditions for the solution of the LQR problem ((A; B; C) stabilizable
and detectable) are not enough to guarantee the solution of the exponentially weighted LQR problem. In
this case we need to assume that any uncontrollable or unobservable states have modes with real parts less
than ¡±.

4
4.1 Time-optimal control
Here we want to ¯nd the optimal control that
Z tf
min 1dt
u;x 0
s.t. x_ 1 = x2
x_ 2 = u
x(0) = x0 arbitrary but given; x(tf ) = [1; 1]>
ju(t)j · 1

The Hamiltonian for this problem is

H = 1 + ¸1 x2 + ¸2 u
from which the necessary conditions for the optimal control are

u : min(H) ) u = ¡sign(¸2 )
u

¸_ 1 = 0 ) ¸1 = const.
¸_ 2 = ¡¸1 ) ¸2 = ¡¸1 t + c

12
From the boundary conditions H(t) = H(tf ) = 0 and for a singular interval ¸2 ´ 0, hence ¸1 = 0, c = 0,
implying H = 1, a contradiction; hence, no singular intervals exist. (This is expected since the system is
controllable.)
Integrating with u = §1 we get

x2 = §t + c1
1
x1 = § t2 + c1 t + c2
2
which yields
1 2
x1 =x + c3 if u = +1
2 2
1
x1 = ¡ x22 + c4 if u = ¡1
2
The trajectories that pass through xf without any switchings are characterized by the constants c3 = 1=2
(for u = +1) and c4 = 3=2 (for u = ¡1). Hence, the switching curve can be de¯ned as

x1 = 12 x22 + 1
2 if x2 < 1
S(x) =
x1 = ¡ 12 x22 + 3
2 if x2 > 1

Notice that S(x) is not di®erentiable at xf . The switching curve divides the state-space into two halves, say
S¡ and S+ , de¯ned as follows:
8
< x1 ¡ 12 x22 ¡ 12 > 0; x2 < 1
S¡ = x + 1 x2 ¡ 3 > 0; x2 > 1
: 1 2 2 2
x1 ¸ 1 x2 = 1
8
< x1 ¡ 12 x22 ¡ 12 < 0; x2 < 1
S+ = x1 + 12 x22 ¡ 32 < 0; x2 > 1
:
x1 < 1 x2 = 1
The optimal control can now be written as
½
¡1; x 2 S¡
u=
+1; x 2 S+

The action at xf was chosen arbitrarily since it is not an equilibrium.

4.2 Weighted time-energy-optimal control

The Hamiltonian is H = ® + u2 + ¸1 x2 + ¸2 u. Then, the costate equations are

¸_ 1 = 0 ) ¸1 = c1 (constant)
¸_ 2 = ¡¸1 ) ¸2 = ¡c1 t + c2

Minimizing H with respect to u we get

u = ¡¸2 =2 ) u = (c1 t ¡ c2 )=2

Now, if tf is ¯xed then the optimal control problem is simply the transfer of an initial state to a ¯nal state
minimizing the energy of the input. (Its solution can also be expressed as a minimum norm problem and is
found in terms of the controllability Gramian, by performing a projection.) Using calculus of variations, we
need to solve the above Euler equations with boundary conditions x(0) = x0 , x(tf ) = xf . That is,

x_ 2 = u ) x2 (t) = x2;0 ¡ c2 t=2 + c1 t2 =4

x_ 1 = x2 ) x1 (t) = x1;0 + x2;0 t ¡ c2 t2 =4 + c1 t3 =12

13
Evaluating these expressions at tf and setting x1 (tf ) = x1;f and x2 (tf ) = x2;f we can solve the two linear
equations for the unknowns c1 ; c2 .
If tf is free, then the additional boundary condition is

H(tf ) = 0 = ® + (c1 tf ¡ c2 )2 =4 + c1 x1;f ¡ (c2 ¡ c1 tf )2 =2

We now have 3 polynomial equations for the unknowns c1 ; c2 ; tf . (Multiple solutions may exist, but the
minimizer can be identi¯ed by, e.g., computing the cost for each case.)

14
5.2 Furnace temperature ramp-up problem [Tsakalis and Stoddard, 6th IEEE
Intl. Conf. ETFA, Los Angeles, 1997]

~y C~x
% STEEPEST DESCENT PROCEDURE
tf = input('tf ');
format short e
ti = [0:3/60:tf]'; N = length(ti);
ri = N:-1:1;
uk = zeros(N,3)+0.8;
niter=0; dd=.5*.5;

load isoc

sys = ss(A,B,C,D);
Ct=[C-C([2,3,1],:)]; Dt = [D-D([2,3,1],:)];
syst = ss(A,B,Ct,Dt);
sysa = ss(A',Ct',B',D');
sysi = ss(zeros(3,3),eye(3,3),eye(3,3),zeros(3,3));
xf = (A\B)*((C*(A\B))\[100;100;100]);
zf = [0;0;0];

[dy,t,x] = lsim(syst,uk,ti);
dy2 = dy.^2 -dd; ig = find(dy2 > 0); dg = 0*dy2; dg(ig) = dy2(ig); g = dg.^2;
z = lsim(sysi,g,ti); Lf = 100*(x(N,:)' - xf); Lzf = z(N,:)';
J0 = [Lf;Lzf]'*[Lf;Lzf]/2
upd=1;

while upd == 1
dg_in = (2*dg.*dy)*diag(Lzf);
dg_in = dg_in(ri,:);
[tem,t,L] = lsim(sysa,dg_in,ti,Lf);
L = L(ri,:); us = -L*B;
us = max(-1e10*L * B,-.2); us = min(us,0.8);

J = inf; gamma = 10;

while J0 < J
if gamma > 1e-10; gamma = gamma/2; else gamma = 0; disp('forced exit'); end
u = (uk + gamma * [us - uk]);
[dy,t,x] = lsim(syst,u,ti);
dy2 = dy.^2 -dd; ig = find(dy2 > 0); dg = 0*dy2; dg(ig) = dy2(ig); g = dg.^2;
z = lsim(sysi,g,ti); Lf = 100*(x(N,:)' - xf); Lzf = z(N,:)';
J = [Lf;Lzf]'*[Lf;Lzf]/2;
end
if J0-J < 1e-10;
upd = 0;
else
niter = niter + 1; J0=J; uk = u;
disp([niter,J])
end
end
y=lsim(sys,u,t);
plot(t,y,t,u*100)
% OBLIQUE PROJECTION SOLUTION
tf = input('tf ');
format short e
TS = 3/60; SSS=4;
ti = [0:TS:tf]'; N = length(ti);
ri = N:-1:1;

load isoc
sys=ss(A,B,C,D);
sysd=c2d(sys,TS);
[y,k]=impulse(sysd);
h1=y(:,:,1);h2=y(:,:,2);h3=y(:,:,3);
h1=h1(1:N,:);h2=h2(1:N,:);h3=h3(1:N,:);
H1 = [ toeplitz(h1(:,1),[h1(1,1),zeros(1,N-1)]);
toeplitz(h1(:,2),[h1(1,2),zeros(1,N-1)]);
toeplitz(h1(:,3),[h1(1,3),zeros(1,N-1)])
];
H2 = [ toeplitz(h2(:,1),[h2(1,1),zeros(1,N-1)]);
toeplitz(h2(:,2),[h2(1,2),zeros(1,N-1)]);
toeplitz(h2(:,3),[h2(1,3),zeros(1,N-1)])
];
H3 = [ toeplitz(h3(:,1),[h3(1,1),zeros(1,N-1)]);
toeplitz(h3(:,2),[h3(1,2),zeros(1,N-1)]);
toeplitz(h3(:,3),[h3(1,3),zeros(1,N-1)])
];
H=[H1 H2 H3];
a=[eye(3*N,3*N);-eye(3*N,3*N)]; b=[ones(3*N,1)*.8;ones(3*N,1)*.2];
c1=[eye(N,N),-eye(N,N),zeros(N,N)];
c2=[eye(N,N),zeros(N,N),-eye(N,N)];
c3=[zeros(N,N),eye(N,N),-eye(N,N)];
bd=ones(N,1)*.5;
a=[a;c1*H;-c1*H;c2*H;-c2*H;c3*H;-c3*H];
b=[b;bd;bd;bd;bd;bd;bd];

HSS=H([[N-SSS:N],[2*N-SSS:2*N],[3*N-SSS:3*N]],:);
yss=ones(3*(SSS+1),1)*100;uss=HSS\yss;
HTH=HSS'*HSS+0*eye(3*N,3*N); disp('starting projection');tic
u=orpr(uss,HTH,a,b,[],[],1,1e-6,1);toc
% OBLIQUE PROJECTION SOLUTION II
tf = input('tf ');
format short e
TS = 5/60; SSS=4;
ti = [0:TS:tf]'; N = length(ti);
ri = N:-1:1;

load isoc
sid=ss(eye(3,3),eye(3,3),eye(3,3),eye(3,3),-1);
sys=ss(A,B,C,D);
sysd=c2d(sys,TS);
[y,k]=step(sysd);
h1=y(:,:,1);h2=y(:,:,2);h3=y(:,:,3);
h1=h1(1:N,:);h2=h2(1:N,:);h3=h3(1:N,:);
H1 = [ toeplitz(h1(:,1),[h1(1,1),zeros(1,N-1)]);
toeplitz(h1(:,2),[h1(1,2),zeros(1,N-1)]);
toeplitz(h1(:,3),[h1(1,3),zeros(1,N-1)])
];
H2 = [ toeplitz(h2(:,1),[h2(1,1),zeros(1,N-1)]);
toeplitz(h2(:,2),[h2(1,2),zeros(1,N-1)]);
toeplitz(h2(:,3),[h2(1,3),zeros(1,N-1)])
];
H3 = [ toeplitz(h3(:,1),[h3(1,1),zeros(1,N-1)]);
toeplitz(h3(:,2),[h3(1,2),zeros(1,N-1)]);
toeplitz(h3(:,3),[h3(1,3),zeros(1,N-1)])
];
H=[H1 H2 H3];
a=[toeplitz(ones(N,1),[1,zeros(1,N-1)]);-toeplitz(ones(N,1),[1,zeros(1,N-1)])];
b=[ones(N,1)*.8;ones(N,1)*.2];
a=[a 0*a 0*a;0*a a 0*a;0*a 0*a a];b=[b;b;b];
c1=[eye(N,N),-eye(N,N),zeros(N,N)];
c2=[eye(N,N),zeros(N,N),-eye(N,N)];c3=[zeros(N,N),eye(N,N),-eye(N,N)];
bd=ones(N,1)*.5;
a=[a;c1*H;-c1*H;c2*H;-c2*H;c3*H;-c3*H]; b=[b;bd;bd;bd;bd;bd;bd];

HSS=H([[N-SSS:N],[2*N-SSS:2*N],[3*N-SSS:3*N]],:);
yss=ones(3*(SSS+1),1)*100;uss=HSS\yss;
HTH=HSS'*HSS+1e-1*eye(3*N,3*N); disp('starting projection');tic
du=orpr(uss,HTH,a,b,[],[],1,1e-6,1);toc
u=lsim(sid,unvector(du,N,3));
plot(ti,unvector(H*du,N,3),ti,u*100)