Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.

5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.11 Bang-bang Control

∂F d ⎛ ∂F ⎞
− ⎜⎜ ⎟=0 ⇒ p& 2 = − p1
11.1 Introduction
∂x 2 dt ⎝ ∂x& 2 ⎟
⎠
(11.10)

∂F d ⎛ ∂F ⎞
This chapter deals with the control with restrictions: is
− ⎜ ⎟=0 ⇒ p 2 = µ ( β − α − 2u )
bounded and might well be possible to have discontinuities.
∂u dt ⎝ ∂u& ⎠
(11.11)

∂F d ⎛ ∂F ⎞
To illustrate some of the basic concepts involved when
− ⎜ ⎟=0 2vµ = 0
∂v dt ⎝ ∂v& ⎠
controls are bounded and allowed to have discontinuities we (11.12)
start with a simple physical problem: Derive a controller such
that a car move a distance a with minimum time.
(11.12) ⇒ v = 0 or µ = 0 . We will consider these two
The motion equation of the car
cases.

d 2x
=u (i) µ =0
⎧p = 0
(11.1)
⇒⎨ 1 ⇒ be impossible.
dt 2

⎩ p2 = 0
(ii) v = 0
where

u = u (t ), − α ≤ u ≤ β (11.2) (11.5): v 2 = (u + α )( β − u ) ⇒ u = −α or u = β
Hence
represents the applied acceleration or deceleration (braking)
⎧β 0 ≤ t ≤τ
x& 2 = ⎨
and x the distance traveled. The problem can be stated as

⎩− α τ <t ≤T
minimize

T = 1 dt
∫ the switch taking place at time τ . Integrating using
T
(11.3)
o
boundary conditions on x 2

⎧β t 0 ≤ t ≤τ
subject to (10.11) and (10.12) and boundary conditions

x(0) = 0, x& (0) = 0, x(T ) = a, x& (T ) = 0 x 2 = x&1 = ⎨

⎩ − α (t − T ) τ <t ≤T
(11.4) (11.13)

The methods we developed in the last chapter would be

Integrating using boundary conditions on x1
appropriate for this problem except that they cannot cope

⎧1 2
with inequality constraints of the form (11.2). We can

⎪⎪ β t 0 ≤ t ≤τ
change this constraint into an equality constraint by
x1 = ⎨ 2
introducing another control variable, v, where
⎪− 1 α (t − T ) 2 + a τ <t ≤T
(11.14)
v = (u + α )( β − u )
2
(11.5) ⎪⎩ 2

state variable notation x1 = x so that

Since v is real, u must satisfy (11.2). We introduce th usual
t = τ , we must have
Both distance, x1 , and velocity, x 2 , are continuous at

x&1 = x2 x1 (0) = 0, x2 (T ) = a (11.6) (11.14) ⇒ βτ = −α (τ − T )

x& 2 = u x 2 (0) = 0, x 2 (T ) = 0
(11.15) ⇒ βτ 2 = a − α (τ − T ) 2
(11.7) 1 1
2 2
We now form the augmented functional

∫
Eliminating T gives the switching time as
T* = {1 + p1 ( x 2 − x&1 ) + p 2 (u − x& 2 ) + µ [v 2
T

2aα
−(u + α )( β − u )]}dt τ=
0

β (α + β )
(11.8) (11.15)

where p1 , p 2 ,η are Lagrange multipliers associated with the

and the final time is
constraints (11.6), (11.7) and (11.5) respectively. The Euler
2a (α + β )
T=
equations for the state variables x1 , x 2 and control variables u

αβ
and v are (11.16)

∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ p& 1 = 0
∂x1 dt ⎜⎝ ∂x&1 ⎟⎠
(11.9) The problem now is completely solved and the optimal
control is specified by

Chapter 11 Bang-bang Control 53

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

⎧β 0 ≤ t ≤τ ∂H
u=⎨ p& i = − (i = 1,2, L , n)
⎩− α τ <t ≤T ∂xi
(11.17) (11.23)

this is illustrated in Fig. 11.1 The Euler equations for the control variables, u i , do not
follow as in section 10.4 as it is possible that there are

β
control u discontinuities in u i , and so we cannot assume that the partial
derivaties ∂H / ∂u i exist. On the other hand, we can apply the
∂F
= 0 to obtain
∂x& i
free end point condition (9.23):

τ
p k (T ) = 0 k = q + 1, L , n
0
T time t (11.24)

−α
that is, the adjoint variable is zero at every end point where
the corresponding state variable is not specified. As before,
we refer to (11.24) as tranversality conditions.
Fig. 11.1 Optimal Control

∂H / ∂u i = 0 for continuous controls. For the moment, let us

Our difficulty now lies in obtaining the analogous equation to

- has a switch (discontinuity) at time t = τ

This figure shows that the control:

consider a small variation δu in the control u such

assume that we can differentiate H with respect to u, and
- only take its maximum and minimum values
that u + δu still belong to U , the admissible control region.
This type of control is called bang-bang control.

change in x , say δx , and in p , say δp . The change in the

Corresponding to the small change in u , there will be small

value of J * will be δJ * , where

11.2 Pontryagin’s Principle (early 1960s)

∫ ∑ p x& }dt
Problem: We are seeking extremum values of the functional

∫
δJ * = δ {H −
n

J =
T
T
i i
i =1
f 0 (x, u, t )dt (11.18) o
0

subject to state equations The small change operator, δ , obeys the same sort of
x& i = f i(x, u, t ) (i = 1,2, L , n) (11.19) properties as the differential operator d / dx .

initial conditions x = x 0 and final conditions on x1 , x 2 , L , x q Assuming we can interchange the small change operator, δ ,
(q ≤ n) and subject to u ∈ U , the admissible control region.
and integral sign, we obtain

δJ * =
∫ [δ ( H − ∑ p x& )]dt
For example, in the previous problem, the admissible control T n
region is defined by i i
i =1
U = {u : −α ≤ u ≤ β }

∫ [δH − ∑ δ ( p x& )]dt

0

=
T n
i i
i =1
As in section 10.4, we form the augmented functional

∑ ∫ ∑ ∑ p δ x& ]dt
0

∫
⎡ ⎤
J* = ⎢ f0 + pi ( f i − x& i )⎥ dt = [δH − x& i δ pi −
n T n n
T

⎢ ⎥
(11.20)
⎣ ⎦
i i
i =1 i =1 i =1
0 0

and define the Hamintonian Using chain rule for partial differentiation

H = f0 + ∑ δH = ∑
∂H
δu j + ∑
∂H
δxi + ∑ ∂p δp
∂H
n m n n

∂u j ∂xi
pi f i (11.21)
i =1
i
j =1 i =1 i =1 i

For simplicity, we consider that the Hamintonian is a so that

vector p, that is, H = H (x, u, p) .

∫ ∑ ∂u
function of the state vector x, control vector u, and adjoint
⎡
⎢ ∂H
δJ * = δu j
T m

⎢
We can express J * as
⎣

∑ p x& ⎟⎟ dt
j =1

∫
0

T⎛ ⎞

∑ ⎜⎜⎝ ∂p δ p
j

J* = ⎜H − ⎛ ∂H ⎞⎤
n

⎜ + − p& i δ xi − xi δ pi − pi δ x& i ⎟⎟⎥ dt

n
(11.22)
⎝ ⎠ ⎠⎥⎦
i i
i =1
0 i
i =1 i

∂H
since p& i = −∂H / ∂xi . Also, from (11.21) = f i = x& i
and evaluating the Euler equations for xi , we obtain as in
section 10.4 the adjoint equations ∂pi

Chapter 11 Bang-bang Control 54

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

using (11.19): x& i = f i(x, u, t ), (i = 1,2, L , n) . Thus We first illustrate its use by examining a simple problem. We
required to minimize

∫ ∑ ∑ ( p& iδ xi + piδ x&i )⎥⎥ dt ∫

⎡ ⎤
⎢ ∂H
δJ * = δu j − J = 1dt
T m n

0 ⎢
T

∂u j
⎣ j =1 i =1 ⎦
0

∫ ∑ ∑
⎡ ⎤ subject to x&1 = x 2 , x& 2 = u where −α ≤ u ≤ β and x1 (T ) = a ,
= ⎢ ∂H
δu j − ( piδ xi )⎥⎥ dt
T m n

0 ⎢ x2 (T ) = 0 , x1 (0) = x 2 (0) = 0 .
d
∂u j
⎣ j =1 i =1 ⎦
dt

Introducing adjoint variables p1 and p 2 , the Hamiltonian is

We can now integrate the second part of the integrand to
yield given by

H = 1 + p1 x 2 + p 2 u

∑ ( p δ x ) + ∫ ⎜⎜ ∑
T ⎛⎜ m ∂H
⎞
⎟
T

δJ * = − δu j ⎟ dt
n

∂u j ⎟
(11.25)
⎝ ⎠
i i
i =1 j =1
u ∈U = [−α , β ] , the admissible control region.. Since H is
0 We must minimize H with respect to u, and where
0

At t = 0 : xi (i = 1,L, n) are specified ⇒ δxi (0) = 0

the control region, that is, either at u = −α or u = β . This
linear in u, it clearly attains its minimum on the boundary of

At t = T : xi (i = 1,L , q ) are fixed ⇒ δxi (T ) = 0

For i = q + 1, L, n, from the transversality conditions, (11,24)
illustrated in Fig.11.3. In fact we can write the optimal
control as

pi (T ) δxi (T ) = 0 ⎧−α if p 2 > 0

u=⎨
⎩ β if p 2 < 0
for i = 1,2, L , q, q + 1, L , n . We now have

∫ ∑
H

T ⎛⎜ ∂H
⎞
⎟
δJ * = ⎜ δu j ⎟ dt
m

0 ⎜ ∂u j ⎟
⎝ j =1 ⎠

where δu j is the small variation in the j th component of the

control u
−α β
and we require δJ * = 0 for a turning point when the controls Fig. 11.3 The case p 2 > 0
control vector u . Since all these variations are independent,

are continuous, we conclude that

But p 2 will vary in time, and satisfies the adjoint equations,
∂H
= 0 ( j = 1,2, L , m) ∂H
∂u j p& 1 = − =0
(11.26)
∂x1
∂H
p& 2 = − = − p1
not constrained. In our present case when u ∈ U , the ∂x 2
But this is only valid when the controls are continuous and

Thus p1 = A , a constant, and p 2 = − At + B , where B is

admissible control region and discontinuities in u are allowed.

way, except that (∂H / ∂u j )du j must be replaced by

The arguments presented above follow through in the same
constant. Since p 2 is a linear function of t, there will at most

H (x; u1 , u 2 ,L, u j + δu j ,L, u m ; p) − H (x, u, p)

be one switch in the control, since p 2 has at most one zero,
and from the physical situation there must be at least one
switch. So we conclude that

∫∑
We thus obtain
(i) the control u = −α or u = β , that is, bang-bang control;
δJ * = [ H (x; u1 ,L, u j + δu j ,L, u m ; p) − H (x, u, p)] dt
T m
(ii) there is one and only one switch in the control.
j =1
Again, it is clear from the basic problem that initially u = β ,
0

followed by u = −α at the appropriate time.

have δJ * ≥ 0 for all admissible controls u + δu . This implies
In order for u to be a minimizing control, we must

that 11.3 Switching Curves

H (x; u1 ,L, u j + δu j ,L, u m ; p) ≥ H (x, u, p) (11.27) In the last section we met the idea of a switch in a control.
The time (and position) of switching from one extremum
for all admissible δu j and for j = 1,L, m . So we have value of the control to another does of course depend on the
initial starting point in the phase plane. Byconsidering a
established that on the optimal control H is minimized with specific example we shall show how these switching
respect to the control variables, u1 , u 2 ,L, u m . This is known positions define a switching curve.
as Pontryagin’s minimum principle.

Chapter 11 Bang-bang Control 55

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Suppose a system is described by the state variables x1 , x 2 x 2 − B = − k log | ( x1 − k ) / A |

Now if u = 1 , that is, k = 1 , then the trajectories are of the

where

x&1 = − x1 + u (11.28) form

x& 2 = u
x 2 − B = − log | ( x1 − 1) / A |
(11.29)

constraints −1 ≤ u ≤ 1 . Given that at t = 0, x1 = a, x 2 = b ,

Here u is the control variable which is subject to the
that is

x 2 = − log | x1 − 1 | +C
x1 = 0, x 2 = 0 in minimum time; that is, we wish to minimize
we wish to find the optimal control which takes the system to
(11.32)

∫
where C is constant. The curves for different values of C are
J = 1dt
T
(11.30) illustrated in Fig.11.4

u =1
0

while moving from (a, b) to (0,0) in the x1 − x 2 phase plane

x2

and subject to (11.28), (11.29) and

−1 ≤ u ≤ 1 (11.31)

Following the procedure outline in section 11.2, we introduce

the adjoint variables p1 and p 2 and the Hamiltonian
0 x1
H = 1 + p1 (− x1 + u ) + p 2 u
= u ( p1 + p 2 ) + 1 − p1 x1
1

Since H is linear in u, and | u |≤ 1 , H is minimized with

respect to u by taking

⎧+1 if p1 + p 2 < 0
u=⎨
A
⎩− 1 if p1 + p 2 > 0

Fig. 11.4 Trajectories for u = 1

depend on the sign changes in p1 + p 2 . As the adjoint
So the control is bang-bang and the number of switches will

equations, (11.23), are Follow the same procedure for u = −1 , giving

∂H x2 = log | x1 − 1 | +C
p& 1 = − = p1
(11.33)
∂x1 p1 = Ae
⇒
t

∂H p2 = B
, A and B are constant
p& 2 = − =0
and the curves are illustrated in Fig. 11.5.

∂x 2

and p1 + p 2 = Ae t + B , and this function has at most one

x2 B

sign change.

So we know that from any initial point (a, b) , the optimal

control will be bang-bang, that is, u = ±1 , with at most one
switch in the control. Now suppose u = k , when k = ±1 ,
then the state equations for the system are x1
−1
x&1 = − x1 + k 0
x& 2 = k
u = −1
We can integrate each equation to give

x1 = Ae −t + k
x2 = k t + B
, A and B are constants

The x1 − x 2 plane trajectories are found by eliminating t ,

Fig. 11.4 Trajectories for u = −1
giving

Chapter 11 Bang-bang Control 56

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

The basic problem is to reach the origin from an arbitrary Following the usual procedure, we form the Hamiltonian

H = 1 + p1 x2 + p 2 u
initial point. All the possible trajectories are illustrated in
Figs. 11.4 and 11.5, and we can see that these trajectories are (11.39)
only two possible paths which reach the origin, namely AO
in Fig. 11.4 and BO in Fig. 11.5. We minimize H with respect to u, where −1 ≤ u ≤ 1 , which
gives
x2 B
u = −1 ⎧+1 if p2 < 0
u=⎨
⎩− 1 if p2 > 0

The adjoint variables satisfy

⎧& ∂H
⎪ p1 = − ∂x = 0 ⎧p = A
⎪
x1
−1 ⎨ ⇒ ⎨ 1
0

∂ ⎩ p 2 = − At + B
1

⎪ p& = −
1
= − p1
⎪⎩ 2
H
∂x2

Since x2 (T ) is not specified, the transversality condition

becomes p 2 (T ) = 0 . Hence 0 = − At + B and p 2 = A(T − t ) .
A u =1 For 0 < t < T , there is no change in the sign of p 2 , and hence
no switch in u. Thus either u = +1 or u = −1 , but with no

Fig. 11.4 Trajectories for u = −1

switches. We have

x22 = 2( x1 − B) when u =1 (11.40)

= −2( x1 − B) u = −1
Combining the two diagrams we develop the Fig. 11.6. The
x22
AOB, we take u = +1 until the switching curve is reached,
curve AOB is called switching curve. For initial points below when (11.41)

followed by u = −1 until the origin is reached. Similarly for

the arrows being determined from x& 2 = u
These trajectories are illustrated in Fig. 11.7, the direction of
the points above AOB, u = −1 until the switching curve is
reached, followed by u = +1 until the origin is reached. So
we have solved the problem of finding the optimal trajectory u = +1 x2 x2 u = −1
from an arbitrary starting point.

Thus the switching curve has equation

⎧ log(1 + x1 ) x1 > 0
x2 = ⎨
for x1 x1
⎩− log(1 + x1 ) x1 < 0
(11.34) 0 0
for

11.4 Transversarlity conditions

To illustrate how the transversality conditions ( pi (T ) = 0 if

xi is not specified) are used, we consider the problem of
finding the optimum control u when | u | ≤ 1 for the system Fig. 11.7 Possible Optimal Trajectories
described by
x2
x&1 = x 2 (11.35) x1
x& 2 = u (11.36) 0
( a, b )

{
x1 (0) = a, x 2 (0) = b to any point on the x2 axis, that is,
which takes the system from an arbitrary initial point

x1 (T ) = 0 but x2 (T ) is not given, and minimize

T+

∫
T−{
J = 1.dt
T
(11.37) ( a, b ) A
0

subject to (11.35), (11.36), the above boundary conditions Fig. 11.8 Initial point below OA

We first consider initial points (a, b) for which a > 0 . For

on x1 and x 2 , and such that

−1 ≤ u ≤ 1 (11.38) points above the curve OA, there is only one trajectory,

Chapter 11 Bang-bang Control 57

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

u = −1 which reaches the x 2 -axis, and this must be the

optimal curve. For points below the curve OA, there are two
possible curves, as shown in Fig. 11.8.

From (11.36): x& 2 = u , that is x& 2 = ±1 , and the integrating

between 0 and T gives

x 2 (T ) − x 2 (0) = ±T

that is

T = | x 2 (T ) − x 2 (0) | (11.42)

Hence the modulus of the difference in final and initial values

of x 2 given the time taken. This is shown in the diagram as
T+ for u = +1 and T− for u = −1 . The complete set of
optimal trajectories is illustrated in Fig. 11.9.

x2
u = +1 u = −1

0 x1

Fig. 11.9 Optimal Trajectories to reach x 2 -axis in minimum

time

11.5 Extension to the Boltza problem

Chapter 11 Bang-bang Control 58