Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.

3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

1. System Dynamics and Differential Equations

1.1 Introduction 1.4 shows a tank with: inflow rate q i , outflow rate q o , head
level h , and the cross-section are of the tank is A .
- Investigation of dynamics behavior of idealized components,
and the collection of interacting components or systems are
equivalent to studying differential equations. inflow
- Dynamics behavior of systems ⇔ differential equations.

1.2 Some System Equations

h (head)
Inductance circuit: In electromagnetic theory it is known that, discharge q0
for a coil, in Fig.1.1, having an inductance L , the
electromotive force (e.m.f.) E is proportional to the rate of valve
change of the current I , at the instant considered, that is, Fig. 1.4

∫ If q = q o − q i is the net rate of inflow into a tank, over a

dI 1
E=L or I = E dt (1.1)
dt L period δ t , and δ h is the corresponding change in the head
L level, then q δ t = A δ h and in the limit as δ t → 0

∫ q dt
dh 1
E q=A or h = (1.4)
dt A
Fig. 1.1
All these equations (1.1)-(1.4) have something in common:
Capacitance circuit: Similarly, from electrostatics theory we
they can all be written in the form
know, in Fig. 1.2, that the voltage V and current I through a
capacitor of capacitance C are related at the instant t by dy
a =x (1.5)
dt
∫ I dt
1 dE
E= or I = C (1.2)
C dt Equation (1.5) is interesting because:
- its solution is also the solution to any one of the systems
C considered.
- it shows the direct analogies which can be formulate
between quite different types of components and systems.
- it has very important implications in mathematical
E modeling because solving a differential equation leads to
Fig. 1.2 the solution of a vast number of problems in different
disciplines, all of which are modeled by the same equation.
Dashpot device: Fig. 1.3 illustrates a dashpot device which
consists of a piston sliding in an oil filled cylinder. The motion Over small ranges of the variables involved the loss of
of the piston relate to the cylinder is resisted by the oil, and accuracy may be very small and the simplification of
this viscous drag can be assumed to be proportional to the calculations may be great. It is known that the flow rate
velocity of the piston. through a restriction such as a discharge valve is of the form
If the applied force is f (t ) and the corresponding displacement
is y (t ) , then the Newton’s Law of motion is q =V P

dy where P is the pressure across the valve and V is coefficient

f =µ (1.3)
dt dependent on the properties of the liquid and the geometry of
the valve. Fig.1.5 shows the relation between the pressure and
where the mass of the piston is considered negligible, and µ is the flow rate.
the viscous damping coefficient. q flow rate
q1
y (t ) assumed
linear
f (t )

Fig.1.3
P (pressure)
Liquid level: To analyze the control of the level of a liquid in P1
a tank we must consider the input and output regulations. Fig. Fig. 1.5
___________________________________________________________________________________________________________
Chapter 1 System Dynamics and Differential Equation 1
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Assume that in the neighborhood of the pressure P = P1 We can also write the set of simultaneous first order equation
(1.7) and (1.8) as one differential equation of the second order.
change in pressure On differentiating (1.7), we obtain
≈ R'
change in flow rate
1 & 1 ⎛ 1 1 ⎞&
h&&2 = h1 − ⎜
⎜R + R
⎟ h2
⎟ (1.10)
where R' is a constant called the resistance of the valve at the
A2 R1 A2 ⎝ 1 2 ⎠
point considered.
This type of assumption, which assumes that an inherently From (1.7) and (1.8),
nonlinear situation can be approximated by a linear one albeit
in a restricted range of the variable, is fundamental to many 1 1 1
applications of control theory. h&1 = q− h1 + h2
A1 A1 R1 A1 R1

At a given point, the pressure P = hρ g , define R = R' /( ρ g ) , 1 A 1 ⎛ 1 1 ⎞ 1

= q − 2 h&2 − ⎜ + ⎟ h2 + h2
we can rewrite the above relation as A1 A1 A1 ⎜⎝ R1 R 2 ⎟⎠ A1 R1
1 A 1
R = h / qo (1.6) = q − 2 h&2 − h2
A1 A1 A1 R 2

A two tank systems with one inflow (q) and two discharge Then
valves is shown in Fig. 1.6. 1 1 & 1 1 ⎛ 1 1 ⎞&
h&&2 = q− h2 − h2 − ⎜
⎜R + R
⎟ h2
⎟
inflow
A1 A2 R1 A1 R1 A1 A2 R1 R2 A2 ⎝ 1 2 ⎠
1 ⎡ 1 ⎛ 1 1 ⎞ 1 ⎤& 1
= q−⎢ ⎜ ⎟+
⎜R + R ⎟ A R ⎥ h2 − A A R R h2
q 1 2
A1 A2 R1 ⎢⎣ A2 ⎝ 1 2 ⎠ 1 1 ⎥⎦ 1 2 1 2
A1 A2 or
h1 ⎡ 1 ⎛ 1
⎞ 1 1 ⎤& 1 1
R1 h2 R2 h&&2 + ⎢ ⎜
⎟+
⎟ A R ⎥ h 2 + A A R R h2 = A A R q
⎜R +R
⎣⎢ A2 ⎝ 1
⎠ 1 1 ⎦⎥
2 1 2 1 2 1 2 1
(1.11)
Fig. 1.6 Equation (1.11) is a differential equation of order 2 of the
form &y& + a1 y& + a 2 y = b0 x .
For tank 1
dh1 1 1.3 System Control
A1 = q− (h1 − h2 )
dt R1
Consider a control system: a tank with an inflow q i and a
dh 1 1 1
⇒ h&1 = 1 = q− h1 + h2 (1.7) discharge q o in Fig. 1.7
dt A1 A1 R1 A1 R1

For tank 2
float
dh 1 1 lever
A2 2 = (h1 − h2 ) − h2
dt R1 R2 R
dh 1 1 ⎛ 1 1 ⎞ inflow outflow
⇒ h&2 = 2 = h1 − ⎜
⎜R + R
⎟ h2
⎟ (1.8)
dt A2 R1 A2 ⎝ 1 2 ⎠ regulator valve
Fig. 1.7
We can write (1.7) and (1.8) in matrix form as follows
The purpose of controller is to maintain the desired level h .
⎡ 1 1 ⎤
− ⎡ 1 ⎤ The control system works very simply. Suppose for some
⎡ h&1 ⎤ ⎢⎢ A1 R1 A1 R1 ⎥
⎥ ⎡ h1 ⎤ + ⎢ A ⎥ q reason, the level of the liquid in the tank rises above h . The
⎢& ⎥ = ⎢ 1 1 ⎛ 1 ⎞ ⎢
1 ⎥⎣ 2⎦ ⎢ 1⎥
h ⎥
⎣ h2 ⎦ − ⎜ ⎟ float senses this rise, and communicates it via the lever to the
⎢ A R ⎜ R + R ⎟⎥ ⎣ 0 ⎦
⎣ 2 1 A2 ⎝ 1 2 ⎠⎦ regulating valve which reduces or stops the inflow rate. This
situation will continue until the level of the liquid again
stabilizes at its steady state h .
that is, in the form
The above illustration is a simple example of a feedback
control system. We can illustrate the situation schematically
h& = A h + B q (1.9) by use of a block diagram as in Fig. 1.8.

where h1 ε x h2
⎡ 1 1 ⎤ regulator plant
⎡ h&1 ⎤ ⎢− A R A1 R1 ⎥ ⎡1 ⎤
h= ⎢& ⎥ , A= ⎢
1 1 ⎥, B=⎢A ⎥ h2
⎢ 1 1 ⎛ 1 1 ⎞⎥ ⎢ 1⎥
⎣ 2⎦
h
− ⎜ ⎟
⎢ A R ⎜ R + R ⎟⎥ ⎣ 0 ⎦
⎣ 2 1 A2 ⎝ 1
Fig. 1.8
2 ⎠⎦

___________________________________________________________________________________________________________
Chapter 1 System Dynamics and Differential Equation 2
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

with theory is to design – in terms of the transfer function – a

system which satisfies certain assigned specifications. This
h1 : desired value of the head objective is primarily achieved by a trial and error approach.
h2 : actual value of head
Modern control theory is not only applicable to linear
ε = h1 − h2 : error autonomous systems but also to time-varying systems and it is
useful when dealing with nonlinear systems. In particularly it
The regulator receives the signal ε which transforms into a is applicable to MIMO systems – in contrast to classical
movement x of the lever which in turn influences the position control theory. The approach is based on the concept of state.
of the regulator valve. It is the consequence of an important characteristic of a
To obtain the dynamics characteristics of this system we dynamical system, namely that its instantaneous behavior is
consider a deviation ε from the desired level h1 over a short dependent on its past history, so that the behavior of the
system at time t > t 0 can be determined given
time interval dt .
If q i and q o are the change in the inflow and outflow, from
(1) the forcing function (that is, the input), and
(1.4), (2) the state of the system at t = t 0

dε
A = qi − q o In contrast to the trial and error approach of classical
dt control, it is often possible in modern control theory to
make use of optimal methods.
where (1.6), q o = ε / R . So that the equation can be written as

dε
AR + ε = R qi
dt

the change in the inflow, depends on the characteristics of the

regulator valve and may be of the form

q i = K ε , K is a constant

Another type of control system is known as an open loop

control system, in which the output of the system is not
involved in its control. Basically the control on the plant is
exercised by a controller as shown in Fig. 1.9.

input output
controller plant

Fig. 1.9

1.4 Mathematical Models and Differential Equations

Many dynamic systems are characterized by differential

equations. The process involved, that is, the use of physical
laws together with various assumptions of linearity, etc., is
known as mathematical modeling.

Linear differential equation

&y& + 2 y& − 3 y = u → time-invariant (autonomous) system

&y& − 2 t y& + y = u → time-varying (non-autonomous) system

Nonlinear differential equation

&y& + 2 y& 2 − 2 y = u
&y& − y 2 y& + y = u

1.5 The Classical and Modern Control Theory

Classical control theory is based on Laplace transforms and

applies to linear autonomous systems with SISO. A function
called transfer function relating the input-output relationship
of the system is defined. One of the objectives of control

___________________________________________________________________________________________________________
Chapter 1 System Dynamics and Differential Equation 3
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

2. Transfer Functions and Block Diagrams

2.1 Introduction u = f (t ) du = d [ f (t )]
Consider ⇒
- Review of Laplace transform dv = e − st dt v = − 1s e − st
- Using Laplace transform to solve a differential equation then
∞
2.2 Review of Laplace Transforms F ( s) =
∫ 0
f (t ) e − st dt
∞ ∞⎛
⎡ ⎛ 1 ⎞⎤ 1 − st ⎞
∫
Definition: The Laplace transform of f (t ) , a sectionally
= ⎢ f (t ) ⎜ − e − st ⎟⎥ − ⎜ − e ⎟ d [ f (t )]
continuous function of time, denoted by L [ f (t )] , is defined as ⎣ ⎝ s ⎠⎦ 0 0 ⎝ s ⎠
∞ ∞
⎡ ⎤
∫
∞ 1 1 d
= − ⎢ f (t ) e − st ⎥ + [ f (t )] e − st dt
∫
− st
L [ f (t )] = f (t ) e dt = F ( s ) (2.1) ⎣ s ⎦0 s 0 dt
0
f ( 0) 1 ⎡ d ⎤
where s = σ + iω , σ and ω are real variables. f (t ) is called = + L ⎢ f (t ) ⎥
s s ⎣ dt ⎦
−1
inverse transform of F (s) , such as L [ F ( s )] = f (t ) . ⎡d ⎤
Hence L ⎢ f (t )⎥ = sF ( s) − f (0)
Example 2.1 _______________________________________ ⎣ dt ⎦

⎧0 for t < 0 (4) Integration

Consider the step functions f (t ) = ⎨ , c is a
⎩c for t ≥ 0
⎡ t ⎤ F (s)
∫
constant as shown in Fig. 2.1. Find the Laplace transform
L ⎢ f ( x )dx ⎥ = (2.5)
⎣ 0 ⎦ s
f (t )
c Proof
⎡ t ⎤ ∞⎛ t ⎞
∫ ∫ ⎜⎜⎝ ∫ f ( x)dx ⎟⎟⎠ e
− st
L ⎢ f ( x)dx ⎥ = dt
⎣ 0 ⎦ 0 0
∞
0 t ⎡⎛ t ⎞⎛ 1 ⎞⎤
Fig. 2.1 = ⎢⎜⎜
⎣⎢⎝
∫0
f ( x)dx ⎟⎟ ⎜ − e − st ⎟⎥
⎠ ⎝ s ⎠⎦⎥ 0
∞
∞ ⎡1 ⎤
∞⎛ 1 − st ⎞ ⎡ t ⎤
∫ ∫ ∫
c c
F (s) = c e − st dt = −c ⎢ e −st ⎥ = − lim e − st + − ⎜ − e ⎟ d ⎢ f ( x)dx ⎥
0 ⎣s ⎦0 s t →∞ s 0 ⎝ s ⎠ ⎣ 0 ⎦
∞⎛ 1 − st ⎞ d ⎡ t ⎤
So long as σ (=real part of s ) is positive, lim e − st = 0 , hence
t →∞
=−
∫ 0
⎜ − e ⎟ ⎢ f ( x )dx ⎥ dt
⎝ s ⎠ dt ⎣ 0 ⎦
∫
1 ∞
F ( s) =
c
s
(so long as σ > 0 ) =
s 0 ∫
f (t ) e − st dt
__________________________________________________________________________________________
F (s)
=
Some properties of Laplace transforms s
A is a constant and L [ f (t )] = F ( s )
(5) The delay function
(1) L [ A f (t )] = A L [ f (t )] = A F ( s )
L [ f (t − α )] = e −α s F ( s ) (2.6)
(2) L [ f1 (t ) + f 2 (t )] = L [ f1 ( s )] + L [ f 2 ( s )] = F1 ( s ) + F2 ( s )
Proof
Given the Laplace transform F (s ) of a function f (t ) such that
(3) Differential
f (t ) = 0 for t < 0 . We wish to find the Laplace transform of
⎡d ⎤ f (t − α ) where f (t − α ) = 0 for t < α .(Fig. 2.2)
L ⎢ f (t )⎥ = sF ( s ) − f (0) (2.2)
⎣ dt ⎦
f (t ) f (t − α )
Proof
d (uv) = udv + vdu
∞ ∞ ∞
⇒
∫ 0 ∫ ∫
d (uv) = udv + vdu
0 0
∞ ∞
0 α
∫ udv = [uv] − ∫ vdu
∞ t t 0
⇒ 0
0 0 Fig. 2.2
___________________________________________________________________________________________________________
Chapter 2 Transfer Functions and Block Diagrams 4
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Consider the function (8) Initial value theorem

∞ This theorem makes it possible to find f (0) directly from
F ( s) =
∫ 0
f ( x) e − sx dx .
F (s ) , without actually evaluating the inverse Laplace
Let x = t − α , then transform. The theorem states that provided
∞ lim sF ( s ) exists, then
F ( s) =
∫ 0
f (t − α ) e − s (t −α ) dt s→∞
f (0) = lim sF ( s ) (2.9)
∞ s→∞

∫
αs − st
=e f (t − α ) e dt
0
∞
= eα s L [ f (t − α )] Since L [ f ' (t )] = sF ( x) − f (0) and lim
s→∞ 0 ∫ f ' (t ) e −st dt = 0 ,

hence 0 = lim sF ( s ) − f (0) , and (2.9) follows.

hence L [ f (t − α )] = e −α s F ( s ) t →∞

(6) The unit impulse function (9) Final value theorem

Consider the function shown in Fig. 2.3 This theorem makes it possible to obtain f (∞) directly from
⎧1 F (s ) . Provided the indicated limits exist
⎪ 0≤t ≤c
f (t ) = ⎨ c
⎪0 t>c lim f (t ) = f (∞) = lim sF ( s ) (2.10)
⎩ s→∞ s→0
f (t )
Indeed, as above
1 ∞
c L [ f ' (t )] =
∫ 0
f ' (t ) e − st dt = sF ( s ) − f (0)

Now
∞ ∞
∫ ∫ f ' (t )dt = f (t )
∞
c 0
t lim f ' (t ) e − st dt = 0
= f ( ∞ ) − f ( 0)
s→0 0 0
Fig. 2.3
We can express f (t ) in terms of step functions as Hence
f (∞) − f (0) = lim sF ( s) − f (0) , and (2.10) follows.
1 1 s→0
f (t ) = u (t ) − u (t − c)
c c
2.3 Applications to differential equations
that is, a step function beginning at t = 0 minus a step
By taking the Laplace transform of a differential equation it is
function beginning at t = c . This leads to the definition of the
transformed into an algebraic one in the variable s . This
unit impulse function or the Dirac delta function denoted
equation is rearranged so that all the terms involving the
by δ (t ) defined by dependent variable are on one side and the solution is obtained
by taking the inverse transform.
u (t ) − u (t − c)
δ (t ) = lim
c→0 c dy d2y
Let denote Dy = , D2 y = 2 , …
dt dt
Using the results of Example 2.1 and (2.6)
Example 2.6 _______________________________________
1
L [δ (t )] = lim (1 − e −c s )
c→0 c s Solve the equation D 2 y − 2 Dy + y = e 3t subject to the initial
d conditions y (0) = 1 and y ' (0) = 2 .
(1 − e −c s )
= lim dc
c→0 d Taking Laplace transform
(c s )
dc
s e −cs L [ D 2 y ] − L [2 Dy ] + L [ y ] = L [e 3t ]
= lim =1 (2.7)
c→0 s 1
⇒ [ s 2Y ( s ) − sy (0) − y ' (0)] − 2[ sY ( s ) − y (0)] + Y ( s ) =
s−3
(7) Time scale change 1
⇒ ( s 2 − 2 s + 1) Y ( s ) − s =
Suppose that, L [ f (t )] = F ( s ) , find L [ f (t / α )] , α > 0 s −3
s 2 − 3s + 1
⎡ ⎛ t ⎞⎤ ∞ ⎛t ⎞ ⇒ Y (s) =
L ⎢ f ⎜ ⎟⎥ =
⎣ ⎝ α ⎠⎦ ∫ 0
f ⎜ ⎟ e − s t dt
⎝α ⎠
( s − 3)( s − 1) 2
3/ 4 1/ 2 1/ 4
∞ ⇒ Y (s) = + +
∫
− sα x t s − 1 ( s − 1) 2 s − 3
= f ( x) e α dx, x=
0 α
= α F (α s ) (2.8) Taking inverse transform

___________________________________________________________________________________________________________
Chapter 2 Transfer Functions and Block Diagrams 5
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

−1 3 −1 ⎡ 1 ⎤ 1 −1
⎡ 1 ⎤ 1 −1 ⎡ 1 ⎤ Hence
L [Y ( s)] = L ⎢ s − 1⎥ + 2 L ⎢ ⎥+ L ⎢ s − 3⎥
4 ⎣ ⎦ ⎣⎢ ( s − 1) ⎦⎥ 4
2
⎣ ⎦ H (s) R
G(s) = =
3 1 1 Q( s) 1 + A R s
⇒ y (t ) = e t + te t + e 3t
4 2 4
__________________________________________________________________________________________
Note that G (s) is dependent only on the system characteristics
2.4 Transfer functions such as A and R .
__________________________________________________________________________________________

u (t ) y (t ) Example 2.8 _______________________________________

system
Find the transfer function of the system described by the
Fig. 2.6
equation D 2 y + 3Dy + 2 y = u where u (t ) is the input and
y (t ) is the output.
In general form the n th order SISO system (Fig. 2.6) may have
an associated differential equation
Taking Laplace transform both sides with zero-initial
conditions, we obtain
a0 D n y + a1 D n−1 y + K + a n y = b0 D m u + b1 D m−1u + K + bm u
(2.11) ( s 2 + 3s + 2) Y ( s ) = U ( s )
where
u (t ) : input H (s) 1
y (t ) : output. Hence G ( s ) = =
Q( s ) ( s + 1)( s + 2)
n ≥ m : proper system __________________________________________________________________________________________

n < m : improper system

2.5 Block Diagrams
If all initial conditions are zero, taking the Laplace transform
of (2.11) yields Block diagrams
The equation Y ( s) = G ( s ) U ( s) is represented by the Fig.2.7
(a0 s n + a1s n−1 + K + a n ) Y ( s ) = (b0 s m + b1s m−1 + K + bm )U ( s )
U (s ) Y (s )
Definition The transfer function G (s ) of a linear autonomous G(s)
system is the ratio of the Laplace transform of the output to
Fig. 2.7
the Laplace transform of the input, with the restriction that all
Sensing diagram
initial conditions are zero.
Using for representation of addition and/or subtraction of
signals.
From the above equation, it follows that
R (s ) E (s) = R( s) − C ( s)
Y ( s ) b0 s m + b1s m−1 + K + bm
G( s) = = (2.12)
U ( s ) a0 s n + a1s n−1 + K + a n
C (s )
The transfer function G (s ) depends only on the system
parameters and is independent of the input. Fig. 2.8

(2.12) is also written as Identity diagram

To show the relation Y ( s ) = Y1 ( s ) = Y2 ( s)
Y ( s) = G ( s)U ( s) (2.13)
Y (s ) Y1 ( s ) = Y ( s )
Example 2.7 _______________________________________
Obtain the transfer function for a fluid tank having a cross-
sectional area A with one inflow q (t ) , a head h(t ) and with Y2 ( s ) = Y ( s )
an outflow valve offering a resistance R .
Fig. 2.9
dh 1 1 1
Using equation (1.7): h&1 = 1 = q− h1 + h2 ,
Reduction of block diagram
dt A1 A1 R1 A1 R1
the system can be described as follows
Y (s)
= G ( s ) = B ( s ) A( s ) (2.15)
dh 1 dh U ( s)
A = q − h or A R +h=qR
dt R dt
U (s ) X (s ) Y (s ) U (s ) Y (s )
On taking the Laplace transform with zero initial conditions A(s) B(s) B(s)A(s)

A R s H ( s) + H ( s) = Q(s) R Fig. 2.10

___________________________________________________________________________________________________________
Chapter 2 Transfer Functions and Block Diagrams 6
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Block in parallel
C ( s ) = A( s ) R ( s ) − A( s ) B ( s)C ( s )
C (s) A( s )
⇒ G(s) = = (2.16)
R( s ) 1 + A( s ) B( s )

R (s ) E (s ) C (s )
A(s) R (s ) C (s )
A( s )
G ( s )=
1 + A( s ) B ( s )
B(s)

Fig. 2.11

Example 2.9 _______________________________________

Obtain the system transfer function for the positive feedback
system shown in Fig. 2.12

R (s ) E (s ) C (s )
A(s)

B(s)

Fig. 2.12

E ( s) = R( s) + B( s) C ( s)
C ( s ) = A( s ) E ( s )
⇒ C ( s ) [1 − A( s ) B ( s )] = A( s ) R ( s )
C (s) A( s )
and G ( s ) = =
R( s ) 1 − A( s ) B ( s )

A special case and an important case of a feedback system is

the unity feedback system shown in Fig. 2.1.3

R (s ) E (s ) C (s )
A(s)

Fig. 2.13

The unity feedback system transfer function is

A( s )
G ( s) = (2.17)
1 + A( s )
__________________________________________________________________________________________

Example 2.10_______________________________________
K
Consider a system having the transfer function A( s ) = .
s−2

Time response of this system to a unit impulse

Y ( s ) = A( s ) U ( s ), U ( s ) = 1
⇒ y (t ) = K e 2t

For a unity feedback closed loop system

A( s ) K
G(s) = =
1 + A( s) s + ( K − 2)
⇒ y (t ) = K e −( K − 2) t
__________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 2 Transfer Functions and Block Diagrams 7
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

3. State Space Formulation

3.1 Introduction (Review) where

⎡λ1 0 L 0 ⎤
⎢ ⎥
Λ = ⎢ 0 λ2
3.1.1 Eigenvalues and eigenvectors L 0
Consider a matrix A of order n × n . If there exists a vector M M O M ⎥ = diag{λ1 , λ2 ,L, λn }
⎢0 0 λn ⎥⎦
x ≠ 0 and a scalar λ such that ⎣ L

Ax = λ x is called eigenvalue matrix of A .

Hence
then x is called an eigenvector of A . λ is called an eigenvalue Λ = X −1 A X
of A .
3.1.2 The Cayley-Hamilton theorem
The above equation can be written in the form Given a square matrix A (of order n × n ) and integers r and s ,
then
[ λ I − A]x = 0
Ar A s = Ar + s = A s+r = A s Ar
where I is the unit matrix (of order n × n ). It is known that the This property leads to the following definition.
above homogeneous equation has non-trivial (that is, non-
zero) solution only if the matrix [ λ I − A ] is singular, that is if Definition
Corresponding to a polynomial in a scalar variable λ
det(λ I − A) = 0
f (λ ) = λk + b1λk −1 + K + bk −1λ + bk
This is an equation in λ of great importance. We denoted it
by c(λ ) , so that define the (square) matrix f ( A) , called a matrix polynomial,
by
c(λ ) = det(λ I − A) = 0
f ( A) = A k + b1 A k −1 + K + bk −1 A + bk I
It is called the characteristic equation of A . Written out in full,
this equation has the form
where I is the unit matrix of order n × n .

⎡λ − a11 a12 L a1n ⎤

⎢ a 21 λ − a 22 L a 2n ⎥ = 0 For example, corresponding to f (λ ) = λ2 − 2λ + 3 and
c (λ ) = ⎢
M ⎥⎥
⎢ M A = ⎡ 1 1⎤ , we have
M O
⎣ a n1 an 2 L λ − a nn ⎦ ⎢⎣− 1 3⎥⎦

On expansion of determinant, c(λ ) is seen to be a polynomial

f ( A) = ⎡⎢ 1 1⎤⎥ ⎡⎢ 1 1⎤⎥ − 2 ⎡⎢ 1 1⎤⎥ + 3⎡⎢1 0⎤⎥ = ⎡⎢ 1 2⎤⎥
of degree n in λ , having the form ⎣− 1 3⎦ ⎣− 1 3⎦ ⎣− 1 3⎦ ⎣0 1⎦ ⎣− 2 5⎦

c(λ ) = λn + b1λn−1 + K + bn−1λ + bn Of particularly interest to us are polynomials f having the

property that f ( A) = 0 . For every matrix A one such
= (λ − λ1 )(λ − λ2 ) L (λ − λn )
polynomial can be found by the Cayley-Hamilton theorem
=0 which states that: Every square matrix A satisfies its own
characteristic equations.
λ1 , λ2 ,L, λn , the roots of c(λ ) = 0 , are the eigenvalues of A .
For the above example, the characteristic equation of A is
Assuming that A has distinct eigenvalue λ1 , λ2 ,L, λn , the
corresponding eigenvectors x1 , x 2 ,L, x n are linearly indepen- c(λ ) = (λ − λ1 )(λ − λ2 ) = λ2 − 4λ + 4
dent. The (partitioned) matrix
where λ1 , λ2 are eigenvalues of A . So that
X = [x1 x 2 L x n ]
c( A) = A 2 − 4 A + 4 I
is called the modal matrix of A. Since = ⎡ 1 1⎤ ⎡ 1 1⎤ − 4 ⎡ 1 1⎤ + 4 ⎡1 0⎤
⎢⎣− 1 3⎥⎦ ⎢⎣− 1 3⎥⎦ ⎣⎢− 1 3⎥⎦ ⎣⎢0 1⎥⎦
A xi = λ xi (i = 1,2,L, n)
= ⎡⎢0 0⎤⎥
⎣0 0 ⎦
it follows that
In fact the Cayley-Hamilton theorem guarantees the existence
AX = ΛX of a polynomial c(λ ) of degree n such that c( A) = 0 .
___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 8
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

3.2 State Space Forms ⎡ x1 ⎤

y = [1 0 0] ⎢ x 2 ⎥
Consider the system equation in the form ⎢x ⎥
⎣ 3⎦

y ( n ) + a1 y ( n −1) + K + a n −1 y& + a n y = u (3.1) (2) Diagonal form

It is assumed that y (0), y& (0), L , y ( n−1) (0) are known. If we

Let
x1 = 1 y + 1 &y&
define x1 = y , x 2 = y& , L , x n = y ( n −1) , (3.1) is written as 5 5
x 2 = 15 (2 + i ) y − 12 iy& + 10
1 ( −1 + 2i ) &y& (3.6)
x&1 = x 2 x 3 = 15 (2 − i ) y + 12 iy& − 10
1 (1 + 2i ) &y&

x& 2 = x 3
where i 2 = −1 . Then
M
x& n −1 = x n ⎡ x&1 ⎤ ⎡2 0 0 ⎤ ⎡ x1 ⎤ 1 ⎡ 2 ⎤
x& n = − a n x1 − a n−1 x 2 − K − − a1 x n + u ⎢ x& 2 ⎥ = ⎢0 i 0 ⎥ ⎢ x 2 ⎥ + ⎢− 1 + 2i ⎥ u (3.7)
⎢ x& ⎥ ⎢0 0 − i ⎥ ⎢ x ⎥ 10 ⎢ − 1 − 2i ⎥
⎣ 3⎦ ⎣ ⎦⎣ 3⎦ ⎣ ⎦
which can be written as a vector-matrix differential equation and
⎡ x1 ⎤
⎡ x&1 ⎤ ⎡ 0 1 L 0 0 ⎤ ⎡ x1 ⎤ ⎡0⎤ y = [1 1 1] ⎢ x 2 ⎥
⎢ x& 2 ⎥ ⎢ 0 0 L 0 0 ⎥ ⎢ x 2 ⎥ ⎢0 ⎥ ⎢x ⎥
⎢ M ⎥=⎢ M ⎣ 3⎦
M O M M ⎥ ⎢ M ⎥ + ⎢M⎥ u (3.2)
⎢ x& ⎥ ⎢ 0 0 L 0 1 ⎥ ⎢ x n −1 ⎥ ⎢0⎥
__________________________________________________________________________________________

⎢ n −1 ⎥ ⎢ L − a 2 − a1 ⎥⎦ ⎢⎢ x ⎥⎥ ⎢⎣1 ⎥⎦
⎢⎣ x& n ⎥⎦ ⎣− a n − a n−1 ⎣ n ⎦ In general form, a MIMO system has state equations of the
form
that is, as x& = A x + B u , where x , A and B are defined in
equation (3.2). The output of the system ⎧ x& = A x + B u
⎨ (3.14)
⎩ y = C x + Du
⎡ x1 ⎤
⎢ ⎥
y = [1 0 L 0] ⎢ x 2 ⎥ (3.3) and a SISO system has state equations of the form as (3.4).
M
⎢x ⎥
⎣ n⎦ 3.3 Using the transfer function to define state variables

that is, as, where C = [1 0 L 0] . The combination of It is sometimes possible to define suitable state variables by
considering the partial fraction expansion of the transfer
equations (3.2) and (3.3) in the form function. For example, given the system differential equation

⎧ x& = A x + B u &y& + 3 y& + 2 y = u& + 3u

⎨ (3.4)
⎩y = C x
The corresponding transfer function is
are known as the state equations of the system considered. The
matrix A in (3.2) is said to be in companion form. The Y ( s) s+3 2 1
G(s) = = = −
components of x are called the state variables x1 , x 2 , L , x n . U ( s ) ( s + 1)( s + 2) s + 1 s + 2
The corresponding n-dimensional space called the state space.
Any state of the system is represented by a point in the state Hence
space.
Y (s) = X 1 (s) + X 2 (s)
Example 3.1 _______________________________________
2U ( s )
X 1 (s) =
Obtain two forms of state equations of the system defined by s +1
U (s)
&y&& − 2 &y& + y& − 2 y = u X 2 (s) = −
s+2

where matrix A corresponding to one of the forms should be On taking the inverse Laplace transforms, we obtain
diagonal.
x&1 = − x1 + 2u
(a) A in companion form
x& 2 = −2 x 2 − u
Let the state variable as x1 = y , x 2 = y& , x 3 = &y& . Then

or
⎡ x&1 ⎤ ⎡0 1 0⎤ ⎡ x1 ⎤ ⎡0⎤
⎢ x& 2 ⎥ = ⎢0 0 1 ⎥ ⎢ x 2 ⎥ + ⎢0⎥ u ⎡ x&1 ⎤ ⎡− 1 0 ⎤ ⎡ x1 ⎤ ⎡ 2 ⎤
⎢ x& ⎥ ⎢2 − 1 2⎥ ⎢ x ⎥ ⎢1 ⎥
(3.5) ⎢⎣ x& 2 ⎥⎦ = ⎢⎣ 0 − 2⎦⎥ ⎢⎣ x 2 ⎥⎦ + ⎢⎣− 1⎥⎦ u
⎣ 3⎦ ⎣ ⎦⎣ 3⎦ ⎣ ⎦
⎡x ⎤
y = [1 1] ⎢ 1 ⎥
and ⎣x2 ⎦
___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 9
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

We can of course make a different choice of state variables, t

∫ e τ dτ
3τ
for example x = 2e −3t + 4e −3t
0

Y (s) = 2 X 1 (s) − X 2 (s) = 2e −3t + 4e −3t ( te − e

1
3
3t 1 3t
9
+ 19 )
U (s) = 22 e −3t + 43 t − 94
X 1 (s) = 9
s +1 __________________________________________________________________________________________

U (s)
X 2 (s) = To use this method to solve the vector matrix equation (3.15),
s+2
we must first define a matrix function which is the analogue of
then
the exponential, and we must also define the derivative and the
integral of a matrix.
x&1 = − x1 + u ∞

∑ n! z
&x 2 = −2 x 2 + u 1
Definition Since e z = n
(all z ), we define
0
Now the state equations are ∞

∑ n! A
1 1 2 1
eA = n
= A0 + A + A + K = I + A + A2 + K
2! 2!
⎡ x&1 ⎤ ⎡− 1 0 ⎤ ⎡ x1 ⎤ ⎡1⎤ 0
⎢⎣ x& 2 ⎥⎦ = ⎢⎣ 0 − 2⎦⎥ ⎢⎣ x 2 ⎥⎦ + ⎢⎣1⎥⎦ u
(where A0 ≡ I ) for every square matrix A .
⎡x ⎤
y = [2 − 1] ⎢ 1 ⎥ Example 3.4 _______________________________________
⎣ x2 ⎦
⎡ 1 0 0⎤
3.4 Direct solution of the state equation Evaluate e At given A = ⎢− 1 0 0⎥
⎢⎣ 1 1 1 ⎥⎦
By a solution to the state equation

x& = A x + B u (3.15) We must calculate A 2 , A3 ,…

⎡ 1 0 0⎤
we mean finding x at any time t given u (t ) and the value of A 2 = ⎢ − 1 0 0 ⎥ = A = A3 = K
x at initial time t 0 = 0 , that is, given x(t 0 ) = x 0 . It is ⎢⎣ 1 1 1⎥⎦
instructive to consider first the solution of the corresponding
scalar differential equation It follows that

x& = a x + b u (3.16) 1 2 2 1 33
e At = I + A t + A t + A t +K
2! 3
given x = x0 at t = 0 . The solution of (3.15) is found by an ⎛ 1 2 1 3 ⎞
= I + A ⎜ t + t + t + K⎟
analogous method. ⎝ 2 ! 3 ⎠
Multiplying the equation by the integrating factor e − at , we = I + A(e t − 1)
obtain
⎡1 0 0⎤ ⎡⎢ e − 1 0 ⎤
t
0
⎥
= ⎢0 1 0⎥ + ⎢ − e t + 1 0 0 ⎥
e − at ( x& − ax ) = e − at b u
⎣⎢0 0 1⎥⎦ ⎢⎣ e − 1 e − 1 e − 1⎦⎥
t t t
d −at
or [e x] = e −at b u ⎡ et
dt 0 0⎤
⎢ ⎥
= ⎢− e t + 1 1 0⎥
t
− t
− t
⎣⎢ ⎦⎥
Integrating the result between 0 and t gives e 1 e 1 e
__________________________________________________________________________________________
t
∫e
−at − aτ
e x − x0 = b u (τ ) dτ In fact, the evaluation of the matrix exponential is not quite as
0
difficult as it may appear at first sight. We can make use of the
that is
Cayley-Hamilton theorem which assures us that every square
t
∫ e b u(τ ) dτ
−a( t −τ )
x= e at x0 + (3.17) matrix satisfies its own characteristic equation.
123 0
complementary function 144424443
particular int ergral One direct and useful method is the following. We know that
if A has distinct eigenvalues, say λ1 , λ2 ,L, λn , there exists a
Example 3.3 _______________________________________ non-singular matrix P such that
Solve the equation
⎡λ1 0 L 0⎤
⎢ 0 ⎥ = diag{λ , λ ,L, λ } = Λ
x& + 3 x = 4t P A P = ⎢ 0 λ2
−1 L
M M O M ⎥ 1 2 n
⎢0 0 L λn ⎥⎦
given that x = 2 when t = 0 . ⎣

We then have A = P Λ P −1 , so that

Since a = −3 , b = 4 and u = t , on substituting into equation
(3.17) we obtain A 2 = ( PΛP −1 )( PΛP −1 ) = PΛ ( P −1P )ΛP −1 = PΛ2 P −1
___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 10
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

e At = ⎡⎢ 2 2 ⎤⎥ e −t + ⎡⎢−1 −2⎤⎥ e −2t

and in general
⎣− 1 − 1⎦ ⎣1 1⎦
A r = PΛr P −1 (r = 1,2,L)
In the solution to the unforced system equation x = e At x 0 , the
If we consider a matrix polynomial, say f ( A) = A − 2 A + I , 2
eigenvalues λ1 , λ2 ,L, λn are called the poles and
we can write it as
e λ1t , e λ2t ,L, e λnt are called the modes of the system.
__________________________________________________________________________________________
f ( A) = PΛ2 P −1 − 2 PΛP −1 + P I P −1
We now define the derivative and the integral of a
= P ( Λ2 − 2Λ + I ) P −1
matrix A(t ) whose elements are functions of t .
= P f (Λ ) P −1
Definition
= P diag{ f (λ1 ), f (λ2 ),L, f (λn )}P −1 Let A(t ) = [aij (t )], then
d d
Since (for the above example) (1) A(t ) = A& (t ) = [ (aij )], and
dt dt
f ( λ ) = λ 2 − 2λ + I
⎡λ12 ⎤
(2)
∫ A(t )dt = [∫ a ij (t ) dt ],
0 L 0 ⎡λ1 0 L 0 ⎤ ⎡1 0 L 0⎤ that is, each element of the matrix is differentiated (or
⎢ ⎥ ⎢
=⎢ 0 λ 2
L 0 ⎥ − 2 ⎢ 0 λ2 L 0 ⎥ + ⎢0 1 L 0⎥ integrated).
M ⎥ ⎢M M
2
⎢M M O M ⎥ M M O O M⎥
⎢ λn ⎥⎦ ⎢⎣0 0 L 1⎥⎦ ⎡ 6t sin 2t ⎤
⎢0 0 L λ2n ⎥⎦ ⎣ 0 0 L
For example, if A = ⎢ 2
⎣ 3 ⎥⎦
, then
⎣t + 2
⎡λ12 − 2 0 L 0 ⎤
⎢
λ22 − 2λ2 + 1
⎥ A& = ⎡⎢ 6 2 cos 2t ⎤⎥ , and
=⎢ 0 L 0 ⎥ ⎣2t 0 ⎦
⎢ M M O M ⎥
⎢ 0 ⎡ 3t 2 − 12 cos 2t ⎤
L λ2n − 2λn + 1⎥⎦
⎣
∫
0 Adt = ⎢ 3 ⎥ + C ( C is a constant matrix)
= diag{ f (λ1 ), f (λ2 ),L, f (λn )} ⎢⎣ 13 t + 2t 3t ⎥⎦

From the definition a number of rules follows: if α and β are

The results holds for the general case when f (x) is a
constants, and A and B are matrices,
polynomial of degree n . In generalizing the above, taking
d
(i) (α A + β B ) = α A& + β B&
dt
f ( A) = e At , we obtain
b b b
e At = P diag{e λ1t , e λ2t ,L, e λnt } P −1 (3.18) (ii)
∫ (α A + β B)dt = α ∫ Adt + β ∫ Bdt
a a a
d
Example 3.5 _______________________________________ (iii) ( A B ) = A B& + A& B
dt
Find e At given A = ⎡ 0 2 ⎤
d At
⎢⎣− 1 − 3⎥⎦ (iv) If A is a constant matrix, then (e ) = Ae At
dt

The eigenvalues of A are λ1 = −1 and λ2 = −2 . It can be ⊗ Note that although the above rules are analogous o the
rules for scalar functions, we must not be dulled into
verified that P = ⎡ 2 1 ⎤ and that P −1 = ⎡ 1 1 ⎤ . ( P can accepting for matrix functions all the rules valid for scalar
⎢⎣− 1 − 1⎥⎦ ⎢⎣− 1 − 2⎥⎦
functions. For example, although d ( x n ) = n x n−1 x& in general, dt
be found from A = P Λ P −1 ). Using (3.18), we obtain
it is not true that d
dt
( A n ) = n A n−1 A& .For example,
⎡ −t 0 ⎤ ⎡ 1 1 ⎤ ⎡ 2 ⎤
e At = ⎡ 2 1 ⎤ ⎢e when A = ⎢t 2t ⎥ ,
⎢⎣− 1 − 1⎥⎦ 0 e −2t ⎥ ⎢⎣− 1 − 2⎥⎦ ⎣0 3⎦
⎣ ⎦
⎡ 3 ⎤
( A 2 ) = ⎢4t 6t + 6⎥
d 2
⎡ 2e −t − e −2t 2e −t − 2e −2t ⎤ then
= ⎢ −t − 2t ⎥ ⎣ 0 0 ⎦
⎣− e + e − e −t + 2e −2t ⎦ dt
& ⎡4t 3 4t 2 ⎤
and 2 AA = ⎢
From the above discussion it is clear that we can also write ⎣ 0 0 ⎥⎦
e At in its spectral or modal form as so that
d 2
A ≠ 2 AA& .
dt
e At = A1e λ1t + A2 e λ2t + K + An e λnt (3.19) We now return to our original problem, to solve equation
(3.15): x& = A x + B u . Rewrite the equation in the form
where λ1 , λ2 ,L, λn are the eigenvalues of A and A1 , A2 ,L ,
x& − A x = B u
An are matrices of the same order as the matrix A . In the
above example, we can write ⇒ e − At (x& − A x) = e − At ( B u )

___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 11
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

d − At With this notation equation (3.20) becomes

or (e x) = e − At B u
dt
t

On integration, this becomes

x(t ) = Φ(t ) x(0) = Φ(t − τ ) B u (τ ) dτ
∫ 0
(3.22)

t 3.5 Solution of the state equation by Laplace transforms

∫e
− Aτ
e − At x(t ) − x(0) = B u (τ ) dτ
0
so that Since the state equation is in a vector form we must first
t
define the Laplace transform of a vector.
∫e
A(t −τ )
x(t ) = e At x(0) = B u (τ ) dτ (3.20)
0 Definition
⎡ x1 ⎤
Example 3.6 _______________________________________ ⎢x ⎥
Let x(t ) = ⎢ 2 ⎥
M
A system is characterized by the state equation ⎢x ⎥
⎣ n⎦
⎡ x&1 ⎤ ⎡ 0 2 ⎤ ⎡ x1 ⎤ ⎡0⎤ ⎡ L [ x1 (t )] ⎤ ⎡ X 1 ( s ) ⎤
⎢⎣ x& 2 ⎥⎦ = ⎢⎣− 1 − 3⎥⎦ ⎢⎣ x2 ⎥⎦ + ⎢⎣1⎥⎦ u ⎢ L [ x2 (t )]⎥ ⎢ X 2 ( s) ⎥
We define L [x(t )] = ⎢ ⎥ = ⎢ M ⎥ = X( s )
M
⎢ L [ x (t )]⎥ ⎢ X ( s)⎥
⎣ ⎦ ⎣ n ⎦
If the forcing function u (t ) = 1 for t ≥ 0 and x(0) = [1 − 1]T .
n

Find the state x of the system at time t . From this definition, we can find all the results we need. For
example,
We have already evaluated e At in Example 3.5. It follows that
⎡ L [ x&1 (t )] ⎤ ⎡ sX 1 ( s) − x1 (0) ⎤
⎢ ⎥ ⎢ ⎥
L [x& (t )] = ⎢ L [ x 2 (t )]⎥ = ⎢ sX 2 ( s) − x2 (0) ⎥
&
⎡ −t −2t −t −2t ⎤
e At x(0) = ⎢ 2e −t − e −2t 2e −t − 2e −2t ⎥ ⎡ 1 ⎤ M M
⎣− e + e − e + 2e ⎦ ⎢⎣− 1⎥⎦ ⎢ L [ x& (t )]⎥ ⎢ sX ( s) − x (0)⎥
⎣ n ⎦ ⎣ n n ⎦
= e −2t ⎡⎢ 1 ⎤⎥
⎣− 1⎦ Now we can solve the state equation (3.15). Taking the
t t transform of x& = A x + B u , we obtain
∫ 0
e A(t −τ ) B u (τ ) dτ = e A(t −τ ) ⎡⎢0⎤⎥ dτ
0 ⎣1⎦ ∫
−(t −τ ) s X( s ) − x(0) = A X( s ) + B U ( s ) where U ( s ) = L [u (t )]
⎡ − 2e −2(t −τ ) ⎤ dτ
t
∫ ⎣ 2e
= ⎢2e
0
−2(t −τ ) ⎥
− e −(t −τ ) ⎦
or ( sI − A) X( s ) = x(0) + B U ( s )
⎡ −t − 2t ⎤
= ⎢1 − 2−et +−e2t ⎥
⎣ e − e ⎦ Unless s happens to be an eigenvalue of A , the matrix
( sI − A) is non-singular, so that the above equation can be
Hence the state of the system at time t is solved giving

⎡ −2t ⎤ ⎡ −t −2t ⎤ ⎡ −t −2t ⎤

X( s) = ( sI − A) −1 x(0) + ( sI − A) −1 B U ( s)
x(t ) = ⎢ e −2t ⎥ + ⎢1 − 2−et +−e2t ⎥ = ⎢1 − 2−et + 2−e2t ⎥ (3.23)
⎣− e ⎦ ⎣ e − e ⎦ ⎣ e − 2e ⎦
__________________________________________________________________________________________
and the solution x(t ) is found by taking the inverse Laplace
At
transform of equation (3.23).
The matrix e in the solution equation (3.20) is of special
interest to control engineers; they call it the state-transition Definition
matrix and denote it by Φ(t ) , that is,
( sI − A) −1 is called the resolvent matrix of the system.

Φ(t ) = e At (3.21) On comparing equation (3.22) and (3.23) we find that

For the unforced system (such as u (t ) = 0 ) the solution (Eq. −1
Φ (t ) = L {( sI − A) −1}
(3.20)) becomes

x(t ) = Φ(t ) x(0) Not only the use of transforms a relatively simple method for
evaluating the transition matrix, but indeed it allows us to
calculate the state x(t ) without having to evaluate integrals.
so that Φ(t ) transforms the system from its state x(0) at some
initial time t = 0 to the state x(t ) at some subsequent time t -
Example 3.7 _______________________________________
hence the name given to the matrix.
Use Laplace transform to evaluate the state x(t ) of the system
Since e At e − At = I . It follows that [e At ]−1 = e − At . describe in Example 3.6.

Hence Φ −1 (t ) = e − At = Φ(−t ) ⎡ x&1 ⎤ ⎡ 0 2 ⎤ ⎡ x1 ⎤ ⎡0⎤

⎢⎣ x& 2 ⎥⎦ = ⎢⎣− 1 − 3⎥⎦ ⎢⎣ x2 ⎥⎦ + ⎢⎣1⎥⎦ u
At − Aτ A(t −τ )
Also Φ(t ) Φ(−τ ) = e e =e = Φ (t − τ ) .
___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 12
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

For this system T z& = A T z + B u

y = CT z
( sI − A) = ⎡⎢ s −2 ⎤⎥ , so that
⎣1 s + 3⎦ or as

( sI − A) −1 =
1 ⎡ s + 3 2⎤ z& = A1 z + B1 u
s ( s + 3) + 2 ⎢⎣ − 1 s ⎥⎦ (3.26)
y = C1 z
⎡ s+3 2 ⎤
⎢ ( s + 1)( s + 2) ( s + 1)( s + 2) ⎥
=⎢ ⎥ where A1 = T −1 A T , B1 = T −1 B, C1 = C T . The transformation
⎢ −1 s ⎥
⎢⎣ ( s + 1)( s + 2) ( s + 1)( s + 2) ⎥⎦ (3.25) is called state-transformation, and the matrices A and
A1 are similar. We are particular interested in the
⎡ 2 1 2 2 ⎤
⎢ s +1 − s + 2 s + 1
−
s + 2 ⎥
transformation when A1 is diagonal (usually denoted by Λ )
=⎢
1 1 1 2 ⎥ and A is in the companion form, such as (3.2)
⎢− + − + ⎥
⎣ s +1 s + 2 s +1 s + 2⎦
So that ⎡ x&1 ⎤ ⎡ 0 1 L 0 0 ⎤ ⎡ x1 ⎤ ⎡0⎤
⎢ x& 2 ⎥ ⎢ 0 0 L 0 0 ⎥ ⎢ x 2 ⎥ ⎢0 ⎥
⎡ −t −2t −t −2t ⎤
⎢ M ⎥=⎢ M
L −1{( SI − A) −1} = ⎢ 2e −t − e −2t 2e −t − 2e −2t ⎥ = Φ(t ) M O M M ⎥ ⎢ M ⎥ + ⎢M⎥ u
− + − + ⎢ x& ⎥ ⎢ 0 1 ⎥ ⎢ x n −1 ⎥ ⎢0⎥
⎣ e e e 2 e ⎦ ⎢ ⎥ ⎢
0 L 0
L − a 2 − a1 ⎥⎦ ⎢⎢ x ⎥⎥ ⎢⎣1 ⎥⎦
n −1
⎢⎣ x& n ⎥⎦ ⎣− a n − a n −1 ⎣ n ⎦
Hence the complementary function is as in Example 3.6. For
the particular integral, we note that since
It is assumed that the matrix A has distinct eigenvalues
1 λ1 , λ2 ,L, λn . Corresponding to the eigenvalue λi there is the
L {u (t )} = , then
s eigenvector x i such that
1 1 ⎡ s + 3 2⎤ ⎡0⎤
( sI − A) −1 BU ( s ) =
( s + 1)( s + 2) s ⎢⎣ − 1 s ⎥⎦ ⎣⎢1 ⎥⎦ A x i = λi xi (i = 1,2,L, n) (3.27)
⎡ 2 ⎤
⎢ s ( s + 1)( s + 2) ⎥ We define the matrix V whose columns are the eigenvectors
=⎢ ⎥
⎢ ⎥ V = [x1 x 2 L x n ]
1
⎣⎢ ( s + 1)( s + 2) ⎦⎥
⎡1 2 1 ⎤
⎢ − + V is called the modal matrix, it is non-singular and can be
⎥
= ⎢ s s +1 s + 2 ⎥ used as the transformation matrix T above. We can write the n
1 1
⎢ − ⎥ equations defined by equation (3.27) as
⎣ s +1 s + 2 ⎦
AV = Λ V (3.28)
On taking the inverse Laplace transform, we obtain
where
⎡ −t −2 t ⎤
L −1
{( sI − A) BU ( s )} = ⎢1 − 2−et +−e2t ⎥
−1 ⎡λ1 0 L 0⎤
− ⎢ 0 ⎥ = diag{λ , λ ,L, λ }
⎣ ⎦ Λ = ⎢ 0 λ2
e e L
M M O M ⎥ 1 2 n
⎢0 0 L λn ⎥⎦
which is the particular integral part of the solution obtained in ⎣
Example 3.6.
__________________________________________________________________________________________
From equation (3.28) we obtain
3.6 The transformation from the companion to the
diagonal state form Λ = V −1 AV (3.29)

Note that the choice of the state vector is not unique. We now
The matrix A has the companion form
assume that with one choice of the state vector the state
equations are
⎡ 0 1 0 L 0 ⎤
⎢ 0 0 1 L 0 ⎥
x& = A x + B u A=⎢ M M M O M ⎥
(3.24) ⎢ 0
y =Cx 0 0 L 1 ⎥
⎢⎣− a0 − a1 − a 2 L − a n−1 ⎥⎦
where A is the matrix of order n × n and B and C are matrices
of appropriate order. so that its characteristic equation is

Consider any non-singular matrix T of order n × n . Let det( sI − A) = λn + a n−1λn−1 + K + a1λ + a0 = 0 ]

x =T z (3.25) By solving this equation we obtain the eigenvalues

λ1 , λ2 ,L, λn . The corresponding eigenvectors have an
then z is also a state vector and equation (3.24) can be written
as interesting form. Consider one of the eigenvalues λ and the
___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 13
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

corresponding eigenvector x = [α1 α 2 L α n ]T . Then the so the transformation V −1x gives the new choice of the state
equation A x = λ x , corresponds to the system of equations vector

⎡ z1 ⎤ ⎡ 4 0 4 ⎤ ⎡ y⎤
⎧α 2 = λ α1 ⎢ z 2 ⎥ = 1 ⎢8 + 4i − 10i − 2 + 4i ⎥ ⎢ y& ⎥
⎪ ⎢ z ⎥ 20 ⎢8 − 4i 10i − 2 − 4i ⎥ ⎢ &y&⎥
⎪α 3 = λα2 ⎣ 3⎦ ⎣ ⎦⎣ ⎦
⎨
⎪ M
⎪α n = λ α n−1 The state equations are now in the form of equation (3.26),
⎩
[ ]
that is
n −1 T
Setting α1 = 1 , we obtain x = 1 λ λ2 L λ . Hence
the modal matrix in this case takes the form z& = A1z + B1u
y = C1z
⎡ 1 1 L 1 ⎤ where
⎢ λ1 λ1 L λ1 ⎥
V =⎢ M A1 = V −1 AV = diag{2, i,−i}
M O M ⎥ (3.30)
⎢ n−1 n−1 ⎥
⎣λ1 λ1 L λ1n−1 ⎦ 1 ⎡ 2 ⎤
B1 = V −1B = ⎢− 1 + 2i ⎥
10 ⎢ − 1 − 2i ⎥
⎣ ⎦
In this form V is called Vandermonde matrix; it is non-
C1 = C V = [1 1 1]
singular since it has benn assumed that λ1 , λ2 ,L, λn are __________________________________________________________________________________________
distinct. We now consider the problem of obtaining the
transformation from the companion form to diagonal form. 3.7 The transfer function from the state equation

Example 3.8 _______________________________________ Consider the system in state space form of Eq.(3.24)
Having chosen the state vector so that
x& = A x + B u
(3.24)
&y&& − 2 &y& _ y& − 2 y = u y =Cx

is written as the equation (in companion form) Taking the Laplace transforms we obtain

⎡ x&1 ⎤ ⎡0 1 0⎤ ⎡ x1 ⎤ ⎡0⎤ s X (s) = A X (s) + B U (s)

⎢ x& 2 ⎥ = ⎢0 0 1 ⎥ ⎢ x 2 ⎥ + ⎢0⎥ u Y (s) = C X (s)
⎢ x& ⎥ ⎢2 − 1 2⎥ ⎢ x ⎥ ⎢1⎥
⎣ 3⎦ ⎣ ⎦⎣ 3⎦ ⎣ ⎦
⇒ X ( s ) = ( sI − A) −1 B U ( s )
⎡ x1 ⎤
y = [1 0 0]⎢ x 2 ⎥
Y (s)
and G ( s ) = = C ( sI − A) −1 B (3.31)
⎢x ⎥ U (s)
⎣ 3⎦
Find the transformation which will transform this into a state
Example 3.9 _______________________________________
equation with A in diagonal form.
Calculate the transfer function from the system whose state
The characteristic equation of A is equations are

det(λ I − A) = λ3 − 2λ2 + λ − 2 = (λ − 2)(λ2 + 1) = 0 x& = ⎡1 −2 ⎤ x + ⎡1 ⎤ u

⎢⎣3 − 4⎥⎦ ⎢⎣2⎥⎦

that is λ1 = 2, λ2 = i and λ3 = −i . y = [1 − 2]x

From (3.30) the modal matrix is −1

G ( s ) = [1 − 2]⎡⎢ s − 1 2 ⎤⎥ ⎡1 ⎤
⎢⎣2⎥⎦
⎣ − 3 s + 4⎦
⎡1 1 1 ⎤
−1
V = ⎢2 i − i ⎥ ⎡ s+4 −2 ⎤
⎢⎣4 − 1 − 1⎥⎦ ⎢ ( s + 1)( s + 2) ( s + 1)( s + 2) ⎥ 1
= [1 − 2]⎢ ⎥ ⎡ ⎤
⎢ 3 s−2 ⎥ ⎢⎣2⎥⎦
and its inverse can be shown to be ⎢⎣ ( s + 1)( s + 2) ( s + 1)( s + 2) ⎥⎦
3s + 2
1 ⎡ 4 0 4 ⎤ =−
V −1 = ⎢8 + 4i − 10i − 2 + 4i ⎥ ( s + 1)( s + 2)
20 ⎢8 − 4i 10i − 2 − 4i ⎥
⎣ ⎦ __________________________________________________________________________________________

Example 3.10_______________________________________
The transformation is defined by equation (3.25), that is
x = V z or z = V −1x . The original choice for x was
Given that the system
x& = A x + B u
x = [ y y& &y&] T y =Cx

___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 14
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

has the transfer function G (s) , find the transfer function of the
system

z& = T −1 AT z + T −1 B u
y = CT z

If the transfer function is G1 ( s )

G1 ( s ) = C T ( sI − T −1 AT ) −1T −1 B
= C T {T −1 ( sI − A)T }−1T −1 B
= C ( sI − A) −1 B
= G(s)

so that G1 ( s ) = G ( s ) .
__________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 3 State Space Formulation 15
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.4 Transient and Steady State Response Analysis

4.1 Introduction Notice that when t = a , then y (t ) = y (a) = 1 − e −1 = 0.63 . The

Many applications of control theory are to servomechanisms response is in two-parts, the transient part e −t / a , which
which are systems using the feedback principle designed so approaches zero as t → ∞ and the steady-state part 1, which is
that the output will follow the input. Hence there is a need for the output when t → ∞ .
studying the time response of the system.
If the derivative of the input are involved in the differential
The time response of a system may be considered in two parts: equation of the system, that is, if a y& + y = b u& + u , then its
• Transient response: this part reduces to zero as t → ∞
transfer function is
• Steady-state response: response of the system as t → ∞
bs + 1 s+z
4.2 Response of the first order systems Y ( s) = U (s) = K U (s) (4.4)
as + 1 s+ p
Consider the output of a linear system in the form where
K =b/a
Y ( s) = G ( s)U ( s) (4.1) z = 1 / b : the zero of the system
where p = 1 / a : the pole of the system
Y (s ) : Laplace transform of the output
G (s ) : transfer function of the system When U ( s ) = 1 / s , Eq. (4.4) can be written as
U (s ) : Laplace transform of the input
K1 K2 z z− p
Y ( s) = − , where K1 = K and K 2 = K
Consider the first order system of the form a y& + y = u , its s s+ p p p
transfer function is Hence,

1 y (t ) = K1 − K 2 e − pt (4.5)
Y ( s) = U (s) { 1
424 3
as + 1 steady − state part transient part

For a transient response analysis it is customary to use a With the assumption that z > p > 0 , this response is shown in
reference unit step function u (t ) for which Fig. 4.2.

1 K1
U (s) =
s y = K1 − K 2e − pt
K2
It then follows that

1 1 1
Y ( s) = = − (4.2) K1 − K 2
(a s + 1) s s s + 1 / a
K 2e − pt
On taking the inverse Laplace of equation (4.2), we obtain t
−t / a Fig. 4.2
y (t ) = 1
{ − e123 (t ≥ 0) (4.3)
steady − state part transient part We note that the responses to the systems (Fig. 4.1 and Fig.
4.2) have the same form, except for the constant terms K1 and
Both of the input and the output to the system are shown in
K 2 . It appears that the role of the numerator of the transfer
Fig. 4.1. The response has an exponential form. The constant
a is called the time constant of the system. function is to determine these constants, that is, the size of
y (t ) , but its form is determined by the denominator.
u (t )
1.00 4.3 Response of second order systems

An example of a second order system is a spring-dashpot

0.63 arrangement shown in Fig. 4.3. Applying Newton’s law, we
find

M &y& = − µ y& − k y + u (t )

where k is spring constant, µ is damping coefficient, y is the

a t distance of the system from its position of equilibrium point,
Fig. 4.1 and it is assumed that y (0) = y& (0) = 0 .
___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 16
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

y K
k y (t ) = (1 − e − pt − p t e − pt ) (4.10)
p2
M µ
u (t ) Case 3: a12 < 4a 2 → under-damped system

Fig. 4.3 In this case, the poles p1 and p 2 are complex conjugate having
Hence u (t ) = M &y& + µ y& + k y the form p1, 2 = α ± i β where α = a1 / 2 and β = 1 4a 2 − a12 .
2
Hence
On taking Laplace transforms, we obtain

1 K1 K2 K3
Y ( s) = U (s) =
K
U (s) Y ( s) = + + ,
s s + p1 s + p 2
M s2 + µ s + k s 2 + a1 s + a 2
where
where K = 1 / M , a1 = µ / M , a 2 = k / M . Applying a unit K K (− β − iα ) K (− β + iα )
K1 = , K2 = , K3 =
step input, we obtain α2 + β2 2 β (α 2 + β 2 ) 2 β (α 2 + β 2 )

K (Notice that K 2 and K 3 are complex conjugates)

Y (s) = (4.6)
s ( s + p1 )( s + p 2 ) It follows that

y (t ) = K1 + K 2 e −(α +iβ )t + K 3 e −(α −iβ )t

a1 ± a12 − 4a 2
where p1, 2 = , p1 and p 2 are the poles of the = K1 + e −αt [( K 2 + K 3 ) cos β t + ( K 3 − K 2 )i sin β t ]
2
K (using the relation e iβ t = cos β t + i sin β t )
transfer function G ( s ) = , that is, the zeros of the
2
s + a1 s + a 2 K K α
= + e −αt (− cos β t − sin β t (4.11)
denominator of G(s). α2 + β2 α2 + β2 β

There are there cases to be considered: K K α2

= − 1+ e −αt sin( β t + ε ) (4.12)
2 2 2 2
α +β α +β β2
Case 1: a12 > 4a 2 → over-damped system
where tan ε = β / α
In this case p1 and p 2 are both real and unequal. Eq. (4.6) can
be written as Notice that when t = 0 , y (t ) = 0 . The there cases discussed
above are plotted in Fig. 4.4.
K1 K2 K3
Y ( s) = + + (4.7)
s s + p1 s + p 2 y (t ) under damped system
u (t )
where 1
K K K K
K1 = = , K2 = , K3 =
p1 p 2 a 2 p1 ( p1 − p 2 ) p 2 ( p 2 − p1 ) critically damped system
over damped system
(notice that K1 + K 2 + K 3 = 0 ). On taking Laplace transform
of Eq.(4.7), we obtain t
Fig. 4.4
y (t ) = K1 + K 2 e − p1t + K 3 e − p2t (4.8)
From Fig. 4.4, we see that the importance of damping (note
The transient part of the solution is seen to be that a1 = µ / M , µ being the damping factor). We would
K 2 e − p1t + K 3 e − p2t . expect that when the damping is 0 (that is, a1 = 0 ) the system
should oscillate indefinitely. Indeed when a1 = 0 , then
Case 2: a12 = 4a 2 → critically damped system
α = 0 , and β = a 2
In this case, the poles are equal: p1 = p 2 = a1 / 2 = p , and
and since sin ε = 1 and cos ε = 0 , then ε = π / 2 , Eq. (4.12)
K KK2 K3 becomes
Y ( s) = = 1 + + (4.9)
s (s + p) 2 s s + p ( s + p) 2
y (t ) =
K ⎡ ⎛ π ⎞⎤ K
⎢1 − sin ⎜ a 2 t + ⎟⎥ =
a2 ⎣ ⎝ 2 ⎠⎦ a 2
1 − cos a 2 t [ ]
Hence y (t ) = K1 + K 2 e − pt + K 3 t e − pt , where K1 = K / p 2 ,
K 2 = − K / p 2 and K 3 = − K / p so that This response of the undamped system is shown in Fig.4.5.

___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 17
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

y (t ) (1) Overshoot defined as

maximum overshoot
× 100%
final desired value

(2) Time delay t d , the time required for a system response

to reach 50% of its final value.

2π 4π t (3) Rise time, the time required for the system response to
0 a2 a2 rise from 10% to 90% of its final value.

Fig. 4.5 (4) Settling time, the time required for the eventual settling
down of the system response to be within (normally) 5%
There are two important constants associated with each second of its final value.
order system.
(5) Steady-state error ess , the difference between the steady
• The undamped natural frequency ω n of the system is the state response and the input.
frequency of the response shown in Fig. 4.5: ω n = a 2 In fact, one can often improve one of the parameters but at the
expense of the other. For example, the overshoot can be
• The damping ratio ξ of the system is the ratio of the decreased at the expense of the time delay.
actual damping µ ( = a1M ) to the value of the damping
In general, the quality of a system may be judged by various
µ c , which results in the system being critically damped standards. Since the purpose of a servomechanism is to make
µ a1 the output follow the input signal, we may define expressions
(that is, when a1 = 2 a 2 ). Hence ξ = = . which will describe dynamics accuracy in the transient state.
µ c 2 a2
Such expression are based on the system error, which in its
simplest form is the difference between the input and the
We can write equation (4.12) in terms of these constants. We output and is denoted by e(t ) , that is, e(t ) = y (t ) − u (t ) , where
note that a1 = 2ω nξ and a 2 = ω n2 . Hence y (t ) is the actual output and u (t ) is the desired output ( u (t ) is
the input).
a12 2 a2 1
1+α 2 / β 2 = 1+ = = The expression called the performance index can take on
4a 2 − a12 4a 2 − a12 1−ξ 2 various forms, typically among them are:

∞
Eq. (4.12) becomes (1) integral of error squared (IES)
∫ 0
e 2 (t )dt
∞
⎛ ⎞
y (t ) =
K ⎜
2 ⎜
1−
1 ⎟
e −ω nξ t sin(ω t + ε ) ⎟ (4.13)
(2) integral of absolute error (IAS)
∫ 0
e (t )dt
ωn ⎜ 1−ξ 2 ⎟ (3) integral of time multiplied absolute error criterion (ITAE)
⎝ ⎠
∞
∫ t e (t )dt
where
0
1−ξ 2
ω = ω n 1 − ξ 2 and tan ε = .
ξ Having chosen an appropriate performance index, the system
which minimizes the integral is called optimal. The object of
It is conventional to choose K / a 2 = 1 and then plot graphs of modern control theory is to design a system so that it is
the ‘normalised’ response y (t ) against ω t for various values of optimal with respect to a performance index and will be
discussed in the part II of this course.
the damping ratio ξ . There typical graphs are shown in Fig.
4.6. 4.4 Response of higher order systems

Some definitions
We can write the transfer function of an n th - order system in
the form
y (t ) maximum overshoot steady-state
error ess
K ( s m + b1 s m −1 + K + b m )
1.0 G (s) = (4.14)
0.9 s n + a1 s n −1 + K + a n

0.5 Example 4.1________________________________________

rise time
With reference to Fig. 2.11, calculate the close loop transfer
0.1 1
function G ( s ) given the transfer functions A( s ) = and
td t s+3
Fig. 4.7 B( s) = 2 / s
___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 18
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

R (s ) E (s ) C (s ) such as resistanceless circuits. We then divide the numerator

until we obtain a proper fraction so that when applying a unit
A(s) R (s ) C (s )
A( s ) step input, we can write Y ( s ) as
G ( s )=
1 + A( s ) B ( s )
B(s) K1 ( s n + d1s n−1 + K + d n )
Y ( s) = K (c1 + c 2 s + K + c s n−m−1 ) +
Fig. 2.11 s ( s n + a1s n−1 + K + a n )
(4.20)
We obtain where c s , d s , K and K1 are all constants.

s s The inverse Laplace transform of the first right term of (4.20)

G(s) = =
s 2 + 3s + 2 ( s + 1)( s + 2) involves the impulse function and various derivatives of it.
__________________________________________________________________________________________
The second term of (4.20) is treated as in Case 1 or Case 2
above.
The response of the system having the transfer function (4.14)
to a unit step input can be written in the form
Example 4.2________________________________________
K ( s + z1 )( s + z 2 ) L ( s + z m ) Find the response of the system having a transfer function
Y ( s) = (4.15)
s ( s + p1 )( s + p2 )L ( s + p n )
where 5( s 2 + 5s + 6)
G(s) =
z1 , z 2 ,L, z m : the zeros of the numerator s + 6 s 2 + 10 s + 8
3

p1 , p 2 ,L, p n : the zeros of the denominator

to a unit step input.
We first assume that n ≥ m in equation (4.14); we then have In this case,
two cases to consider:
5( s 2 + 5s + 6) 5( s + 2)( s + 3)
Y ( s) = =
Case 1: p1 , p 2 ,L, p n are all distinct numbers. The partial s + 6 s 2 + 10 s + 8
3 s( s + 4)[ s + (1 + i )][s + (1 − i )]
fraction expansion of equation (4.15) has the form
The partial fraction expansion as
K1 K2 K
Y ( s) = + + L + n+1 (4.16) K1 K K3 K4
s s + p1 s + pn Y ( s) = + 2 + +
s s + 4 s + (1 + i ) s + (1 − i )
K1 , K 2 ,L, K n+1 are called the residues of the expansions. The
15 1 −7 + i −7 − i
response has the form where K1 = , K 2 = − , K3 = , K4 = .
4 4 4 4

y (t ) = K1 + K 2 e − p1t + K + K n+1e − pnt Hence

15 1 −4t 1 −t
y (t ) = − e + e (−14 cos t + 2 sin t )
Case 2: p1 , p 2 ,L, p n are not distinct any more. Here at least 4 4 4
one of the roots, say p1 , is of multiplicity r , that is 15 1 −4t 2000 −t
= − e + e sin(t + 352)
4 4 4
K ( s + z1 )( s + z 2 ) L ( s + z m ) __________________________________________________________________________________________
Y ( s) = (4.17)
s( s + p1 ) ( s + p 2 )L ( s + pn )
r
4.5 Steady state error

The partial fraction expansion of equation (4.17) has the form Consider a unity feedback system as in Fig. 4.8

K1 K K 2r K n −r + 2
Y ( s) = + 21 + L + +L+ (4.18)
s s + p1 ( s + p1 ) r s + p n−r +1 R (s ) E (s ) C (s )
A(s)

⎧ K ⎫⎪ −1 ⎪ K
Since L ⎨ ⎬= t j −1e − pt , ( j = 1,2,L, r ) , the
⎪⎩ ( s + p ) j ⎪⎭ ( j − 1)!
Fig. 4.8
response has the form
where
r (t ) : reference input
K 2 r r −1 − p1t
y (t ) = K1 + K 21e − p1t + K 22 e − p1t + K + t e + c(t ) : system output
(r − 1)!
e(t ) : error
K 3e − p2t + L + K n−r + 2 e − pn −r +1t (4.19)
We define the error function as
We now consider that n < m in equation (4.14); which is the
case when the system is improper; that is, it can happen when e (t ) = r (t ) − c (t ) (4.21)
we consider idealized and physically non-realisable systems,
___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 19
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

hence, ess = lim e(t ) . Since E ( s ) = R( s ) − A( s ) E ( s ) , it c(t ) steady-state

t →∞
error ess
R( s)
follows that E ( s ) = and by the final value theorem
1 + A( s )
sR ( s )
ess = lim sE ( s ) = lim (4.22)
s →0 s →0 1 + A( s ) k u (t )

We now define three error coefficients which indicate the

steady state error when the system is subjected to three
different standard reference inputs r (s ) . t
Fig. 4.11
(1) step input, r (t ) = k u (t ) ( k is a constant)
Example 4.3________________________________________
sk /s k
ess = lim = Find the (static) error coefficients for the system having a
s →0 1 + A( s) 1 + lim A( s)
s →0 8
open loop transfer function A( s ) =
s ( 4 s + 2)
lim A( s ) = K p , called the position error constant, then
s→0
K p = lim A( s) = ∞
s→0
k k − ess K v = lim sA( s ) = 4
ess = or K p = (4.23) s→0
1+ K p ess
K a = lim s 2 A( s ) = 0
s→0
c(t ) steady-state __________________________________________________________________________________________
k u (t )
error ess
From the definition of the error coefficients, it is seen that ess
k depends on the number of poles at s = 0 of the transfer
function. This leads to the following classification. A transfer
function is said to be of type N if it has N poles at the origin.
Thus if

K ( s − z1 ) L ( s − z m )
A( s ) = (4.24)
t s j ( s − p1 )L ( s − p n )

Fig. 4.9 K (− z1 )L (− z m )
K1
At s = 0 , A( s ) = lim where K1 = (4.25)
sj (− p1 )L (− p n )
s →0
(2) Ram input, r (t ) = k t u (t ) ( k is a constant)
K1 is called the gain of the transfer function. Hence the steady
k k k
In this case, R ( s) = 2
, so that ess = or K v = , where state error ess depends on j and r (t ) as summarized in Table
s Kv ess
4.1
K v = lim sA( s ) is called the velocity error constant.
s→0 Table 4.1
ess
c(t ) steady-state j System
error ess r(t)=ku(t) r(t)=ktu(t) r(t)=½kt2u(t)

0 Type 1 Finite ∞ ∞
1 Type 2 0 finite ∞
k u (t ) 2 Type 3 0 0 finite

4.6 Feedback Control

Consider a negative feedback system in Fig. 4.12

t
Fig. 4.10 R (s ) E (s ) C (s )
A(s)
1 2
(3) Parabolic input, r (t ) = k t u (t ) ( k is a constant)
2 B(s)
k k k
In this case, R ( s ) = 3 , so that ess = or K a = , where Fig. 4.12
s Ka ess
K a = lim s 2 A( s) is called the acceleration error constant. The close loop transfer function is related to the feed-forward
s→0 transfer function A(s ) and feedback transfer function B (s) by
___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 20
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

A( s ) where q (t ) is output signal from the controller

G(s) = (4.26)
1 + A( s ) B ( s ) Q1 , Q2 are some constants

We consider a simple example of a first order system for The on-off controller is obviously a nonlinear device and
K it cannot be described by a transfer function.
which A( s ) = and B ( s ) = c , so that
as + 1
(2) Proportional Controller
For this control action
KK /a
G(s) = = q (t ) = K p e(t )
Kc + 1
as + Kc + 1
s+
a where K p is a constant, called the controller gain. The
On taking Laplace inverse transform, we obtain the impulse transfer function of this controller is
response of the system, where Q( s)
B( s) = = Kp (4.28)
E (s)
K
(1) c = 0 (response of open loop system): g (t ) = e −t / a
a (3) Integral Controller
K − K ca+1 t K − αt a t
(2) c ≠ 0 : g (t ) = e
a
= e , where α =
a K c +1 In this case q(t ) = K e(t )dt , hence
∫ 0
a and α are respectively the time-constants of the open loop
and closed loop systems. a is always positive, but α can be B( s) = K / s (4.29)
either positive or negative.
(4) Derivative Controller
Fig. 4.13 shows how the time responses vary with different de
values of K c . In this case q (t ) = K , hence
dt
g (t )
Kc ≤ −1 Kc = −5 Kc = −3 B( s ) = Ks (4.30)

(5) Proportional-Derivative Controller (PD)

K Kc = −1 de
In this case q (t ) = K p e(t ) + K1 , hence
a Kc = 0 dt
⎛ K ⎞
Kc = 4 B ( s ) = K p ⎜1 + 1 s ⎟ = K p (1 + K s ) (4.30)
stable region ⎜ Kp ⎟
⎝ ⎠
t
Fig. 4.13 (6) Proportional-Integral Controller (PI)
t
If the impulse response does not decay to zero as t increase, In this case q(t ) = K p e(t ) + K1 e(t )dt , hence
∫ 0
the system is unstable. From the Fig. 4.13, the instability
⎛ K 1⎞ ⎛ K⎞
region is defined by Kc ≤ −1 . B ( s ) = K p ⎜1 + 1 ⎟ = K p ⎜ 1 + ⎟ (4.31)
⎜ Kp s⎟ ⎝ s⎠
⎝ ⎠
In many applications, the control system consists basically of
a plant having the transfer function A(s ) and a controller (7) Proportional-Derivative-Integral Controller (PID)
having a transfer function B (s) , as in Fig. 4.14. t
∫
de
In this case q (t ) = K p e(t ) + K1 + K 2 e(t )dt , hence
dt 0
R (s ) E (s ) Q (s) C (s ) ⎛ K K 1⎞
B ( s) = K p ⎜1 + 1 s + 2 ⎟ = K p (1 + k1s + k 2 / s )
B(s) A(s)
⎜ Kp K p s ⎟⎠
controller plant ⎝

Example 4.4________________________________________
Fig. 4.14
Design a controller for a plant having the transfer function
With the closed loop transfer function
A( s ) = 1 /( s + 2) so that the resulting closed loop system has a
A( s) B ( s ) zero steady state error to a reference ramp input.
G(s) = (4.27)
1 + A( s ) B( s ) For zero steady state error to a ramp input, the system must be
of type 2. Hence if we choose an integral controller with
The controllers can be of various types. B ( s ) = K / s then the transfer function of the closed loop
system including the plant and the controller is
(1) The on-off Controller
The action of such a controller is very simple.
A( s ) B ( s ) K
⎧Q if e(t ) > 0 =
q (t ) = ⎨ 1 1 + A( s ) B ( s ) s 3 + 2 s 2 + K
⎩Q2 if e(t ) < 0
___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 21
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

The response of this control system depends on the roots of

the denominator polynomial s 3 + 2 s 2 + K = 0 .

If we use PI controller, B ( s) = K p (1 + K / s ) the system is of

type 2 and response of the system depends on the roots of
s 3 + 2 s 2 + K p s + KK p = 0
__________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 4 Transient and Steady State Response Analysis 22
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.5 Stability

5.1 Introduction K ( s m + b1s m−1 + K + bm )

G(s) = (5.1)
5.2 The Concept of Stability s n + a1s n−1 + K + a n

The properties of a system are characterized by its weighting and the system response is determined, except for the residues,
function g (t ) , that is, its response to a unit impulse or by the poles of G (s ) , that is by the solution of
equivalently by the Laplace transform of the weighting
function – the transfer function. s n + a1s n−1 + K + a n = 0 (5.2)
A system is said to be asymptotically stable if its weighting
function response decays to zero as t tends to infinity. Eq. (5.2) is called characteristic equation of the system. It
follows that the system is asymptotically stable if and only if
the zeros of the characteristic equation (that is the finite poles
If the response of the system is indefinitely oscillatory, the
system is said to be marginally stable. of the transfer function) p1 , p 2 ,L, p n are negative or have
negative real parts.
Back to the system transfer function has the form Fig. 5.1 illustrates graphically the various cases of stability.

Type of root s-plane graph ( s = σ + jω ) Response graph Remark

ω y
Asymptotically
Real and negative
stable
σ t
ω y
Real and positive Unstable
σ t
ω y
Marginally
Zero
stable
σ t
ω y
Conjugate complex
Asymptotically
with negative real σ t
stable
part
ω y
Conjugate imaginary Marginally
σ t
(multiplicity r=1) stable

ω y
Conjugate imaginary
σ t Unstable
(multiplicity r=2)

ω y
Conjugate with
σ t Unstable
positive real part

ω y
Roots of multiplicity
r=2 at the origin
y=t Unstable
σ t

Fig. 5.2

5.3 Routh Stability Criterion Lyapunov’s method is the most general method we know for
system stability analysis. It is applicable for both linear and
5.4 Introduction to Lyapunov’s Method nonlinear systems of any order. For linear system, it provides
both the necessary and sufficient conditions, whereas for
___________________________________________________________________________________________________________
Chapter 5 Stability 23
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

nonlinear systems it provides only the sufficient conditions 1

for asymptotically stable. x12 + x 22 = A 2 (5.9)
g /l
Lyapunov considered the solution x(t ) of the state equations
to trace out a trajectory (or an orbit or a path) which, for an The trajectories (for various values of A and ε ) are ellipses as
asymptotically stable system, terminates at the origin of the shown in Fig. 5.3
n-dimensional state-space, known as the phase plane when
n=2. x2

Take a simple example: consider the motion of an undamped

pendulum. The equation of motion is x1

g
θ&& + sin θ = 0 (5.4)
l Fig. 5.3
which is nonlinear system. Every trajectory shown in Fig. 5.3 is closed, showing that the
pendulum, assumed to have no resistance to its motion, will
swing indefinitely. A closed trajectories is typical of a
periodic motion, that is, one for which the solution x(t ) has
the property x(t + T ) = x(t ) , where T is the period of the
motion.
θ l
When the oscillations are damped, the trajectories would
terminate at the point x = 0 so that it may have the form
shown in Fig. 5.4.

x2
mg x0
Fig. 5.2 x1

Assume that the displacements are small, we have sin θ ≈ θ

and (5.4) can be rewritten as
Fig. 5.4
g
θ&& + θ = 0 (5.5)
In general autonomous system, the state equation correspond-
l
ding to equation (5.7) takes the form
This is well known linear equation of simple harmonic
motion, and has the solution ⎧ x&1 = P ( x1 , x2 )
⎨ (5.9)
⎩ x& 2 = Q( x1 , x2 )
θ = A sin( g / l t + ε ) (5.6)
The solutions corresponding to Eq. (5.8) are
where A and ε are constants determined by the initial
conditions. ⎧ x1 = φ (t )
⎨ (5.10)
⎩ x2 = ψ (t )
Let define the state variables: x1 = θ , x 2 = θ& . (5.5) can be
written as and can be plotted to give the trajectory.

⎧ x&1 = x 2 For an n-dimensional state equation, the algebra is similar.

⎪
⎨ g (5.7)
⎪ x& 2 = − l x1 Definition
⎩
A point ( x10 , x20 ) , for which P ( x10 , x20 ) = 0 = Q( x10 , x20 ) is
and the solution called a critical point.

⎧⎪ x1 = A sin( g / l t + ε ) For a linear system, there is only one equilibrium point and it
⎨ (5.8) is the point x = 0 .
⎪⎩ x 2 = A g / l cos( g / l t + ε )
A system is asymptotically stable when it is stable and the
and obtain the trajectory by eliminating t in (5.8) trajectory eventually approaches x = 0 as t → ∞ . In
mathematical terminology, these definitions take the form
shown in Fig. 5.5
___________________________________________________________________________________________________________
Chapter 5 Stability 24
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

The equilibrium point x = 0 (Fig. 5.5) is said to be Lyapunov’s method depends on the state trajectory – but not
directly. We illustrate his method using the pendulum
(1) stable if , given any t > 0 , there exists δ > 0 so that example.
every solution of (5.10) which at t = 0 is such that
The potential energy U of the system
x12 + x22 = ϕ 2 (0) + ψ 2 (0) < δ 2
implies U = mgl (1 − cosθ ) ≈ 12 mglθ 2 (for small θ )
ϕ 2 (0) + ψ 2 (0) < ε 2 for t ≥ 0
The kinetic energy of the system
(2) asymptotically stable if,
(a) it is stable, and
K = 12 ml 2θ& 2
⎧ lim ψ (t ) = 0
⎪t →∞
(b) ⎨ or lim [ψ 2 (t ) + ψ 2 (t )] = 0
⎪⎩tlim ψ (t ) = 0 t →∞ Hence the system energy is
→∞

(3) unstable if it is not stable V = 12 mglθ 2 + 12 ml 2θ& 2

= 12 mglx12 + 12 ml 2 x22 (5.13)
x2
asymptotically = V ( x1 , x2 )
stable
Notice that V (0,0) = 0 . It follows that
ε
δ
x12 x22 2 2
0 + = 1 , where a 2 = and b 2 = 2
x1 a V2
b 2V mgl ml
x ( 0)
stable The trajectory is (again) seen to be an ellipse (Fig. 5.6) with
major and minor axes having lengths 2a 2V and 2b 2V
unstable
respectively.

x2
Fig. 5.5

In the definition, starting ‘near’ the equilibrium point is

defined by the ‘ δ -neighborhood’, whereas the ‘reasonable’
14243

distance is the ‘ ε -neighborhood’. b 2V

If we consider an unforced system, having the state equation 1442443 x1

0
a 2V
x& = A x (5.11)

we have found the solution

x = e At x(0)
(5.12) Fig. 5.6
= ( A1e λ1t + A2 e λ2t + K + An e λnt ) x(0)
The rate of change of the energy along a trajectory is
For asymptotic stability, lim x(t ) = 0 , where x(t ) is the
t →∞ dV (x) dV dx1 dV dx2
norm of the (solution) vector x defined by V& = = + (5.14)
dt dx1 dt dx2 dt
dx1 dx
x = (x, x) = x12 + x22 + K + xn2 = mglx1 + ml 2 x2 2
dt dt
= mglx1 x2 + ml 2 x2 (− g / l ) x1
Hence for asymptotic stability all the state variables
x1 , x2 ,L, xn must decrease to zero as t → ∞ . Form (5.12), =0
we see that this is the case when Re λi < 0 (i = 1,2,L, n) , Hence the total energy is a constant along any trajectory. If
where λi is the eigenvalues of A , that is, the solution of the dV / dt < 0 , the energy is decreasing monotonically along
characteristic equation det( sI − A) = 0 . every trajectory. This means that every trajectory moves
toward the point of minimum energy, that is the equilibrium
point (0,0). This is an essence the Lyapunov’s criterion for
asymptotic stability.

___________________________________________________________________________________________________________
Chapter 5 Stability 25
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Lyapunov’s function V (x) is similar to the energy function, it V ( x) = x ′ P x

has the following properties: ⎡ p11 p12 L p12 ⎤ ⎡ x1 ⎤
⎢p L p 2n ⎥ ⎢ x2 ⎥
= [x1 x 2 L x n ]⎢ 21 22
p
O M ⎥⎢ M ⎥
(1) V (x) and its partial derivatives ∂V / ∂xi (i = 1,2,L, n) , (5.15)
M M
⎢p L p nn ⎥⎦ ⎢⎣ xn ⎥⎦
are continuous. ⎣ n1 p n 2
(2) V (x) > 0 for x ≠ 0 in the neighborhood of x = 0 , and n n
V ( 0) = 0 = ∑∑ p x x
i =1 j =1
ij i j
(3) V& (x) < for x ≠ 0 (in the neighborhood of x = 0 ), and
V& (0) = 0 The matrix P is of order n × n and is symmetric.

V (x) satisfying the above properties is called a Lyapunov Sylvester’s Criterion: the necessary and sufficient
function. conditions for a quadratic form V ( x) = x′ P x to be positive-
definite is that all the successive principal minors of the
Definition
symmetric matrix P are positive, that is
V (x) is
• positive-definite if V (x) > 0 for x ≠ 0 , and V (0) = 0 p11 p12 p13
• positive-semi-definite if V (x) ≥ 0 for all x p11 p12
p11 > 0 , > 0 , p 21 p 22 p 23 > 0 , etc.
p 21 p 22
• negative-definite if −V (x) is positive-definite p31 p32 p33
• negative-semi-definite if −V (x) is positive-semi-definite
Example 5.6_______________________________________
Example 5.5_______________________________________ Consider the function
Classify the Lyapunov function V (x) = 4 x12 + 3 x 22 + x32 + 4 x1 x 2 − 2 x1 x3 − 2 x 2 x3

a. V ( x) = x12 + 3 x22 Rewrite V (x) in the form

V (x) > 0, ∀x ≠ 0 and V (0) = 0 ⇒ V (x) is positive-definite
V (x) = 4 x12 + 3 x 22 + x32 + 4 x1 x 2 − 2 x1 x3 − 2 x 2 x3
b. V (x) = −( x1 + x2 ) 2 − 2 x12
⎡ 4 2 − 1⎤ ⎡ x1 ⎤
V (x) < 0, ∀x ≠ 0 and V (0) = 0 ⇒ V (x) is negative-definite = [x1 x 2 x3 ]⎢ 2 3 − 1⎥ ⎢ x2 ⎥
⎢ ⎥
⎣⎢− 1 − 1 1 ⎥⎦ ⎣ x3 ⎦
c. V ( x) = ( 2 x1 + x 2 ) 2 4 2 −1
Since 4>0, 4 2 = 8 > 0 , 2 3 − 1 = 5 > 0 , Sylvester’s
Whenever x2 = −2x1 , V (0) = 0 , otherwise V (0) > 0 . Hence 2 3 −1 −1 1
V (x) ≥ 0 ⇒ V (x) is positive-semi-definite
criterion assures us that V (x) is positive-definite.
_________________________________________________________________________________________

d. V (x) = x12 + 2 x 22 − 4 x1 x 2
5.6 Determination of Lyapunov’ Functions
V (1,1) < 0 and V (1,3) > 0 . V (x) assumes both positive and
negative values ⇒ V (x) is indefinite Definition
_________________________________________________________________________________________
The matrix A is said to be stable, if the corresponding system
defined by equation x& = A x is stable.
Lyapunov’s Theorem
The origin (that is the equilibrium point) is asymptotically
stable if there exists a Lyapunov function in the neighbor- As explained above, to determine whether A is stable we seek
hood of the origin. If V& (x) ≤ 0 , then the origin is a stable a symmetric matrix P so that V ( x) = x′ P x is a positive-
point. definite function, and

This is sometimes referred to as stability in the sense of V& ( x) = x& ′ P x + x′ P x&

Lyapunov. = ( A x)′ P x + x′ PA x (5.16)
= x′[ A′ P + PA] x
5.5 Quadratic Forms

Definition is negative. Since P is symmetric, [ A′ P + PA] is symmetric.

A quadratic form is a scalar function V (x) of variables Hence V& ( x) is a quadratic form.
x = [x1 x 2 L x n ]T defined by
Let −Q = A′P + PA (5.17)

for V& ( x ) to be negative-definite, x′ Q x must be positive-

definite.
___________________________________________________________________________________________________________
Chapter 5 Stability 26
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Lyapunov’s theorem can now be expressed in the following 5.7 The Nyquist Stability Criterion
form:
If the transfer function of a system is not known, it is often
Given a positive-definite matrix Q , the matrix A is stable if possible to obtain it experimentally by exciting the system
the solution to the equation (5.17) results in a positive- with a range of sinusoidal inputs of fixed amplitude by
definite matrix P . varying frequencies, and observing the resulting changes in
amplitude and phase shift at the output.
Example 5.7_______________________________________
The information obtained in this way, known as the system
Use Lyapunov’s direct method to determine whether the frequency response, can be used in various ways for system
following systems are asymptotically stable: analysis and design.

⎧ x& = x 2 5.8 The Frequency Response

a. ⎨ 1
⎩ x& 2 = 2 x1 − x 2 Consider the system Y ( s ) = G ( s ) U ( s ) . For a sinusoidal input
Solving A′P + PA = − I , we obtain
of unit amplitude u (t ) = sin ω t so that U ( s) = ω /( s 2 + ω 2 )
P = ⎡ −3 −1⎤
1
4 ⎢⎣− 1 1 ⎥⎦ and
which, by Sylvester’s criterion, is not positive definite. ω
Y ( s0 = G (s)
Hence the system is not stable. s +ω2
2

⎧ x&1 = x 2 Assuming that G (s ) has poles at s = − p1 ,− p 2 , L ,− p m ,

b. ⎨
⎩ x& 2 = −5 x1 − 2 x 2 Y ( s ) can be expanded in partial fractions as

We obtain P = ⎡17 1⎤ which is positive definite. Hence

1
10 ⎢⎣ 1 3⎥⎦ n

∑s+ p
Kj Q1 Q2
Y (s) = + − (5.18)
the system is asymptotically stable.
j s + iω s − iω
_________________________________________________________________________________________ j =1
and
Example 5.8_______________________________________
Determine using Lyapunov’s method, the range of K for the (s + p j ) ω G(s)
system in Fig. 5.8 to be asymptotically stable Kj =
s2 + ω 2 s =− p j
U (s ) K X 2 (s) 3 X 1 (s) (s + p j ) ω G(s) 1
s +1 s+2 Q1 = =− G (−i ω )
s2 + ω 2 s = −i ω
2i

(s + p j ) ω G(s) 1
Fig. 5.8 Q2 = = G (i ω )
s2 + ω 2 s =i ω
2i

From Fig. 5.8 On taking the inverse Laplace transform of equation (5.18),
3 we obtain
X1 = X2 ⇒ x&1 = −2 x1 + 3 x 2
s+2
n
X2 =
K
s +1
(− X 1 + U ) ⇒ x& 2 = − Kx1 − x 2 + Ku y (t ) = ∑K e
j =1
j
− p jt
+ Q1e −iω t + Q2 e i ω t (5.19)
1 42
4 43
4 144424443
transient form steady − state form
The state equation is
⎡ x&1 ⎤ ⎡ − 2 3 ⎤ ⎡ x1 ⎤ ⎡ 0 ⎤
⎢⎣ x& 2 ⎥⎦ = ⎢⎣− K − 1⎥⎦ ⎢⎣ x 2 ⎥⎦ + ⎢⎣ K ⎥⎦ u The steady-state term can be written as

1
Solving A′P + PA = − I [G (iω ) e iω t − G (−iω ) e −iω t ] = Im{G (iω ) e iω t } (5.20)
2i

⎡− 2 − K ⎤ ⎡ p11 p12 ⎤ + ⎡ p11 p12 ⎤ ⎡ − 2 3 ⎤ = ⎡− 1 0 ⎤

⎢⎣ 3 − 1 ⎥⎦ ⎢⎣ p 21 p 22 ⎥⎦ ⎢⎣ p 21 p 22 ⎥⎦ ⎣⎢− K − 1⎥⎦ ⎣⎢ 0 − 1⎥⎦ In polar form we can express G (iω ) = G (iω ) e iφ , where
Im{G (iω )}
φ = tan −1 , and Eq. (5.12) becomes
1 ⎡ K 2 + 3K + 3 3 − 2 K ⎤ Re{G (iω )}
We obtain P =
18 K + 12 ⎢⎣ 3 − 2 K 3K + 15⎥⎦
Im{ G (iω ) e i (ω t +φ ) } = G (iω ) sin(ωt + φ ) (5.21)
Sylvester’s criterion leads us to conclude that K > −2 / 3 .
_________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 5 Stability 27
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Im{ G (iω ) e i (ω t +φ ) } = G (iω ) sin(ωt + φ ) (5.21) where

Im{G j (iω )}
φ j (k ) = tan −1 ( j = 1,2, L , k )
Eq. (5.21) shows that the steady-state output is also Re{G j (iω )}
sinusoidal, and relative to the input the amplitude is
multiplied by G (iω ) . Example 5.10______________________________________
Construct the frequency diagram for the system having the
The frequency diagram is the plot of G (iω ) in the complex transfer function
plane as the frequency varies form −∞ to +∞ . Since for the
systems considered the transfer function G (s ) are the ratios of K
G(s) = ( K is constant)
two polynomials with real coefficients, it follows that s ( s + 1)

G (iω ) = the complex conjugate of G (−iω )

⎛ 1 ⎞⎛ K ⎞
Since G (iω ) = ⎜ ⎟ ⎜ ⎟ = G1 (iω ) G2 (iω ) , we obtain
⎝ iω ⎠ ⎝ iω + 1 ⎠
and it is sufficient to plot G (iω ) for 0 ≤ ω < ∞ ; the
1 1
remainder of the locus is symmetrical about the real axis to G1 (iω ) = and φ = −π / 2 − tan −1 ω . The plot is
this plot. ω 1+ω2
given in Fig. 5.10
Example 5.9_______________________________________
Imaginary
Determine the frequency diagrams for the following transfer axis
functions

=0
ω
a. G ( s ) = Ks
ω = −∞
G (iω ) = iKω , hence G (iω ) = Kω and ϕ (ω ) = 90 0 ⇒ the
plot is the positive imaginary axis (for ω ≥ 0 ) Real axis

b. G ( s ) = K / s ϕ
G ω = +∞
G (iω ) = K /(iω ) , hence G (iω ) = K / ω and ϕ (ω ) = −90 0

⇒ the plot is the negative imaginary axis (for ω ≥ 0 ) ω

=0
c. G ( s ) = K /(a + s ) Fig. 5.10
________________________________________________________________________________________
G (iω ) = K /( a + iω ) = K (a − iω ) /(a 2 + ω 2 ) , hence
To use the Nyquyist criterion we must plot the frequency
G (iω ) = K / a 2 + ω 2 and ϕ (ω ) = − tan −1 (ω / a) response of the system’s open-loop transfer function.
⇒ the plot for ω ≥ 0 is given in Fig. 5.9
Consider the closed loop system in the Fig. 5.11
Imaginary Locus corresponding
axis to negative frequency R (s ) E (s ) C (s )
G(s)

C1 ( s ) = H ( s ) C ( s )
ω = −∞
K K H(s)
2a a
Real axis
0 ϕ ω =0 Fig. 5.11
G (iω
ω = +∞ )
The closed loop transfer function
increasing ω
C (s) G ( s)
= (5.22)
G( s) 1 + G ( s) H ( s)
Fig. 5.9
_________________________________________________________________________________________
The ratio of the feedback signal C1 ( s) to the actuating error
If the transfer function involves several factors, the above C1 ( s )
procedure can be used for each factor individually, and the signal E ( s ) is = G ( s ) H ( s ) and is called the system
E (s)
overall result is obtained by the following rules
open-loop transfer function.
If G ( s ) = G1 ( s ) G 2 ( s ) L G k ( s ) , then
If we consider the unity negative feedback system shown in
G (iω ) = G1 (iω ) G2 (iω ) L Gk (iω ) , and C (s)
Fig. 5.12 we again find that 1 = G(s) H (s)
φ (ω ) = φ1 (ω ) + φ 2 (ω ) + K + φ k (ω ) , E (s)
___________________________________________________________________________________________________________
Chapter 5 Stability 28
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

R (s ) E (s ) C (s ) s = 1 + iω −1 ≤ ω ≤ 1
G(s)H(s)
and corresponding along D ' A'
C1( s )
1 5 − iω
F ( s) = = = u + iv
Fig. 5.12 5 + iω 25 + ω 2

And both of the systems in Fig. 5.11 and Fig. 5.12 have the 5 ω
same open-loop transfer functions. where u = and v = − . They relationship is
25 + ω 2
25 + ω 2
Assume that if there are any factors which cancel in the
product G ( s) H ( s) they involve only poles and zeros in the (u − 1 / 10) 2 + v 2 = (1 / 10) 2
left haft s-plane. We will consider the system to be stable so
long as the close loop transfer function equation (5.22) has no This is the equation of a circle, center (1 / 10,0) and
poles in the RHP. We note that s = s1 is a pole of equation radius 1 / 10 . In Fig. 5.13 we note that
(5.22) if
(1) the closed contour γ in the s -plane is mapped into a
(1) 1 + G ( s1 ) H ( s1 ) = 0 and G ( s1 ) ≠ 0 closed contour Γ in the F ( s) -plane
A( s) B( s) (2) Any point inside γ maps into a point inside Γ
(2) G(s) = and 1 + G ( s ) H ( s ) = , where
( s − s1 ) n ( s − s1 ) m (3) We can define a positive direction in which s traces out
its path. It does not matter which direction is chosen to
n > m A( s1 ) ≠ 0 and B ( s1 ) ≠ 0 .
be positive.
From (1) and (2), we conclude that the system having the It is interesting to consider the direction in which F ( s )
transfer function equation (5.22) is stable if and only if
sweeps out its path, as s moves along a contour which
[1 + G ( s ) H ( s )] has no zeros in RHP.
encloses the pole ( s = −4) of F ( s ) . This is shown in Fig.
5.9 An Introduction to Conformal Mappings 5.14.

To establish the Nyquist criterion we regard the open loop iω Im{F ( s )}

s − plane F ( s ) − plane
transfer function G ( s ) H ( s ) = F ( s ) of a feedback system as a
mapping from the s-plane to the F ( s) -plane. As an example, B A C’ D’
i
consider F ( s ) = 1 /( s + 4) . We consider two planes: σ
-5 -4 -3 -2 0
Re{F ( s )}
(1) the s-plane, having a real axis, Re{s} = σ and imaginary
-i
axis Im{s} = ω , where s = σ + iω C D B’ A’

γ − path Γ − path
(2) the F ( s) -plane which has similarly a real axis
Fig. 5.14
Re{F ( s)} and imaginary axis Im{F ( s )}
In this case we note that as s traces the γ -contour in the
Consider corresponding paths in the two planes as in Fig.
positive direction, F ( s) traces the corresponding Γ -contour
5.13
in the opposite, that is, negative direction..
s − plane F ( s ) − plane
Cauchy’s Theorem
iω Im{F ( s )} Let γ be a closed contour in the s -plane enclosing P poles
C’ and Z zeros of F ( s ) , but not passing through any poles or
B i A 0 .1
zeros of F ( s ) . As s traces out γ in the positive direction,
D’ F ( s) traces out a contour Γ in the F ( s ) − plane, which
1
σ Re{F ( s )} encircles the origin ( Z − P ) times in the positive direction.
-1 0 0 .1 0 .2 0 .3
A’
In Fig. 5.13, γ does not enclose any poles or zeros of F ( s ) ,
C -i D − 0 .1 so that P = 0 = Z ⇒ the origin of the F ( s ) -plane is not
B’
γ − path Γ − path encircled by the Γ -contour.
Fig. 5.13
In Fig. 5.14, γ encircles a pole of F ( s ) , so that P = 1 ,
In Fig. 5.13 it is shown that F ( s ) traces out circular arcs as Z = 0 , hence Z − P = −1 ⇒ Γ -contour encircles the origin
s moves along the insides of the square. This can of course (-1) times in the positive direction, that is, Γ encircles the
be proved analytically. For example, along DA origin once in the negative direction.

___________________________________________________________________________________________________________
Chapter 5 Stability 29
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

5.10 Application of Conformal Mappings to the Example 5.11______________________________________

Frequency Response
Use the Nyquist criterion to determine the stability of the
closed-loop system having an open-loop transfer function
In section 5.8 we discussed the frequency response, which
required s to trace out the imaginary axis in the s-plane. In
K
section 5.9, Cauchy’s theorem calls for a closed contour in F ( s) =
the s-plane avoiding the poles and zeros of F ( s ) . We now s ( s + 1)( s + 2)
combine these two contours and define the Nyquist path.
Nyquist path consist of the entire imaginary axis, and a when (1) K = 1 and (2) K = 10
semicircular path of large radius R in the right half of s-plane,
as shown in Fig. 5.15 Since F ( s) has a pole at the origin, the Nyquist contour has a
semicircular indentation as shown in Fig. 5.18(a)
s − plane F ( s ) − plane
iω Im{F ( s )} s − plane F ( s ) − plane
Nyquist
iω F (i 0 − ) ω = −0
path i∞
∞
R→ i 0+

R
α ω = 2 ω = +∞
σ 0
Re{F ( s )} r
σ
θ 0.2 0.1 0
ω = −∞
Γ i0 −
γ
Fig. 5.15 −i∞
ω = +0 F (i 0 + )
The small semicircular indentations are used to avoid poles (a) (b)
and zeros of F ( s) which lie along the imaginary axis. The Fig. 5.18
semi circle in the s-plane is assumed large enough to enclose
all the poles and zeros of F ( s) which may lie in the RHP. The F ( s) contour for K = 1 corresponding to the Nyquist path
is shown in Fig. 5.18(b).
The Nyquist Criterion (1) For K = 1 , the point (−1,+∞) is not encircled by the F ( s )
To determine the stability of a unity feedback system, we contour. Since F ( s ) has no poles in the RHP, we can
consider the map Γ in the F ( s) -plane of the Nyquist contour conclude that the system is stable.
traversed in a positive direction. (2) For K = 10 , the magnitude F (iω ) is 10 times larger than
The system is stable if when K = 1 for each finite value of ω while the phase φ is
(1) Γ does not encircle the point (-1,0) when the open-loop unchanged. This means that Fig. 5.18 is valid for K = 10 if
transfer function G ( s ) H ( s ) has no poles in the RHP. we change the coordinate scales by a factor of 10. This time
(2) Γ encircles the point (-1,0) P times in the positive the critical point (−1,+∞) is not encircled twice. We can
direction when the open-loop transfer function conclude that there are two closed-loop poles in the RHP, and
G ( s ) H ( s ) has P poles in the RHP. the system is unstable.
________________________________________________________________________________________
Otherwise, the system is unstable.

Fig. 5.7 shows examples of a stable and an unstable system

for the case where P = 0 .

F ( s ) − plane F ( s ) − plane

-1 0 -1 0

stable system unstable system

Fig. 5.17

___________________________________________________________________________________________________________
Chapter 5 Stability 30
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.6 Controllability and Observability

6.1 Introduction
It is seen from equation (6.6) that if b′i , the i th row of B1 ,
6.2 Controllability
has all zero components, then z&i = λi z i + 0 , and the input
We consider a system described by the state equations u(t ) has no influence on the i th mode (that is e λit ) of the
system. The mode is said to be uncontrollable, and a system
⎧ x& = A x + B u having one or more such modes is uncontrollable. Otherwise,
⎨ (6.1)
⎩y = C x where all the modes are controllable the system is said to be
completely state controllable, or just controllable.
where A is n × n , B is n × m and C is r × n .
By definition, the system equation (6.6) is controllable if only
With the transformation if a control function u(t ) can be found to transform the initial
state z 0 = [z 01 z 02 L z 0 n ]T to a specifies state
x = Pz (6.2)
z1 = [z11 z12 L z1n ] . T

we can transform equation (6.1) into the form

Example 6.1_______________________________________
⎧ x& = A1z + B1u Check whether the system having the state-space
⎨ (6.3)
⎩y = C1x
representation

where A1 = P −1 A P , B1 = P −1 B and C1 = C P . Assuming x& = ⎡− 1 2⎤ ⎡ 4⎤

⎢⎣ 3 4⎥⎦ x + ⎢⎣6⎥⎦ u
that A has distinct eigenvalues λ1 , λ2 ,L, λn we can choose y = [1 − 2]x
P so that A1 is a diagonal matrix, that is,
is controllable.
A1 = diag{λ1 , λ2 ,L, λn }
The eigenvalues λ1 = 1 , λ2 = 2 and the corresponding
If n = m = r = 2 the first order of (6.3) has the form eigenvectors x1 = [1 2]T , x 2 = [2 3]T , so that the modal
matrix is
⎡ z&1 ⎤ ⎡λ1 0 ⎤ ⎡ z1 ⎤ ⎡ b11 b12 ⎤ ⎡ u1 ⎤
⎢⎣ z& 2 ⎥⎦ = ⎢⎣ 0 λ2 ⎥⎦ ⎢⎣ z 2 ⎥⎦ + ⎢⎣b21 b22 ⎥⎦ ⎢⎣u 2 ⎥⎦
P = ⎡1 2⎤ and P −1 = ⎡ 3 −2⎤
⎢⎣1 3⎥⎦ ⎢⎣− 1 1 ⎥⎦
which is written as
Using transformation x = P z , the sate equation becomes
⎧ z& 1 = λ1z1 + b1′ u
⎨ (6.4)
⎩z& 2 = λ2 z 2 + b ′2 u z& = ⎡⎢1 0⎤⎥ z + ⎡⎢0⎤⎥ u
⎣0 2 ⎦ ⎣ 2⎦
where b′1 and b′2 are the row vectors of the matrix B1 . The y = [− 1 − 4]z
output equation is
This equation shows that the first mode is uncontrollable and
so the system is uncontrollable.
⎡ y1 ⎤ ⎡ c11 c12 ⎤ ⎡ z1 ⎤ ⎡ c11 ⎤ ⎡ c 21 ⎤
⎢⎣ y 2 ⎥⎦ = ⎢⎣c21 c22 ⎥⎦ ⎢⎣ z 2 ⎥⎦ = ⎢⎣c21 ⎥⎦ z1 + ⎢⎣c 22 ⎥⎦ z 2
________________________________________________________________________________________

or Definition
y = c1z1 + c 2 z 2 (6.5) The matrix Q = [ B | A B | L | A n−1 B ] is called the system
controllability matrix.
where c1 ,c 2 are the vectors of C1 . So in general, equation
(6.3) can be written in the form The Controllability Criterion
rank{Q} = n ⇔ the system is controllable
⎧z& i = λi z i + b ′i u (i = 1,2,L, n)
⎪⎪ n
Example 6.2_______________________________________
⎨
∑
(6.6)
⎪ y = ci z i Using the controllability criterion verify that the system in
⎪⎩ i =1 example 6.1 is uncontrollable.

___________________________________________________________________________________________________________
Chapter 6 Controllability and Observability 31
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

For the system The above example illustrates the importance of the
observability concept. In this case we have a non-stable
b = ⎡⎢4⎤⎥ and Ab = ⎡⎢ −1 2⎤⎥ ⎡⎢4⎤⎥ = ⎡⎢ 8 ⎤⎥ ,
system, whose instability is not observed in the output
⎣6⎦ ⎣− 3 4⎦ ⎣6⎦ ⎣12⎦ measurement.

so that Definition
The matrix R = [ C | C T A | L | C T A n−1 ]T is called the system
r{Q} = r {[b Ab]} = r{⎡4 8 ⎤} = 1
⎢⎣6 12⎥⎦ observability matrix.
Since the rank of Q is less than 2, the system is
uncontrollable. The Observability Criterion
________________________________________________________________________________________ rank{R} = n ⇔ the system is observable

Example 6.3_______________________________________ Example 6.5_______________________________________

Determine whether the system, governed by the equation Using observability criterion, verify that the system examined
in example 6.4 is unobservable
⎡ x&1 ⎤ ⎡ 0 6 ⎤ ⎡ x1 ⎤ ⎡ 3 6 ⎤ ⎡ u1 ⎤
⎢⎣ x& 2 ⎥⎦ = ⎢⎣− 1 − 5⎥⎦ ⎢⎣ x 2 ⎥⎦ + ⎢⎣− 1 − 2⎥⎦ ⎢⎣u 2 ⎥⎦ For this system

is controllable. c ′ = [3 − 2] and A = ⎡ −5 4⎤ ,
⎢⎣− 6 5⎥⎦
Using the controllability criterion
⎡ ⎤
r{Q} = r {[B A B ]} = r{⎢ 3 6 − 6 − 12⎥}
hence
⎣− 1 − 2 2 4 ⎦
It is obvious that the rank of this matrix is 1. The system is R = ⎡ 3 −2⎤ , so that r {R} = 1
therefore uncontrollable. ⎢⎣− 3 2 ⎦⎥
________________________________________________________________________________________

Since the rank of R is less than 2, the system is unobservable.

6.3 Observability ________________________________________________________________________________________

⎧ z& = A1z + B1u 6.4 Decomposition of the system state

Consider the system in the form ⎨ . If a row of
⎩y = C1x From the discussion in the two previous sections, the general
the matrix C1 is zero, the corresponding mode of the system state variables of a linear system can be classified into four
will not appear in the output y . In this case the system is exclusive groups as follows:
unobservable, since we cannot determine the state variable
• controllable and observable
corresponding to the row of zeros in C1 from y .
• controllable and unobservable
• uncontrollable and observable
Example 6.4_______________________________________ • uncontrollable and unobservable
Determine whether the system have the state equations
Assuming that the system has distinct eigenvalues, by
appropriate transforms the state equations can be reduced to
⎡ x&1 ⎤ ⎡ − 5 4⎤ ⎡ x1 ⎤ ⎡1 ⎤
⎢⎣ x& 2 ⎥⎦ = ⎢⎣− 6 5⎥⎦ ⎢⎣ x 2 ⎥⎦ + ⎢⎣2⎥⎦ u the form below

⎡x ⎤
y = [3 − 2]⎢ 1 ⎥ ⎡ x& 1 ⎤ ⎡ A1 0 0 0 ⎤ ⎡ x1 ⎤ ⎡ B1 ⎤
⎣ x2 ⎦ ⎢x& 2 ⎥ ⎢ 0 A2 0 0 ⎥ ⎢x 2 ⎥ ⎢ B ⎥
⎢ x& ⎥ = ⎢ 0 0 A 0 ⎥ ⎢ x ⎥ + ⎢ 2 ⎥ u
is observable. ⎢ 3⎥ ⎢ 3 ⎥⎢ 3⎥ ⎢ 0 ⎥
⎣⎢x& 4 ⎦⎥ ⎣⎢ 0 0 0 A4 ⎥⎦ ⎣⎢x 4 ⎦⎥ ⎣ 0 ⎦
(6.11)
Using the modal matrix P = ⎡1 2⎤ , the transform matrix ⎡ x1 ⎤
⎢⎣1 3⎥⎦ ⎢x ⎥
y = [C1 0 C3 0] ⎢ 2 ⎥
x = P z , transform the state-equations into x
⎢ 3⎥
⎣⎢x 4 ⎦⎥
⎡ z&1 ⎤ ⎡− 1 0⎤ ⎡ z1 ⎤ ⎡− 1⎤
⎢⎣ z& 2 ⎥⎦ = ⎢⎣ 0 1⎥⎦ ⎢⎣ z 2 ⎥⎦ + ⎢⎣ 1 ⎥⎦ u The (transformed) systems matrix A has been put into a
“block-diagonal” form where each Ai (i = 1,2,3,4) is in
⎡z ⎤
y = [1 0]⎢ 1 ⎥ diagonal form. The suffix i of the state-variable
⎣z2 ⎦
vector xi implies that the elements of this vector are the state
This results shows that the system is unobservable because variables corresponding to the i th group defined above.
the output y does not influenced by the state variable z 2 .
________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 6 Controllability and Observability 32
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Example 6.6______________________________________ Example 6.7_______________________________________

Consider the system Illustrate the above discussion with the systems considered in
examples 6.1 and 6.4
⎡ x&1 ⎤ ⎡− 2 0 0 ⎤ ⎡ x1 ⎤ ⎡1
0 0 0 − 1⎤
⎢ x& 2 ⎥ ⎢ 0 0 0 ⎥ ⎢⎢ x 2 ⎥⎥ ⎢0
−1 0 0 0⎥
In example 6.1, the state equations were transferred into the
⎢ x& ⎥ ⎢
⎢ 3⎥ = ⎢ 0 0 0 ⎥ ⎢ x3 ⎥ + ⎢1
0 1 0 1 ⎥ ⎡ u1 ⎤ diagonal form
⎢ x& 4 ⎥ ⎢ 0 0 0 ⎥ ⎢ x 4 ⎥ ⎢0
0 0 −3 0 ⎥ ⎢⎣u 2 ⎥⎦
⎢ x&5 ⎥ ⎢ 0 0 0 ⎥ ⎢ x ⎥ ⎢1
0 0 0 0⎥ ⎡ z&1 ⎤ ⎡1 0⎤ ⎡ z1 ⎤ ⎡0⎤
0 − 4⎥⎦ ⎢ x ⎥ ⎢⎣0 2 ⎥⎦ ⎢⎣ z& 2 ⎥⎦ = ⎢⎣0 2⎦⎥ ⎢⎣ z 2 ⎥⎦ + ⎢⎣2⎥⎦ u
5
⎢⎣ x& 6 ⎥⎦ ⎣ 0 0
⎣ 6⎦
0 0

⎡ x1 ⎤ ⎡z ⎤
⎢ x2 ⎥ y = [− 1 − 4] ⎢ 1 ⎥
⎣z2 ⎦
⎡ y1 ⎤ ⎡0 0 − 1 2 0 1⎤ ⎢⎢ x3 ⎥⎥
⎢⎣ y 2 ⎥⎦ = ⎢⎣1 0 1 − 1 0 1⎥⎦ ⎢ x4 ⎥ and can be represented by Fig.6.2
⎢ x5 ⎥
⎢⎣ x6 ⎥⎦
z&1 = z1 -1 y
By inspection we can rewritten the above system in the u
following form z&2 = 2 z2 + 2u -4

Fig. 6.2
⎡ x&1 ⎤ ⎡− 2 0 0 0 0 0 ⎤ ⎡ x1 ⎤ ⎡1 − 1⎤
⎢ x&3 ⎥ ⎢ 0 1 0 0 0 0 ⎥ ⎢⎢ x3 ⎥⎥ ⎢1 1⎥
⎢ x& ⎥ ⎢ The system has two modes corresponding to the poles λ = 1
⎢ 6⎥ = ⎢ 0 0 −4 0 0 0 ⎥ ⎢ x 6 ⎥ + ⎢0 2 ⎥ ⎡ u1 ⎤
0 ⎥ ⎢ x5 ⎥ ⎢1 0 ⎥ ⎢⎣u 2 ⎥⎦ and λ = 2 . The transfer function
⎢ x&5 ⎥ ⎢ 0 0 0 0 0
⎢ x& 4 ⎥ ⎢ 0 0 0 0 −3 0 ⎥ ⎢ x ⎥ ⎢0 0⎥
⎢⎣ x& 2 ⎥⎦ ⎣ 0 0 0 0 0 − 1⎥⎦ ⎢ x 4 ⎥ ⎢⎣0 0 ⎥⎦ ⎡ 1 ⎤
⎣ 2⎦
⎢ s − 2 0 ⎥ ⎡0⎤ −8
⎡ x1 ⎤ G ( s) = C [ sI − A] B = [− 1 − 4] ⎢ −1
=
⎢ x3 ⎥ 1 ⎥ ⎢⎣2⎥⎦ s − 2
⎢ 0 ⎥
⎡ y1 ⎤ ⎡0 − 1 1 0 2 0⎤ ⎢⎢ x6 ⎥⎥ ⎣ s − 1⎦
⎢⎣ y 2 ⎥⎦ = ⎢⎣1 1 1 0 − 1 0⎥⎦ ⎢ x5 ⎥
⎢ x4 ⎥ It is seen that the transfer function involves only the mode
corresponding to λ = 2 - the controllable and observable one.
⎣⎢ x2 ⎥⎦
The uncontrollable mode λ = 1 does not appear in the
hence transfer function.

• controllable and observable x1, x 3 , x 6 In example 6.4 the transformed state equations are

• controllable and unobservable x5 ⎡ z&1 ⎤ ⎡− 1 0⎤ ⎡ z1 ⎤ ⎡− 1⎤

• uncontrollable and observable x4 ⎢⎣ z& 2 ⎥⎦ = ⎢⎣ 0 1⎥⎦ ⎢⎣ z 2 ⎥⎦ + ⎢⎣ 1 ⎥⎦ u
• uncontrollable and unobservable x2 ⎡z ⎤
________________________________________________________________________________________ y = [1 0] ⎢ 1 ⎥
⎣z2 ⎦
We can represent the decompositions of the state variables
into four groups by a diagram and can be represented in Fig.6.3

y
S1 z&1 = z1 + u
u u
y
S2 z&2 = 2 z2 + 2u

Fig. 6.3
S3
In this case
⎡ 1 ⎤
S4 ⎢ s + 1 0 ⎥ ⎡− 1⎤ −1
G ( s) = C [ sI − A] B = [1 0] ⎢ −1
=
1 ⎥ ⎢⎣ 1 ⎥⎦ s + 1
⎢ 0 ⎥
Fig. 6.1 ⎣ s − 1⎦
In general, a transfer function G ( s ) represents only the
subsystem S1 of the system considered, and indeed on adding This result shows that the unobservable mode λ = 1 does not
to S1 the subsystem S 2 , S 3 , S 4 will not effect G ( s ) . involved in the transfer function.
________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 6 Controllability and Observability 33
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Definition
⎧ x& = A x + B u ⎡ 0 1 0 L ⎤ 0 ⎡0 ⎤
The state equation ⎨ are said to be a realization ⎢ ⎥ ⎢ ⎥
⎩y = C x ⎢ 0 0 1 0 0 ⎥ ⎢0 ⎥
of the transfer function G (s ) if G ( s ) = C [ sI − A] B . −1 C=⎢ M M M O M ⎥ and d = T −1b = ⎢0⎥
⎢ ⎥ ⎢ ⎥
⎢ 0 0 0 L 1 ⎥ ⎢M⎥
⎢− a − a1 − a2 L − a n −1 ⎥⎦ ⎢1⎥
Definition ⎣ 0 ⎣ ⎦
⎧ x& = A x + B u (6.15)
A realization equation ⎨ of a transfer function
⎩y = C x Also the characteristic equation of A yields
G ( s) is said to be minimal if it is both controllable and λ I − A = λn + a n−1λn−1 + L + a 0 = λ I − C = 0
observable.

Example 6.8_______________________________________ We construct the transformation matrix T in the following

way:
Obtain a minimal realization of a system having the transfer
function Let p be a vector of order n × 1 . Then p ′A, p ′A 2 , L , p ′A n −1 are
n row vectors, which we collect to make up the n rows of the
1 ⎡ − s −( s + 1)⎤
G ( s) = matrix T −1 , that is
( s + 1)( s + 2) ⎢⎣3s + 2 2( s + 1) ⎥⎦
⎡ p′ ⎤
The system has two modes λ = −1 and λ = −2 . Hence we ⎢ ′ ⎥
pA ⎥
can write T −1 =⎢ (6.16)
⎢ M ⎥
⎢ n −1 ⎥
1 ⎡ − s −( s + 1)⎤ ⎢⎣p ′A ⎥⎦
G ( s) =
( s + 1)( s + 2) ⎢⎣3s + 2 2( s + 1) ⎥⎦
= C [ sI − A] −1 B Assume for the moment that T −1 is a non-singular matrix, so
that T exists and has the portioned form
⎡ 1 ⎤
0 ⎥
⎡ c11 c12 ⎤ ⎢ s + 1 ⎡ b11 b12 ⎤
=⎢ ⎢ 1 ⎥ ⎢⎣b21 b22 ⎥⎦ T = [q1 q1 L q n ]
⎣c 21 c 22 ⎥⎦ ⎢ 0
(6.17)
⎥
⎣ s + 2⎦
so that T −1T = I , hence
And the minimal realization is therefore
⎡ p ′q1 p ′q 2 L p ′q n ⎤ ⎡1 0 L 0⎤
⎡ x&1 ⎤ ⎡− 1 0 ⎤ ⎡ x1 ⎤ ⎡1 0⎤ ⎢ ′ ′Aq 2 ⎥ ⎢ ⎥
⎢ p A q p L p ′Aq n ⎥ ⎢0 1 L 0⎥
⎢⎣ x& 2 ⎥⎦ = ⎢⎣ 0 − 2⎥⎦ ⎢⎣ x 2 ⎥⎦ + ⎢⎣2 1⎥⎦ u
1
=
⎢ M M O M ⎥ ⎢M M O M⎥
⎢ n−1 n −1 ⎥ ⎢ ⎥
⎡x ⎤
y = ⎡⎢ 1 − 1⎤⎥ ⎢ 1 ⎥ ′ ′
⎣⎢p A q1 p A q 2 L p ′A n−1q n ⎦⎥ ⎣⎢0 0 L 1 ⎦⎥
⎣− 1 2 ⎦ ⎣ x 2 ⎦ (6.18)
________________________________________________________________________________________
On taking the product
6.5 A Transformation into the Companion Form
⎡ p′ ⎤
There exists a transformation ⎢ ′ ⎥
A [q1 q1 L q n ]
pA ⎥
T −1 AT = ⎢
⎢ M ⎥
x =Tz (6.12) ⎢ n−1 ⎥
⎢⎣p ′A ⎥⎦
transforming the system
we obtain the matrix
x& = A x + b u (6.13)
⎡ p ′Aq1 p ′Aq 2 L p ′Aq n ⎤
⎢ ′ 2 ′A 2 q 2 ⎥
where A is n × n matrix assumed to have n distinct ⎢ p A q 1 p L p ′A 2 q n ⎥
(6.19)
eigenvalues, into ⎢ M M O M ⎥
⎢ n ⎥
⎣⎢p ′A q1 p ′A q 2 L p ′A n q n ⎦⎥
n
z& = C z + d u (6.14)
Note that the first (n − 1) rows of equations (6.19) are the
where C = T −1 AT same (row by row) as the last (n − 1) rows of the matrix
equation (6.18) ⇒ the matrix equation (6.19) is in the
is in a companion form, that is
companion form, and
___________________________________________________________________________________________________________
Chapter 6 Controllability and Observability 34
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

⎧ − a0 = p ′A n q1
⎪
⎪ − a1 = p ′A n q 2
⎨ (6.20)
⎪
⎪
⎩− a n−1 = p ′A q n
n

We must also have

⎡ p′ ⎤ ⎡0 ⎤ ⎧ p ′b = 0
⎢ ′ ⎥ ⎢ ⎥ ⎪ ′Ab = 0
⎢ p A ⎥ b = ⎢0⎥ or ⎪⎨
p
⎢ M ⎥ ⎢M⎥ ⎪
⎢ n−1 ⎥ ⎢ ⎥ ⎪p ′A n −1b = 0
⎢⎣p ′A ⎥⎦ ⎢⎣1 ⎥⎦ ⎩

that is
p′ b [ Ab L A n −1b = d ′ ] (6.21)

Eq. (6.21) is a system of n simultaneous equations in n

unknowns (the components of p ) and will have a unique
solution if and only if the rank of the system controllability
matrix is n , that is if

rank b [ Ab L A n −1b = n ] (6.22)

(then the system is controllable).

Example 6.9_______________________________________

The state equation of a system is

⎡ 13 35 ⎤ ⎡−2⎤
x& = ⎢ ⎥x + ⎢ ⎥u
⎣− 6 − 16⎦ ⎣1⎦

Find the transformation so that the state matrix of the

transformed system is in a companion form.

⎧ p ′b = 0
Using ⎨ , we obtain
⎩p ′Ab = 0

⎡−2⎤ ⎡ 13 35 ⎤ ⎡−2⎤
[ p1 p 2 ] ⎢ ⎥ = 0 and [ p1 p2 ]⎢ ⎥⎢ ⎥ =1
⎣1⎦ ⎣− 6 − 16⎦ ⎣ 1 ⎦

that is

⎧−2 p1 + p 2 = 0 ⎧ p1 = 1
⎨ ⇒ ⎨
⎩9 p1 − 4 p 2 = 1 ⎩ p2 = 2

and the transformation matrix

⎡ 3 −2⎤ −1 ⎡1 2⎤
T =⎢ ⎥ and T = ⎢ ⎥
⎣ − 1 1 ⎦ ⎣1 3⎦

The required transformation is

⎡ 3 −2⎤
x& = ⎢ ⎥z
⎣− 1 1 ⎦
________________________________________________________________________________________
___________________________________________________________________________________________________________
Chapter 6 Controllability and Observability 35
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.7 Multivariable Feedback and Pole Location

7.1 Introduction Example 7.1_______________________________________

7.2 State Feedback of a SISO System Given the system x& = ⎡ −13 8 ⎤ x + ⎡1 ⎤ u , find the feedback
⎢⎣− 25 15⎥⎦ ⎢⎣2⎥⎦
Consider a SISO system described by the state equations control law so that the resulting closed loop system has poles
at λ = −2 and λ = −3 .
x& = A x + b u (7.1)
With the transformation matrix T = ⎡⎢1 1 ⎤⎥ and
The dynamics of this system are determined by the poles, that ⎣1 2⎦
is, the eigenvalues λ1 , λ 2 , L , λ n (assumed distinct) of the
T −1 = ⎡ 2 −1⎤ the state equation becomes
matrix A . ⎢⎣− 1 1 ⎥⎦

If system is controllable, there exists a matrix T such that

x = T z transforms the system defined by equation (7.1) to z& = T −1 A T z + T −1b u = ⎡ 0 1 ⎤ z + ⎡0⎤ u
⎢⎣− 5 2⎥⎦ ⎢⎣1⎥⎦

z& = C z + d u (7.2)
the characteristic equation in this case is λ12 − 2λ + 5 = 0 and
where
the poles are at λ = 1 ± 2 i so that the system is unstable.
⎡ 0 1 0 L 0 ⎤ ⎡0 ⎤
⎢ 0 L 0 ⎥ ⎢0 ⎥ ⎡z ⎤
Apply feedback law in the form u = [k1 k 2 ] ⎢ 1 ⎥ and using
0 1
C = T −1 A T = ⎢ M M M O M ⎥ , d = T −1b = ⎢ M ⎥
⎢ 0 ⎣z2 ⎦
0 0 L 1 ⎥ ⎢0 ⎥
⎢⎣− a1 − a 2 − a3 L − a n ⎥⎦ ⎢⎣1 ⎥⎦ equation (7.5)

k1 = a1 − r1 = 5 − 6 = −1
Now apply the feedback control of the form
k 2 = a 2 − r2 = −2 − 5 = −7
u (t ) = k′z , k ′ = [k1 , k 2 , L , k n ] (7.3)
and the control law is
yields
u (t ) = k ′ z = k ′ T −1 x = [− 1 − 7 ] ⎡⎢ 2 −1⎤⎥ x = [5 − 6] x
z& = [C + d k ′] z (7.4) ⎣− 1 1 ⎦

which is the system whose dynamics are determined by the • Checking the poles of the closed loop system
eigenvalues of [C + d k′] , that is by the eigenvalues of After assigned poles

⎡ 0 1 0 L 0 ⎤ ⎡0 0 0 L 0⎤ x& = Ax + bu = ( A + bk ′T −1 ) x
⎢ 0 0 1 L 0 ⎥ ⎢0 0 0 L 0⎥
⎢ M M M O M ⎥+⎢M M M O M⎥=
⎢ 0 0 0 L 1 ⎥ ⎢0 0 0 L 0⎥ and
⎢⎣− a1 − a 2 − a3 L − a n ⎥⎦ ⎢⎣k1 k 2 k 3 L k n ⎥⎦
⎡ −13 8 ⎤ ⎡1 ⎤ ⎡ −8 2⎤
⎡ 0 1 0 L 0 ⎤ A + bk ′T −1 = ⎢ ⎥ + ⎢ ⎥ [5 − 6] = ⎢ ⎥
⎢ 0 0 1 L 0 ⎥ ⎣ − 25 15 ⎦ ⎣ ⎦2 ⎣− 15 3⎦
⎢ M M M O M ⎥
⎢ 0 0 0 L 1 ⎥
⎢⎣− r1 − r2 − r3 L − rn ⎥⎦ The corresponding characteristic equation is

(7.5)
λ 2 + 5λ + 6 = (λ + 2)(λ + 3) = 0
where − ri = k i − ai , (i = 1,2, L , n)

hence the closed loop poles were assigned at λ = −2 and

The characteristic equation of [C + d k ′] is
λ = −3 as desired.
________________________________________________________________________________________

s n + rn s n −1 + K + r2 s + r1 = 0
7.3 Multivariable Systems
By appropriate choice of the components k1 , k 2 , L , k n of k
The pole assignment for a multivariable system is more
in equation (7.5) we can assign arbitrary values to the
complicated than for a scalar system since a multitude of
coefficients r1 , r2 , L , rn and so achieve any desired pole inputs are to be controlled, and not just one !
configurations. Since z = T −1 x it follows that
We will discuss a relatively simple method dealing with this
−1 problem.
u (t ) = k ′ z = k ′ T x (7.6)
___________________________________________________________________________________________________________
Chapter 7 Multivariable Feedback and Pole Location 36
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

• Matrix property Now we consider a multivariable system having n state

If P and Q are matrices of orders m × n and n × m variables and m inputs. To simplify the mathematical
respectively, then manipulations we shall initially assume that the state matrix
is in a diagonal form, that is the state equation has the form
| I m − PQ | = | I n − Q P | (7.7)
z& = Λz + B1u (7.10)
where the notation | A | stands for the determinant of A .
where
I m and I n are the unit matrices of order m × n and n × m Λ = diag{λ1 , λ 2 , L , λ n }
respectively. B1 ∈ Rn×m , z ∈ Rn×1 , u ∈ Rm×1

• Proof The system open loop characteristic polynomial is

Firstly, take a notice about some matrix properties
| sI − Λ | = ( s − λ1 )( s − λ 2 ) L ( s − λ n ) (7.11)
For any A, B :
| AB | = | BA | where the system poles are assumed to be distinct. To
correspond to equation (7.2) we assume a feedback law of the
For any square matrix A form
I 0 A 0
= =| A | u = K1 z (7.12)
0 A 0 I
I A I 0 where K1 ∈ Rm×n is a proportional gain matrix.
= =1
0 I A I
The closed loop system dynamics
When n = m z& = (Λ + B1 K1 ) z (7.13)
| I − PQ | = | P( P −1 − Q) | = | ( P −1 − Q) P | = | I − QP |
The objective of this procedure is to choose K1 so that the
When n ≠ m closed loop system poles ρ1 , ρ 2 , L , ρ n have prescribed
Im P I P Im 0 values. These poles are the solution of the closed loop
= m characteristic equation, that is the roots of
Q In Q In − Q In
I m − PQ P | sI − Λ + B1 K1 | = ( s − ρ1 )( s − ρ 2 ) L ( s − ρ n ) = 0 (7.14)
= (7.9)
0 In
= | I m − PQ | Select K1 to have the so-called dyadic form
Im P I 0 Im P
= m K1 = f d ′ (7.15)
Q In − Q In Q In
Im P where f = [ f1 f2 L f m ]′ and d ′ = [d1 d2 L dn ]
= (7.8)
0 I n − QP
= | I n − QP | with this choice, (7.14) becomes
⇒ | I m − PQ | = | I n − Q P | (7.7)
| sI − Λ + B1f d ′ | = | sI − Λ | | I − ( sI − Λ ) −1 B1f d ′ |
Example 7.2_______________________________________
Taking the determinant of both sides and using (7.11) and
Given P = [ p1 p 2 ] and (7.14) we obtain
show that | I m − P Q | = | I n − Q P |
n n
π ( s − ρ i ) = π ( s − λi ) | I − ( sI − λ ) −1 B1f d ′ | (7.17)
⎡q ⎤ i =1 i =1
P = [ p1 p2 ] and Q = ⎢ 1 ⎥ ⇒ m = 1, n = 2
⎣q 2 ⎦ Let
B1f = r = [r1 r2 L rn ]′
Hence
| I m − PQ | = 1 − ( p1q1 + p 2 q 2 ) (7.19)
−1
P = ( sI − Λ ) B1f ∈ Rn×1
Also
Q = d ′ ∈ R1×n ,
⎡q p q1 p 2 ⎤ ⎡1 − q1 p1 − q1 p 2 ⎤
| I n − PQ | = I 2 − ⎢ 1 1 ⎥=⎢ ⎥
⎣q 2 p1 q 2 p 2 ⎦ ⎣ − q 2 p1 1 − q 2 p 2 ⎦ It follows that
Hence
| I n − PQ | = (1 − q1 p1 )(1 − q 2 p 2 ) − q 2 p1 q1 p 2 | sI − Λ + B1f d ′ | = | 1 − d ′( sI − Λ ) −1 B1f | (7.18)
= 1 − ( p1q1 + p 2 q 2 )
we then have
________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 7 Multivariable Feedback and Pole Location 37
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

d ′( sI − Λ ) −1 B1f Example 7.3_______________________________________

⎡ 1 ⎤ Given a system defined by the state equation

⎢s − λ 0 L ⎥ 0
⎢ 1 ⎥ ⎡ r1 ⎤ ⎡−1 0 ⎤ ⎡2⎤
⎢ 0 1 ⎥ ⎢r2 ⎥ x& = ⎢ ⎥x + ⎢ ⎥u
= [d1 L d n ]⎢
L
d2 s − λ2 ⎥⎢ ⎥ ⎣ 0 − 2 ⎦ ⎣1 ⎦
⎢ M ⎢M⎥
M O M ⎥⎢ ⎥
⎢ 1 ⎥ ⎣⎢rn ⎦⎥
⎢ 0 0 L ⎥ find a proportional gain matrix K1 such that the closed loop
⎣⎢ s − λ n ⎦⎥ system poles are both at λ = −3 .
d1r1 d r d r In this case n = 2, m = 1 so that K1 = f d ′ is a one-row matrix,
= + 2 2 +L+ n n
s − λ1 s − λ 2 s − λn and f is one-element matrix
n

∑s−λ
d i ri
= (7.20) The open loop characteristic polynomial is
i =1 i
( s − λ1 )( s − λ 2 ) = ( s + 1)( s + 2) = s 2 + 3s + 2
Substituting Eqs. (7.18) and (7.20) into Eq. (7.17) we obtain
The closed loop characteristic polynomial is
⎛ n
d i ri ⎞⎟
∑
n n
π ( s − ρ i ) = π ( s − λi )⎜⎜1 − ( s − ρ1 )( s − ρ 2 ) = ( s + 3) 2 = s 2 + 6 s + 9
i =1 i =1 s − λi ⎟
⎝ i =1 ⎠
The residuals are
Hence
( s + 1)(−3s − 7)
n n R1 = lim = −4
π ( s − λi ) − π ( s − ρ i ) n s →−1 ( s + 1)( s + 2)
∑s−λ
i =1 i =1 d i ri
= (7.21)
n
π ( s − λi ) i =1 i and
i =1 ( s + 2)(−3s − 7)
R2 = lim =1
s →−2 ( s + 1)( s + 2)
On taking partial fraction

n n Hence (Eq. (7.24)): r1d1 = −4 and r2 d 2 = 1

π ( s − λi ) − π ( s − ρ i ) n

∑s−λ
i =1 i =1 Ri
= (7.22) From Eq. (7.19)
n
π ( s − λi ) i =1 i
i =1
⎡ 2⎤ ⎡ r1 ⎤ ⎡2 f ⎤ ⎡ r1 ⎤
⎢ ⎥ f = ⎢ ⎥ so that ⎢ ⎥ = ⎢ ⎥
where the residues Ri at the eigenvalue λ j ( j = 1,2, L , n) can ⎣ ⎦
1 ⎣ 2⎦
r ⎣ f ⎦ ⎣r2 ⎦
be evaluated as
⎧r1 = 2 ⎧d1 = −2
Choose f = 1 , ⇒ ⎨ and ⎨ and K1 = [−2 1] .
⎡ n ⎤ n
⎩r2 = 1 ⎩d 2 = 1
( s − λ j ) ⎢ π ( s − λi ) − π ( s − ρ i )⎥
⎣i =1 i =1 ⎦
R j = lim (7.23) Check
s →λ j n
π ( s − λi ) The closed loop characteristic polynomial is
i =1

⎡ s 0⎤ ⎡− 1 0 ⎤ ⎡2⎤
| sI − Λ − B1 K1 | = ⎢ ⎥−⎢ ⎥ − ⎢ ⎥ [− 2 1]
The procedure for the pole assignment can be summarized as
follows: ⎣0 s ⎦ ⎣ 0 − 2 ⎦ ⎣1 ⎦
⎡ s 0⎤ ⎡ − 1 0 ⎤ ⎡− 4 2⎤
• Calculate the residues R j ( j = 1,2, L , n) from Eq. (7.23) = ⎢ ⎥−⎢ ⎥−⎢ ⎥
⎣0 s ⎦ ⎣ 0 − 2 ⎦ ⎣ − 2 1 ⎦
⎡n n ⎤
( s − λ j ) ⎢ π ( s − λi ) − π ( s − ρ i ) ⎥ s+5 −2
⎣i =1 i =1 ⎦ =
R j = lim 2 s +1
s →λ j n
π ( s − λi ) = ( s + 3) 2
i =1
________________________________________________________________________________________
• Calculate ri after choosing f i using Eq. (19)
B1f = r = [r1 r2 L rn ]′ In general case, if the state matrix is not necessarily in a
diagonal form. Assume that
• Calculate d i from the relation between (7.21) and (7.22)
d jrj = R j (7.24) x& = A x + B u (7.25)
• Calculate gain K1 from Eq. (7.15)
Using the transformation x = T z , Eq. (7.25) becomes
K1 = f d ′
___________________________________________________________________________________________________________
Chapter 7 Multivariable Feedback and Pole Location 38
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

z& = T −1 AT z + T −1 B u And the closed loop system poles are the roots of the
(7.26) characteristic equation
= Λ z + B1u
s + 59 98
where Λ = T −1 AT and B1 = T −1 B . = s 2 + 6s + 9 = 0
− 32 s − 53

Applying the inverse transformation, z = T −1x to the as required.

________________________________________________________________________________________
resulting state equation, we obtain
7.4 Observers
x& = T ΛT −1x + TB1 K1T −1x
(7.27)
= A x + BK x In may control system it is not possible to measure entire
state vector, although the measurement of some of the state
where variables is practical. Fortunately, Luenberger has shown that
a satisfactory estimate of the state vector can be obtained by
using the available system inputs and outputs to derive a
K = K1T −1 (7.28) second linear system known as an observer.

is the proportional gain matrix associated with the state of the The observer is a dynamic system whose state vector,
system specified by Eq. (7.25). say z (t ) is an approximation to a linear transformation of the
state x(t ) , that is
Example 7.4_______________________________________
Given z (t ) ≈ T x(t ) (7.29)
⎡ 4 10 ⎤ ⎡9 ⎤
x& = ⎢ ⎥x + ⎢ ⎥u The entire state vector z is of course, available for feed back.
⎣ − 3 − 7 ⎦ ⎣ − 5⎦
Consider the system
Find the proportional gain matrix K such that the closed loop
poles are both at λ = −3 . x& = Ax + Bu
(7.30)
On applying the transformation y = c ′x

⎡2 5⎤ where A ∈ Rn×n , B ∈ Rn×m and c ′ ∈ R1×n .

x =Tz = ⎢ ⎥z
⎣ − 1 − 3⎦
To simplify the mathematical manipulation, the system has
been assumed to have a single output. But the results are
for which applicable to multi-output systems.

⎡3 5 ⎤
T −1 = ⎢
u x y
⎥ system
⎣ − 1 − 2⎦ x& = Ax + Bu output

It is found that the system is the same as the one considered

in example 7.3. Hence
z
K1 = [−2 1]
observer
z& = Fz + hy + Nu

and Fig. 7.1

⎡3 5 ⎤ If the dynamics of the observer when regarded as a free

K = K1T −1 = [− 2 1] ⎢ ⎥ = [− 7 − 12] system (that is when the input is zero) are defined by z& = F z
⎣ − 1 − 2⎦
then, when the inputs are applied, using equation
so the feedback control
z& = F z + h y + N u (7.31)
u = [−7 12] x
where h is an arbitrary vector (of order n × 1 ). Using (7.30)
which achieve the desired pole assignment. and (7.31), we form the difference

Check z& - T x& = F z + h y + N u - TAx-TBu

(7.32)
The closed loop system yields = F (z − Tx) + (hc ′ − TA + FT )x + ( N − TB )u

⎡ 4 10 ⎤ ⎡−63 −108⎤ ⎡−59 −98⎤ If we choose

A + BK = ⎢ ⎥+⎢ ⎥=⎢ ⎥
⎣− 3 − 7 ⎦ ⎣ 35 60 ⎦ ⎣ 32 53 ⎦ hc ′ = TA − FT
(7.33)
N = TB
___________________________________________________________________________________________________________
Chapter 7 Multivariable Feedback and Pole Location 39
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Eq. (7.32) becomes that is

d
( z − Tx) = F ( z − Tx) λ 2 + (h1 + 3)λ + (2h1 + h2 + 2) = 0
dt
Since the observer eigenvalues are at λ = −3 , the above
which has the solution equation is identical to

z − Tx = e Ft [z (0) − T x(0)] λ2 + 6λ + 9 = 0

or hence
z = Tx + e Ft [z (0) − T x(0)] (7.34)
h1 + 3 = 6
If it is possible to chose z (0) = T x(0) then z = T x for all 2h1 + h2 + 2 = 9
t ≥ 0 . Generally it is more realistic to assume that z (0) is
only approximately equal to T x(0) , in which case it is so that

required that e Ft approaches the zeros matrix as rapidly as ⎡ h ⎤ ⎡3⎤

possible. This can be achieved by choosing the eigenvalues h = ⎢ 1⎥ = ⎢ ⎥
⎣h2 ⎦ ⎣1⎦
of F to have enough large negative real parts.

Having obtained an estimate of T x , now we construct a It follows that the state matrix of the required observer is

system having the transfer function T −1 to obtain x . This can ⎡ −4 1 ⎤

be avoided if we choose T to be the identity F =⎢ ⎥
transformation I . The resulting system is called an identity ⎣ − 1 − 2⎦
observer.
Check
In this case equations (7.33) becomes On computing the difference z& − x& , we find

hc ′ = A − F ⎡ −4 1 ⎤ ⎡3 0⎤ ⎡1 −1⎤
(7.35) z& − x& = ⎢ ⎥z + ⎢ ⎥x+ ⎢ ⎥u
N=B ⎣− 1 − 2⎦ ⎣1 0⎦ ⎣0 1 ⎦
⎡− 1 1 ⎤ ⎡1 − 1⎤
From Eq. (7.35) we obtain −⎢ ⎥x− ⎢ ⎥u
⎣ 0 − 2 ⎦ ⎣0 1 ⎦
F = A − hc ′ (7.36) ⎡− 4 1 ⎤
=⎢ ⎥ ( z − x)
⎣ − 1 − 2⎦
so that the observer eigenvalues are seen to be determined by
the choice of h .
as required.
________________________________________________________________________________________
Example 7.7_______________________________________
Design an identity observer having both eigenvalues at
λ = −3 for the system defined by the following state
equations

⎡ −1 1 ⎤ ⎡1 −1⎤
x& = ⎢ ⎥x+ ⎢ ⎥u
⎣ 0 − 2 ⎦ ⎣0 1 ⎦
y = [1 0] x

y = [1 0] x ⇒ only the state variable x1 is available at the

output.

F = A − hc ′
⎡− 1 1 ⎤ ⎡ h1 ⎤
=⎢ ⎥ − ⎢ ⎥ [1 0]
⎣ 0 − 2 ⎦ ⎣ h2 ⎦
⎡− 1 − h1 1 ⎤
=⎢ ⎥
⎣ − h2 − 2⎦

The observer characteristic equation is

| λI − F | = 0
___________________________________________________________________________________________________________
Chapter 7 Multivariable Feedback and Pole Location 40
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.8 Introduction to Optimal Control

8.1 Control and Optimal Control Here v = (v1 , v 2 ) and the external force is the gravity force
only Fext = (0,−mg ) .
Take an example: the problem of rocket launching a satellite
into an orbit about the earth.
• A control problem would be that of choosing the thrust The minimum fuel problem is to choose the controls, β and
angle and rate of emission of the exhaust gases so that the φ , so as to take the rocket from initial position to a
rocket takes the satellite into its prescribed orbit. prescribed height, say, y , in such a way as to minimize the
• An associated optimal control problem is to choose the
fuel used. The fuel consumed is
controls to affect the transfer with, for example, minimum
expenditure of fuel, or in minimum time. T

8.2 Examples ∫ β dt
o
(8.25)

8.2.1 Economic Growth where T is the time at which y is reached.

8.2.2 Resource Depletion
8.2.3 Exploited Population 8.2.6 Servo Problem
8.2.4 Advertising Policies A control surface on an aircraft is to be kept at rest at a
8.2.5 Rocket Trajectories position. Disturbances move the surface and if not corrected
The governing equation of rocket motion is it would behave as a damped harmonic oscillator, for
example
dv
m = m F + Fext (8.22)
dt θ&& + a θ& + w 2θ = 0 (8.26)
where
m : rocket’s mass where θ is the angle from the desired position (that is, θ = 0 ).
v : rocket’s velocity
The disturbance gives initial values θ = θ 0 , θ& = θ 0′ , but a
F : thrust produced by the rocket motor
Fext : external force servomechanism applies a restoring torque, so that (8.26) is
modified to
It can be seen that
θ&& + a θ& + w 2θ = u (8.27)
c
F = β (8.23) The problem is to choose u (t ) , which will be bounded by
m
where u (t ) ≤ c , so as to bring the system to θ = 0, θ& = 0 in
c : relative exhaust speed
minimum time. We can write this time as
β = − dm / dt : burning rate
T
Let φ be the thrust attitude angle, that is the angle between ∫ 1.dt
0
(8.28)
the rocket axis and the horizontal, as in the Fig. 8.4, then the
equations of motion are and so this integral is required to be minimized.
dv1 cβ
= cos φ 8.3 Functionals
dt m
(8.24)
dv 2 cβ All the above examples involve finding extremum values of
= sin φ − g integrals, subject to varying constraints. The integrals are all
dt m
of the form

t1
v path of rocket J=
∫ t0
F ( x, x& , t )dt (8.29)

r where F is a given function of function x(t ) , its derivative

v x& (t ) and the independent variable t . The path x(t ) is defined
for t 0 ≤ t ≤ t1
• given a path x = x1 (t ) ⇒ give the value J = J 1
φ • given a path x = x 2 (t ) ⇒ give the value J = J 2
x in general, J 1 ≠ J 2 , and we call integrals of the form (8.29)
Fig. 8.4 Rocket Flight Path functional.

___________________________________________________________________________________________________________
Chapter 8 Introduction to Optimal Control 41
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

8.4 The Basic Optimal Control Problem

Consider the system of the general form x& = f (x, u, t ) , where

f = ( f1 , f 2 , L , f n ) ′ and if the system is linear

x& = A x + B u (8.33)

The basic control is to choose the control vector u ∈ U such

that the state vector x is transfer from x 0 to a terminal point
at time T where some of the state variables are specified.
The region U is called the admissible control region.

If the transfer can be accomplished, the problem in optimal

control is to effect the transfer so that the functional

T
J=
∫f 0
0 ( x, u, t ) dt (8.34)

is maximized (or minimized).

___________________________________________________________________________________________________________
Chapter 8 Introduction to Optimal Control 42
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.9 Variational Calculus

9.1 The Brachistochrone Problem Suppose we are looking for the path y = y (x) between A
and B (Fig.9.3) which gives an extremum value for the
Consider the Johann Bernoulli problem: Find the trajectory functional
from point A to point B in a vertical plane such that a body
slides down the trajectory (under constant gravity and no t1
friction) in minimum time.
A b y
∫
J = F ( y, dy / dx, x)dx
t0
(9.5)

y B
β
r r
g v
Cε

a εη
B ( a , b) C0
A
x α
Fig.9.1 The Brachistochrone Problem
a b x
Introduce coordinates as in Fig.9.1, so that A is the point Fig.9.3 Possible paths between A and B
A(0,0) and B is B (a, b) . Assuming that the particle is
released from rest at A , conservation of energy gives Let C 0 be the required curve, say y 0 ( x) , and let Cε be a
neighboring curves defined by
1 m v2 − m g x = 0 (9.1)
2
yε ( x) = y 0 ( x) + ε η ( x) (9.6)
where,
The particle speed is v = x& 2 + y& 2 . From Fig.9.2, we can ε : small parameter
see that an element of arc length, δ s , is given approximately η (x) : arbitrary differential function of x ,
η (a) = η (b) = 0 .
by δ s ≈ δ x 2 + δ y 2
Thus equation (9.6) generates a whole class of neighboring
curve Cε , depending on the value of ε . The value ε = 0
δs gives the optimal curve C 0 .
δy
The value of J for the path Cε for the general problem is
δx given by
Fig.9.2 Arc Length Element
b dη
∫ F(y
dy
J= 0 + ε η , y 0′ + ε η ′, x)dx , y′ = , η′ =
Hence, a dx dx

δ s ds ⎛ dx ⎞ ⎛ dy ⎞
2 2 For a given function y (x) , J = J (ε ) . For extremum values of
v = lim = = ⎜ ⎟ +⎜ ⎟ (9.2)
δ t →0 δ t dt ⎝ dt ⎠ ⎝ dt ⎠ J , we must have

dJ
From (9.1) and (9.2), dt / ds = (2 gx) −
1
2 so that on integrating, =0 (9.7)
dε ε =0
the time of descent is given by
Taking the differentiation yields
a 1 + (dy / dx) 2
∫ (2 gx) ∫
ds 1
t= = dx (9.3)
∫ [η F ( y
1 1
x dJ b
0
2 (2 g ) 2 = + ε η , y 0′ + ε η ′, x) +
dε
y 0
a

Our problem is to find the path y = y (x) such that y (0) = 0, y ′F y′ ( y 0 + ε η , y 0′ + ε η ′, x ) dx ]

y (a ) = b and which minimizes (9.3). where
dF dF
Fy = , F y′ =
9.2 Euler Equation dy dy ′

___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 43
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

dF dF dy dF dy ′ Example 9.1_______________________________________
= + = η F y + η ′F y′
dε dy dε dy ′ dε Find the curve y (x) which gives an extremum value to the
y = y0 + εη functional
y ′ = y 0′ + ε η ′
1

Applying condition (9.7) we obtain ∫

J = ( y ′ 2 + 1)dx
0

∫ [η F ( y , y′ , x) +η′F ]dx = 0
b with
y 0 0 y ′ ( y0 , y0′ , x )
a
y (0) = 1, y (1) = 2
From now on all the partial derivations of F are evaluated on
the optimal path C 0 , so we will leave off the ( y 0 , y 0′ , x) Here, F = y ′ 2 + 1 ⇒ F y = 0 . Hence the Euler equation
dependence. With this notation, on the optimal path, (9.10) becomes

b
∫ (η F
d
y + η ′F y′ )dx = 0 ( F y′ ) = 0
a dx

and the integrating the second term by parts gives and on integrating, F y′ = A constant, that is

b
⎡ ⎤ F y′ = 2 y ′ = A
∫
b d
η F y′ + ⎢η F y − η ( F y′ )⎥ dx = 0
a ⎣ dx ⎦
a
Integrating again,
or
b
⎡ ⎤ y = Ax / 2 + B
∫
d
η (b) F y′ − η ( a ) F y′ + ⎢η F y − η ( F y′ )⎥ dx = 0
x =b x =a ⎣ dx ⎦
a From boundary conditions A = 2, B = 1 , and so the optimal
(9.8) curve is the straight line
But η (a ) = η (b) = 0 , so we are left with
y = x +1
b
⎡ ⎤
∫
d
⎢η F y − η dx ( F y′ )⎥ dx = 0 (9.9) as illustrated in Fig. 9.5
⎣ ⎦
a
y
This must hold for all differentiable functions η (x) such that
2
η (a) = η (b) = 0

We now require the following Lemma.

1
Lemma
Let η (x) be a differentiable function for a ≤ x ≤ b such that
η (a) = η (b) = 0 . If f is continuous on a ≤ x ≤ b and x
0 1
b Fig. 9.5 Optimal Path
∫ f ( x)η ( x) dx = 0
a
The corresponding extremum value of J is

1 1
for all such η (x) , then f = 0 on a ≤ x ≤ b .
∫
J = ( y ′ 2 + 1)dx = 2 dx = 2
∫
Applying the result of this Lemma to (9.9), immediately 0 0
_________________________________________________________________________________________
gives
Example 9.2_______________________________________
d
Fy − ( F y′ ) = 0 (9.10) Find the curve y (x) which has minimum length between
dx
(0,0) and (1,1)
on the optimal path. This is known as Euler’s equation (or
the Euler-Lagrange equation) and must be satisfied by Let y = y ( x), 0 ≤ x ≤ 1 be any curve between (0,0) and (1,1) .
paths y (x ) which yield extremum values of the functional J . The length, L , is given by

___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 44
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

1 This is a question of a cycloid, the constant c being chosen so

L=
∫
0
1 + y ′ 2 dx that the curve passes through the point (a, b) . A typical
example of such a curve is shown in Fig. 9.6

The integrand F = 1 + y ′ 2 is independent of y , that is, A(0,0) x

F y = 0 . Hence (9.10) becomes

d
( F y′ ) = 0
dx

and on the integrating F y′ = A , constant. Thus B ( a , b)

y
y′ Fig. 9.6 (a,b) Cycloid from A and B
=A
1 + y′ 2
⊗ Note that: Our theory has only shown us that we have a
stationary value of t on this path. The path found does in fact
which implies that y ′ = B = constant. Hence y = B x + C , corresponding to minimum t .
________________________________________________________________________________________
where C is a constant. To pass through (0,0) and (1,1), we
obtain y = x . The Euler equation, (9.9), takes special forms when the
________________________________________________________________________________________
integrand F is independent of either y or x :
Example 9.3 The Brachristochrone problem_____________
1) F independent of y
Determine the minimum value of the functional (9.3), that is
d d
From (9.10): F y − ( F y′ ) = 0 ⇒ ( F y′ ) = 0 , hence
a dx dx
1 + y′2
2g ∫
1
t= dx
x F y′ = constant (9.12)
0

such that y (0) = 0 and y (a ) = b . 2) F independent of x

In general

1 + y′2 dF ∂F dy ∂F dy ′ ∂F
F= = + + = y ′F y + y ′′F y
x dx ∂y dx ∂y ′ dx {∂x
d =0
⇒ ( F y′ ) = F y = 0 d d
dx Using (9.10): F y − ( F y′ ) = 0 ⇒ F y = ( F y′ ) , give
y′ dx dx
⇒ F y′ = = c , constant. dF d d
y (1 + y ′ 2 ) = y ′ ( F y′ ) + y ′′F y = ( y ′F y′ ) , therefore
dx dx dx
c2x
⇒ y′ = F − y ′F y′ = constant (9.13)
1− c2x

∫
x
and y=c dx Example 9.4 Minimum Surface Area___________________
x − c2 x2
A plane curve, y = y ( x ) passes through the points (0, b) and
( a, c), (b ≠ c ) in the x − y plane ( a, b, c are positive constants)
1
Let define x ≡ (1 − cos θ ) ⇒ dx / dθ = sin θ /( 2c 2 ) , and the curve lies above the x-axis as shown in Fig. 9.7.
2c 2
therefore y
1 (a, c )
y= (θ − sin θ ) + A , A is a constant c
2 c2

Initial condition: ( x, y ) = (0,0) ≡ θ = 0 ⇒ A = 0 . y = y (x )

b
Hence
1
x= (1 − xosθ )
2 c2
(9.11)
1 a x
y= (θ − sin θ )
2c2 Fig. 9.7 Curve rotates about x-axis

___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 45
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Determine the curve y ( x) which, when rotated about the x-

axis, yields a surface of revolution with minimum surface y
area. B

The area of the surface of revolution is given by C0

2 A
a ⎛ dy ⎞
A=
∫ 2πy ds = 2π
∫ 0
y 1 + ⎜ ⎟ dx
⎝ dx ⎠

so we wish to find the curve y ( x) to minimizes this integral.

a b x
The integrand F = y 1 + y ′ 2 . This is independent of x so we Fig. 9.8 Optimal and Neighboring Curves
can apply (9.13)
d
Fy − ( F y′ ) = 0 (9.17)
F − y ′F y′ = A , constant dx

y′2 y Eq. (9.6) becomes η (b) F y′ − η ( a ) F y′ = 0 . This result

⇒ y 1 + y′2 − =A x =b x=a
1 + y′ 2 must be true for all differentiable functions η (x) . In fact for
the class of all differentiable functions η (x) with the
⇔ y = A (1 + y ′ )
2 2 2
restriction
dy 1
⇔ y′ = = y 2 − A2
dx A η (a) = 0

∫ ∫
dy 1
⇔ = dx
y −A
2 2 A which means that

⇔ cosh −1 ( y / A) = x / A + B , B: constant η (b) F y′ =0

x =b
e x + e−x
Hyperbolic cosine of x = cosh x ≡
2 Since the value of η (b) is arbitrary, we conclude that
so that
∂F
y = A cosh( x / A + B) (9.14) = 0 at x = b (9.18)
∂y ′

The constants are chosen so that the curve passes through

Similarly
(0,b) and (a,c).
_________________________________________________________________________________________

∂F
= 0 at x = a (9.19)
9.3 Free End Conditions ∂y ′
This section generalizes the result obtained in section 9.2 for
and if only free at one end point, then it can be shown that
finding extremum values of functionals.
∂F
= 0 at that end point. We can summarize these results by
Find the path y = y ( x), a ≤ x ≤ b which gives extremum ∂y ′
values to the functional
∂F
= 0 at end point at which y is not specified. (9.20)
b ∂y ′
J=
∫ a
F ( y , y ′, x)dx (9.15)
Example 9.5 ______________________________________
with no restriction on the value of y at either end point.
Find the curve y = y (x) which minimizes the functional
The analysis used in 9.2 follows through directly, except that
1
we no longer have the restriction η (a ) = η (b) = 0 . The J =
∫ ( y′ + 1)dx
2

optimal curve, C∈ , are illustrated in Fig.9.8. 0

where y (0) = 1 and y is free at x = 1 .

Form Eq. (9.8)
b
⎛ ⎞
∫
d We have already seen, from the example 9.1, that the Euler
η (b) F y′ − η ( a ) F y′ + η ⎜ Fy − ( F y′ ) ⎟ dx = 0
equation gives
x =b x=a ⎝ dx ⎠
a
(9.16) x
y= A +B
The free end point also satisfy the Euler equation 2
___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 46
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Apply Eq. (9.20) to give Hence x 2 + &x&1 = 0 and x1 + &x&2 = 0 . Eliminating x 2 gives

∂F
= 0 at x = 1 that is, 2 y ′ = 0 at x = 1 . d 4 x1 ⎧⎪ x1 = Ae t + Be −t + C sin t + D cos t
∂y ′ − x1 = 0 ⇒ ⎨
dt 4 ⎪⎩ x 2 = Ae t + Be −t − C sin t − D cos t
⎧ y (0) = 1 ⎧ A = 0
The boundary conditions ⎨ ⇒⎨ . Hence the
⎩ y ′(1) = 0 ⎩ B = 1
optimal path is ⎧ e −π / 2
⎪A =
⎪ 1 − e −π
y ( x) = 1, 0 ≤ x ≤ 1 ⎪
Applying the boundary conditions gives ⎨ B = − A ,
⎪C = 0
⎪
and the optimum value of J is J = 1 , which is clearly a ⎪⎩ D = 0
minimum.
_________________________________________________________________________________________ Hence the optimum path is

9.4 Constrains
e t − e −t
x1 = x 2 = e −π / 2 0 ≤ t ≤π /2
The results obtained in the last two sections can be further 1 − e −π
_________________________________________________________________________________________
generalized
- firstly to the case where the integrand of more than one
In many optimization problems, we have constraints on the
independent variable,
relevant variables. We first deal with extremum values of a
- secondly to the case of optimization with constraints.
function, say f ( x1 , x 2 ) subject to a constraint of the form
If we wish to find extremum values of the functional g ( x1 , x 2 ) = 0 . Assuming that it is not possible to eliminate
one of x1 or x 2 using the constraint, we introduce a Lagrange
b
J =
∫ F ( x , x ,L, x , x& , x& ,L, x& , t )dt
a
1 2 n 1 2 n (9.21) multiplier, λ , and form the augmented function

f * = f ( x1 , x 2 ) + λ g ( x1 , x 2 ) (9.24)
where, x1 , x 2 , L , x n are independent functions of t
x&1 , x& 2 , L , x& n are differentiation with respect to t We now look for extremum values of f * , that is

We obtain n Euler equations to be satisfied by the optimum ∂f * ∂f *

path, that is = =0 (9.25)
∂x1 ∂x 2

∂F d ⎛ ∂F ⎞
− ⎜ ⎟ = 0 (i = 1,2, L , n) (9.22) Solving these equations, x1 , x 2 will be functions of the
∂xi dt ⎜⎝ ∂x& i ⎟
⎠ parameter λ , that is, x1 (λ ), x 2 (λ ) . And appropriate value of
λ is found by satisfying g ( x1 (λ ), x 2 (λ )) = 0 . Since
Also at any end point where xi is free
f * ≡ f , we have the condition for the extremum values
∂F for f .
=0 (9.23)
∂x& i
Example 9.7 ______________________________________
Example 9.6 ______________________________________ Find the extremum value of
Find the extremals of the functional
f ( x1 , x 2 ) = x12 + x 22
π /2
J =
∫ 0
( x&12 + x& 22 + 2 x1 x 2 )dt subject to x1 + 3 x 2 = 4 .

subject to boundary conditions x1 (0) = 0 , x1 (π / 2) = 1 , We first note that we can easily eliminate x1 from the
x 2 (π / 2) = 1 . constraint equation to give

Using Eq. (9.22) to give f = (4 − 3 x 2 ) 2 + x 22

⎧ ∂F d ⎛ ∂F ⎞ and we can then solve the problem by evaluating df / dx 2 = 0 .

⎪ − ⎜⎜ ⎟=0 ⎛ d
⎟ ⎜ 2 x 2 + (2 x&1 ) = 0
⎪ ∂x1 dt ⎝ ∂x&1 ⎠ ⎜ dt
⎨ ⇒ Using a Lagrange multiplier, we consider the augmented
⎪ ∂F d ⎛ ∂F ⎞ ⎜ d
⎪ ∂x − ⎜ ⎟ = 0 ⎜ 2 x1 + (2 x& 2 ) = 0 function
⎜ ⎟ ⎝
⎩ 2 dt ⎝ ∂x& 2 ⎠
dt
f * = x12 + x 22 + λ ( x1 + 3 x 2 − 4)
Hence
___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 47
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

d ⎛⎜ ⎞
For extremum value of f * ,
z& ⎟=0
y: λ −
⎜
dx 2 2 g x (1 + y& z& ) ⎟
⎧ ∂f * ⎝ ⎠
⎪ ∂x = 0
⎪ 1 ⎧ 2 x1 + λ = 0
⇒⎨ d ⎛⎜ y& ⎞
⎟=0
⎨ z: −λ −
⎪ ∂f *
=0 ⎩2 x 2 + 3λ = 0 dx ⎜ 2 2 g x (1 + y& z& )
⎝
⎟
⎠
⎪⎩ ∂x 2
Eliminating λ , we obtain
Hence x1 = λ / 2, x 2 = −3λ / 2 and to satisfy the constraint

d ⎛⎜ y& + z& ⎞
⎟=0
(−λ / 2) + [3(−3λ / 2)] = 4 ⇒ λ = −4 / 5
dx ⎜ 2 2 g x(1 + y& z& ) ⎟
⎝ ⎠
and the extremum value of f occurs at
y& + z&
⇒ =A
x1 = 2 / 5, x2 = 6 / 5 x(1 + y& z& )
_________________________________________________________________________________________

We can now return to the main problem: Suppose we wish using the constraint z = y − 1 , we obtain for the optimum path
for extremum values of (9.21)
y& A
b b =
∫ F ( x , x ,L, x , x& , x& ,L, x& , t )dt = ∫ F (x, x& , t )dt
J= x(1 + y ) &2 2
1 2 n 1 2 n
a a _________________________________________________________________________________________
(9.21)
subject to a constraint between x1 , x 2 , L , x n , x&1 , x& 2 , L , x& n , t Example 9.9 ______________________________________
which we write as Minimize the cost functional
g (x, x& , t ) = 0 (9.26) 2
∫
1
J= &x&2 dt
2 0
then we introduce a Lagrange multiplier, λ , and form the
augmented functional where x(0) = 1, x& (0) = 1; x(2) = 0, x& (2) = 0
b
∫
J * = ( F + λ g )dt
a
(9.27) To use Lagrange multiplier, we introduce the state variable
x1 = x, x 2 = x& . The functional is now

We now find the extremum values of J * in the usual way, 2

∫
1
regarding the variables as independent. The Lagrange J= x& 2 dt
multiplier can then be eliminated to yield the required 2 0
optimum path.
and we have the constraint
Example 9.8 ______________________________________
x 2 − x&1 = 0
Minimize the cost functional
The augmented functional is
a 1 + y&z&
∫
1
J= dx
2⎛ 1 ⎞
∫ ⎜⎝ 2 x&
2g 0 x
J* = 2
2 + λ ( x 2 − x&1 ) ⎟ dt
0 ⎠
where the variables y = y ( x), z = z ( x) are subject to the
constraint Euler equation yields

y = z +1 d
x1 : 0 − ( −λ ) = 0
dt λ& = 0
⇒
and where y (0) = 0, y (a ) = b d λ − &x&2 = 0
x 2 : λ − ( x& ) = 0
dt
Introducing the Lagrange multiplier λ , the augmented
functional is d 3 x2
Eliminating λ , we obtain = 0 , and
dt 3
b⎡ 1 + y& z& ⎤
∫ ⎢⎣⎢
1
J* = + λ ( y − z − 1)⎥ dx t3 t2
x1 = A +B +Ct + D
⎦⎥
a 2g x
3 2
x2 = A t 2 + B t + C
Euler equation yields
___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 48
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.4
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Applying the end conditions implies

1 3 7 2
x1 = t + t +t +1
2 4
3 2 7
x2 = t + t + 1
2 2

y& + z&
⇒ =A
x(1 + y& z& )

and we obtain for the optimum path

2
∫ (3t − 7 / 2) dt = 13 / 4
1
J= 2
2 0
_________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 9 Variational Calculus 49
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.10 Optimal Control with Unbounded Continuous Controls

10.1 Introduction and the corresponding optimal control is recovered from

(10.2), giving
The methods appropriate for variational calculus can readily
extended to optimal control problems, where the control sinh 2 (T − t ) − 2 cosh 2 (T − t )
u = x0
vector u is unbounded. Technique for dealing with the sinh 2T
general case u ∈ U , a bounded admissible control region, will
be discussed in the next chapter. 10.2 The Hamiltonian

We start with a simple example. Suppose we wish to find Consider the general problem of minimizing
extremum values of the functional
T

J=
∫
T
( x + u )dt
2 2
(10.1)
J=
∫fo
0 ( x, u , t ) dt (10.6)
o
subject to the differential equation
where x : the state variable
u : the control variable x& = f ( x, u , t ) (10.7)
satisfy the differential equation
and boundary condition x(0) = a . With Lagrange multiplier,
x& + x = u (10.2) λ , we form the augmented functional
T
with the boundary conditions x(0) = x 0 , x(T ) = 0 . J* =
∫ {fo
0 ( x, u , t ) + λ [ f ( x, u , t ) − x& ]} dt

A direct method of solving this problem would be to

The integrand F = f 0 ( x, u , t ) + λ [ f ( x, u , t ) − x& ] is a function
eliminate u from (10.2) and (10.1), which reduces it to a
straightforward calculus of variations problem. In general, it of the two variables x and u, and so we have two Euler
will not able to eliminate the control, so we will develop equations
another method based on the Lagrange multiplier technique
for dealing with constraints. ⎧ ∂F d ⎛ ∂F ⎞ ⎧ ∂f 0 ∂f
− ⎜ ⎟=0 &
⎪
⎪ ∂x dt ⎝ ∂x& ⎠ ⎪⎪ ∂x + λ ∂x + λ = 0
We introduce the Lagrange multiplier, λ , and from the ⎨ ⇒ ⎨ (10.8-9)
⎪ ∂F − d ⎛⎜ ∂F ⎞⎟ = 0 ⎪ ∂f 0 ∂f
augmented functional +λ =0
⎪⎩ ∂u dt ⎝ ∂u& ⎠ ⎩⎪ ∂u ∂u
T
J* =
∫ 0
[ x 2 + u 2 + λ ( x& + x − u )] dt (10.3) Now defining the Hamintonian

H = f0 + λ f (10.10)
The integrand F = x 2 + u 2 + λ ( x& + x − u ) is a function of two
variables x and u. We can evaluate Euler’s equation for x and
u, that is (10.8) and (10.9) can be rewritten as

∂H
∂F d ⎛ ∂F ⎞ λ& = − (10.11)
− ⎜ ⎟=0 ⇒ 2 x + λ − λ& = 0 (10.4) ∂x
∂x dt ⎝ ∂x& ⎠
∂H
∂F d ⎛ ∂F ⎞ 0= (10.12)
− ⎜ ⎟=0 ⇒ 2u − λ = 0 (10.5) ∂u
∂u dt ⎝ ∂u& ⎠
These are the equations, together with (10.2), which govern
From (10.2), (10.4) and (10.5), we obtain the optimal paths.

&x& − 2 x = 0 Example 10.1______________________________________

− 2t
⇒ x = Ae 2t
+ Be Using Hamintonian method, find the optimal control u which
yields a stationary value for the functional
or, alternatively x = a cosh 2T + b sinh 2T .With the 1
boundary conditions, we obtain
∫
J = ( x 2 + u 2 )dt
0
where x& = u

x 0 sinh 2 (T − t )
x=
sinh 2T with x(0) = 1 and not fixed at t = 1 .
___________________________________________________________________________________________________________
Chapter 10 Optimal Control with Unbounded Continuous Controls 50
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Here f 0 = x 2 + u 2 , f = u and the Hamintonian is with α = 4 , (10.20) has solutions of the form x1 = e mt , where
m satisfies
H = x2 + u2 + λ u 2
1 ⎛ 2 1⎞
m4 − m2 + = ⎜m − ⎟ = 0
2 ⎝ 2⎠
∂H
(10.11): λ& = − ⇒ λ& = −2 x
∂x with the boundary conditions, we get
∂H
(10.12): 0 = ⇒ 2u + λ = 0
∂u x1 = [a + (b + a / 2 ) t ] e −t / 2
(10.23)
−t / 2
x 2 = [b − (b + a / 2 ) t / 2 ] e (10.24)
Hence we obtain &x& − x = 0 with the solution is
u = [− a / 2 − ( 2 − 1)b − (b + a / 2 )(1 − 1 / 2 ) t / 2 ] e −t / 2
x = a sinh t + b cosh t
(10.25)
General Problem
From the boundary conditions
The basic problem is to find the optimal controls u which
cosh(1 − t ) sinh(1 − t )
x= and u = − yield extremal values of
cosh 1 cosh 1
T
∫f
_________________________________________________________________________________________
J= 0 (x, u, t ) dt (10.27)
0
10.3 Extension to Higher Order Systems
subject to the constraints
The method described in the last section can be readily
extended to higher order systems. For example, we wish to
minimize x& i = f i (x, u, t ) (i = 1,2, L , n) (10.28)

∞ where, x = [x1 x 2 L x n ]T and u = [u1 u 2 L u n ]T

∫
1
J= ( x + α u ) dt
2 2
(10.13)
2 0
We introduce the Lagrange multipliers, pi (i = 1,2, L , n)
where, usually called the adjoint variables and form the augmented
functional
&x& = − x& + u , α is constant (10.14)
⎡ n ⎤
∑p (f
T
and initially x = a , x& = b , and as t → ∞ , both x and x& → 0 . J* =
∫ 0
⎢ f0 +
⎢
⎣
i i − x& i )⎥ dt
⎥
⎦
Following the usual technique, we define x1 ≡ x, x 2 ≡ x& so i =1

that we have two constraints

We also define the Hamintonian, H, is
x 2 − x&1 = 0 (10.15)
n
− x 2 + u − x& 2 = 0 (10.16) H = f0 + ∑p f
i =1
i i (10.29)

The augmented function is so that

⎡ n ⎤
∑ p x& ⎥⎥ dt
T
∫
∞ ⎡1 ⎤ ⎢H −
J* =
∫ ⎢ 2 ( x1 + α u ) + λ1 ( x 2 − x&1 ) + λ 2 (− x 2 + u − x& 2 )⎥ dt
J* =
2 2 i i
⎣ ⎦ 0 ⎢
0 ⎣ i =1 ⎦

We now have three Euler equations, namely, n

∂F d ⎛ ∂F ⎞
The integrand, F = H − ∑ p x&
i =1
i i depends on x, u, t, and we

− ⎜ ⎟=0 ⇒ x1 + λ&1 = 0 (10.17)

∂x1 dt ⎜⎝ ∂x&1 ⎟⎠ form (n+m) Euler equations, namely

∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ λ1 − λ 2 + λ&2 = 0 (10.18) ∂F d ⎛ ∂F ⎞ ∂H
∂x 2 dt ⎜⎝ ∂x& 2 ⎟ − ⎜ ⎟ = 0 (i = 1,2, L , n) that is, p& i = −
⎠ ∂xi dt ⎜⎝ ∂x& i ⎟
⎠ ∂xi
∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ α u + λ2 = 0 (10.19) (10.30)
∂u dt ⎝ ∂u& ⎠
which are known as the adjoint equations; and

Eliminating λi , we obtain
∂F d ⎛ ∂F ⎞
⎟ = 0 ( j = 1,2, L , m) that is, ∂H = 0
− ⎜
∂u j dt ⎜⎝ ∂u& j ⎟
⎠ ∂u j
d 4 x1 d 2 x1 x1
− + =0 (10.20) (10.31)
dt 4
dt 2 α
___________________________________________________________________________________________________________
Chapter 10 Optimal Control with Unbounded Continuous Controls 51
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

The optimal solutions are determined from (10.28,30,31). Example 10.3______________________________________

Using boundary conditions given in section 8.4, we have
Find the optimal control u which gives an extremum value of
xi (0) given (i = 1,2, L , n) and xl (T ), (l = 1,2, L , q ) are given.
the functional
The remaining values x q +1 (T ), L , x n (T ) are free, and so we
1
must apply the free end condition (9.23):

∂F
J = u 2 dt
∫
0
where x&1 = x 2 , x& 2 = u

= 0 (k = q + 1, L , n) at t = T
∂x& i
with x1 (0) = 1 , x1 (1) = 0 , x 2 (0) = 1 and x 2 (1) is not specified.
This gives
The Hamintonian is
p k (T ) = 0 (k = q + 1, L , n) (10.32)
H = u 2 + p1 x 2 + p 2 u
which is known as the transversality condition. In general
∂H 1
pi (T ) = 0 for any xi not specified at t = T . (10.31): = 0 = 2u + p 2 ⇒ u=− p2
∂u 2
p& 1 = 0 p1 = A
Example 10.2______________________________________ (10.30): ⇒
p& 2 = − p1 = − A p 2 = − At + B
Find the control u which minimizes
A B
1 Hence u= t−
∫
2 2
J = u dt 2
where x& = u + ax , a is constant, and
⎧ A 2 B
0
A B ⎪⎪ x 2 = 4 t − 2
t +C
x& 2 = t − ⇒ ⎨
(i) x(0) = 1 2 2 ⎪x = A t 3 − B 2
t + Ct + D
⎪⎩ 1 12 4
(ii) x(0) = 1 , x(1) = 0

Boundary conditions give A = 12, B = 12, C = 1, D = 1 . The

The Hamintonian is H = u 2 + p (u + ax) where p is the
optimal control is
adjoint variable.
u = 6(t − 1), 0 ≤ t ≤ 1
(10.31): 2u + p = 0 _________________________________________________________________________________________

(10.30): p& = − ap
⎧ p = Ae − at
⎪ A − at
⇒ ⎨ A −at ⇒ x& − ax = − 2 e
⎪ u = − e
⎩ 2
A −at
and x = Be + at
e , the constants A and B will be
4a
calculated from boundary conditions.

(i). x(0) = 1 and x(1) is not specified

Here B + A / 4a = 1 and since x(1) is not specified, we
apply the transversality condition (10.32) at t = 1 ,
which is p (1) = 0 , that is A = 0 . The optimal control
u = 0, 0 ≤ t ≤ 1 . This gives J = 0 , which is clearly a
minimum as J ≥ 0 .
(ii) x(0) = 1 , x(1) = 0
Now, we have

B + A / 4a = 1
Be a + Ae − a / 4a = 0

Hence, A = 4a /(1 − e −2a ) and the optimal control is

u = −2ae − at /(1 − e −2a ), 0 ≤ t ≤ 1

_________________________________________________________________________________________

___________________________________________________________________________________________________________
Chapter 10 Optimal Control with Unbounded Continuous Controls 52
Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.11 Bang-bang Control

11.1 Introduction ∂F d ⎛ ∂F ⎞
− ⎜⎜ ⎟=0 ⇒
⎟ p& 2 = − p1 (11.10)
∂x 2 dt ⎝ ∂x& 2 ⎠
This chapter deals with the control with restrictions: is
bounded and might well be possible to have discontinuities. ∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ p 2 = µ ( β − α − 2u ) (11.11)
∂u dt ⎝ ∂u& ⎠
To illustrate some of the basic concepts involved when
controls are bounded and allowed to have discontinuities we ∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 2vµ = 0 (11.12)
start with a simple physical problem: Derive a controller such ∂v dt ⎝ ∂v& ⎠
that a car move a distance a with minimum time.
(11.12) ⇒ v = 0 or µ = 0 . We will consider these two
The motion equation of the car
cases.

d 2x (i) µ =0
=u (11.1)
dt 2 ⎧p = 0
⇒⎨ 1 ⇒ be impossible.
where ⎩ p2 = 0
(ii) v = 0
u = u (t ), − α ≤ u ≤ β (11.2) (11.5): v 2 = (u + α )( β − u ) ⇒ u = −α or u = β
Hence
represents the applied acceleration or deceleration (braking)
and x the distance traveled. The problem can be stated as
⎧β 0 ≤ t ≤τ
minimize x& 2 = ⎨
⎩− α τ <t ≤T
T
T = 1 dt
∫o
(11.3) the switch taking place at time τ . Integrating using
boundary conditions on x 2
subject to (10.11) and (10.12) and boundary conditions
⎧β t 0 ≤ t ≤τ
x(0) = 0, x& (0) = 0, x(T ) = a, x& (T ) = 0 (11.4) x 2 = x&1 = ⎨ (11.13)
⎩ − α (t − T ) τ <t ≤T
The methods we developed in the last chapter would be
Integrating using boundary conditions on x1
appropriate for this problem except that they cannot cope
with inequality constraints of the form (11.2). We can
change this constraint into an equality constraint by ⎧1 2
introducing another control variable, v, where ⎪⎪ β t 0 ≤ t ≤τ
x1 = ⎨ 2 (11.14)
⎪− 1 α (t − T ) 2 + a τ <t ≤T
v = (u + α )( β − u )
2
(11.5) ⎪⎩ 2

Since v is real, u must satisfy (11.2). We introduce th usual Both distance, x1 , and velocity, x 2 , are continuous at
state variable notation x1 = x so that
t = τ , we must have

x&1 = x2 x1 (0) = 0, x2 (T ) = a (11.6) (11.14) ⇒ βτ = −α (τ − T )

x& 2 = u x 2 (0) = 0, x 2 (T ) = 0 (11.7) 1 1
(11.15) ⇒ βτ 2 = a − α (τ − T ) 2
2 2
We now form the augmented functional
Eliminating T gives the switching time as
T
T* =
∫ 0
{1 + p1 ( x 2 − x&1 ) + p 2 (u − x& 2 ) + µ [v 2
2aα
−(u + α )( β − u )]}dt (11.8) τ= (11.15)
β (α + β )

where p1 , p 2 ,η are Lagrange multipliers associated with the

and the final time is
constraints (11.6), (11.7) and (11.5) respectively. The Euler
equations for the state variables x1 , x 2 and control variables u
2a (α + β )
and v are T= (11.16)
αβ
∂F d ⎛ ∂F ⎞
− ⎜ ⎟=0 ⇒ p& 1 = 0 (11.9) The problem now is completely solved and the optimal
∂x1 dt ⎜⎝ ∂x&1 ⎟⎠ control is specified by

Chapter 11 Bang-bang Control 53

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

⎧β 0 ≤ t ≤τ ∂H
u=⎨ (11.17) p& i = − (i = 1,2, L , n) (11.23)
⎩− α τ <t ≤T ∂xi

this is illustrated in Fig. 11.1 The Euler equations for the control variables, u i , do not
follow as in section 10.4 as it is possible that there are
control u discontinuities in u i , and so we cannot assume that the partial
β
derivaties ∂H / ∂u i exist. On the other hand, we can apply the
∂F
free end point condition (9.23): = 0 to obtain
∂x& i
0 τ T time t p k (T ) = 0 k = q + 1, L , n (11.24)

that is, the adjoint variable is zero at every end point where
−α the corresponding state variable is not specified. As before,
we refer to (11.24) as tranversality conditions.
Fig. 11.1 Optimal Control
Our difficulty now lies in obtaining the analogous equation to
This figure shows that the control: ∂H / ∂u i = 0 for continuous controls. For the moment, let us
- has a switch (discontinuity) at time t = τ assume that we can differentiate H with respect to u, and
- only take its maximum and minimum values consider a small variation δu in the control u such
that u + δu still belong to U , the admissible control region.
This type of control is called bang-bang control.
Corresponding to the small change in u , there will be small
11.2 Pontryagin’s Principle (early 1960s) change in x , say δx , and in p , say δp . The change in the
value of J * will be δJ * , where
Problem: We are seeking extremum values of the functional
n

∑ p x& }dt
T
δJ * = δ
∫
T
{H −
J =
∫ 0
f 0 (x, u, t )dt (11.18) o
i =1
i i

subject to state equations The small change operator, δ , obeys the same sort of
x& i = f i(x, u, t ) (i = 1,2, L , n) (11.19) properties as the differential operator d / dx .

initial conditions x = x 0 and final conditions on x1 , x 2 , L , x q Assuming we can interchange the small change operator, δ ,
and integral sign, we obtain
(q ≤ n) and subject to u ∈ U , the admissible control region.
For example, in the previous problem, the admissible control n

∑ p x& )]dt
T
region is defined by δJ * =
∫ 0
[δ ( H −
i =1
i i

U = {u : −α ≤ u ≤ β } n

∫ [δH − ∑ δ ( p x& )]dt

T
= i i
As in section 10.4, we form the augmented functional 0
i =1
⎡ ⎤ n n

∑ ∑ p δ x& ]dt
n T

∑ ∫
T
J* =
∫ 0
⎢ f0 +
⎢
⎣ i =1
pi ( f i − x& i )⎥ dt
⎥
⎦
(11.20) =
0
[δH −
i =1
x& i δ pi −
i =1
i i

and define the Hamintonian Using chain rule for partial differentiation

n m n n
H = f0 + ∑i =1
pi f i (11.21) δH = ∑
∂H
∂u j
δu j + ∑
∂H
∂xi
δxi + ∑ ∂p δp
∂H
i
i
j =1 i =1 i =1

For simplicity, we consider that the Hamintonian is a so that

function of the state vector x, control vector u, and adjoint
vector p, that is, H = H (x, u, p) . ⎡ m
∂H
∫ ∑ ∂u
T
We can express J * as δJ * = ⎢ δu j
0 ⎢ j
⎣ j =1
T⎛ n ⎞
∫ ⎜H −
∑ p x& ⎟⎟ dt ⎞⎤
n
J* = ⎛ ∂H
∑ ⎜⎜⎝ ∂p δ p
(11.22)
0 ⎜ i i
+ i − p& i δ xi − xi δ pi − pi δ x& i ⎟⎟⎥ dt
⎝ i =1 ⎠ i ⎠⎥⎦
i =1

and evaluating the Euler equations for xi , we obtain as in ∂H

section 10.4 the adjoint equations since p& i = −∂H / ∂xi . Also, from (11.21) = f i = x& i
∂pi

Chapter 11 Bang-bang Control 54

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

using (11.19): x& i = f i(x, u, t ), (i = 1,2, L , n) . Thus We first illustrate its use by examining a simple problem. We
required to minimize
⎡ m n ⎤
∂H
∫ ∑ ∑ ( p& iδ xi + piδ x&i )⎥⎥ dt
T T
δJ * = ⎢
0 ⎢
⎣ j =1
∂u j
δu j −
i =1 ⎦
J = 1dt
∫ 0

T⎡ m
∂H
n ⎤ subject to x&1 = x 2 , x& 2 = u where −α ≤ u ≤ β and x1 (T ) = a ,
∫ ∑ ∑ ( piδ xi )⎥⎥ dt
⎢ d
= δu j −
0 ⎢ ∂u j dt x2 (T ) = 0 , x1 (0) = x 2 (0) = 0 .
⎣ j =1 i =1 ⎦
Introducing adjoint variables p1 and p 2 , the Hamiltonian is
We can now integrate the second part of the integrand to
yield given by

H = 1 + p1 x 2 + p 2 u
T ⎛⎜ m ⎞
T
n
∂H
δJ * = − ∑ ( p δ x ) + ∫ ⎜⎜ ∑
i i
0 ∂u j
⎟
δu j ⎟ dt
⎟
(11.25)
We must minimize H with respect to u, and where
i =1 0 ⎝ j =1 ⎠ u ∈U = [−α , β ] , the admissible control region.. Since H is
linear in u, it clearly attains its minimum on the boundary of
At t = 0 : xi (i = 1,L, n) are specified ⇒ δxi (0) = 0
the control region, that is, either at u = −α or u = β . This
At t = T : xi (i = 1,L , q ) are fixed ⇒ δxi (T ) = 0 illustrated in Fig.11.3. In fact we can write the optimal
For i = q + 1, L, n, from the transversality conditions, (11,24) control as

pi (T ) δxi (T ) = 0 ⎧−α if p 2 > 0

u=⎨
⎩ β if p 2 < 0
for i = 1,2, L , q, q + 1, L , n . We now have
H

T ⎛⎜ m
∂H
⎞
∫ ∑
⎟
δJ * = ⎜ δu j ⎟ dt
0 ⎜ ∂u j ⎟
⎝ j =1 ⎠
control u
where δu j is the small variation in the j th component of the
control vector u . Since all these variations are independent, −α β
and we require δJ * = 0 for a turning point when the controls Fig. 11.3 The case p 2 > 0
are continuous, we conclude that
But p 2 will vary in time, and satisfies the adjoint equations,
∂H
= 0 ( j = 1,2, L , m) (11.26) ∂H
∂u j p& 1 = − =0
∂x1
But this is only valid when the controls are continuous and ∂H
p& 2 = − = − p1
not constrained. In our present case when u ∈ U , the ∂x 2
admissible control region and discontinuities in u are allowed.
The arguments presented above follow through in the same Thus p1 = A , a constant, and p 2 = − At + B , where B is
way, except that (∂H / ∂u j )du j must be replaced by constant. Since p 2 is a linear function of t, there will at most
be one switch in the control, since p 2 has at most one zero,
H (x; u1 , u 2 ,L, u j + δu j ,L, u m ; p) − H (x, u, p)
and from the physical situation there must be at least one
switch. So we conclude that
We thus obtain
(i) the control u = −α or u = β , that is, bang-bang control;
T m
δJ * =
∫∑0
j =1
[ H (x; u1 ,L, u j + δu j ,L, u m ; p) − H (x, u, p)] dt (ii) there is one and only one switch in the control.

Again, it is clear from the basic problem that initially u = β ,

In order for u to be a minimizing control, we must followed by u = −α at the appropriate time.
have δJ * ≥ 0 for all admissible controls u + δu . This implies
that 11.3 Switching Curves

H (x; u1 ,L, u j + δu j ,L, u m ; p) ≥ H (x, u, p) (11.27) In the last section we met the idea of a switch in a control.
The time (and position) of switching from one extremum
for all admissible δu j and for j = 1,L, m . So we have value of the control to another does of course depend on the
initial starting point in the phase plane. Byconsidering a
established that on the optimal control H is minimized with specific example we shall show how these switching
respect to the control variables, u1 , u 2 ,L, u m . This is known positions define a switching curve.
as Pontryagin’s minimum principle.

Chapter 11 Bang-bang Control 55

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Suppose a system is described by the state variables x1 , x 2 x 2 − B = − k log | ( x1 − k ) / A |

where
Now if u = 1 , that is, k = 1 , then the trajectories are of the
x&1 = − x1 + u (11.28) form
x& 2 = u (11.29)
x 2 − B = − log | ( x1 − 1) / A |
Here u is the control variable which is subject to the
constraints −1 ≤ u ≤ 1 . Given that at t = 0, x1 = a, x 2 = b , that is
we wish to find the optimal control which takes the system to x 2 = − log | x1 − 1 | +C (11.32)
x1 = 0, x 2 = 0 in minimum time; that is, we wish to minimize
where C is constant. The curves for different values of C are
T
J = 1dt
∫ 0
(11.30) illustrated in Fig.11.4

u =1 x2
while moving from (a, b) to (0,0) in the x1 − x 2 phase plane
and subject to (11.28), (11.29) and

−1 ≤ u ≤ 1 (11.31)

Following the procedure outline in section 11.2, we introduce

the adjoint variables p1 and p 2 and the Hamiltonian
0 x1
H = 1 + p1 (− x1 + u ) + p 2 u 1
= u ( p1 + p 2 ) + 1 − p1 x1

Since H is linear in u, and | u |≤ 1 , H is minimized with

respect to u by taking

⎧+1 if p1 + p 2 < 0 A
u=⎨
⎩− 1 if p1 + p 2 > 0

So the control is bang-bang and the number of switches will Fig. 11.4 Trajectories for u = 1
depend on the sign changes in p1 + p 2 . As the adjoint
equations, (11.23), are Follow the same procedure for u = −1 , giving

∂H x2 = log | x1 − 1 | +C (11.33)
p& 1 = − = p1
∂x1 p1 = Ae t
⇒ , A and B are constant and the curves are illustrated in Fig. 11.5.
∂H p2 = B
p& 2 = − =0
∂x 2
x2 B
and p1 + p 2 = Ae t + B , and this function has at most one
sign change.

So we know that from any initial point (a, b) , the optimal

control will be bang-bang, that is, u = ±1 , with at most one
switch in the control. Now suppose u = k , when k = ±1 ,
then the state equations for the system are x1
−1 0
x&1 = − x1 + k
x& 2 = k
u = −1
We can integrate each equation to give

x1 = Ae −t + k
, A and B are constants
x2 = k t + B

The x1 − x 2 plane trajectories are found by eliminating t ,

Fig. 11.4 Trajectories for u = −1
giving

Chapter 11 Bang-bang Control 56

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

The basic problem is to reach the origin from an arbitrary Following the usual procedure, we form the Hamiltonian
initial point. All the possible trajectories are illustrated in
Figs. 11.4 and 11.5, and we can see that these trajectories are H = 1 + p1 x2 + p 2 u (11.39)
only two possible paths which reach the origin, namely AO
in Fig. 11.4 and BO in Fig. 11.5. We minimize H with respect to u, where −1 ≤ u ≤ 1 , which
gives
x2 B
u = −1 ⎧+1 if p2 < 0
u=⎨
⎩− 1 if p2 > 0

The adjoint variables satisfy

⎧& ∂H
0 x1 ⎪ p1 = − ∂x = 0 ⎧p = A
⎪ 1
−1 1 ⎨ ⇒ ⎨ 1
⎪ p& = − ∂H ⎩ p 2 = − At + B
= − p1
⎪⎩ 2 ∂x2

Since x2 (T ) is not specified, the transversality condition

becomes p 2 (T ) = 0 . Hence 0 = − At + B and p 2 = A(T − t ) .
A u =1 For 0 < t < T , there is no change in the sign of p 2 , and hence
no switch in u. Thus either u = +1 or u = −1 , but with no
switches. We have
Fig. 11.4 Trajectories for u = −1
x22 = 2( x1 − B) when u =1 (11.40)
Combining the two diagrams we develop the Fig. 11.6. The
curve AOB is called switching curve. For initial points below x22 = −2( x1 − B) when u = −1 (11.41)
AOB, we take u = +1 until the switching curve is reached,
followed by u = −1 until the origin is reached. Similarly for These trajectories are illustrated in Fig. 11.7, the direction of
the points above AOB, u = −1 until the switching curve is the arrows being determined from x& 2 = u
reached, followed by u = +1 until the origin is reached. So
we have solved the problem of finding the optimal trajectory u = +1 x2 x2 u = −1
from an arbitrary starting point.

Thus the switching curve has equation

⎧ log(1 + x1 ) for x1 > 0 x1 x1

x2 = ⎨ (11.34) 0 0
⎩− log(1 + x1 ) for x1 < 0

11.4 Transversarlity conditions

To illustrate how the transversality conditions ( pi (T ) = 0 if

xi is not specified) are used, we consider the problem of
finding the optimum control u when | u | ≤ 1 for the system Fig. 11.7 Possible Optimal Trajectories
described by
x2
x&1 = x 2 (11.35) x1
x& 2 = u (11.36) 0
( a, b )
which takes the system from an arbitrary initial point
x1 (0) = a, x 2 (0) = b to any point on the x2 axis, that is,

{
x1 (T ) = 0 but x2 (T ) is not given, and minimize
T+
T T−{
∫
J = 1.dt
0
(11.37) ( a, b ) A

subject to (11.35), (11.36), the above boundary conditions Fig. 11.8 Initial point below OA
on x1 and x 2 , and such that
We first consider initial points (a, b) for which a > 0 . For
−1 ≤ u ≤ 1 (11.38) points above the curve OA, there is only one trajectory,

Chapter 11 Bang-bang Control 57

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

u = −1 which reaches the x 2 -axis, and this must be the

optimal curve. For points below the curve OA, there are two
possible curves, as shown in Fig. 11.8.

From (11.36): x& 2 = u , that is x& 2 = ±1 , and the integrating

between 0 and T gives

x 2 (T ) − x 2 (0) = ±T

that is

T = | x 2 (T ) − x 2 (0) | (11.42)

Hence the modulus of the difference in final and initial values

of x 2 given the time taken. This is shown in the diagram as
T+ for u = +1 and T− for u = −1 . The complete set of
optimal trajectories is illustrated in Fig. 11.9.

x2
u = +1 u = −1

0 x1

Fig. 11.9 Optimal Trajectories to reach x 2 -axis in minimum

time

11.5 Extension to the Boltza problem

Chapter 11 Bang-bang Control 58

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C.12 Applications of Optimal Control

12.6 Rocket Trajectories Problem

Minimizing the fuel (cost function)
The governing equation of rocket motion is
T

m
dv
= m F + Fext (8.22)
J=
∫ β dt
o
(12.41)
dt
subject to the differential constraints
where
m : rocket’s mass dv1 cβ
= cos φ (12.42)
v : rocket’s velocity dt m
F : thrust produced by the rocket motor dv 2 cβ
Fext : external force = sin φ − g (12.43)
dt m

It can be seen that and also

c dx
F = β (8.23) = v1 (12.44)
m dt
dy
where = v2 (12.45)
dt
c : relative exhaust speed
β = −dm / dt : burning rate The boundary conditions

Let φ be the thrust attitude angle, that is the angle between t = 0 : x = 0, y = 0, v1 = 0, v 2 = 0

the rocket axis and the horizontal, as in the Fig. 8.4, then the t = T : x not specified, y = y , v1 = v , v 2 = 0
equations of motion are

dv1 cβ Thus we have four state variables, namely x, y , v1 , v 2 and two

= cos φ controls β (the rate at which the exhaust gases are emitted)
dt m
(8.24)
dv 2 cβ and φ (the thrust attitude angle). In practice we must have
= sin φ − g
dt m bounds on β , that is,

Here v = (v1 , v 2 ) and the external force is the gravity force 0≤β ≤β (12.46)
only Fext = (0,− mg ) .
so that β = 0 corresponds to the rocket motor being shut
y path of rocket down and β = β corresponds to the motor at full power.

The Hamilton for the system is

r
v cβ ⎛ cβ ⎞
H = β + p1v1 + p 2 v 2 + p 3 cos φ + p 4 ⎜ sin φ − g ⎟
m ⎝ m ⎠
(12.47)
φ where p1 , p 2 , p 3 , p 4 are the adjoint variables associated
with x, y , v1 , v 2 respectively.
x
Fig. 8.4 Rocket Flight Path
(12.47) ⇒
The minimum fuel problem is to choose the controls, β and ⎛ c c ⎞
H = p1v1 + p 2 v 2 − p 4 g + β ⎜1 + p 3 cos φ + p 4 sin φ ⎟
φ , so as to take the rocket from initial position to a ⎝ m m ⎠
prescribed height, say, y , in such a way as to minimize the
fuel used. The fuel consumed is ⎛ c c ⎞
If we assuming that ⎜1 + p 3 cos φ + p 4 sin φ ⎟ ≠ 0 , we
⎝ m m ⎠
T
see that H is linear in the control β , so that β is bang-bang.
∫ β dt
o
(8.25)
That is β = 0 or β = β . We must clearly start with β = β so
that, with one switch in β , we have
where T is the time at which y is reached.

Chapter 12 Applications of Optimal Control 59

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

⎧⎪ β 0 ≤ t < t1
T
∫
for T = 1dt
β =⎨ (12.48) (12.50)
⎪⎩0 for t1 < t ≤ T 0

subject to
Now H must also be maximized with respect to the second
control, φ , that is, ∂H / ∂φ = 0 giving
d 2θ dθ
+α + ω 2θ = u (12.51)
cβ cβ dt 2 dt
− p3 sin φ + p 4 cos φ = 0
m m
Boundary conditions
which yields tan φ = p 4 / p 3 .
t = 0 : θ = θ 0 , θ& = θ 0′
The adjoint variables satisfy the equations t = T : θ = 0, θ& = 0

∂H Constraint on control: | u | ≤ 1 .
p& 1 = − =0 ⇒ p1 = A
∂x
∂H Introduce the state variables: x1 = θ , x 2 = θ& . Then (12.51)
p& 1 = − =0 ⇒ p2 = B
∂y
becomes
∂H
p& 1 = − = − p1 = − A ⇒ p 3 = C − At
∂v1 x&1 = x 2
(12.52)
∂H x& 2 = a x 2 − ω 2 x1 + u
p& 1 = − = − p2 ⇒ p 4 = D − Bt
∂v 2
As usual, we form the Hamiltonian
where, A, B, C, D are constant. Thus
H = 1 + p1 y 2 + p 2 (−a x 2 − ω 2 x1 + u ) (12.53)
D − Bt
tan φ =
C − At where p1 , p 2 are adjoint variables satisfying

Since x is not specified at t = T , the transversality condition

∂H
is p1 (T ) = 0 , that is, A = 0 , and so p& 1 = − = ω 2 p2 (12.54)
∂x
∂H
tan φ = a − bt (12.49) p& 1 = − = − p1 + ap 2 (12.55)
∂y
where a = D / C , b = B / C .
Since H is linear in u, and | u | ≤ 1 , we again immediately see
The problem, in principle, I now solved. To complete it that the control u is bang-bang, that is, u = ±1 , in fact
requires just integration and algebraic manipulation. With φ
given by (12.49) and β given by (12.48), we can integrate ⎧+1 if p2 < 0
u=⎨
(12.42) to (12.45). This will bring four further constants of ⎩− 1 if p2 > 0
integration; together with a, b and the switchover time t1 we
have seven unknown constants. These are determined from To ascertain the number of switches, we must solve for p 2 .
the seven end-point conditions at t = 0 and t = T . A typical
From (12.54) and (12.55), we see that
trajectory is shown in Fig. 12.5.
r &p& 2 = − p& 1 + a p& 2 = −ω 2 p 2 + a p& 2
y y=y v
β =0 that is,
null thrust arc
t = t1 &p&2 − a p& 2 + ω 2 p 2 = 0 (12.56)
β =β

maximum thrust arc The solution of (12.56) gives us the switching time.

0 x
Fig. 12.5 Typical rocket trajectory

12.7 Servo Problem

The problem here is to minimize

Chapter 12 Applications of Optimal Control 59