Linear System Theory, Fall 2011

Solutions to Examples Paper 6

Hilbert Spaces, Discrete-time systems, Controller design
(due date: December 22nd)

Exercise 1. (Properties

of Inner Products, 30%)
Let H, F, h·, ·i be a Hilbert space. Recall that a nonempty set C ⊆ H is called convex
if and only if for every λ ∈ [0, 1] and x1 , x2 ∈ C one has (1 − λ)x1 + λx2 ∈ C. Assume
the following:
Projection Theorem. Let C ⊆ H be a nonempty, closed & convex set.
Then for every x ∈ H there exists a unique PC (x) ∈ C such that
kx − PC (x)k = inf kx − yk .
y∈C

Use the Projection Theorem to show that if C ⊆ H is a nonempty, closed and convex set,
then:
1. For every z ∈ C we have <e hx − PC (x), z − PC (x)i ≤ 0. Provide an interpretation
of this fact for the case F = R.
2. For all x1 , x2 ∈ H we have kPC (x1 ) − PC (x2 )k ≤ kx1 − x2 k.

Solution 1.
1. Fix x ∈ H. From the Projection Theorem we know that there exists a unique element
PC (x) ∈ C such that kx − PC (x)k ≤ kx − yk for every y ∈ C. If z ∈ C and λ ∈ (0, 1],
then (1 − λ)PC (x) + λz ∈ C due to convexity of C, and in view of the last statement
we know that kx − (1 − λ)PC (x) − λzk2 ≥ kx − PC (x)k2 . This inequality upon expansion
gives λ2 kz − PC (x)k2 +2λ<e hx − PC (x), PC (x) − zi ≥ 0. Dividing by λ and letting λ ↓ 0
(λ ∈ (0, 1] is arbitrary,) we see that <e hx − PC (x), PC (x) − zi ≥ 0.
If the field is R, then the claim says that the angle between the vectors x − PC (x) and
z − PC (x) is at least π/2.
2. The result trivially holds if P (x1 ) = P (x2 ). If P (x1 ) 6= P (x2 ), set y1 = PC (x1 ) and
y2 = PC (x2 ). We know that
<e hx1 − x2 , y1 − y2 i = <e hx1 − y2 + y2 − x2 , y1 − y2 i
= <e hx1 − y2 , y1 − y2 i + <e hy2 − x2 , y1 − y2 i
= <e hx1 − y1 + y1 − y2 , y1 − y2 i + <e hy2 − x2 , y1 − y2 i
= <e hx1 − y1 , y1 − y2 i + ky1 − y2 k2 + <e hy2 − x2 , y1 − y2 i
≥ ky1 − y2 k2 ,
where the last inequality follows from part 1. Thus, applying the Cauchy-Schwarz in-
equality we get ky1 − y2 k2 ≤ kx1 − x2 k ky1 − y2 k, which gives the desired result.

1
 2

Exercise 2. (Discrete-time systems, 30%)

Consider a linear, time-invariant, discrete-time system:

xt+1 = Axt + But

yt = Cxt + Dut

where t ∈ N, xt ∈ Rn , ut ∈ Rm , yt ∈ Rp , and A, B, C, D are matrices of appropriate

dimensions.
(1) Let x0 = x̄. Show that, given an input signal {ut }+∞
t=0 , the corresponding state and
output trajectories are given by
t−1
X
xt = At x̄ + At−1−i Bui
i=0
t−1
X
t
yt = CA x̄ + CAt−1−i Bui + Dut
i=0

(2) Suppose now that the matrix A is semi-simple and ut = 0 for all t (Zero input
response). The system is said to be asymptotically stable if for all ε > 0 there
exists δ > 0 such that if kx̄k < δ, then kxt k < ε for all t ≥ 0 (stability) and
moreover, for all initial conditions x̄, the corresponding state trajectory xt → 0 as
t → +∞ (convergence). Show that the system is asymptotically stable if and only
if |λ| < 1 for all the eigenvalues λ of A.
(3) The system is said to be reachable if for any terminal state x̄ ∈ Rn there exists
an input sequence u0 , u1 , · · · , un , · · · such that, starting from the initial condition
x0 = 0, it holds xn = x̄. (Here, n is precisely the dimension of the state space.)
Show that if the matrix
 
B AB · · · An−1 B

has full rank, then the system is reachable.

(4) The system is said to be controllable to zero if, for any initial state x̄ ∈ Rn , there
exists an input sequence u0 , u1 , · · · , un , · · · such that, starting from the initial
condition x0 = x̄, it holds xn = 0. Show that if the system is reachable, then it
is also controllable to zero. Show that the converse is not in general true, that is,
provide an example of a system which is controllable but not reachable.
(5) Optional: Show that, if the system is controllable to zero and A is invertible, then
the system is also reachable.

Solution 2.
(1) Let us consider the expression for xt . The proof easily done by induction on t. At
time t = 0,
−1
X
0
x0 = A x̄ + A0−1−i Bui = x̄
i=0
 3

Pt−1
which is correct. Suppose that xt = At x̄ + i=0 At−1−i Bui . Then,

xt+1 = Axt + But

t−1
!
X
= A At x̄ + At−1−i Bui + But
i=0
t−1
X
t+1
=A x̄ + A At−1−i Bui + But
i=0
t−1
X
= At+1 x̄ + At−i Bui + At−t But
i=0
Xt
= At+1 x̄ + At−i Bui
i=0

so that the expression is correct also at time t+1. By induction on t, the expression
is correct for all t. Note that no difficult proof of existence and uniqueness is needed
here. The follow from the fact that, given the input, the system equation dictates
recursively the evolution of the state.
To obtain the expression for y(t), just substitute the expression for the state into
yt = Cxt + Dut .
(2) Since A is semi-simple, there exists an invertible matrix T such that A = T −1 ΛT ,
where Λ is diagonal, the elements on the diagonal being the eigenvalues of A. Let
ξt = T xt (change of basis in the state space). Then the evolution of the system
reads

xt+1 = Axt = T −1 ΛT xt
T xt+1 = ΛT xt
ξt+1 = Λξt
1  1 
ξt+1 λ1 0 · · · 0 ξt
  
2
 ξt+1   0 λ2 · · · 0   ξt2 
 ..  = . .. . . ..   . 
 .   .. . . .   .. 
n
ξt+1 0 0 ··· λn ξtn

Taking into account the expression for the evolution of a discrete-time system,
given above, we have

ξt = Λt ξ0
ξt1
  t
ξ01
  t 1
λ1 0 · · · 0 λ1 ξ0
  
 ξt2   0 λt2 · · · 0   ξ0   λt2 ξ02
2

 . = . .. . . .  .  =  . 
 ..   .. . . ..   ..   .. 
ξtn 0 0 · · · λtn ξ0n λtn ξ0n

If now |λ| < 1 for all the eigenvalues, then all the components of ξt tend to
zero exponentially irrespective of the initial condition. Therefore ξt → 0, and
consequently xt = T −1 ξt → 0. This proves convergence.
 4

To establish stability, consider that

q
1 n
kξt+1 k = |ξt+1 |2 + · · · + |ξt+1 |2
q
= |λ1 ξt1 |2 + · · · + |λn ξtn |2
q
= |λ1 |2 |ξt1 |2 + · · · + |λn |2 |ξtn |2
q
≤ |λ|2 |ξt1 |2 + · · · + |λ|2 |ξtn |2
q
= |λ| |ξt1 |2 + · · · + |ξtn |2
= |λ|kξt k

where λ is an eigenvalue with maximum modulus, and k · k is the Euclidean norm.

If all the eigenvalues have modulus less than 1, then kξt+1 k < kξt k for all t. It
follows (by induction) that kξt k ≤ kξ0 k for all t. Given ε > 0, it now suffices to
choose δ = ε/(kT −1 kkT k) to attain

kxt k ≤ kT −1 kkξt k ≤ kT −1 kkξ0 k ≤ kT −1 kkT kkx0 k

< kT −1 kkT kδ = ε

for all t, whenever kx0 k < δ. This proves stability.

We have shown that if all the eigenvalues have modulus less than 1, then the system
is asymptotically stable. Suppose, on the other hand, that A has an eigenvalue λ
with |λ| ≥ 1. Then A has an eigenvector xλ with respect to λ, and without loss
of generality let kxλ k = δ. Choosing x0 = xλ ,

xt = At xλ = λt xλ
kxt k = |λ|t δ

Hence, kxt k is always equal to δ if |λ| = 1, and diverges if |λ| > 1; in any case it
fails to converge to zero; therefore the system is not asymptotically stable. This
concludes the proof.
Note: There is nothing wrong in working with complex eigenvectors and/or changes
of basis; but if the reader wanted to restrict his attention to real matrices, vectors,
and bases, he or she could still adapt the proofs exploiting the real Jordan form
of the restrictions of A over the 2-dimensional subspaces generated by the real and
imaginary part of complex eigenvectors, along the lines of what we have seen in
assignment 4. Also, note that the claim is true also when A is not semi-simple,
although in this case the proof must deal with Jordan forms in full generality.
(3) It suffices to notice that

un−1
 
n−1
X   un−2 
An−1−i Bui = B AB · · · An−1 B 

 ... 

i=0
u0
 5

Hence, if we impose xn = x̄,

un−1
 
  un−2 
x̄ = An x0 + B AB · · · An−1 B 

 ... 


u0
un−1
 
   un−2 
= B AB · · · An−1 B   ... 


u0
 
Assuming that B AB · · · An−1 B has full rank is the same as to assume that
for all x̄ there exists a linear combination of its columns, the coefficients being
the respective components of un−1 , un−2 , · · · , u0 , that equals x̄. This is enough to
establish reachability.
(4) If the system is reachable, then for all x̄ there exists an input u0 , · · · , un−1 such
that if x0 = 0, then xn = −An x̄. Explicitly,
n−1
X n−1
X
n n−1−i
xn = A x0 + A Bui = An−1−i Bui = −An x̄
i=0 i=0

But then, exploiting the expression above for the evolution of the state, if we apply
the same input to the initial state x̄, we attain
n−1
X
n
xn = A x̄ + An−1−i Bui = An x̄ − An x̄ = 0
i=0

This ensures that, if the system is reachable, it is also controllable to zero. The
converse is not in general true. For example (just to avoid the trivial example
where all the system matrices are equal to zero), consider the system
   
0 1 1
xt+1 = xt + ut
0 0 0
This system is controllable to zero; indeed just choosing the input u0 = u1 = 0 we
obtain
 2
0 1
x2 = x̄ = 0
0 0
for all initial states x̄. On the other hand, starting from x0 = 0, the explicit
expression for x2 is
 2    
0 1 1 0
x2 = 0+ u1 + u0
0 0 0 0
 
1
= u1
0
 
1
Thus, only multiples of can be reached from 0, hence the system is not
0
reachable.
 6

(5) Suppose that the n × n matrix A is invertible. Then, of course, A−1 is also
invertible, and so are An and A−n . The trick is now to show that, if a certain
n × q matrix C has rank n, where q ≥ n, then the matrix An C has also rank n.
This is trivial: Since C has full rank (= n), then for all x̄ there exists u such that
Cu = A−n x̄ (linear combination of the columns of C!), hence for the same u it
holds An Cu = x̄, which proves the claim. Now suppose also that the system is
controllable to zero. For all initial conditions −x̄, there exist a sequence of inputs
 >
un−1 un−2 · · · u0 = u such that
n−1
X
n
xn = A (−x̄) + An−1−i Bui
i=0
un−1

  un−2 
= −An x̄ + B AB · · · An−1 B 

 ...  = 0


u0
un−1
 
  un−2 
An x̄ = B AB · · · An−1 B 

 ... 


u0
x̄ = A−n B AB · · · An−1 B
 

u
Therefore, C = A−n B AB · · · An−1 B has full rank. Then An C = B AB · · · An−1 B
   
has full rank as well, therefore the system is reachable.
The crucial fact, here, is that At , which is the discrete-time “transition matrix”
(that is, Φ(t, 0) = At in discrete-time), is invertible. When this is not the case at
time t = n, the system may be controllable to zero but it cannot be reachable.
Now recall that the transition matrix of a continuous-time linear system (e.g.,
Φ(t, 0) = eAt ) is always invertible, irrespective of t; precisely for this reason, in
continuous time reachability and controllability (to zero) are actually the same
property. This is why, during the course, we did not make the distinction, and we
introduced the general notion of “controllability (over an interval)” as a substitute
for both.
 7

Exercise 3. (Stabilization of the inverted pendulum on a cart, 40%)

ϕ L

mg

u M

The figure depicts an inverted pendulum standing over a cart.

The arm of the pendulum consists of a rod of length 2L, with mass m and uniform linear
density, and is free to rotate around a pivot fixed to the cart. The center of mass of the
2
rod is at distance L from the pivot, and the moment of inertia of the rod is J = mL 3
. The
cart has mass M , and is free to move on a horizontal track. A horizontal force u applied
to the cart acts as an input. Suppose that friction is negligible, and that M >> m, so
that the reactive force of the rod against the cart is also negligible.
(1) Provide a 2-dimensional state-space model for the motion of the rod, choosing as
coordinates the angle ϕ(t) of the arm with respect to the vertical position, and
the angular velocity ϕ̇(t). Let the output be y(t) = ϕ(t).
Hint: The model is supposed to describe the motion of the sole rod. The motion of
the cart is described by an equation like M ẍ(t) = −h(t) + u(t), where −h(t) is the
reactive force of the rod over the cart, hence, normally, we should care about the
cart position and velocity as well. But since we are not interested in the cart, and
since we consider the reactive force negligible, we can approximate the equation
as M ẍ(t) = u(t) and substitute in the equations of the motions of the rod, i.e.
the dynamics of the cart itself should not appear in the model, only the input u(t)
should.
(2) Show that ϕ = 0, that is the vertical position, is an equilibrium of the system.
(3) Linearize the system around this equilibrium. Show that the equilibrium is unsta-
ble. What do you guess about the stability properties of the vertical equilibrium
of the nonlinear system?
(4) Show that the linearized system is controllable. Design a feedback controller of
the form u(t) = Kx(t) to stabilize the linear system placing all the eigenvalues of
the closed-loop system at a certain negative number −α.
(5) Now recall that only the angle of the pendulum is observed. Show that the lin-
earized system is observable. Design an asymptotic observer to provide an estimate
x̂(t) of the entire state, in such a way that all the modes of the estimation error
decay exponentially as e−βt , where β = 10α.
(6) Fix L = 10cm, m = 10g, M = 1kg, and α = 2. Simulate the system in Matlab,
applying to the nonlinear system the feedback u(t) = K x̂(t). Verify that, from
small (but non-vertical) initial positions, your controller steers the rod back to the
vertical position. Do some experiment with different values of α and β.
 8

Solution 3.
(1) Let x(t) be the horizontal position of the cart (we will soon get rid of this variable).
Let h(t) and v(t) be, respectively, the horizontal and vertical forces applied to the
center of mass of the rod. We have
d2
h(t) = m (x(t) + L sin ϕ(t))
dt2
d
= mẍ(t) + mL (cos ϕ(t))ϕ̇(t)
dt
= mẍ(t) − mL(sin ϕ(t))ϕ̇(t)2 + mL(cos ϕ(t))ϕ̈(t)
d2
v(t) − mg = m (L cos ϕ(t))
dt2
d
= −mL (sin ϕ(t))ϕ̇(t)
dt
v(t) = mg − mL(cos ϕ(t))ϕ̇(t)2 − mL(sin ϕ(t))ϕ̈(t)
The equation of angular motion of the rod around its center of mass reads
J ϕ̈(t) = Lv(t) sin ϕ(t) − Lh(t) cos ϕ(t)
Substituting the expressions for h(t) and v(t), we get
J ϕ̈(t) = L mg − mL(cos ϕ(t))ϕ̇(t)2 − mL(sin ϕ(t))ϕ̈(t) sin ϕ(t)

− L mẍ(t) − mL(sin ϕ(t))ϕ̇(t)2 + mL(cos ϕ(t))ϕ̈(t) cos ϕ(t)

= mgL sin ϕ(t) − mL2 sin ϕ(t)2 + cos ϕ(t)2 ϕ̈(t) − mLẍ(t) cos ϕ(t)

= mgL sin ϕ(t) − mL2 ϕ̈(t) − mLẍ(t) cos ϕ(t)

or
(J + mL2 )ϕ̈(t) = mgL sin ϕ(t) − mLẍ(t) cos ϕ(t)
Now, the equation for the motion of the cart is
M ẍ(t) = −h(t) + u(t)
but, following the hint, we disregard the effect of −h(t) and substitute the approx-
imate expression ẍ(t) = u(t)/M into the above equation, obtaining finally:
mgL mL u(t)
ϕ̈(t) = 2
sin ϕ(t) − cos ϕ(t)
J + mL J + mL2 M
It is now easy to get a nonlinear model for the pendulum. Letting x1 (t) =
ϕ(t), x2 (t) = ϕ̇(t), we have
ẋ1 (t) = f1 (x1 (t), x2 (t), u(t)) = x2 (t)
mgL mL
ẋ2 (t) = f2 (x1 (t), x2 (t), u(t)) = 2
sin x1 (t) − u(t) cos x1 (t)
J + mL M (J + mL2 )
y(t) = h(x1 (t), x2 (t), u(t)) = x1 (t)
(the last equation means that only the position of the pendulum is observed).
 9

(2) If u(t) = 0, the origin is an equilibrium point:

ẋ1 (t) = f1 (0, 0, 0) = 0
mgL
ẋ2 (t) = f2 (0, 0, 0) = sin 0 = 0
J + mL2
(3) We linearize the system around the origin:
  " ∂f1 ∂f1 #    ∂f1 
ẋ1 (t) x 1 (t)
= ∂x 1
∂f2
∂x2
∂f2 + ∂f ∂u u(t)
ẋ2 (t) ∂x1 ∂x2
x 2 (t) 2
∂u x1 =x2 =u=0
x1 =x2 =u=0
    
0 1 x1 (t) 0
= mgL + mL u(t)
J+mL2
0 x2 (t) − M (J+mL 2)
    
0 1 x1 (t) 0
= + u(t)
a 0 x2 (t) b
 
 ∂h ∂h  x1 (t)
y(t) = ∂x1 ∂x2 x =x =u=0
1 2 x2 (t)
 
  x1 (t)
= 1 0
x2 (t)
mgL mL
where a = J+mL 2 > 0 and b = − M (J+mL2 ) < 0. This linear system is unstable,
√
because the system matrix has a positive eigenvalue ( a). As such, the nonlinear
equilibrium is unstable as well.
(4) The system is controllable; indeed its controllability matrix
 
0 b
C=
b 0
 
has full rank. Letting K = k1 k2 and u(t) = Kx(t), the equation of dynamics
becomes
       
ẋ1 (t) 0 1 0   x1 (t)
= + k1 k2
ẋ2 (t) a 0 b x2 (t)
  
0 1 x1 (t)
=
a + bk1 bk2 x2 (t)
The characteristic polynomial of the system matrix is χ(s) = s(s−bk2 )−(a+bk1 ) =
s2 − (bk2 )s − (a + bk1 ). The problem asks to design k1 , k2 in such a way that this
polynomials equals (s + α)2 = s2 + 2αs + α2 . This is easily attained choosing
−bk2 = 2α and −a − bk1 = α2 , that is, k1 = (−a − α2 )/b and k2 = −2α/b.
(5) The system is also observable, because its observability matrix
 
1 0
O=
0 1
has full rank. The problem asks to design an observer of the kind
˙
x̂(t) = Ax̂(t) + Bu(t) + L(y(t) − ŷ(t))
ŷ(t) = C x̂(t)
The error dynamics (see section 9.3 in the lecture notes) is then
ė(t) = (A − LC)e(t)
 10

 
l1
so that we are asked to place all the eigenvalues of A − LC at −β. Let L = .
l2
Then
     
0 1 l1   −l1 1
A − LC = − 1 0 =
a 0 l2 a − l2 0
The caracteristic polynomial is χ(s) = (s + l1 )s − (a − l2 ) = s2 + l1 s + (l2 − a). The
problem asks to design l1 , l2 in such a way that χ(s) = (s + β)2 = s2 + 2βs + β 2 .
This is easily attained by choosing l1 = 2β and l2 = a + β 2 .