16-299 Spring 2021: State Space Control Systems

George Kantor

16-25 February 2021

1 Moving to Control: Nonlinear State Space Systems

A continuous nonlinear state space system looks like this:

ẋ(t) = f (x(t), u(t)), x(t0 ) = x0

y(t) = h(x(t))
where:

• the state x ∈ Rn

• the input u ∈ Rm
• the output y ∈ Rp

And the continuous functions f and h are such that

f : Rn × Rm → Rn

and

h : Rn → Rp

We think of f and h as a vector of functions, each of which are real valued:

 
f1 (x, u)
 f2 (x, u) 
f (x, u) = 
 
.. 
 . 
fn (x, u)
 
h1 (x)
h2 (x)
h(x) =  . 
 
 .. 
hp (x)
2 Equilibrium Points and Stability

a point xe is an equilibrium point if ẋ is zero at xe for the unforced system, in other words

f (xe , 0) = 0

Consider a pendulum modeled as a point mass at the end of a massless rod with a frictionless pivot:

We saw earlier that the state space equations of motion for this system are:
 
x2
ẋ = = f (x, u).
− g` sin x1 − γx2 + u

Suppose we had a sensor that could measure position but not velocity, then the output of this system would be given
by

y(t) = x(t),

in other words, h(x) = x.

The pendulum has an equilibrium point at
 
0
x=
0

which corresponds to the pendulum hanging straight down.

There is an equilibrium point at
 
π
x=
0

which corresponds to the pendulum standing straight up.

There are actually an infinite number of points at
 
nπ
x=
0

2
for any n.
stability: From experience, we know that the equilibrium point at the top is unstable and the equilibrium point at the
bottom is stable. If I add a little friction the equilibrium point at the bottom becomes asymptotically stable.
Let’s make some more formal stability definitions.
First, we need to review the mathematical notion of open sets: A set D ⊂ Rn is said to be open if every point in D is
in the interior of D. More rigorously, D is open if for any x ∈ D there exists  > 0 so that every point that is within a
ball of radius is also in D, i.e., B (x) ⊂ D, where

B (x) = {y ∈ Rn | kx − yk < }.

The equilibrium point xe is said to be

• stable if for every  > 0 there exists δ > 0 so that if kx0 − xe k < δ then kx(t) − xe k <  for all time t > 0.
Note that this is inherently a local property.
• asymptotically stable if there exists an open D with xe ∈ D such that kx(t) − xe k → 0 for any initial condition
x0 ∈ D.
• unstable if it is not stable, i.e., there exists an  > 0 so that for any δ > 0 there is an initial condition x0 ∈
Bδ (xe ) so that kx(t) − xe k >  for some t > 0.

3 Linearization

It is usually possible to approximate a nonlinear system with a linear system near one of its equilibrium points (nonlin-
ear systems, unlike linear systems, can have multiple distinct equilibrium points). Here we show how to “linearize”,
i.e., find a linear approximation of a nonlinear system about a general equilibrium point.

3.1 Taylor Series

First, we need to revisit grade school to remind ourselves about the Taylor series and the approximations that can result
from it.
Consider a function g : R → R, g : x 7→ g(x). The Taylor series says that

dg d2 g 2 d3 g 3
g(x0 + ) = g(x0 ) + (x0 ) + 2 (x0 ) + 3 (x0 ) + · · ·
dx dx 2 dx 3!
This can also be written as
∞
X dk g k
g(x0 + ) = k
(x0 ) .
dx k!
k=0

When  is small, then the series is dominated by the lower order terms. This leads to the idea of approximating g near
x0 with a truncated Taylor series. The nth order Taylor approximation of g is then
n
X dk g k
g(x0 + ) = k
(x0 ) .
dx k!
k=0

3
The one we are particularly interested in is the first order approximation:
dg
g(x0 + ) = g(x0 ) + (x0 ).
dx
Of course, we are going to be applying this approximation to a vector-valued function f in order to linearize our
nonlinear system. So we need to do a few things to get the answer we need out of this last equation:

1. we need the g(x0 ) term to disappear. This is why we linearize about equilibrium points. By definition,
f (xe , 0) = 0 at an equilibrium point.
2. we need x0 to also be zero, which (when combined with 1.) makes the last equation look like
dg
g() = (x0 ).
dx
3. we need to generalize Taylor series to work for vector valued function with multiple inputs. In particular, we
need to generalize the notion of the derivative.

We won’t go into the details of these things, we will just state how linearization works for nonlinear systems in the
following section. Hopefully you will be able to make the connection.

3.2 Back to Nonlinear Systems

Consider the nonlinear system

ẋ = f (x, u)
y = h(x)
that has an equilibrium point xe . Here are the steps to find a linear approximation:

1. define new coordinates such that the xe becomes zero in the new coordinates:
z = x − xe

2. rewrite f and h in these new coordinates

4
ż = ẋ = f (x) = f (z + xe ) = fz (z)
y = h(z + xe ) = hz (z)
The result is a new system that has an equilibrium point at zero.
3. the linear approximation is then
∂fz ∂fz
ż ≈ z+ u
∂z z=0,u=0 ∂u z=0,u=0

∂hz
y≈ z
∂z z=0
where for any function g(x), g : Rn → Rm :
 ∂g ∂g1 ∂g1

∂x
1
∂x2 · · · ∂xn
 ∂g21 ∂g2 ∂g2 
∂g 4  ∂x 1 ∂x 2
· · · ∂x 
n 
= . . . .
∂x  ..
 .. .. .. 
∂gm ∂gm ∂gm
∂x1 ∂x2 · · · ∂xn

4
Thus, the system is approximated by a linear state space system around an equilibrium point:

ż = Az + Bu,

where A is an n × n matrix and B is an n × m matrix.

4 Some Background on Matrices

We won’t cover this in class, but it is here if you need to brush up on some basic facts about matrices.
Matrix Invertibility: Let A be any square matrix. TFAE (the following are equivalent):

1. A is invertible
2. A is nonsingular
3. det(A) 6= 0

4. all of the eigenvalues of A are non-zero

5. all of the rows of A are linearly independent
6. all of the columns of A are linearly independent
7. Ax = 0 if and only if x = 0.

Eigenvalues and Eigenvectors: Let A ∈ Rn×n and let e ∈ Rn . Then Ae is also in Rn . So it is conceivable that
λ ∈ R exists such that

Ae = λe

In fact, it is more than conceivable: there are always n (e, λ) pairs for every A. Both e and λ might be complex.
Technically, there are an infinite number of pairs, because if (e, λ) solve the above equation, then (γe, λ) also solve
the above equation for any γ ∈ R. But if we only consider e’s that are unit vectors (i.e., kek = 1) then there are n.

• the collection of e’s are called (unit) eigenvectors of A

• the collection of λ0 s are callec eigenvalues of A

What is an eigenvalue? I’ll start by talking about eigenvectors. We can think of a square matrix as a map that takes an
n dimensional vector as input and returns an n dimensional vector as an output, You put in a vector x ∈ Rn and a new
vector Ax ∈ Rn comes out. For most values of x, the vector Ax is completely different from x; the two vectors have
different magnitudes and point in a different directions. But for a very special subset of the possible x’s, the vector
Ax points in the same direction as x and differs only in magnitude. For these vectors, multiplication by the matrix A
is the same as multiplication by a scalar λ. These special vectors are called “eigenvectors”, and they’re often denoted
with an e (not to be confused with the exponential!). The scalar λ is called the eigenvalue, and it is associated with the
eigenvector e.

Ae = λe

So how do you find the eigenvalues?

Ae = λe

5
is the same as

(λI − A)e = 0

Using the invertibility facts above, we see that a solution can only exist if the matrix (λI − A) is not invertible, so we
can solve for λ using the equation

det(λI − A) = 0

You can also use the eig command in MATLAB.

5 Linear State Space Systems

Linear state space systems are a special case of nonlinear systems. We’ll get back to nonlinear systems a few lectures
from now, but first we will focus on systems that are exactly linear. Linear systems are important because (1) there are
many standard tools for doing anaylsis, control, and estimation for them and (2) almost any system can be reasonably
approximated as a linear system when operating near an equilibrium point (i.e., by linearization).
We start by defining a state vector x ∈ Rn , i.e.,
 
x1 (t)
 x2 (t) 
x(t) =  .  .
 
 .. 
xn (t)

We also define an input vector u ∈ Rm and output vector y ∈ Rp :

   
u1 (t) y1 (t)
 u2 (t)  y2 (t)
u(t) =  .  , y(t) =  .  .
   
 ..   .. 
um (t) yp (t)

Note that there can be more than one input and one ouput. A linear state space is represented by the following
equations:

ẋ(t) = Ax(t) + Bu(t)

y(t) = Cx(t).
where the matrix A ∈ Rn×n , B ∈ Rn×m , and C ∈ Rp×n . The first equation is called the state equation and the
second equation is called the output equation. Sometimes you will see the output equation written as

y(t) = Cx(t) + Du(t).

This is a more general form of the state space representation. We will only deal with systems that have D = 0.
If the matrices (A, B, C) are constant, then the system is linear time invariant, or LTI for short. In this section of the
class, we will deal exclusively with LTI systems. State space is just one of many ways of representing LTI systems.

6
6 Unforced System

It turns out that when we set u = 0 we can learn a lot about the behavior of a state space system. We call this case the
“unforced” system:

ẋ(t) = Ax(t), x(0) = x0 ,

6.1 Equilibrium Points

An equilibrium point is defined to be any state such that if the initial condition is that state, then the solution to the state
space ODE remains at that state for all time. The only way this can happen is if ẋ is zero, so we can mathematically
define an equilibrium point xe to be any point that satisfies:

ẋ = Axe = 0.

This means three important things for linear systems:

1. There is always an equilibrium point at x = 0.

2. If A is nonsingular, then the equilibrium point at x = 0 is unique.

3. If xe 6= 0 is an equlibrium point, then γxe is also an equilibrium point for any γ ∈ R.

6.2 Solution

The solution to an unforced state space system is

x(t) = eAt x0 ,

where the matrix exponential eAt is defined by the infinite series

(At)2 (At)3
eAt = I + At + + + ··· .
2 3!
Here I is the n×n identity matrix. Using MATLAB, we can compute the matrix exponential with the expm command.

7
7 Stability Criteria

Let xe be the equilibrium point of the linear state space system

ẋ(t) = Ax(t), x(0) = 0

and let {λ1 , λ2 , . . . , λn } be the eigenvalues of A.

The equilibrium point xe = 0 is:

• globally asymptotically stable if and only if Re(λi ) < 0 ∀i.

• unstable if and only if Re(λi ) > 0 for some i.
• if A has no repeated eigenvalues with zero real part, then xe = 0 is stable if and only if Re(λi ) ≤ 0 ∀i.
• more generally, xe = 0 is stable if and only if both of the following hold:
– Re(λi ) ≤ 0 for all i
– every eigenvalue with Re(λi ) = 0 has an associated Jordan block of order 1.

Jordan blocks are beyond the scope of this class, so you are only responsible for the first stability result. Use the
second (stronger) result if you already know what a Jordan block is or if you feel like looking it up (Google Jordan
normal form).

8 Some Examples

8.1 Mass-spring-damper

The second order LTI ODE, which you could get from applying Newton’s law or following the Euler-Lagrange process:

mÿ + γ ẏ + ks y = u,

where y is the position of the mass, m is the mass, γ is the viscous friction coefficient, ks is the linear spring constant,
and u is the input force applied to the mass. Now lets get the state space representation. Define
   
y x
x= = 1
ẏ x2

then
 
x2
ẋ = γ
− kms x1 − m x2 + 1
mu

which can be written in (A, B, C) form as

   
0 1 0
ẋ = γ x + 1 u
− kms − m m
 
y= 1 0 x
To check for stability, we find the eigenvalues of A, i.e., we look for the values of λ such that

det(λI − A) = 0

8
Let’s do it:
   
λ 0 0 1
det(λI − A) = det − γ
0 λ − kms
−m
   
λ −1  γ ks
= det ks γ =λ λ+ − −
m λ+ m m m
γ ks
= λ2 + λ+
m m
so the eigenvalues of A are the solutions to the equation
γ ks
λ2 + λ+ =0
m m
which are the same as the solutions of the equation

mλ2 + γλ + ks = 0.

We can solve for λ using the quadratic formula:

p
−γ ± γ 2 − 4mks
λ= .
2m
From this we can see that the eigenvalues take different forms depending on the sign of the term under the radical. If
4mks > γ 2 (i.e., if the damping coefficient is small compared to the mass and the spring constant) then the eigenvalues
γ
will be a complex pair with real part − 2m . Assuming that γ and m are both positive (which is almost always true
for real physical systems) then the real parts of the eigenvalues are negative and the equilibrium point at zero is
asymptotically stable. In the case where γ > 4mks, then there are two real eigenvalues. In this case, we can reason
that the square root term is always less than γ (assuming m and ks are positive) so again both eigenvalues are negative
and the systems is asymptotically stable.

8.2 Mass-damper (no spring)

This is the same as above with ks = 0, so the eigenvalues are the roots of

mλ2 + γλ = 0

which yields
γ
λ1 = 0, λ2 = −
m
Here we have all eigenvalues less then or equal to zero, and the eigenvalue with zero real part is not repeated, so we
can conclude that this system is stable (and has a continuum of equilibrium points!)

8.3 Double integrator

Consider an unforced double integrator:

ÿ = 0

Defining
 
y
x=
ẏ

9
we get
 
0 1
ẋ = x
0 0

so the eigenvalues of A are both zero. Since they are repeated, we cannot use the weaker stability result. In fact,
this system is unstable, which we can see just by imagining how the solution will evolve for an initial condition with
x2 6= 0. (We could also use the stronger stability result if we knew what a Jordan block was).

9 Controllability

A continuous time state space system is said to be controllable if for any two states x0 and xf , there exists a control
signal u(t), t ∈ [0, tf ] such that if x(0) = x0 , then x(tf ) = xf .
Loosely speaking, a system is controllable if you can get from any state to any other state.

9.1 Controllability Test

A continuous state space is controllable if and only if the matrix

Q = [ B | AB | A2 B | · · · | An−1 B ]

has rank n (i.e., Q has n linearly independent columns). Equivalently, a system is controllable iff det(QQT ) 6= 0.
Since controllability only depends on A and B, we sometimes say “the pair (A, B) is contollable”.

10 Linear State Feedback

OK, forget about the output for awhile, and lets only worry about the state equation:

ẋ = Ax + Bu

The goal of linear state feedback is to find a control law

u = −Kx,

K ∈ Rm×n , so that the system behaves in some “nice” way. Note that under this feedback law, the state equation
becomes

ẋ = Ax − BKx = (A − BK)x

so we can understand how the closed loop system behaves using the eigenvalues of the matrix A − BK.
eigenvalue Placement Theorem: Let Λ = {λ1 , λ2 , . . . , λn } be an allowable set of eigenvalues, i.e., if λi = a + bi,
b 6= 0, is in Λ, then λj = a − bi must also be in Λ. Then the pair (A, B) is controllable if and only if there exists a
K ∈ Rm×n such that

eig(A − BK) = Λ.

This is huge!!!. It means that I can find a gain matrix K to place the eigenvalues of a controllable system anywhere I
want to!

10
10.1 Example: mass-spring-damper

mÿ + γ ẏ + ks y = u
we’ve already seen that this becomes
   
0 1 0
ẋ = γ x + 1 u
− kms − m m
 
y= 1 0 x
Is the system controllable?
     
  0 0 1 0
W = B AB = 1 γ 1
m − kms −m m

1
 
0 m
= 1
m − mγ2
We can check the rank of W by looking at its determinant:
1
det(W ) = − 6= 0
m2
so W is full rank and the pair (A, B) is controllable.
Let Λ = {λ1 , λ2 } be an allowable pair of eigenvalues.
The controller design problem is then to find a matrix
 
K = k1 k2

such that the eigenvalues of A − BK are λ1 and λ2 . The eigenvalue placement theorem tells us we can do it, but it
doesn’t tell us how! We can figure it out though because we are smart.
   
0 1 0  
A − BK = ks γ − 1 k1 k2
− m −m m
 
0 1
= −ks −k1 −γ−k2
m m
The eigenvalues of A − BK are

det(λI − (A − BK)) = 0,

i.e.,

Λ = roots(det(λI − (A − BK))).

Subbing in our values:

  
λ −1
Λ = roots det ks +k1
m λ + γ+k
m
2

   
γ + k2 ks + k1
= roots λ λ + +
m m
 
2 γ + k2 ks + k1
= roots λ + λ +
m m

11
Using the quadratic formula, this becomes
r  2
− γ+k
m ±
2 γ+k2
m − 4 ksm
+k1

Λ=
2
If the desired eigenvalues are complex, Λ = a ± bi, then we can solve as follows:

γ + k2
a=− → k2 = −2ma − γ
2m
imaginary part: it is tedious, but it can be done!
MATLAB is our friend: Fortunately, the MATLAB controls toolbox provides some eigenvalue placement functions:
place and acker.

11 Complex Plane Intuition

It is useful to look at the locations of the eigenvalues of a system in the complex plane. Right now, we can look at the
eigenvalue locations and quickly tell whether or not the system is stable: the system is stable if all of the eigenvalues
lie in the left half plane. If any of the eigenvalues lie in the right half plane or on the imaginary axis, then the system
is unstable.

11.1 Second Order Systems

In order to get an intuitive understanding how systems respond as a function of eigenvalue placement, it can be
useful to study the mass-spring-damper system. After working through the example (see Section 8.1 in Lecture 7) we
determined that the eigenvalues of the system are:
p
−γ ± γ 2 − 4mks
λ= .
2m
In the “normal” case, γ > 0 and ks > 0, and we have good intuition for how we expect the system to behave. This
case can be broken down even further into the following three cases:
case 1 (underdamped): in this case, the damping is small, which we quantify by the condition γ 2 < 4mks . This
means that the sign of the term under the square root is negative, which means that the eigenvalues are complex. The
real part is −γ/2m, so the system is asymptotically stable. As γ increases, the real parts of the eigenvalues are pushed

12
further in the left half plane, and the system converges faster. As the spring gets stiffer (i.e., as ks gets bigger), the
magnitude of the imaginary parts of the eigenvalues increases and the frequency of oscillation increases.
These two observations can be generalize to apply to all stable second order systems with complex eigenvalues: as the
real part gets more negative, the rate of convergence increases; and as the imaginary part gets bigger, the frequency of
oscillation increases.
case 2 (critically damped): in this case, γ 2 = 4mks , which means that there are two eigenvalues on top of each other
at −γ/2m. In this case, there is no oscillatory component to the system response. The system will converge directly to
the origin without any overshoot. As in case 1, the rate of convergence will increase this pair of eigenvalues is pushed
leftward in the complex plane.
case 3 (overdamped): in this case, damping is large, which we quantify by the condition γ 2 > 4mks . Here the sign of
the term under the square root is positive, which means the two eigenvalues are separate and real. As with the critically
damped case, there is no oscillatory component to the system response. The rate of convergence is determined by the
rightmost eigenvalue. As the damping gets larger, this eigenvalue moves rightward along the real axis, making the rate
of convergence slower. Note that the square root term is guaranteed to be less than γ, which means that the rightmost
eigenvalue is guaranteed to be negative, which means that the system is always stable.
The non-normal cases are less physically meaningful, but still give good intuition and are fun to think about.
First, consider the case of negative damping. This means that there is a force that pushes the mass in the direction
of its current velocity. The faster the mass moves, the harder the force pushes. Intuitively, this will obviously lead to
instability. And the math matches: since γ is negative, the real parts of the eigenvalues will be positive in all cases,
and the system is indeed unstable.
Next, consider the case of a negative spring constant. This means that the spring pushes the mass away from zero,
and the further away the mass is the harder it pushes. Again, this intuitively will lead to instability. And the math
matches. Since 4mks is negative, the term under the square root will be positive and greater than γ 2 , which means the
numerator will always yield one positive and one negative eigenvalue, which means the system is unstable.

11.2 Dominant Eigenvalues

If there is a set of eigenvalues that are much closer to the imaginary axis than the other eigenvalues, those eigenvalues
will dominate the response. It is not unusual for one or two eigenvalues to be dominant, allowing a higher order system
to be approximated by a first or second order system. This is part of the reason why it is important to thoroughly
understand second orders systems: many higher order systems act like second order systems!

13
12 Linear Quadratic Regulator

We know a lot about eigenvalues, and it is nice to be able to place them wherever we want. However, it can be
overwhelming to figure out the “best” place to put the eigenvalues, especially when the order to the system is large.
Optimal control theory can help simplify things, or at least put them into a more intuitive framework.
The basic idea of any optimal control strategy is to

1. define a cost function J : X × U → R, where X and U represent the set of continuous functions of time in the
state and input spaces, respectively.
2. find a feedback control law u = k(x) that minimizes J while possibly also satisfying some constraints (such as
x(tf ) = xf .

If you can solve the above problem, then instead of directly placing eigenvalues, we can design the controller by
adjusting the parameters of J.
Let’s make that more concrete, and look at an optimal control problem called infinite time horizon linear quadradic
regulation, or just LQR for short.
LQR problem statement:
Define the cost function
Z ∞
J(x(t), u(t)) = (x(t)T Qx(t) + u(t)T Ru(t))dt
0

Where Q and R are

1. symmetric, i.e. Q = QT or qij = qji

2. positive definite, i.e. xT Qx > 0 for any x ∈ Rn and uT Ru > 0 for any u ∈ Rm . Alternatively, all of the
eigenvalues of Q and R are positive.

Now find the state feedback matrix K ∗ so that the closed loop system under the feedback law u = −K ∗ x results in a
lower value of J than any other choice of K.
What does this mean intuitively? To simplify lets examine the case where n = m = 1. Then J becomes
Z ∞
J(x, u) = (Qx(t)2 + Ru(t)2 )dt
0

Now we can use this cost function to evaluate “how good” a feedback controller is. Assume that the feedback law
being considered is u = −Kx, i.e., linear state feedback for some matrix K, and further assume that the resulting
closed loop system is asymptotically stable. This means that both x and u go to zero at t → ∞. The cost function
integrates the weighted squares of these functions, capturing both some combination of how fast the functions go to
zero and how large they are in the mean time.
Now, more generally, we can think of

xT Qxdt is a penalty for not having x = 0, i.e., penalty for not getting to the goal.
R
1.
uT Rudt is a penalty for not having u = 0, i.e., penalty for using input energy (“price of gas”).
R
2.

Fortunately, the problem of finding the linear feedback matrix K that minimizes J when the pair (A, B) is controllable
has already been solved.

14
And guess what? MATLAB has a command that solves it! Namely

K = lqr(A, B, Q, R)

Gives the K that minimizes J.

Now we can think about controller design in terms of a tradeoff between Q and R.

• making Q larger will drive x to zero faster.

• making R larger will reduce the control energy expended.

LQR feedback law for the double mass-spring example can be found in lec16 example.m.

13 Quick Review

So we have two extremely powerfull tools for doing state feedback that are guaranteed to work as long as (A, B) is
controllable:

• Eigenvalue placement: We can find a K to put the eigenvalues of the the closed loop system into any desired set
of locations. Use the matlab place command.

• linear quadratic regulator (LQR): We can get good closed loop performance by defining the symmetric positive-
definite matrices Q (i.e., cost for being away from goal) and R (i.e., the “price of gas”). LQR finds the feedback
matrix K so that the closed loop system optimizes the integral of the sum of the two cost components. Use the
matlab lqr command.

These tools assume we can directly measure the state, which is not always true. So what do you do when the state
cannot directly be measured? We’ll get to that later today. But first, let’s look at how we can use state-feedback to
create a controller that causes the output to track a reference input (like PID did).

14 State-Space Tracking Controllers

Suppose that we add the ability to include an external input v into our state feedback controller by making u = v −Kx,
resulting in the following block diagram:

Further, assume that v is constant. What happens? The addition of v does not affect the transient response of the
system, that is determined by the eigenvalues, which are unchanged by adding v. But the equilibrium point that the

15
state will converge to (assuming the closed loop system is asymptotically stable) will change. Before adding v, the
equilibrium point was at zero. We can find the equilibrium point with v added by solving for the x where ẋ is zero,
i.e.:

ẋ = 0 = Axe − BKxe + Bv,

which gives

xe = −(A − BK)−1 Bv.

The corresponding equilibrium output is then

ye = Cxe = −C(A − BK)−1 Bv.

So, for some desired output value yd , we can set

−1
v = − C(A − BK)−1 B yd ,

which will cause the output of the system to converge to ye = yd . We can add this into the block diagram as follows:

For constant yd , we are guaranteed that y → yd , hence we can “track” constant desired outputs. We can generalize
this to varying yd signals that vary much slower than the transients in the closed-loop system. We loose any kind of
guarantee, but when yd varies slowly, the actual y will roughly track it.

16