Linear Control Systems

Linear Quadratic Regulator (LQR)

– p. 1/22
Consider the system

ẋ = Ax + Bu

and suppose we want to design state feedback control

u = F x to stabilize the system.

– p. 2/22
Consider the system

ẋ = Ax + Bu

and suppose we want to design state feedback control

u = F x to stabilize the system. The design of F is a
tradeoff between the transient response and the control
effort.

– p. 2/22
Consider the system

ẋ = Ax + Bu

and suppose we want to design state feedback control

u = F x to stabilize the system. The design of F is a
tradeoff between the transient response and the control
effort. The optimal control approach to this design tradeoff
is to define the performance index (cost functional)
Z ∞
J = [xT (t)Qx(t) + uT (t)Ru(t)] dt
0

and search for the control u = F x that minimizes this index.

– p. 2/22
Consider the system

ẋ = Ax + Bu

and suppose we want to design state feedback control

u = F x to stabilize the system. The design of F is a
tradeoff between the transient response and the control
effort. The optimal control approach to this design tradeoff
is to define the performance index (cost functional)
Z ∞
J = [xT (t)Qx(t) + uT (t)Ru(t)] dt
0

and search for the control u = F x that minimizes this index.

Q is an n × n symmetric positive semidefinite matrix and R
is an m × m symmetric positive definite matrix

– p. 2/22
The matrix Q can be written as Q = M T M , where M is a
p × n matrix, with p ≤ n. With this representation

xT Qx = xT M T M x = z T z

where z = M x can be viewed as a controlled output

– p. 3/22
The matrix Q can be written as Q = M T M , where M is a
p × n matrix, with p ≤ n. With this representation

xT Qx = xT M T M x = z T z

where z = M x can be viewed as a controlled output

Optimal Control Problem: Find u(t) = F x(t) to minimize J

subject to the constraint ẋ = Ax + Bu

– p. 3/22
The matrix Q can be written as Q = M T M , where M is a
p × n matrix, with p ≤ n. With this representation

xT Qx = xT M T M x = z T z

where z = M x can be viewed as a controlled output

Optimal Control Problem: Find u(t) = F x(t) to minimize J

subject to the constraint ẋ = Ax + Bu

Since J is defined by an integral over [0, ∞), the first

question we need to address is: Under what conditions will
J exist and be finite?

– p. 3/22
Write J as
¯ f)
J = lim J(t
tf →∞
Z tf
¯ f) =
J(t [xT (t)Qx(t) + uT (t)Ru(t)] dt
0

– p. 4/22
Write J as
¯ f)
J = lim J(t
tf →∞
Z tf
¯ f) =
J(t [xT (t)Qx(t) + uT (t)Ru(t)] dt
0
¯ f ) is a monotonically increasing function of tf .
J(t

– p. 4/22
Write J as
¯ f)
J = lim J(t
tf →∞
Z tf
¯ f) =
J(t [xT (t)Qx(t) + uT (t)Ru(t)] dt
0
¯ f ) is a monotonically increasing function of tf . Hence,
J(t
as tf → ∞, J¯(tf ) either converges to a finite limit or
diverges to infinity

– p. 4/22
Write J as
¯ f)
J = lim J(t
tf →∞
Z tf
¯ f) =
J(t [xT (t)Qx(t) + uT (t)Ru(t)] dt
0
¯ f ) is a monotonically increasing function of tf . Hence,
J(t
as tf → ∞, J¯(tf ) either converges to a finite limit or
diverges to infinity

¯ f ) be finite?
Under what conditions will limtf →∞ J(t

– p. 4/22
Recall that (A, B) is stabilizable if the uncontrollable
eigenvalues of A, if any, have negative real parts

– p. 5/22
Recall that (A, B) is stabilizable if the uncontrollable
eigenvalues of A, if any, have negative real parts

Notice that (A, B) is stabilizable if (A, B) is controllable or

Re [λ(A)] < 0

– p. 5/22
Recall that (A, B) is stabilizable if the uncontrollable
eigenvalues of A, if any, have negative real parts

Notice that (A, B) is stabilizable if (A, B) is controllable or

Re [λ(A)] < 0

Definition: (A, C) is detectable if the unobservable

eigenvalues of A, if any, have negative real parts

– p. 5/22
Recall that (A, B) is stabilizable if the uncontrollable
eigenvalues of A, if any, have negative real parts

Notice that (A, B) is stabilizable if (A, B) is controllable or

Re [λ(A)] < 0

Definition: (A, C) is detectable if the unobservable

eigenvalues of A, if any, have negative real parts

Lemma 1 : Suppose (A, B) is stabilizable, (A, M ) is

detectable, where Q = M T M , and u(t) = F x(t). Then, J
is finite for every x(0) ∈ Rn if and only if
Re [λ(A + BF )] < 0

– p. 5/22
Remarks:

– p. 6/22
Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [λ(A + BF )] < 0

– p. 6/22
Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [λ(A + BF )] < 0
2. To see why detectability of (A, M ) is needed, consider
Z ∞
ẋ = x + u, J = u2 (t) dt
0

– p. 6/22
Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [λ(A + BF )] < 0
2. To see why detectability of (A, M ) is needed, consider
Z ∞
ẋ = x + u, J = u2 (t) dt
0

A = 1, B = 1, M = 0, R = 1

– p. 6/22
Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [λ(A + BF )] < 0
2. To see why detectability of (A, M ) is needed, consider
Z ∞
ẋ = x + u, J = u2 (t) dt
0

A = 1, B = 1, M = 0, R = 1
(A, B) is controllable, but (A, M ) is not detectable

– p. 6/22
Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [λ(A + BF )] < 0
2. To see why detectability of (A, M ) is needed, consider
Z ∞
ẋ = x + u, J = u2 (t) dt
0

A = 1, B = 1, M = 0, R = 1
(A, B) is controllable, but (A, M ) is not detectable

F = 0 ⇒ u(t) = 0 ⇒ J = 0

– p. 6/22
Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [λ(A + BF )] < 0
2. To see why detectability of (A, M ) is needed, consider
Z ∞
ẋ = x + u, J = u2 (t) dt
0

A = 1, B = 1, M = 0, R = 1
(A, B) is controllable, but (A, M ) is not detectable

F = 0 ⇒ u(t) = 0 ⇒ J = 0

This control is clearly optimal and results in a finite J but it

does not stabilize the system because A + BF = A = 1
– p. 6/22
Lemma 2: For any stabilizing control u(t) = F x(t), the
cost is given by
J = x(0)T W x(0)
where W is a symmetric positive semidefinite matrix that
satisfies the Lyapunov equation

W (A + BF ) + (A + BF )T W + Q + F T RF = 0

– p. 7/22
Lemma 2: For any stabilizing control u(t) = F x(t), the
cost is given by
J = x(0)T W x(0)
where W is a symmetric positive semidefinite matrix that
satisfies the Lyapunov equation

W (A + BF ) + (A + BF )T W + Q + F T RF = 0

Remark: The control u(t) = F x(t) is stabilizing if

Re [λ(A + BF )] < 0

– p. 7/22
Theorem: Consider the system ẋ = Ax + Bu and the
performance index
Z ∞
J = [xT (t)Qx(t) + uT (t)Ru(t)] dt
0

where Q = M T M , R is symmetric and positive definite,

(A, B) is stabilizable, and (A, M ) is detectable. The
optimal stabilizing control is

u(t) = −R−1 B T P x(t)

where P is the symmetric positive semidefinite solution of

the Algebraic Riccati Equation (ARE)

0 = P A + AT P + Q − P BR−1 B T P
– p. 8/22
Remarks:

– p. 9/22
Remarks:

1. Since the control is stabilizing,

Re [λ(A − BR−1 B T P )] < 0

– p. 9/22
Remarks:

1. Since the control is stabilizing,

Re [λ(A − BR−1 B T P )] < 0

2. The control is optimal among all square integrable

signals u(t), not just among u(t) = F x(t)

– p. 9/22
Remarks:

1. Since the control is stabilizing,

Re [λ(A − BR−1 B T P )] < 0

2. The control is optimal among all square integrable

signals u(t), not just among u(t) = F x(t)

3. The Riccati equation can have more than one solution,

but there is only one solution that is positive semidefinite

– p. 9/22
Example:
ẋ1 = x2 , ẋ2 = u
Z ∞
J = [x21 + ρu2 ] dt, ρ > 0
0

– p. 10/22
Example:
ẋ1 = x2 , ẋ2 = u
Z ∞
J = [x21 + ρu2 ] dt, ρ > 0
0
" # " # " #
0 1 0 1 0
A= , B= , Q= , R=ρ
0 0 1 0 0

– p. 10/22
Example:
ẋ1 = x2 , ẋ2 = u
Z ∞
J = [x21 + ρu2 ] dt, ρ > 0
0
" # " # " #
0 1 0 1 0
A= , B= , Q= , R=ρ
0 0 1 0 0
" #
1 h i h i
Q= 1 0 ⇒ M = 1 0
0

– p. 10/22
Example:
ẋ1 = x2 , ẋ2 = u
Z ∞
J = [x21 + ρu2 ] dt, ρ > 0
0
" # " # " #
0 1 0 1 0
A= , B= , Q= , R=ρ
0 0 1 0 0
" #
1 h i h i
Q= 1 0 ⇒ M = 1 0
0
" #
0 1
rank[B, AB] = rank = 2 ⇒ (A, B) is controllable
1 0

– p. 10/22
 " # " #
M 1 0
rank = rank =2
MA 0 1

– p. 11/22
 " # " #
M 1 0
rank = rank =2
MA 0 1

⇒ (A, M ) is observable

– p. 11/22
 " # " #
M 1 0
rank = rank =2
MA 0 1

⇒ (A, M ) is observable
0 = P A + AT P + Q − P BR−1 B T P

– p. 11/22
 " # " #
M 1 0
rank = rank =2
MA 0 1

⇒ (A, M ) is observable
0 = P A + AT P + Q − P BR−1 B T P
" # 0 = 1 − ρ1 p212
p11 p12
P = ⇒ 0 = p11 − ρ1 p12 p22
p12 p22
0 = 2p12 − ρ1 p222

– p. 11/22
 " # " #
M 1 0
rank = rank =2
MA 0 1

⇒ (A, M ) is observable
0 = P A + AT P + Q − P BR−1 B T P
" # 0 = 1 − ρ1 p212
p11 p12
P = ⇒ 0 = p11 − ρ1 p12 p22
p12 p22
0 = 2p12 − ρ1 p222
√ p
p12 = ± ρ, p22 = 2ρ p12

– p. 11/22
 " # " #
M 1 0
rank = rank =2
MA 0 1

⇒ (A, M ) is observable
0 = P A + AT P + Q − P BR−1 B T P
" # 0 = 1 − ρ1 p212
p11 p12
P = ⇒ 0 = p11 − ρ1 p12 p22
p12 p22
0 = 2p12 − ρ1 p222
√ p
p12 = ± ρ, p22 = 2ρ p12

√ √ 3/4 1 √
p12 = ρ, p22 = 2ρ , p11 = p12 p22 = 2ρ1/4
ρ

– p. 11/22
 " √ #
2ρ1/4 ρ1/2
P = 1/2
√ 3/4
ρ 2ρ

– p. 12/22
 " √ #
2ρ1/4 ρ1/2
P = 1/2
√ 3/4
ρ 2ρ

p11 > 0, det(P ) = ρ > 0 ⇒ P is positive definite

– p. 12/22
 " √ #
2ρ1/4 ρ1/2
P = 1/2
√ 3/4
ρ 2ρ

p11 > 0, det(P ) = ρ > 0 ⇒ P is positive definite

1 h √ i
F = −R−1 B T P = − ρ1/2 2ρ3/4
ρ

– p. 12/22
 " √ #
2ρ1/4 ρ1/2
P = 1/2
√ 3/4
ρ 2ρ

p11 > 0, det(P ) = ρ > 0 ⇒ P is positive definite

1 h √ i
F = −R−1 B T P = − ρ1/2 2ρ3/4
ρ
" #
0 1
A + BF = −1/2
√ −1/4
−ρ − 2ρ

– p. 12/22
 " √ #
2ρ1/4 ρ1/2
P = 1/2
√ 3/4
ρ 2ρ

p11 > 0, det(P ) = ρ > 0 ⇒ P is positive definite

1 h √ i
F = −R−1 B T P = − ρ1/2 2ρ3/4
ρ
" #
0 1
A + BF = −1/2
√ −1/4
−ρ − 2ρ

−1/4 1
λ1,2 = ρ √ (−1 ± j)
2

– p. 12/22
Notice how F and the eigenvalues depend on ρ

– p. 13/22
Notice how F and the eigenvalues depend on ρ
h √ i
F = − ρ−1/2 2ρ−1/4

−1/4 1
λ1,2 = ρ √ (−1 ± j)
2

– p. 13/22
Notice how F and the eigenvalues depend on ρ
h √ i
F = − ρ−1/2 2ρ−1/4

−1/4 1
λ1,2 = ρ √ (−1 ± j)
2
What happens as you change ρ?

– p. 13/22
Matlab Calculations:

– p. 14/22
Matlab Calculations:
X = lyap(A,N) solves the Lyapunov equation

AX + XAT + N = 0

– p. 14/22
Matlab Calculations:
X = lyap(A,N) solves the Lyapunov equation

AX + XAT + N = 0

To solve the equation

W (A + BF ) + (A + BF )T W + Q + F T RF = 0

use W = lyap((A+B*F)′ ,Q+F′ *R*F) ( Notice the transpose)

– p. 14/22
X = are(A,S,Q) solves the Riccati equation

XA + AT X + Q − XSX = 0

S = BR−1 B T

– p. 15/22
X = are(A,S,Q) solves the Riccati equation

XA + AT X + Q − XSX = 0

S = BR−1 B T

[K,P,E] = lqr(A,B,Q,R) solves the Riccati equation

P A + AT P + Q − P BR−1 B T P = 0

K = R−1 B T P and E is a vector whose elements are the

eigenvalues of (A − BR−1 B T P )

– p. 15/22
X = are(A,S,Q) solves the Riccati equation

XA + AT X + Q − XSX = 0

S = BR−1 B T

[K,P,E] = lqr(A,B,Q,R) solves the Riccati equation

P A + AT P + Q − P BR−1 B T P = 0

K = R−1 B T P and E is a vector whose elements are the

eigenvalues of (A − BR−1 B T P )
Notice that F = −K

– p. 15/22
LQR Design

– p. 16/22
LQR Design

Given the system

ẋ = Ax + Bu
where (A, B) is stabilizable, we want to design state
feedback control u = F x to stabilize the system while
meeting
Transient response specifications
Magnitude constraints on x and u

– p. 16/22
Procedure:

– p. 17/22
Procedure:
1. Choose Q and R such that Q = M T M , with (A, M )
detectable, and R = RT > 0

– p. 17/22
Procedure:
1. Choose Q and R such that Q = M T M , with (A, M )
detectable, and R = RT > 0
2. Solve the Riccati equation

P A + AT P + Q − P BR−1 B T P = 0

and compute F = −R−1 B T P

– p. 17/22
Procedure:
1. Choose Q and R such that Q = M T M , with (A, M )
detectable, and R = RT > 0
2. Solve the Riccati equation

P A + AT P + Q − P BR−1 B T P = 0

and compute F = −R−1 B T P

3. Simulate the initial response of ẋ = (A + BF )x for
different initial conditions

– p. 17/22
Procedure:
1. Choose Q and R such that Q = M T M , with (A, M )
detectable, and R = RT > 0
2. Solve the Riccati equation

P A + AT P + Q − P BR−1 B T P = 0

and compute F = −R−1 B T P

3. Simulate the initial response of ẋ = (A + BF )x for
different initial conditions
4. If the transient response specifications and/or the
magnitude constraints are not met, go back to step 1 and
re-choose Q and/or R

– p. 17/22
Typical Choice:
   
q1 r1
 q2   r2 
Q= , R = ρ
   
 ...   ... 

qn rm

1 1
qi = , ri = , ρ>0
tsi (ximax )2 (uimax )2
tsi is the desired settling time of xi
ximax is a constraint on |xi |
uimax is a constraint on |ui |
ρ is chosen to tradeoff regulation versus control effort

– p. 18/22
Example:
   
0 1 0 0
A= 0 0 1 , B =  0 
   
−0.4 −4.2 −2.1 1

Open-loop eigenvalues are: −0.1, −1 ± 1.7321j

Desired settling time is 4 sec.

– p. 19/22
Example:
   
0 1 0 0
A= 0 0 1 , B =  0 
   
−0.4 −4.2 −2.1 1

Open-loop eigenvalues are: −0.1, −1 ± 1.7321j

Desired settling time is 4 sec.

Q = I3 ; Try R = 0.01, 0.1, and 1

– p. 19/22
Example:
   
0 1 0 0
A= 0 0 1 , B =  0 
   
−0.4 −4.2 −2.1 1

Open-loop eigenvalues are: −0.1, −1 ± 1.7321j

Desired settling time is 4 sec.

Q = I3 ; Try R = 0.01, 0.1, and 1

R λ
0.01 −9.7364, −0.9039 ± 0.4593j
0.1 −0.6568, −1.9548 ± 1.0158j
1 −0.2599, −1.1433 ± 1.6840j

– p. 19/22
 Q = diag(1,1,1)
2 1.5
R=0.01
R=0.1
R=1 1
1.5

0.5
x1

x2
1
0

0.5
−0.5

0 −1
0 1 2 3 4 5 0 1 2 3 4 5

1 5

0.5 0

0 −5

−0.5 −10
x3

u
−1 −15

−1.5 −20

−2 −25

−2.5 −30
0 1 2 3 4 5 0 0.2 0.4 0.6 0.8 1

– p. 20/22
 R=0.1
1.5
Q11 = 1
1
Q11 = 10
1
0.5
x1

x2
0.5
0

0 −0.5

−1
−0.5
0 1 2 3 4 5 0 1 2 3 4 5

1 5

0 0

−5
−1
x3

u
−10
−2
−15
−3
−20
0 1 2 3 4 5 0 1 2 3 4 5

– p. 21/22
 Q11=1;R=0.1
1.5 Q =10;R=0.1
11 1
Q11=10;R=0.4

1 0.5
x1

x2
0.5 0

−0.5
0

−1
−0.5
0 1 2 3 4 5 0 1 2 3 4 5

1 5

0 0

−5
−1
x3

−10
−2
−15
−3
−20
0 1 2 3 4 5 0 0.2 0.4 0.6 0.8 1

– p. 22/22