Modern Optimal Control

Matthew M. Peet
Arizona State University

Lecture 22: H2 , LQG and LGR

Conclusion

To solve the H∞ -optimal state-feedback problem, we solve

min γ such that

γ,X1 ,Y1 ,An ,Bn ,Cn ,Dn
 
X1 I
>0
I Y1
AY1 +Y1 AT +B2 Cn +CnT B2T ∗T ∗T ∗T
 
T T T T T T
 A + An + [B2 Dn C2 ] X1 A+A X1 +Bn C2 +C2 Bn ∗ ∗T 
 [B1 + B2 Dn D21 ]T [XB1 + Bn D21 ]T −γI
< 0
C1 Y1 + D12 Cn C1 + D12 Dn C2 D11 +D12 Dn D21 −γI

M. Peet Lecture 22: 2 / 21

Conclusion
Then, we construct our controller using

DK = (I + DK2 D22 )−1 DK2

BK = BK2 (I − D22 DK )
CK = (I − DK D22 )CK2
AK = AK2 − BK (I − D22 DK )−1 D22 CK .

where
   −1      T −1
AK2 BK2 X2 X1 B2 An Bn X1 AY1 0 Y2 0
= − .
CK2 DK2 0 I Cn Dn 0 0 C2 Y1 I

and where X2 and Y2 are any matrices which satisfy X2 Y2 = I − X1 Y1 .

• e.g. Let Y2 = I and X2 = I − X1 Y1 .
• The optimal controller is NOT uniquely defined.
• Don’t forget to check invertibility of I − D22 DK

M. Peet Lecture 22: 3 / 21

Conclusion

The H∞ -optimal controller is a dynamic system.

 
AK B K
• Transfer Function K̂(s) =
CK DK
Minimizes the effect of external input (w) on external output (z).

kzkL2 ≤ kS(P, K)kH∞ kwkL2

• Minimum Energy Gain

M. Peet Lecture 22: 4 / 21

H2 -optimal control
Motivation

H2 -optimal control minimizes the H2 -norm of the transfer function.

• The H2 -norm has no direct interpretation.
Z ∞
1
kGk2H2 = Trace(Ĝ∗ (ıω)Ĝ(ıω))dω
2π −∞

Motivation: Assume external input is Gaussian noise with spectral density Sw

Z ∞
1
E[w(t)2 ] = Trace(Ŝw (ıω))dω
2π −∞

Theorem 1.
For an LTI system P , if w is noise with spectral density Ŝw (ıω) and z = P w,
then z is noise with density

Ŝz (ıω) = P̂ (ıω)Ŝ(ıω)P̂ (ıω)∗

M. Peet Lecture 22: 5 / 21

H2 -optimal control
Motivation

Then the output z = P w has signal variance (Power)

Z ∞
1
E[z(t)2 ] = Trace(Ĝ∗ (ıω)S(ıω)Ĝ(ıω))dω
2π −∞
≤ kSkH∞ kGk2H2

If the input signal is white noise, then Ŝ(ıω) = I and

E[z(t)2 ] = kGk2H2

M. Peet Lecture 22: 6 / 21

H2 -optimal control
Colored Noise

If the noise is colored, then we can still use the same approach to H2 optimal
control. Let Ĥ(ıω)Ĥ(ıω)∗ = Ŝw (ıω). Then design an H2 -optimal controller for
the plant  
P̂ (s)Ĥ(s) P̂12 (s)
P̂s (s) = 11
P̂21 (s)Ĥ(s) P̂22 (s)
Now, using the controller and filtered plant,
S(Ps , K)(s) = P11 H + P12 (I − KP22 )−1 KP21 H = S(P, K)H

For noise with spectral density Sw , let z = S(P, K)w. Then if Sz is the spectral
density of z, we have
Sz (s) = S(P, K)(s)Ŝw (s)S(P, K)(s)∗
= S(P, K)(s)Ĥ(s)Ĥ(s)∗ S(P, K)(s)∗ = S(Ps , K)(s)Ŝ(Ps , K)(s)∗

and hence E[z(t)2 ] = kŜ(Ps , K)k2H2 = γ. Thus minimization of kŜ(Ps , K)k2H2

achieves the optimal system response to noise with density Ŝw .

M. Peet Lecture 22: 7 / 21

H2 -optimal control

Theorem 2.
Suppose P̂ (s) = C(sI − A)−1 B. Then the following are equivalent.
1. A is Hurwitz and kP̂ kH2 < γ.
2. There exists some X > 0 such that

trace CXC T < γ

AX + XAT + BB T < 0

M. Peet Lecture 22: 8 / 21

H2 -optimal control

Proof.
Suppose A is Hurwitz and kP̂ kH2 < γ. Then the Controllability Grammian is
defined as Z ∞
T
Xc = eAt BB T eA dt
0
Now recall the Laplace transform
Z ∞
At
eAt e−ts dt


Λe (s) =
Z0 ∞
= e−(sI−A)t dt
0
t=−∞
−1 −(sI−A)t

= −(sI − A) e dt
t=0
= (sI − A)−1

Hence ΛCeAt B (s) = C(sI − A)−1 B.

M. Peet Lecture 22: 9 / 21

H2 -optimal control

Proof.
ΛCeAt B (s) = C(sI − A)−1 B implies

kP̂ k2H2 = kC(sI − A)−1 Bk2H2

Z ∞
1
= Trace((C(ıωI − A)−1 B)∗ (C(ıωI − A)−1 B))dω
2π 0
Z ∞
1
= Trace((C(ıωI − A)−1 B)(C(ıωI − A)−1 B)∗ )dω
2π 0
Z ∞
∗
= Trace CeAt BB ∗ eA t C ∗ dt
−∞
= TraceCXc C T

Thus Xc ≥ 0 and TraceCXc C T = kP̂ k2H2 < γ.

M. Peet Lecture 22: 10 / 21

H2 -optimal control

Proof.
Likewise TraceB T Xo B = kP̂ k2H2 . To show that we can take strict the
inequality X > 0, we simply let
Z ∞

T
X= eAt BB T + I eA dt
0

for sufficiently small  > 0. Furthermore, we already know the controllability

grammian Xc and thus X satisfies the Lyapunov inequality.
AT X + X A + BB T < 0

These steps can be reversed to obtain necessity.

M. Peet Lecture 22: 11 / 21

H2 -optimal control
Full-State Feedback

Lets consider the full-state feedback problem

 
A B1 B2
Ĝ(s) =  C1 0 D12 
I 0 0

• D12 is the weight on control effort.

• D11 = 0 is neglected as the feed-through term.
• C2 = I as this is state-feedback.
 
0 0
K̂(s) =
0 K

M. Peet Lecture 22: 12 / 21

H2 -optimal control
Full-State Feedback

Theorem 3.
The following are equivalent.
1. kS(K, P )kH2 < γ.
2. K = ZX −1 for some Z and X > 0 where
 AT
   
 X
+ X ZT + B1 B1T < 0
 
A B2
Z B2T
Trace C1 X + D12 Z X −1 C1 X + D12 Z < γ
   

However, this is nonlinear, so we need to reformulate using the Schur

Complement.

M. Peet Lecture 22: 13 / 21

H2 -optimal control

Applying the Schur Complement gives the alternative formulation convenient for
control.
Theorem 4.
Suppose P̂ (s) = C(sI − A)−1 B. Then the following are equivalent.
1. A is Hurwitz and kP̂ kH2 < γ.
2. There exists some X, Z > 0 such that
 T
CT
  
A X + XA XB X
< 0, > 0, TraceZ < γ
BT X −γI C Z

M. Peet Lecture 22: 14 / 21

H2 -optimal control
Full-State Feedback

Theorem 5.
The following are equivalent.
1. kS(K, P )kH2 < γ.
2. K = ZX −1 for some Z and X > 0 where
 AT
   
 X T
+ B1 B1T < 0
 
A B2 + X Z
Z B2T
(C1 X + D12 Z)T
 
X
>0
C1 X + D12 Z W
TraceW < γ

Thus we can solve the H2 -optimal static full-state feedback problem.

M. Peet Lecture 22: 15 / 21

H2 -optimal control
Relationship to LQR

Thus minimizing the H2 -norm minimizes the effect of white noise on the power
of the output noise.
• This is why H2 control is often called Least-Quadratic-Gaussian (LQG).
LQR:
• Full-State Feedback
• Choose K to minimize the cost function
Z ∞
x(t)T Qx(t) + u(t)T Ru(t)dt
0

subject to dynamic constraints

ẋ(t) = Ax(t) + Bu(t)

u(t) = Kx(t), x(0) = x0

M. Peet Lecture 22: 16 / 21

H2 -optimal control
Relationship to LQR

To solve the LQR problem, let

 1
Q2
• C1 =
0
 
0
• D12 = 1
R2
• B2 = B and B1 = I.
So that
 
  A + BK I
A + B2 K B1 1
S(P̂ , K̂) = = Q2 
C1 + D12 K D11 1 0
R2 K
And solve the H2 full-state feedback problem. Then if
ẋ(t) = ACL x(t) = (A + BK)x(t) = Ax(t) + Bu(t)
u(t) = Kx(t), x(0) = x0

Then x(t) = eACL t x0

M. Peet Lecture 22: 17 / 21
H2 -optimal control
Relationship to LQR

If

ẋ(t) = ACL x(t) = (A + BK)x(t) = Ax(t) + Bu(t)

u(t) = Kx(t), x(0) = x0

then x(t) = eACL t x0 and

Z ∞ Z ∞
T
x(t)T Qx(t) + u(t)T Ru(t)dt = xT0 eACL t (Q + K T RK)eACL t x0 dt
0 0
Z ∞  1 T  1
T Q 2 Q 2
= Trace xT0 eACL t 1 1 eACL t x0 dt
0 R2 K R2 K
Z ∞
T
= kx0 k2 Trace B1 eACL t (C1 + D12 K)T (C1 + D12 K)eACL t B1T dt
0
= kx0 k2 kS(K, P )k2H2

Thus LQR reduces to a special case of H2 static state-feedback.

M. Peet Lecture 22: 18 / 21

H2 -optimal output feedback control

Theorem 6 (Lall).
The following are equivalent.
 
AK BK
• There exists a K̂ = such that kS(K, P )kH2 < γ.
CK DK
 
X1 I
• There exist X1 , Y1 , Z, An , Bn , Cn , Dn such that >0
I Y1
AY1 +Y1 AT +B2 Cn +CnT B2T ∗T ∗T
 
 AT + An + [B2 Dn C2 ]T X1 A+AT X1 +Bn C2 +C2T BnT ∗T  < 0,
T T
[B1 + B2 Dn D21 ] [XB1 + Bn D21 ] −γI
∗T
 
X1 I
 I Y1 ∗T  > 0,
C1 Y1 + D12 Cn C1 + D12 Dn C2 Z
D11 + D12 Dn D21 = 0, trace(Z) < γ

M. Peet Lecture 22: 19 / 21

H2 -optimal output feedback control

As before, the controller can be recovered as

−1  −1
Y2T
      
AK2 BK2 X2 X1 B2 An B n X1 AY1 0 0
= −
CK2 DK2 0 I C n D n 0 0 C2 Y1 I
for any full-rank X2 and Y2 such that
   −1
X1 X2 Y1 Y2
=
X2T X3 Y2T Y3

To find the actual controller, we use the identities:

DK = (I + DK2 D22 )−1 DK2

BK = BK2 (I − D22 DK )
CK = (I − DK D22 )CK2
AK = AK2 − BK (I − D22 DK )−1 D22 CK

M. Peet Lecture 22: 20 / 21

Robust Control
Before we finish, let us briefly touch on the use of LMIs in Robust Control.

p M q

Questions:
• Is S(∆, M ) stable for all ∆ ∈ ∆?
• Determine
sup kS(∆, M )kH∞ .
∆∈∆

M. Peet Lecture 22: 21 / 21