Topics in Undergraduate Control Systems Design

Joo P. Hespanha a

April 9, 2006

Disclaimer: This is an early draft and probably contains many typos. Comments and information about typos are welcome. Please contact the author at (hespanha@ ece.ucsb.edu).

c Copyright to Joo Hespanha. Please do not distribute this document without the authors a consent.

Contents
System Identication
1 Computer-Controlled Systems 1.1 Computer control . . . . . . . . . . . . . . . . . 1.2 Continuous-time systems . . . . . . . . . . . . . 1.2.1 Steady-state response to sine waves . . . 1.3 Discrete-time systems . . . . . . . . . . . . . . 1.3.1 Steady-state response to sine waves . . . 1.3.2 Step and impulse responses . . . . . . . 1.4 Discrete vs. continuous-time transfer functions 1.5 MATLAB hints . . . . . . . . . . . . . . . . . . 1.6 To probe further . . . . . . . . . . . . . . . . . 2 Parametric identication using 2.1 Parametric identication . . . 2.2 Least-squares line tting . . . 2.3 Vector least-squares . . . . . 2.4 MATLAB hints . . . . . . . . 2.5 To probe further . . . . . . . 2.6 Exercises . . . . . . . . . . . least-squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3
5 5 6 7 8 9 10 10 11 13 15 15 16 18 20 20 22 25 25 26 28 28 30 31 31 33 33 34 35

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

3 Parametric identication of an ARX model 3.1 ARX Model . . . . . . . . . . . . . . . . . . . 3.2 Identication of an ARX model . . . . . . . . 3.3 Known parameters . . . . . . . . . . . . . . . 3.4 MATLAB hints . . . . . . . . . . . . . . . . . 3.5 Exercises . . . . . . . . . . . . . . . . . . . . 4 Practical considerations in parametric 4.1 Choice of inputs . . . . . . . . . . . . 4.2 Signal scaling . . . . . . . . . . . . . . 4.3 Choice of sampling frequency . . . . . 4.4 Choice of model order . . . . . . . . . 4.5 Exercises . . . . . . . . . . . . . . . . i

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

identication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

ii 5 Nonparametric identication 5.1 Nonparametric methods . . . . . 5.2 Time-domain identication . . . 5.2.1 Impulse response method 5.2.2 Step response . . . . . . . 5.2.3 Other inputs . . . . . . . 5.3 Frequency response identication 5.3.1 Sine-wave testing . . . . . 5.3.2 Correlation method . . . 5.4 MATLAB hints . . . . . . . . . . 5.5 To probe further . . . . . . . . . 5.6 Exercises . . . . . . . . . . . . .

Joo P. Hespanha a 37 37 38 38 39 40 40 40 41 42 42 43

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

Robust Control
6 Robust stability 6.1 Model uncertainty . . . . . . . . 6.1.1 Additive uncertainty . . . 6.1.2 Multiplicative uncertainty 6.2 Nyquist stability criterion . . . . 6.2.1 The Nyquist Plot . . . . . 6.2.2 The Nyquist criterion . . 6.3 Small gain condition . . . . . . . 6.4 MATLAB hints . . . . . . . . . . 6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47
49 49 50 52 52 52 53 54 56 57 61 61 62 65 66

7 Control design by loop-shaping 7.1 The loop-shaping design method . . . . 7.2 Open-loop vs. closed-loop specications 7.3 Open-loop gain shaping . . . . . . . . . 7.4 Exercises . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

LQG/LQR controller design

8 Review of State-space models 8.1 State-space models . . . . . . . . 8.2 Input-output relations . . . . . . 8.3 Realizations . . . . . . . . . . . . 8.4 Controllability and observability 8.5 Stability . . . . . . . . . . . . . . 8.6 MATLAB hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71
73 73 74 75 76 77 77

Control Systems Design 9 Linear Quadratic Regulation (LQR) 9.1 Feedback conguration . . . . . . . . 9.2 Optimal Regulation . . . . . . . . . 9.3 State-Feedback LQR . . . . . . . . . 9.4 Stability and Robustness . . . . . . . 9.5 Loop-shaping control using LQR . . 9.6 MATLAB hints . . . . . . . . . . . . 9.7 To probe further . . . . . . . . . . . 9.8 Exercises . . . . . . . . . . . . . . . 10 LQG/LQR Output Feedback 10.1 Output Feedback . . . . . . 10.2 Full-order observers . . . . 10.3 LQG estimation . . . . . . . 10.4 LQG/LQR output feedback 10.5 Separation Principle . . . . 10.6 Loop-gain recovery . . . . . 10.7 MATLAB hints . . . . . . . 10.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii 79 79 80 81 82 85 87 88 89 91 91 91 93 94 95 95 97 97 99 99 100 101 102

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

11 Set-point control 11.1 Nonzero equilibrium state and input 11.2 State-feedback . . . . . . . . . . . . 11.3 Output-feedback . . . . . . . . . . . 11.4 To probe further . . . . . . . . . . .

Attention! When a margin sidebar nishes with . . . , more information about the topic can be found at the end of the lecture in the To probe further section.

iv

Joo P. Hespanha a

System Identication

Introduction to System Identication

The goal of system identication is to utilize input/output experimental data to determine a systems model. For example, one may apply to the system a specic input u, measure the observed discrete-time output y an try to determine the systems transfer function (cf. Figure 1).
u y

? Figure 1. System identication from input/output experimental data

Pre-requisites 1. Laplace transform, continuous-time transfer functions, stability, and frequency response (briey reviewed here). 2. z-transform, discrete-time transfer functions, stability, and impulse/step/frequency responses (briey reviewed here). 3. Computer-control system design (briey reviewed here). 4. Familiarity with basic vector and matrix operations. 5. Knowledge of MATLAB/Simulink.

Joo P. Hespanha a

Lecture #1

Computer-Controlled Systems
Summary
This lecture reviews basic concepts in computer-controlled systems: 1. Input-output models for continuous-time systems (time and Laplace domains) 2. Input-output models for discrete-time systems (time and z-transform domains) 3. Transfer functions for system with sample and hold Contents 1.1 1.2 1.3 1.4 1.5 1.6 Computer control . . . . . . Continuous-time systems . . Discrete-time systems . . . Discrete vs. continuous-time MATLAB hints . . . . . . . To probe further . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . transfer functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 . 6 . 8 . 10 . 11 . 13

1.1

Computer control

Figure 1.1 show the typical block diagram of a control system implemented using a digital computer. Although the output y(t) of most physical systems vary continuously as a function of time, it can only be measured at discrete time instants. Typically, it is sampled periodically as shown in the left plot of Figure 1.2. Moreover, the control signal u(t) cannot be changed continuously and typically is held constant between sampling times as shown in the right plot of Figure 1.2. It is therefore sometimes convenient to regard the process to be controlled as a discrete-time system whose inputs and outputs are the discrete-time signals yd (k) = y(kTs ), ud (k) = u(kTs ), 5 k {1, 2, 3, . . . }.

Joo P. Hespanha a

r(t) computer

ud (k) hold

u(t) process

y(t)

yd (k) sample

Figure 1.1. Computer control architecture yd (2) yd (3) yd (1) Ts y(t) ud (2) ud (3) ud (1) Ts u(t)

... Ts 2Ts 3Ts Ts 2Ts 3Ts Figure 1.2. Sampling (left) and holding (right)

...

The identication techniques that we will study allow us to estimate the transfer function of this discrete-time system. We can then use standard techniques from digital control to (approximately) recover the underlying continuous-time transfer function. This is discussed in Section 1.4 below.

1.2

Continuous-time systems
u(t) Hc (s) =? Figure 1.3. Continuous-time system y(t)

Consider the continuous-time system in Figure 1.3 and assume that the input and output signals satisfy the following dierential equation: y (n) + n1 y (n1) + + 2 y + 1 y + 0 y
Denition. The (unilateral) Laplace transform of a signal x(t) is given by Z X(s) := est x(t)dt.
0

= m u(m) + m1 u(m1) + + 2 u + 1 u + 0 u.

(1.1)

We recall that, given a signal x(t) with Laplace transform X(s), the Laplace transform of

System Identication the th derivative of x(t) is given by s X(s) s1 x(0) s2 x(0) x(1) (0).

In particular, when x and all its derivatives are zero at time t = 0, the Laplace transform of the th derivative x() (t) simplies to s X(s). Taking Laplace transforms to both sides of (1.1) and assuming that both y and u are zero at time zero (as well as all their derivatives), we obtain sn Y (s) + n1 sn1 Y (s) + + 2 s2 Y (s) + 1 sY (s) + 0 Y (s) This leads to Y (s) = Hc (s)U (s), where Hc (s) := m sm + m1 sm1 + + 1 s + 0 , sn + n1 sn1 + + 1 s + 0 (1.2)

= m sm U (s) + m1 sm1 U (s) + + 2 s2 U (s) + 1 sU (s) + 0 U (s).

MATLAB R Hint 1. tf(num,den) creates a continuous-time transfer function with numerator and denominator specied by num, den. . . (p. 11) MATLAB R Hint 2. zpk(z,p,k) creates a continuous-time transfer function with zeros, poles, and gain specied by z, p, k. . . (p. 12) MATLAB R Hint 3. pole(sys) and zero(sys) compute the poles and zeros of a system, respectively. Sidebar 1. Since there are other notions of stability, one should use the term BIBO stable to clarify that it refers to the property that Bounded-Inputs lead to Bounded-Outputs. Sidebar 2. The angular frequency is measured in radian per second, and is related to the (regular) frequency f by the formula = 2f , where f is given in Hertz or cycles per second.

is called the (continuous-time) transfer function of the system. The roots of the denominator are called the poles of the system and the roots of the numerator are called the zeros of the system. The system is (BIBO) stable if all poles have negative real part. In this case, for every input bounded signal u(t), the output y(t) remains bounded.

1.2.1

Steady-state response to sine waves

Suppose that we apply a sinusoidal input with amplitude A and angular frequency , given by u(t) = A cos(t), t 0,

to the system (1.1) with transfer function (1.2). Assuming that the system is BIBO stable, the corresponding output is of the form y(t) = gA cos(t + ) + (t) ,
steady-state transient

t 0,

(1.3)

where the gain g and the phase are given by the following formulas g := |Hc (j)|, The Bode plot of a transfer function Hc (s) depicts := Hc (j), (1.4)

and (t) is a transient signal that decays to zero at t .

MATLAB R Hint 4. bode(sys) draws the Bode plot of the system sys. . . (p. 12)

Joo P. Hespanha a 1. the norm of Hc (j) in decibels [dB], which is given by 20 log10 Hc (j) ; and 2. the phase of Hc (j)

both as a function of the angular frequency (in a logarithmic scale). In view of (1.3)(1.4), the Bode plot provides information about how much a sinusoidal signal is amplied and by how much its phase is changed.

1.3

Discrete-time systems
Hc (s)

u(k) = uc (kT )

uc (t)

process

yc (t)

y(k) = yc (kT )

Hd (z) Figure 1.4. Discrete-time system

Sidebar 3. Since the output cannot depend on future inputs, we must have n m for (1.5) to be physically realizable.

Consider now the discrete-time system in Figure 1.3 and assume that the output can be computed recursively by the following dierence equation: y(k + n) = n1 y(k + n 1) 2 y(k + 2) 1 y(k + 1) 0 y(k) To obtain an equation that mimics (1.1), we can re-write (1.5) as y(k + n) + n1 y(k + n 1) + + 2 y(k + 2) + 1 y(k + 1) + 0 y(k) = m u(k + m) + m1 u(k + m 1) + + 2 u(k + 2) + 1 u(k + 1) + 0 u(k).

+ m u(k + m) + m1 u(k + m 1) + + 2 u(k + 2) + 1 u(k + 1) + 0 u(k).

(1.5)

(1.6)

Denition. The (unilateral) z-transform of a signal x(k) is given by X(z) :=

X

We recall that, given a signal x(k) with z-transform X(z), the z-transform of x(k + ) is given by z X(z) z x(0) z 1 x(1) zx( 1). In particular, when x is equal to zero before time k = , the z-transform of x(k +) simplies to z X(z). Taking z-transforms to both sides of (1.6) and assuming that both y and u are zero before time n m, we obtain

z k x(k).

k=0

System Identication z n Y (z) + n1 z n1 Y (z) + + 2 z 2 Y (z) + 1 zY (z) + 0 Y (z) This leads to Y (z) = Hd (z)U (z), where Hd (z) := m z m + m1 z m1 + + 1 z + 0 , z n + n1 z n1 + + 1 z + 0

= m z m U (z) + m1 z m1 U (z) + + 2 z 2 U (z) + 1 zU (z) + 0 U (z).

MATLAB R Hint 5. tf(num,den,Ts) creates a discrete-time transfer function with sampling time Ts and numerator and denominator specied by num, den. . . (p. 11)

(1.7)

is called the (discrete-time) transfer function of the system. The roots of the denominator are called the poles of the system and the roots of the numerator are called the zeros of the system. The system is (BIBO) stable if all poles have magnitude strictly smaller than one. In this case, for every input bounded signal u(k), the output y(k) remains bounded.

MATLAB R Hint 6. zpk(z,p,k,Ts) creates a transfer function with sampling time Ts zeros, poles, and gain specied by z, p, k. . . (p. 12) Sidebar 4. Since there are other notions of stability, one should use the term BIBO stable to clarify that it refers to the property that Bounded-Inputs lead to Bounded-Outputs. Sidebar 5. Discrete-time angular frequencies only take values from 0 to , since for = , we already get the fastest discrete-time signal: u(k) = A cos(k) = A(1)k .

1.3.1

Steady-state response to sine waves

Suppose that we apply a sinusoidal input with amplitude A and discrete-time angular frequency [0, ], given by u(k) = A cos(k), k {0, 1, 2 . . . }, to the system (1.1) with transfer function (1.2). Assuming that the system is BIBO stable, the corresponding output is of the form y(k) = gA cos(k + ) + (k) ,
steady-state transient

k {0, 1, 2 . . . },

(1.8)

where the gain g and phase are given by g := |Hd (ej )|, The Bode plot of a transfer function Hd (z) depicts 1. the norm of Hd (ej ) in decibels [dB], which is given by 20 log10 Hd (ej ) ; and 2. the phase of Hd (ej ) as a function of the angular frequency (in a logarithmic scale). In view of (1.8)(1.9), the Bode plot provides information about how much a sinusoidal signal is amplied (magnitude) and by how much its phase is changed. Exercise 1. Suppose that you sample at 1KHz a unit-amplitude continuous-time sinusoid uc (t) with frequency 10Hz. Write the expression for the corresponding discrete-time sinusoid ud (k). := Hd (ej ), (1.9)

and (t) is a transient signal that decays to zero at k .

Attention! For discrete-time systems the argument to H is ej and not just j, as in continuous-time systems. This means that H(z) will be evaluated over the unit circle, instead of the imaginary axis. MATLAB R Hint 7. bode(sys) draws the Bode plot of the system sys. . . (p. 12)

10

Joo P. Hespanha a

1.3.2

Step and impulse responses

Suppose that we apply a discrete-time impulse, given by u(k) = (k) := 1 k=0 0 k=0

to the system (1.1) with transfer function (1.2). The corresponding output h(k) is called the impulse response and its z-transform is precisely the systems transfer function: Hd (z) =
k=0

z k h(k).

Consider now an input equal to a discrete-time step with magnitude , i.e., u(k) = s(k) := 0 k<0 k 0,

and let y(k) be the corresponding output, which is called the step response. Since the the impulse (k) can be obtained from the step s(k) by (k) = s(k) s(k 1) , k {0, 1, 2 . . . },

the impulse response h(k) (which is the output to (k)) can be obtained from the output y(k) to s(k) by h(k) = y(k) y(k 1) , k {0, 1, 2 . . . }.

This is a consequence of linearity.

1.4

Discrete vs. continuous-time transfer functions

When the sampling time Ts is small, one can easily go from continuous- to discrete-time transfer functions shown in Figure 1.5. To understand how this can be done, consider a continuous-time integrator y=u with transfer function Hc (s) = Y (s) 1 = . U (s) s (1.11) (1.10)

Suppose that the signals y and u are sampled every Ts time units as shown in Figure 1.2 and that we dene the discrete-time signals yd (k) = y(kTs ), ud (k) = u(kTs ), k {1, 2, 3, . . . }.

System Identication
Hc (s)

11

ud (k)

u(t)

process

y(t) S

yd (k)

Hd (z) Figure 1.5. Discrete vs. continuous-time transfer functions

Assuming that u(t) is approximately constant over an interval of length Ts , we can take a nite dierences approximation to the derivative in (1.10), which leads to y(t + Ts ) y(t) = u(t). Ts In particular, for t = kTs we obtain yd (k + 1) yd (k) = ud (k). Ts Taking the z-transform we conclude that zYd (z) Yd (z) = Ud (z) Ts Hd (z) = Yd (z) 1 = z1 . Ud (z) Ts (1.13) (1.12)
Sidebar 6. In discrete-time, an integrator has a pole at z = 1, whereas in continuous-time the pole is at s = 0.

Comparing (1.11) with (1.13), we observe that we can go directly from a continuous-time transfer function to the discrete-time one using the so called Euler transformations: s z1 Ts z 1 + sTs .

MATLAB R Hint 8. c2d and d2c convert transfer functions from continuousto discrete-time and vice-versa. . . (p. 13) Sidebar 7. Why?. . . (p. 13)

A somewhat better approximation is obtained by taking the right-hand-side of (1.12) to be the average of ud (k) and ud (k + 1). This leads to the so called Tustin or bilinear transformation: s 2(z 1) Ts (z + 1) z 2 + sTs . 2 sTs

1.5

MATLAB hints

MATLAB R Hint 1, 5 (tf). The command sys tf=tf(num,den) assigns to sys tf a MATLAB continuous-time transfer function. num is a vector with the coecients of the numerator of the systems transfer function and den a vector with the coecients of the 2s denominator. The last coecient must always be the zero-order one. E.g., to get s2 +3 you should use num=[2 0];den=[1 0 3];

12

Joo P. Hespanha a

The command sys tf=tf(num,den,Ts) assigns to sys tf a MATLAB discrete-time transfer function, with sampling time equal to Ts. For transfer matrices, num and den are cell arrays. Type help tf for examples. Optionally, one can specify the names of the inputs, outputs, and state to be used in subsequent plots as follows: sys tf=tf(num,den,... InputName,{input1,input2,...},... OutputName,{output1,output2,...},... StateName,{input1,input2,...}) The number of elements in the bracketed lists must match the number of inputs,outputs, and state variables. MATLAB R Hint 2, 6 (zpk). The command sys tf=zpk(z,p,k) assigns to sys tf a MATLAB continuous-time transfer function. z is a vector with the zeros of the sys2s tem, p a vector with its poles, and k the gain. E.g., to get (s+1)(s+3) you should use z=0;p=[1,3];k=2; The command sys tf=zpk(z,p,k,Ts) assigns to sys tf a MATLAB discrete-time transfer function, with sampling time equal to Ts. For transfer matrices, z and p are cell arrays and k a regular array. Type help zpk for examples. Optionally, one can specify the names of the inputs, outputs, and state to be used in subsequent plots as follows: sys tf=zpk(z,p,k,... InputName,{input1,input2,...},... OutputName,{output1,output2,...},... StateName,{input1,input2,...}) The number of elements in the bracketed lists must match the number of inputs,outputs, and state variables. MATLAB R Hint 4, 7 (bode). The command bode(sys) draws the Bode plot of the system sys. To specify the system one can use: 1. sys=tf(num,den), where num is a vector with the coecients of the numerator of the systems transfer function and den a vector with the coecients of the denominator. 2s The last coecient must always be the zero-order one. E.g., to get s2 +3 one should use num=[2 0];den=[1 0 3]; 2. sys=zpk(z,p,k), where z is a vector with the zeros of the system, p a vector with its 2s poles, and k the gain. E.g., to get (s+1)(s+3) one should use z=0;p=[1,3];k=2; 3. sys=ss(A,B,C,D), where A,B,C,D are a realization of the system.

System Identication

13

MATLAB R Hint 8 (c2d and d2c). The command c2d(sysc,Ts,tustin) converts the continuous-time LTI model sysc to a discrete-time model with sampling time Ts using the Tustin transformation. To specify the continuous-time system sysc one can use: 1. sysc=tf(num,den), where num is a vector with the coecients of the numerator of the systems transfer function and den a vector with the coecients of the denominator. 2s The last coecient must always be the zero-order one. E.g., to get s2 +3 you should use num=[2 0];den=[1 0 3]; 2. sysc=zpk(z,p,k), where z is a vector with the zeros of the system, p a vector with 2s its poles, and k the gain. E.g., to get (s+1)(s+3) you should use z=0;p=[1,3];k=2; 3. sysc=ss(A,B,C,D), where A,B,C,D are a realization of the system. The command d2c(sysd,tustin) converts the discrete-time LTI model sysd to a continuoustime model, using the Tustin transformation. To specify the discrete-time system sysd one can use: 1. sysd=tf(num,den,Ts), where Ts is the sampling time, num is a vector with the coefcients of the numerator of the systems transfer function and den a vector with the coecients of the denominator. The last coecient must always be the zero-order 2s one. E.g., to get s2 +3 you should use num=[2 0];den=[1 0 3]; 2. sysd=zpk(z,p,k,Ts), where Ts is the sampling time, z is a vector with the zeros of 2s the system, p a vector with its poles, and k the gain. E.g., to get (s+1)(s+3) you should use z=0;p=[1,3];k=2; 3. sysd=ss(A,B,C,D,Ts), where Ts is the sampling time, A,B,C,D are a realization of the system.

1.6

To probe further

Sidebar 7 (Tustin Transformation). Consider a continuous-time integrator in (1.10) and assume that u(t) is approximately linear on the interval [kTs , (k + 1)Ts ], i.e., u(t) = Then yd (k + 1) = yd (k) + t kTs (k + 1)Ts t ud (k) + ud (k + 1) dt Ts Ts Ts Ts = yd (k) + ud (k) + ud (k + 1). 2 2
kTs (k+1)Ts

(k + 1)Ts t t kTs ud (k) + ud (k + 1),t [kTs , (k + 1)Ts ]. Ts Ts

Taking the z-transform we conclude that zYd (z) Yd (z) Ud (z) + zUd (z) = Ts 2 Hd (z) = Yd (z) Ts (1 + z) = . Ud (z) 2(z 1)

14

Joo P. Hespanha a

Comparing this with (1.11), we observe that we can go directly from a continuous-time transfer function to the discrete-time one using the so called Tustin or bilinear transformation: s 2(z 1) Ts (z + 1) z 2 + sTs . 2 sTs

Lecture #2

Parametric identication using least-squares

Summary
This lecture presents the basic tools needed for parametric identication using the method of least-squares: 1. Formulation of the parametric transfer function identication problem 2. Line tting using the least-squares method 3. Surface tting using the least-squares method Contents 2.1 2.2 2.3 2.4 2.5 2.6 Parametric identication Least-squares line tting Vector least-squares . . MATLAB hints . . . . . To probe further . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 16 18 20 20 22

2.1

Parametric identication

In parametric identication one attempts to determine a nite number of unknown parameters that characterize the model of the system. The following is a typical problem in this class: Problem 1 (Parametric transfer function identication). Determine the coecients i and i of the systems rational transfer function H(z) = m z m + m1 z m1 + + 1 z + 0 . z n + n1 z n1 + + 1 z + 0 15

16 The number of poles n and the number of zeros m are assumed known.

Joo P. Hespanha a

This can be done using the methods of least-squares, which we introduce next, starting from a simple line tting problem and eventually building up to Problem 1 in Lecture 3. The methods of least-squares is generally attributed to Gauss [3] but the American mathematician Adrain [1] seems to have arrived at the same results independently [4]. A detailed treatment of least-squares estimation can be found, e.g., in [5, Chapter 7].

2.2

Least-squares line tting

Suppose that we have reasons to believe that two physical scalar variables x and y are approximately related by a linear equation of the form y = ax + b, (2.1)

but we do not know the value of the constants a and b. The variables y and x could be, e.g., a voltage and a current in a simple electric circuit, or the height of a uid in a tank and the pressure at the bottom of the tank (cf. Figure 2.1).
x = current

y = voltage

R y = pressure

x = height

(a) Voltage vs. current

(b) Pressure vs. height

Figure 2.1. Linear models

Numerical values for a and b can be determined by conducting several experiments and measuring the values obtained for x and y. We will denote by (xi , yi ) the measurements obtained on the ith experiment of a total of N experiments. As illustrated in Figure 2.2, it
y y = ax + b y y = ax + b

(xi , yi ) x
(a) Measurement noise

(xi , yi ) x
(b) Nonlinearity

Figure 2.2. Main causes for discrepancy between experimental data and linear model

is unlikely that we have exactly yi = axi + b, i {1, 2, . . . , N }, (2.2)

System Identication

17

because of measurement errors and also because often the model (2.1) is only approximate. Least-squares identication attempts to nd values for a and b for which the left- and the right-hand-sides of (2.2) dier by the smallest possible error. More precisely, the values of a and b leading to the smallest possible sum of squares for the errors over the N experiments. This motivates the following problem: Problem 2 (Basic Least-Squares). Find values for a and b that minimize the following Sum-of-Squares Error (SSE):
N

Sidebar 8. Statistical interpretation of least-squares. . . (p. 20)

SSE :=
i=1

(axi + b yi )2 .

for the unknowns a and b. This is a simple task because it amounts to solving a system of linear equations: 2
N N N

Solution to Problem 2. This problem can be solved using standard calculus. All we need to do is solve N SSE 2xi (axi + b yi ) = 0 = a i=1 N SSE 2(axi + b yi ) = 0 = b
i=1

x2 a + 2 i
i=1 N i=1

xi b = 2
i=1 N

xi yi

In the above expressions, we used a and to denote the solution to the least-squares mini b mization. These are called the least-squares estimates of a and b, respectively. Attention! The above estimates for a and b are not valid when N(
i

xi a + 2N b = 2
i=1 i=1

yi

a = N ( i xi yi ) ( i xi )( i yi ) N ( i x2 ) ( i xi )2 i ( i x2 )( i yi ) ( i xi yi )( i b = N ( i x2 ) ( i xi )2 i

xi )

x2 ) ( i

xi )2 = 0,
i

(2.3)

because this would lead to a division by zero. It turns out that (2.3) only holds when the xi obtained in all experiments are exactly the same, as shown in Figure 2.3. Such set of measurements does not provide enough information to estimate the parameter a and b that characterize the line dened by (2.1).

Sidebar 9. Why?. . . (p. 21)

18
y

Joo P. Hespanha a

x Figure 2.3. Singularity in least-squares line tting due to poor data.

2.3

Vector least-squares

Notation 1. a b denotes the inner product of a and b. . . . Sidebar 10. The line tting problem is a special case of this one for z = [x 1] and = [a b]. Attention! We are now using (i) to denote the ith experiment since the subscript in zj is needed to denotes the jth entry of the vector z.

The least-square problem dened in Section 2.2 can be extended to a more general linear model, where y is a scalar, z := z1 z2 zm an m-vector, and these variables are related by a linear equation of the form
m

y=
j=1

zj j = z ,

(2.4)

where := 1 to determine.

m is an m-vector with parameters whose values we would like

To determine the parameter vector , we conduct N experiments from which we obtain measurements z(i), y(i) , i {1, 2, . . . , N }, where each z(i) denotes one m-vector and each y(i) a scalar. The least-squares problem is now formulated as follows: Problem 3 (Vector Least-Squares). Find values for := 1 the following Sum-of-Squares Error (SSE):
N

m that minimize

SSE :=
i=1

z(i) y(i) .

Solution to Problem 3. This problem can also be solved using standard calculus. Essentially we have to solve SSE SSE SSE = = = = 0, 1 2 m
Notation 2. x f (x) denotes the gradient of f (x). . . Sidebar 11. Vector notation is convenient both for algebraic manipulations and for ecient MATLAB computations.

for the unknown i . The equation above can also be re-written as SSE =
SSE 1 SSE 2

SSE m

= 0.

However, to simplify the computation it is convenient to write the SSE in vector notation. Dening E to be an N -vector obtained by stacking all the errors z(i) y(i) on top of each other, i.e., z(1) y(1) z(2) y(2) E := , . . . z(N ) y(N )
N 1

System Identication we can write SSE = E

2

19
Notation 3. Given a matrix M , we denote the transpose of M by M .

= E E.

Moreover, by viewing as a column vector, we can write E = Z Y, where Z denotes a N m matrix obtained by stacking all the row vectors z(i) on top of each other, and Y an m-vector obtained by stacking all the y(i) on top of each other, i.e., z(1) y(1) z(2) y(2) Z= . , Y = . . . . . . z(N )
N m

y(N )

N 1

Therefore SSE = ( Z Y )(Z Y ) = Z Z 2Y Z + Y Y. (2.5)

Sidebar 12. The gradient of a quadratic function f (x) = x Qx + cx + d, is given by x f (x) = x (Q + Q ) + c. . . (p. 22) MATLAB R Hint 9. Z\Y computes directly (Z Z)1 Z Y in a very ecient way, which avoids inverting Z Z by solving directly the linear system of equations in (2.6). This is desirable, because this operation is often ill-conditioned.

It is now straightforward to compute the gradient of the SSE and nd the solution to SSE = 0: SSE = 2 Z Z 2Y Z = 0 = Y Z(Z Z)1 , (2.6)

which yields the following least-squares estimate for : = (Z Z)1 Z Y. Since Z Z =

i

(2.7)

z(i) z(i) and Z Y = R :=

i N

z(i) y(i), we can re-write the above formula as

N

= R1 f,

z(i) z(i),
i=1

f :=
i=1

z(i) y(i).

One can gauge the quality of the t by computing the SSE achieved by the estimate (i.e., the minimum achievable squared-error). Replacing the estimate (2.7) in (2.5) we obtain Z Y SSE = Y 2 Y 2
2

=1

Y Z(Z Z)1 Z Y Y Z =1 . Y Y Y Y

(2.8)

When this error-to-signal ratio is much smaller than one, the mismatch between the leftand right-hand-sides of (2.4) has been made signicantly smaller than the output. Attention! The above estimate for is not valid when the mm matrix R is not invertible. Singularity of R is a situation similar to (2.3) in line tting and it means that the z(i) are not suciently rich to estimate . In practice, even when R is invertible, poor estimates are obtained if R is close to a singular.
MATLAB R Hint 10. svd(A) provides a measure of how close A is to being singular. . . (p. 20)

20

Joo P. Hespanha a

2.4

MATLAB hints

MATLAB R Hint 10 (svd). The command svd(A) computes the singular values of the matrix A. The singular values of a matrix A are the square-roots of the eigenvalues of A A. Singular values are always real and nonnegative. 1. The largest singular value (written max [A]) provides a measure of how much a matrix can amplify a vector. In particular, Ax max [A] x . 2. The largest singular value (written min [A]) provides a measure of how little a matrix can amplify a vector. In particular, Ax min [A] x . For singular matrices min [A] = 0. 3. The ration between the largest and smallest singular values is called the condition number of a matrix: [A] := max [A] min [A]

provides a measure of how close a matrix is to being singular. It is equal to for singular matrices. Inverting matrices with large condition numbers is very dicult and often results in large numerical errors due to round-o.

2.5

To probe further
am and a =

Notation 1 (Inner product). Given two m-vectors, a = a1 a2 a1 a2 am , a b denotes the inner product of a and b, i.e.,
m

ab=

ai bi .
i=1

Notation 2 (Gradient). Given a scalar function of n variables f (x1 , . . . , xm ), x f denotes the gradient of f , i.e., x f (x1 , x2 , . . . , xm ) =
f x1 f x2

f xm

Sidebar 8 (Statistical interpretation of least-squares). Suppose that the mismatch between left- and the right-hand sides of (2.2) is due to uncorrelated zero-mean noise. In particular, that yi = axi + b + ni , i {1, 2, . . . , n},

System Identication

21

where all the ni are uncorrelated zero-mean random variables with the same variance 2 , i.e., E[ni ] = 0, E[n2 ] = 2 , i E[ni nj ] = 0,i, j = i.

The Gauss-Markov Theorem stated below justies the wide use of least-squares estimation. Theorem 1 (Gauss-Markov). The best linear unbiased estimator (BLUE) for the parameters a and b is the least-squares estimator. Some clarication is needed to understand this theorem 1. The Gauss-Markov Theorem 1 compares all linear estimators, i.e., all estimators of the form a = 1 y1 + + n yn , = 1 y1 + + n yn , b

where the i , i are coecients that may depend on the xi . 2. It then asks what is the choice for the i , i that lead to an estimator that is best in the sense that it is (i) unbiased, i.e., for which E[] = a, a E[ = b, b]

and (ii) results in the smallest possible variance for the estimation error, i.e., that minimizes E[( a)2 + ( b)2 ]. a b The conclusion is that the least-squares estimator satises these requirements. Unbiasedness, means that when we repeat the identication procedure many time, although we may never get a = a and = b, the estimates obtained will be clustered around the b true values. Minimum variance, means that the clusters so obtained will be as narrow as possible. Sidebar 9 (Singularity of line t). The estimates for a and b are not valid when N(
i

x2 ) ( i

xi )2 = 0.
i

To understand when this can happen let us compute N

i

(xi )2 = N

x2 2N i

xi + n2 2
i

but N =

xi , so N
i

(xi )2 = N

x2 ( i

xi )2 .
i

This means that we run into trouble when

i

(xi )2 = 0,

which can only occur when all the xi are the same and equal to .

22

Joo P. Hespanha a

Sidebar 12 (Gradient of a quadratic function). Given a m m matrix Q and a m-row vector c, the gradient of the quadratic function
m m m

f (x) = x Qx + cx =
i=1 j=1

qij xi xj +
i=1

ci xi .

To determine the gradient of f , we need to compute: f (x) = xk

x m m

qij xi
i=1 j=1

xj xi xi qij xj ci + + . xk i=1 j=1 xk i=1 xk

xi j However, xk and xk are zero whenever i = k and j = k, respectively, and 1 otherwise. Therefore, the summations above can be simplied to

f (x) = xk Therefore x f (x) = =

qik xi +
i=1 j=1

qkj xj + ck =
i=1

(qik + qki )xi + ck .

= x (Q + Q ) + c.

f (x) f (x) f (x) x1 x2 xm m i=1 (qi1 + q1i )xi + c1

m i=1 (qim

+ qmi )xi + cm

2.6

Exercises

Exercise 2 (Electrical circuit). Consider the electrical circuit in Figure 2.4, where the voltage U across the source and the resistor R are unknown. To determine the values of
R
+

Ii ri

Vi

B Figure 2.4. Electrical circuit

U and R you place several test resistors ri between the terminals A, B and measure the voltages Vi and currents Ii across them. 1. Write a MATLAB script to compute the voltages Vi and currents Ii that would be obtained when U = 10V , R = 1k, and the ri are equally spaced in the interval [100, 10k]. Add to the correct voltage and current values measurement noise.

System Identication

23

Use for the currents and for the voltages Gaussian distributed noise with zero mean and standard deviation .1mA and 10mV , respectively. The script should take as input the number N of test resistors. 2. Use least-squares to determine the values of R and U from the measurements for N {5, 10, 50, 100, 1000}. Repeat each procedure 5 times and plot the average error that you obtained in your estimates versus N . Use a logarithmic scale. What do you conclude? Exercise 3 (Nonlinear spring). Consider a nonlinear spring with restoring force F = (x + x3 ), where x denotes the spring displacement. Use least-squares to determine linear models for the spring of the form F = ax + b. Compute values for the parameters a and b for 1. forces evenly distributed in the interval [.1, .1], 2. forces evenly distributed in the interval [.5, .5], and 3. forces evenly distributed in the interval [1, 1]. For each case 1. calculate the SSE, 2. plot the actual and estimated forces vs. x, and 3. plot the estimation error vs. x.

24

Joo P. Hespanha a

Lecture #3

Parametric identication of an ARX model

Summary
This lecture explains how the methods of least-squares can be used to identify ARX models. 1. Auto-Regression model with eXogeneous inputs (ARX) 2. Identication of an ARX model 3. Partial identication (known parameters) Contents 3.1 3.2 3.3 3.4 3.5 ARX Model . . . . . . . Identication of an ARX Known parameters . . . MATLAB hints . . . . . Exercises . . . . . . . . . . . . model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 26 28 28 30

3.1

ARX Model

Suppose that we want to determine the transfer function H(z) of the discrete-time system in Figure 3.1. In least-squares identication, one converts the problem of estimating H(z) into the vector least-squares problem considered in Section 2.3. This is done using the ARX model that will be constructed below. The z-transforms of the input and output of the system in Figure 3.1 are related by m z m + m1 z m1 + + 1 z + 0 Y (z) = H(z) = , U (z) z n + n1 z n1 + + 1 z + 0 25

26
u(k) H(z) =? y(k)

Joo P. Hespanha a

Figure 3.1. System identication from input/output experimental data

where the i and the i denote the coecients of the numerator and denominator polynomials of H(z). Multiplying the numerator and denominator of H(z) by z n , we obtain the transfer function expressed in negative powers of z: m z n+m + m1 z n+m1 + + 1 z n+1 + 0 z n Y (z) = U (z) 1 + n1 z 1 + + 1 z n+1 + 0 z n and therefore Y (z) + n1 z 1 Y (z) + + 1 z n+1 Y (z) + 0 z n Y (z) = Taking inverse z-transforms we obtain y(k) + n1 y(k 1) + + 1 y(k n + 1) + 0 y(k n) = This can be re-written compactly as y(k) = (k) (3.2)

m z n+m U (z) + m1 z n+m1 U (z) + + 1 z n+1 U (z) + 0 z n U (z).

m u(k n + m) + m1 u(k n + m 1) + + 1 u(k n + 1) + 0 u(k n).

(3.1)

where the (n+m+1)-vector contains the coecient of the transfer function and the vector (k) the past inputs and outputs, i.e., := n1 1 0 m m1 1 0

(k) := y(k 1)

y(k n)

u(k n + m)

u(k n) .

The vector (k) is called the regression vector and the equation (3.2) is called an ARX model, a short form of Auto-Regression model with eXogeneous inputs.

3.2

Identication of an ARX model

We are now ready to solve the system identication Problem 1 introduced in Lecture 2, by applying the least-squares method to the ARX model: Solution to Problem 1. 1. Apply a probe input signal u(k), k {1, 2, . . . , N } to the system. 2. Measure the corresponding output y(k), k {1, 2, . . . , N }.

System Identication

27

3. Determine the values for the parameter that minimize the discrepancy between the left- and the right-hand-sides of (3.2) in a least-squares sense. According to Section 2.3, the least-squares estimate of is given by = ( )1 Y, where (2) := . = . .
(N )

(3.3)

MATLAB R Hint 11. PHI\Y computes directly, from the matrix PHI= and the vector Y= Y . Attention! When the system is at rest before k = 1, u(k) = y(k) = 0 k 0. Otherwise, if the past inputs are unknown, one needs to remove from and Y all rows with unknown data. MATLAB R Hint 12. arx is very convenient to perform least-squares identication of ARX models because it does not require the need to explicitly construct the matrix and the vector Y . . . (p. 28)

(1)

y(0) y(1)

y(N 1) y(N n) u(N n+m) u(N n)

. . .

y(1n) u(1n+m) u(1n) y(2n) u(2n+m) u(2n)

. . .

. . .

. . .

or equivalently

y(2) Y := . , . .
y(N ) N

y(1)

= R1 f,

R := =
k=1

(k) (k),

f := Y =
k=1

(k) y(k).

The quality of the t can be assessed by computing the error-to-signal ratio SSE Y =1 . 2 Y Y Y When this quantity is much smaller than one, the mismatch between the left- and righthand-sides of (3.2) has been made signicantly smaller than the output. Attention! One may combine several input/output pairs of signals to identify one ARX model, by stacking all the data available in one matrix and one vector Y . Suppose, for example, that one wants to identify an ARX model with two poles and two zeros, based on an input/output pair u1 (k), y1 (k) : k = 1, 2, . . . , N1 of length N1 and another input/output pair u2 (k), y2 (k) : k = 1, 2, . . . , N2 of length N2 . One would construct y1 (2) y1 (1) u1 (3) y1 (3) y1 (2) u1 (4) . . . . . . . . . y1 (N1 1) y1 (N1 2) u1 (N1 ) = y2 (2) y2 (1) u2 (3) y2 (3) y2 (2) u2 (4) . . . . . . . . . y2 (N2 1) y2 (N2 2)

MATLAB R Hint 13. compare is useful to validate the quality of the t. . . (p. 30) MATLAB R Hint 12. merge allows one to combine data for this type of multi-input processing. . . (p. 28)

u1 (2) u1 (3) . . . u1 (N1 1) u2 (2) u2 (3) . . . u2 (N2 1)

u2 (N2 )

u1 (N1 2) , u2 (1) u2 (2) . . . u2 (N2 2)

u1 (1) u1 (2) . . .

y1 (N1 ) Y = y2 (3) . y2 (4) . . . y2 (N1 )

y1 (3) y1 (4) . . .

28

Joo P. Hespanha a

Attention! Two common causes for errors in least-squares identications of ARX models are: 1. incorrect construction of the matrix and/or vector Y ; 2. incorrect construction of the identied transfer function from the entries in the least squares estimate . Both errors can be avoided using the MATLAB command arx.

3.3

Known parameters

Due to physical considerations, one often knows one or more poles/zeros of the process. For example: 1. one may know that the process has an integrator, which corresponds to a continuoustime pole at s = 0 and consequently to a discrete-time pole at z = 1; or 2. that the process has a dierentiator, which corresponds to a continuous-time zero at s = 0 and consequently to a discrete-time zero at z = 1. In this case, it suces to identify the remaining poles/zeros, which can be done as follows: Suppose that it is known that the transfer function H(z) has a pole at z = , i.e., H(z) =
Sidebar 13. The transfer function H(z) is proper only if H(z) is strictly proper. However, even if H(z) happens not to be proper, this introduces no diculties for this method of system identication.

1 H(z), z

where H(z) corresponds to the unknown portion of the transfer function. In this case, the z-transforms of the input and output of the system are related by Y (z) 1 H(z), = U (z) z and therefore Y (z) = H(z), U (z) where Y (z) := (z )Y (z) y (k) = y(k + 1) y(k).

Sidebar 14. The new output y (k) is not a causal function of the original output y(k), but this is of no consequence since we have the whole y(k) available when we carry out identication.

This means that we can directly estimate H(z) by computing y (k) prior to identication and then regarding this variable as the output, instead of y(k).

3.4

MATLAB hints

MATLAB R Hint 12 (arx). The command arx from the identication toolbox performs least-squares identication of ARX models. To use this command one must

System Identication 1. Create a data object that encapsulates the input/output data using: data=iddata(y,u,Ts);

29

where u and y are vectors with the input and output data, respectively, and Ts is the sampling interval. Data from multiple experiments can be combined using the command merge as follows: dat1=iddata(y1,u1,Ts); dat2=iddata(y1,u1,Ts); data=merge(dat1,dat2); 2. Compute the estimated model using: model=arx(data,[na,nb,nk]) where data is the object with input/output data; na, nb, nk are integers that dene the degrees of the numerator and denominator of the transfer function (see below); and model a MATLAB object with the coecients of the transfer function. The coecient are returned for negative powers of z. In particular, the estimated transfer function is given by z nk model.a(1) + model.a(2)z 1 + + model.a(na + 1)z na Y (z) = . U (z) model.b(1) + model.b(2)z 1 + + model.b(nb + 1)z nb The estimated discrete-time transfer function can be recovered using the MATLAB command sysd = tf(model, Measured ) The object model contains a lot more information about the estimated transfer function, which includes 1. model.da and model.db give the standard deviations associated with the estimates of the coecient. A large value for one of the standard deviations indicates that the matrix R := is close to singular and points to little condence on the identied model. 2. model.noise provides a measure of the quality of t and is directly related to the SSE. However, this measure is not normalized, opposite to (2.8). You can type idprops idpoly at the MATLAB prompt to obtain more information about other information that is available in model. The MATLAB command arx uses a more sophisticated algorithm than the one described in these notes so the results obtained using arx may not match exactly the ones obtained using the formula (3.3).

Sidebar 15. MATLAB is an object oriented language and model.a is the property a of the object model. Sidebar 16. Typically, one chooses nb equal to the (expected) system dimension, nk equal to the (expected) input delay, and na = nb nk. This corresponds to y(k) being a function of y(k 1), . . . , y(k nb) and of u(k nk), . . . , u(k nb).

30

Joo P. Hespanha a

MATLAB R Hint 13 (compare). The command compare from the identication toolbox allows one to compare experimental outputs with simulated values from an estimated model. This is useful to validate model identication. The command compare(data,model) plots the measured output from the input/output data object data and the predictions obtained from the estimated model model. See MATLAB hints 12, 19 for details on these objects.

3.5

Exercises

Exercise 4 (Known zero). Suppose that the process is known to have a zero at z = , i.e., that H(z) = (z )H(z), where H(z) corresponds to the unknown portion of the transfer function. How would you estimate H(z)? What if you known that the process has both a zero at and a pole at ? Exercise 5 (Selected parameters). The data provided was obtained from a system with transfer function H(z) = z .5 , (z .3)(z p)

where p is unknown. Use the least-squares method to determine the value of p.

Lecture #4

Practical considerations in parametric identication

Summary
This lecture discusses several practical considerations that are crucial for successful identications. 1. Signal scaling 2. Choice of inputs 3. Choice of model order 4. Choice of sampling frequency Contents 4.1 4.2 4.3 4.4 4.5 Choice of inputs . . . . . . . Signal scaling . . . . . . . . . Choice of sampling frequency Choice of model order . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 33 33 34 35

4.1

Choice of inputs

The quality of least-squares estimates depends signicantly on the input u(k) used. The choice of inputs should take into account the following requirements: 1. The input should be suciently rich so that the matrix R is nonsingular (and is well conditioned). 31

32
Sidebar 17. Why? 2 H1 (z) = 2z+1 and H2 (z) =
4 2+24z 5+2 2

Joo P. Hespanha a (a) Single sinusoids should be avoided because a sinusoid of frequency will not allow us to distinguish between dierent transfer functions with exactly the same value at the point z = ej . (b) Good inputs include: square-waves, the sum of many sinusoids, or a chirp signal (i.e., a signal with time-varying frequency). 2. The amplitude of the resulting output y(k) should be much larger than the measurement noise. In particular, large inputs are needed when one wants to probe the system at frequencies for which the noise is large. However, when the process is nonlinear but is being approximated by a linear model, large inputs may be problematic because the linearization may not be valid. As shown in Figure 4.1, there is usually an optimal input level that needs to be determined by trial-and-error.
SSE y 2

have the

same value for z = ej/4 . Therefore they will lead to the same y(k) when u(k) is a sinusoid with frequency /4. This input will not allow us to distinguish between H1 and H2 .

noise dominates optimal ]

nonlinear behavior

input magnitude

Figure 4.1. Optimal choice of input magnitude

3. The input should be representative of the class of inputs that are expected to appear in the feedback loop. (a) The input should have strong components in the range of frequencies at which the system will operate. (b) The magnitude of the input should be representative of the magnitudes that are expected in the closed loop. Validation The choice of input is perhaps the most critical aspect in good system identication. After any identication procedure the following steps should be followed to make sure that the data collected is adequate: 1. Check if the matrix R is far from singular. If not a dierent richer input should be used. 2. Check the quality of t by computing following is probably occurring:
SSE Y 2.

If this quantity is not small, one of the

(a) There is too much noise and the input magnitude needs to be increased. (b) The inputs are outside the range for which the process is approximately linear and the input magnitude needs to be decreased.

System Identication

33

(c) The assumed degrees for the numerator and denominator are incorrect (see Section 4.4). 3. After an input signal u(k) passed the checks above, repeat the identication experiment with the input u(k) with = 1, = .5. If the process is in the linear region, the measured outputs should roughly by equal to y(k) and the two additional experiments should approximately result in the same transfer function. When one obtains larger gains in the experiment with = .5, this typically means that the process is saturating and one needs to decrease the magnitude of the input.

4.2

Signal scaling
Attention! After identication, one must then adjust the system gain to reect the scaling performed on the signals used for identication.

For the computation of the least-squares estimate to be numerically well conditioned, it is important that both inputs and outputs have roughly the same order of magnitude. Because the units of these variable are generally quite dierent, this often requires scaling of these variables. It is good practice to scale both inputs and outputs so that all variable take normal values in the same range, e.g., the interval [1, 1].

4.3

Choice of sampling frequency

For a discrete-time model to accurately capture the process continuous-time dynamics, the sampling frequency should be as large as possible. However, this typically leads to diculties in system identication due to a couple of factors: 1. If the poles of the continuous-time system are equal to s1 , s2 , . . . , sn , then the poles of the discrete-time system obtained using the Tustin transformation are given by z1 := 2 + s2 Ts 2 + sn Ts 2 + s1 Ts , z2 := , . . . , zn := . 2 s1 Ts 2 s2 Ts 2 sn Ts

When Ts is much smaller than the poles, all the discrete-time poles zi become very close to one regardless of the continuous-time dynamics. The same is true for the zeros. Therefore, when on identies a discrete-time system using a very high sampling frequency, it is common to obtain discrete-time transfer functions with several poles and zeros close to one (these poles correspond to the actual poles of the continuous-time process, warped by the Tustin transformation) as well as several other poles and zeros away from one that arise due to high-frequency noise. Generally these need to be discarded. 2. When the sampling frequency is very large, the values of y(k) and y(k + 1) are very close to each other and the dierence between them is very much aected by the

34

Joo P. Hespanha a quantization interval introduced by the analogue to digital conversion. This leads to a very poor predictive power by ARX models that try to predict the value of y(k) based on the last few past values of the input and output. With larger sampling times, the time-horizon over which an ARX model attempts to make predictions is larger and consequently the outputs exhibit larger variations, making them less sensitive to quantization noise.

In practice, one wants to chose a sampling rate that is large enough so that the Tustin transformation is valid but suciently small so as to avoid the problems discussed above. A good rule of thumb is to sample the system at a frequency 20-40 times larger than the desired closed-loop bandwidth for the system. One should look at the range of values taken by y(k) over the horizon of the ARX model to make sure that quantization eect are not too noticeable. Attention! It sometimes happens that the hardware used to collect data allow us to sample the system at frequencies much larger than what is needed for identicationoversampling. In this case, one generally needs to down-sample the signals but this actually provides an opportunity to remove measurement noise from the signals. Suppose that y(k) and u(k) have been sampled with a period Tlow but one wants to obtain signals y (k) and u(k) sampled with period Thigh := L Tlow where L is some integer larger than one. A simple method to obtain y (k) and u(k) consists of extracting only one out of each L samples of the original signals: y (k) = y(Lk), u(k) = u(Lk). (4.1)

MATLAB R Hint 15. A signal y can by down-sampled as in (4.1) using bary=y(1:L:end). Down-sampling can also be achieved with the command bary=downsample(y,L). This command requires MATLABs signal processing toolbox. MATLAB Hint 16. A signal y can by down-sampled as in (4.2) using bary=(y(1:L:end-2) + y(2:L:end-1) + y(3:L:end))/3. MATLAB R Hint 17. Down-sampling with more sophisticated (and longer) ltering can be achieved with the command resample, e.g., bary=resample(y,1,L,1). This command requires MATLABs signal processing toolbox.
R

However, instead of discarding all the remaining samples, one can achieve some noise reduction by averaging. E.g., y(Lk 1) + y(Lk) + y(Lk + 1) , 3 u(Lk 1) + u(Lk) + u(Lk + 1) u(k) = , 3 y (k) = (4.2) (4.3)

or even longer averaging. The down-sampled signals obtained in this way exhibit lower noise than the original ones.

4.4

Choice of model order

A signicant diculty in parametric identication of ARX models is that to construct the regression vector (k), one needs to know the degree n of the denominator. In fact, an incorrect choice for n will generally lead to diculties. 1. Selecting a value for n too small will lead to mismatch between the measured data and the model and SSE will be large. Y 2

System Identication

35

2. Selecting a value for n too large is called over-parameterization and it generally leads to R being close to singular. To understand why, suppose we have a transfer function H(z) = 1 , z1

but for estimation purposes we assumed that n = 2 and therefore attempted to determine constants i , i such that H(z) = 2 z 2 + 1 z + 0 . z 2 + 1 z + 0

If the model was perfect, it should be possible to match the data with any i , i such that zp 2 z 2 + 1 z + 0 = z 2 + 1 z + 0 (z 1)(z p) 2 = 0, 1 = 1, 0 = p, 1 = 1 p, 0 = p,

where p can be any number. This means that the data is not sucient to determine the values of the parameters 0 , 0 , 1 , which translates into R being singular. When there is noise, it will never be possible to perfectly explain the data and the smallest SSE will always be positive (either with n = 1 or n = 2). However, in general, dierent values of p will result in dierent values for SSE. In this case, least-squares estimation will produce the specic value of p that is better at explaining the noise, which is not physically meaningful. When one is uncertain about which values to choose for m and n, the following procedure should be followed 1. Select for n the smallest possible integer that is physically meaningful (conceivably 0). 2. Do least-square estimation with m = n. (a) If the SSE is large and R is far from singular increase n and repeat the identication. (b) If R is close to singular or the SSE did not decrease signicantly, then n got too large, go back to the previous n and stop. One needs to exercise judgment in deciding when the SSE is large or when R is far/close to singular. Physical knowledge about the model should play a major role in deciding model orders. Moreover, one should always be very concerned about identifying noise, as opposed to actually identifying the process model.
Sidebar 18. When n is chosen correctly, m too large will generally not lead to trouble so one may as well chose the largest one that is physically meaningful. If it is known that there is a delay of at least d, one should make m = n d [cf. equation (3.1)].

4.5

Exercises

Exercise 6 (Input magnitude). A Simulink block that models a nonlinear spring-massdamper system is provided.

36

Joo P. Hespanha a 1. Use the Simulink block to generate the systems response to step inputs with amplitude 0.25 and 1.0 and no measurement noise. 2. For each set of data, use the least-squares method to estimate the systems transfer function. Try a few values for the degrees of the numerator and denominator polynomials m and n. Check the quality of the model by following the validation procedure outlines above. Important: write MATLAB scripts to automate these procedures. These scripts should take as inputs the simulation data u(k), y(k), and the integers m, n. 3. Use the Simulink block to generate the systems response to step inputs and measurement noise with intensity 0.01. For the best values of m and n determined above, plot the SSE vs. the step size. Which step-size leads to the best model?

Exercise 7 (Model order). Use the data provided to identify the transfer function of the system. Use the procedure outlined above to determine the order of the numerator and denominator polynomials. Plot the largest and smallest singular value of R and the SSE as a function of n. Important: write MATLAB scripts to automate this procedure. These scripts should take as inputs the simulation data u(k), y(k), and the integers m, n. Exercise 8 (Sampling frequency). Consider a continuous-time system with transfer function P (s) = s2 4 2 . + s + 4 2

1. Build a Simulink model for this system and collect input/output data for an input square wave with frequency .25Hz and unit amplitude for two sampling frequencies Ts = .25sec and Ts = .0005sec. 2. Identify the systems transfer function without down-sampling. 3. Identify the systems transfer function using the data collected with Ts = .0005sec but down-sampled. Important: write MATLAB scripts to automate this procedure. These scripts should take as inputs the simulation data u(k), y(k), the integers m, n, and the down-sampling period L. 4. Briey comment on the results.

Lecture #5

Nonparametric identication
Summary
This lecture presents basic techniques for nonparametric identication. 1. Time-domain identication 2. Frequency-domain identication Contents 5.1 5.2 5.3 5.4 5.5 5.6 Nonparametric methods . . . . . Time-domain identication . . . Frequency response identication MATLAB hints . . . . . . . . . . To probe further . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 38 40 42 42 43

5.1

Nonparametric methods

Nonparametric identication attempts to directly determine the model of a system, without assuming that its transfer function is rational and that we known the number of poles or zeros. The following are typical problems in this class: Problem 4 (Nonparametric frequency response identication). Determine the frequency response H(ej ) over a range of frequencies [min , max ]. or Problem 5 (Nonparametric time response identication). Determine the impulse response h(k) from time k = 0 to k = N . 37

38

Joo P. Hespanha a

MATLAB R Hint 18. fft(h) computes the fast-Fourier-transform (FFT) of the signal h(k), k {1, 2, . . . , N }, which provides the values of H(ej ) for equally spaced frequencies = 2k , N k {0, 1, . . . , N 1}.

Problem 4 is useful for controller design based on loop-shaping or the Nyquist criterion, whereas Problem 5 is useful for controller design based on the Ziegler-Nichols rules to tune PID controllers [2]. For N large, one can also recover the transfer function from the impulse response by taking the z-transform: H(z) = Z[h(k)] =
k=0 N

h(k)z k

h(k)z k ,
k=0

assuming that N is large enough so that h(k) 0 for k > N . Throughout this chapter we will assume that the system is BIBO stable, i.e., that all poles of the transfer function H(z) have magnitude smaller than one or equivalently that h(k) converges to zero exponentially fast. A detailed treatment of this subject can be found, e.g., in [5, Chapter 7].

5.2

Time-domain identication

We start by discussing a few methods used to solve the time response identication Problem 5.

5.2.1
Sidebar 19. The main weakness of the impulse-response method is that impulses are rarely representative of typical inputs that appear in closed loop so, especially for nonlinear systems, we estimate a regimen that may be far from the dynamics that will appear in closed loop.

Impulse response method

The impulse response of a system can be determines directly by starting with the system at rest and applying an impulse at the input: u(k) = 0 k=0 k = 0.

The output will be a (scaled) version of the impulse response, possibly corrupted by some measurement noise n(k): y(k) = h(k) + n(k). Therefore h(k) = y(k) n(k)

Hopefully the noise term n(k)/ is small when compared to h(k) and one can use the following estimate for the impulse response: y(k) h(k) = , k {1, 2, . . . , N }.

Attention! The choice of is generally critical to obtain a good estimate for the impulse response:

System Identication

39

(i) || should be large to make sure that n(k)/ is indeed negligible when compared to h(k); (ii) || should be small to make sure that the process does not leave the region where a linear model is valid and where it is safe to operate it open-loop. As one increases , a simple practical test to check if one is leaving the linear region of operation is to do identication both with some > 0 and < 0. In the linear region of operation, the identied impulse-response does not change. See also the discussion in Section 4.1.

5.2.2

Step response

The impulse response of a system can also be determined by starting with the system at rest and applying an step at the input: u(k) = 0 k0 k < 0.

Sidebar 20. In general, steps are more representative than pulses in feedback loops so, in that respect, the step response is a better method than the impulse response.

The z-transform of the output will be given by Y (z) = H(z)U (z) + N (z) = H(z) + N (z), 1 z 1

where N (z) denotes the z-transform of measurement noise. Solving for H(z) we obtain: H(z) = Y (z) z 1 Y (z) N (z) z 1 N (z)

Taking inverse z-transforms, we conclude that h(k) = y(k) y(k 1) n(k) n(k 1) .

Hopefully the noise term n(k)n(k1) is small when compared to h(k), and we can use following estimate for the impulse response: y(k) y(k 1) h(k) = , k {1, 2, . . . , N }.

Attention! The main weakness of the step-response method is that it can amplify noise because in the worst case n(k)n(k1) can be twice as large as n(k) (when n(k 1) = n(k)). Although this may seem unlikely, it is not unlikely to have all the n(k) independent and identically distributed with zero mean and standard deviation . In this case, StdDev n(k) = , StdDev 1.41 n(k) n(k 1) = ,

Sidebar 21. Why?. . . (p. 42)

which means that the noise is amplied by approximately 41%.

40

Joo P. Hespanha a

5.2.3

Other inputs

One can determine the impulse response of a system using any input and not just a pulseor step-input. Take an arbitrary input u(k), with z-transform U (z). The z-transform of the output will be given by Y (z) = H(z)U (z) + N (z), where N (z) denotes the z-transform of measurement noise. Solving for H(z) we obtain: H(z) = U 1 (z) Y (z) N (z) Taking inverse z-transforms, we conclude that h(k) = Z 1 [U 1 (z)] y(k) n(k) ,
MATLAB R Hint 19. The identication toolbox command impulse performs impulse response estimation for arbitrary inputs. . . (p. 42)

where the denotes the convolution operation. Hopefully the noise term is small when compared to h(k), and we can use following estimate for the impulse response: h(k) = Z 1 [U 1 (z)] y(k), k {1, 2, . . . , N }.

5.3

Frequency response identication

We now consider a couple of methods used to solve the frequency response identication Problem 4.

5.3.1

Sine-wave testing
u(k) = cos(k), k {1, 2, . . . , N }. (5.1)

Suppose one applies an sinusoidal input of the form

Assuming that H(z) has no poles at z = ej , the measured output is given by y(k) = A cos k + + (k) + n(k), where 1. A := |H(ej | and := H(ej ) are the magnitude and phase of the transfer function H(z) at z = ej ; 2. (k) is a transient signal that converges to zero as fast as c k , where is the magnitude of the pole of H(z) with largest magnitude and c some constant; and
Sidebar 22. To minimize the errors caused by noise, the amplitude should be large. However, it should still be suciently small so that the process does not leave the linear regime.

(5.2)

3. n(k) corresponds to measurement noise. When n(k) is much smaller than A and for k suciently large so that (k) is negligible, we have y(k) A cos k + . This allows us to recover both A := |H(ej | and := H(ej ) from the magnitude and phase of y(k). Repeating this experiment for several inputs with dierent frequencies , one can estimate the frequency response H(ej ) over the range of frequencies of interest.

System Identication

41

5.3.2

Correlation method

Especially when there is noise, it may be dicult to determine the amplitude and phase of y(k) by inspection. The correlation method aims at accomplishing this task. Suppose that the input u(k) in (5.1) was applied and that after some time N0 , we can neglect the transient term (k) from the output equation (5.2). In the correlation method we compute
N N

K :=
k=1

ejk y(k) =
k=1 N

cos(k) j sin(k) y(k)

N

= A
k=1

cos(k) cos(k + ) j sin(k) cos(k + ) +

1 2 (cos(a

ejk (k) + n(k) .

k=1

Using the trigonometric formulas cos a cos b = 1 2 (sin(a b) + sin(a + b)), we conclude that K = A 2 +
k=1 N

b) + cos(a + b)) and sin a cos b =

k=1

cos + cos 2k + + j sin j sin 2k +

ejk (k) + n(k)

N

A 2

ej +
k=1

A 2

e2jkj +
k=1 N k=1 N

ejk (k) + n(k) ejk (k) + n(k) .

N A ej + A ej 2 2

e2jk +
k=1 k=1

For large N , the rst term dominates over the remaining ones as long as n(k) and e(k) do not contain pure sinusoidal terms of frequency . Therefore, K N N A ej = H(ej ). 2 2

Sidebar 23. Why?. . . (p. 42)

This leads to the following estimate for H(ej ): H(ej ) = 2K . N

Attention! The main weaknesses of the frequency response methods are: (i) To estimate H(ej ) at a single frequency, one still needs a long input (i.e., N large). In practice, this leads to a long experimentation period to obtain the Bode plot over a wide range of frequencies. (ii) It requires that we apply very specic inputs (sinusoids) to the process. This may not be possible (or safe) in certain applications. Especially for processes that are open-loop unstable.

42

Joo P. Hespanha a

5.4

MATLAB hints

MATLAB R Hint 19 (impulse). The command impulse from the identication toolbox performs impulse response estimation from data collected with arbitrary inputs. To use this command one must 1. Create a data object that encapsulates the input/output data using: data=iddata(y,u), where u and y are vectors with the input and output data. 2. Plot the estimated impulse response using: impulse(dat), or compute the response using model=impulse(data), where m is a MATLAB object with an impulse-response model. The impulse response is given by model.b.

5.5

To probe further

Sidebar 21 (Noise in step-response method). When n(k) and n(k 1) are independent, we have Var[n(k) n(k 1)] = Var[n(k)] + Var[n(k 1)]. Therefore StdDev n(k) n(k 1) = = Var n(k) n(k 1) Var[n(k)] + Var[n(k 1)] 2

When, both variables have the same standard deviation , we obtain 2 2 n(k) n(k 1) 2 = sqrt 2 = . StdDev Sidebar 23 (Correlation method). In the correlation method one computes K = N H(ej ) + A ej 2 2
N N

e2jk +
k=1 k=1

ejk (k) + n(k) .

System Identication The rst summation is given by

N

43

e2jk =
k=1

1 e2jN 1 e2j

and is therefore bounded as N (as long as = 0). As N , the second summation converges to
k=1

ejk (k) + n(k) = E(ej ) + N (ej ),

where E(z) and N (z) denote the z-transforms of e(k) and n(k), respectively. Therefore this term is also bounded as long as n(k) and e(k) do not contain pure sinusoidal terms of frequency . We therefore conclude that the term N H(ej ) dominates as N . 2

5.6

Exercises

Exercise 9 (Step response). A Simulink block that models a pendulum with viscous friction is provided. 1. Use the Simulink block (without noise) to estimate the systems impulse response h(k) using both the impulse and step response methods. What is the largest value of the input magnitude for which the system still remains within the approximately linear region of operation? Hint: to make sure that the estimate is good you can compare the impulses response from steps and impulses. 2. Turn on the noise in the Simulink block and repeat the identication procedure above for a range of values for . For both methods, plot the error k h (k) h(k) vs. , (k) denotes the estimate obtained for an input with magnitude . What is where h your conclusion? Take the estimate h(k) determined in 1 for the noise-free case as the ground truth h(k). Exercise 10 (Correlation method). Modify the Simulink block provided for Exercise 9 to accept a sinusoidal input. 1. Estimate the systems frequency response H(ej ) at a representative set of (logspaced) frequencies i using the correlation method. Select an input amplitude for which the system remains within the approximately linear region of operation. 2. Turn on the noise in the Simulink block and repeat the identication procedure above ji ) H(eji ) vs. , where for a range of values for . Plot the error i H (e

44

Joo P. Hespanha a H (eji ) denotes the estimate obtained for an input with magnitude . What is your conclusion? Take the estimate H(ej ) determined in 1 for the noise-free case as the ground truth H(ej ). 3. Compare your best frequency response estimate H(ej ), with the frequency responses that you would obtain by taking the Fourier transform of the best impulse response h(k) that you obtained in Exercise 9. Hint: Use the MATLAB command fft to compute the Fourier transform. This command gives you H(ej ) from = 0 to = 2 so you should discard the second half of the Fourier transform (corresponding to > ).

Robust Control

45

Introduction to Robust Control

For controller design purposes it is convenient to imagine that we know an accurate model for the process, e.g., its transfer function. In practice, this is hardly ever the case: 1. When process models are derived from rst principles, they always exhibit constants that can only be determined up to some error. E.g., the precise values of masses, moments of inertia, and friction coecients in models derived from Newtons laws; or resistances, capacitances, and gains, in electrical circuits. 2. When one identies a model experimentally, noise and disturbances generally lead to dierent results as the identication experiment is repeated multiple times. Which experiment gave the true model? The short answer is none, all models obtained have some error. 3. Processes change due to wear and tear so even if a process was perfectly identied before starting operation, its model will soon exhibit some mismatch with respect to the real process. The goal of this chapter is to learn how to take process model uncertainty into account, while designing a feedback controller. Pre-requisites 1. Laplace transform, continuous-time transfer functions, frequency responses, and stability. 2. Classical continuous-time feedback control design using loop-shaping. 3. Knowledge of MATLAB/Simulink.

47

48

Joo P. Hespanha a

Lecture #6

Robust stability
Summary
This lecture introduces the basic concepts of robust control 1. Characterizations of unstructured uncertainty (additive and multiplicative) 2. Nyquist stability criterion 3. Small-gain stability condition Contents 6.1 6.2 6.3 6.4 6.5 Model uncertainty . . . . Nyquist stability criterion Small gain condition . . . MATLAB hints . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 52 54 56 57

6.1

Model uncertainty
111111111111 000000000000 111111111111 000000000000 111111111111 000000000000
k m 1 0 1 0 u 1 0 b

Suppose we want to control the spring-mass-damper system in Figure 6.1, which has the

Measuring the mass vertical position y with respect to the rest position of the spring, we obtain from Newtons law: m = by ky + u y

1 0

111 000 1 0 y 1 0 1 0

Figure 6.1. Spring-mass-damper system.

49

50

Joo P. Hespanha a

following transfer function from the applied force u to the spring position y P (s) = 1 . ms2 + bs + k (6.1)

Typically, the mass m, the friction coecient b, and the spring constant k would be measured experimentally (or taken from some specications sheet) leading to condence intervals for these parameters and not just a single value: m [m0 1 , m0 + 1 ], b [b0 2 , b0 + 2 ], k [k0 3 , k0 + 3 ].

The values with subscript 0 are called the nominal values for the parameters and the i are called the maximum deviations from the nominal value.
MATLAB R Hint 20. bode(sys) draws the Bode plot of the system sys. . . (p. 56)

In practice, this means that there are many admissible transfer functions for the processone for each possible combination of m, b, and k in the given intervals. Figure 6.2 shows the Bode plot of (6.1) for dierent values of the parameters m, b, and k. The idea
Bode Diagram 20 10 0 Magnitude (dB) Phase (deg) 10 20 30 40 50 60 0

45

90

135

180 1 10

10 Frequency (rad/sec)

10

1 Figure 6.2. Bode plot of P (s) = ms2 +bs+k , for dierent values of m [.9, 1.1], b [.1, .2], and k [2, 3]. Note how the dierent values of the parameters lead to dierent values for the resonance frequency. One can see in the phase plot that it may be dangerous to have the cross-over frequency near the uncertain resonance frequency since the phase margin may very a lot depending on the system parameters.

behind robust control is to design a single controllers that achieves acceptable performance (or at least stability) for every admissible process transfer function.

6.1.1

Additive uncertainty

In robust control one starts by characterizing the uncertainty in terms of the process frequency response. To this eect on selects a nominal process transfer function P0 (s) and, for each admissible process transfer function P (s), one denes: a (s) := P (s) P0 (s), (6.2)

Robust Control

51

which measures how much P (s) deviates from P0 (s). This allows us to express any admissible transfer function P (s) as in Figure 6.3. Motivated by the diagram in Figure 6.3, a (s) is called an additive uncertainty block. P0 (s) should correspond to the most likely transfer
a (s) u P0 (s) P (s) a (s) := P (s) P0 (s)
+ +

y P (s) = P0 (s) + a (s)

Figure 6.3. Additive uncertainty

function, so that the additive uncertainty block is as small as possible. In the example above, one would typically choose P0 (s) = 1 . m0 s2 + b0 s + k0

To obtain a characterization of the uncertainty purely in the frequency domain, one needs to specify how large a (j) may be for each frequency . This is done by determining a function a () suciently large so that for every admissible process transfer function P (s) we have |a (j)| = |P (j) P0 (j)| a (), .

When one has available all the admissible P (s) (or a representative set of themsuch as in Figure 6.2), one can determine a () by simply plotting |P (j) P0 (j)| vs. for all the P (s) and choosing for a () a function larger than all the plots. Since in general it is not feasible to plot all |P (j) P0 (j)|, one should provide some safety-cushion when selecting a (). Figure 6.4 shows a () for the Bode plots in Figure 6.2.
20

20

Frequency (rad/sec)

40

60

80

100 1 10

10 Magnitude (dB)

10

Figure 6.4. Additive uncertainty bounds for the process Bode plots in Figure 6.2 with P0 (s) := 1 , m0 = 1, b0 = 1.5, k0 = 2.5. The solid lines represent possible plots for |P (j)P0 (j)| m0 s2 +b0 s+k0 and the dashed one represents the uncertainty bound a (). Note the large uncertainty bound near the resonance frequency. We shall show shortly how to design a controller that stabilizes all processes for which the additive uncertainty falls below the dashed line.

52

Joo P. Hespanha a

6.1.2

Multiplicative uncertainty

The additive uncertainty in (6.2) measures the dierence between P (s) and P0 (s) in absolute terms and may seem misleadingly large when both P (s) and P0 (s) are largee.g., at low frequencies when the process has a pole at the origin. To overcome this diculty one often denes instead m (s) := P (s) P0 (s) , P0 (s) (6.3)

which measures how much P (s) deviates from P0 (s), relative to the size of Po (s). We can now express any admissible transfer function P (s) as in Figure 6.5, which motivates calling m (s) a multiplicative uncertainty block. To characterize a multiplicative uncertainty one
P (s)
+ +

m (s) u

m (s) := y

P (s) P0 (s) P0 (s)

P0 (s)

` P (s) = P0 (s) 1 + m (s)

Figure 6.5. Multiplicative uncertainty

determines a function m () suciently large so that for every admissible process transfer function P (s) we have |m (j)| = |P (s) P0 (s)| m (), |P0 (s)| .

One can determine m () by plotting |P (s)P0 (s)| vs. for all admissible P (s) (or a repre|P0 (s)| sentative set of them) and choosing m () to be larger than all the plots. Figure 6.6 shows m () for the Bode plots in Figure 6.2.

6.2

Nyquist stability criterion

The rst question we address is: Given a specic feedback controller C(s), how can we verify that it stabilizes every admissible process P (s). When the admissible processes are described in terms of a multiplicative uncertainty block, this amounts to verifying that the closed-loop system in Figure 6.7 is stable for every m (j) with norm smaller than m (). This can be done using the Nyquist stability criterion, which we review next.
Sidebar 24. Figure 6.8 can represent the closed-loop system in Figure 6.7 if we choose G(s):= ` 1 + m (s) P (s)C(s). MATLAB R Hint 21. nyquist(sys) draws the Nyquist plot of the system sys. . .

The Nyquist criterion is used to investigate the stability of the negative feedback connection in Figure 6.8. We briey summarize it here. The textbook [2, Section 6.3] provides a more detailed description of it with several examples.

6.2.1

The Nyquist Plot

The rst step consists of drawing the Nyquist plot, which is done by evaluating G(j) from

Robust Control
20

53

20

Frequency (rad/sec)

40

60

80

100 1 10

10 Magnitude (dB)

10

Figure 6.6. Multiplicative uncertainty bounds for the process Bode plots in Figure 6.2 with 1 P0 (s) := m0 s2 +b0 s+k0 , m0 = 1, b0 = 1.5, k0 = 2.5. The solid lines represent possible plots for and the dashed one represents the uncertainty bound m (). Note the large uncertainty bound near the resonance frequency. We shall show shortly how to design a controller that stabilizes all processes for which the multiplicative uncertainty falls below the dashed line. m (s) r
+
|P (j)P0 (j)| |P0 (j)|

|m (j)| m (),
+

u C(s)
+

P0 (s)

Figure 6.7. Unity feedback conguration with multiplicative uncertainty

= to = + and plotting it in the complex plane. This leads to a closed-curve that is always symmetric with respect to the real axis. This curve should be annotated with arrows indicating the direction corresponding to increasing . Attention! Any poles of G(s) on the imaginary axis should be moved slightly to the left of the axis to avoid divisions by zero. E.g., G(s) = s+1 s(s 3) s s G(s) = 2 = s +4 (s + 2j)(s 2j) s+1 (s + )(s 3) s s G (s) = , (s + + 2j)(s + 2j) (s + )2 + 4 G (s)

for a small > 0. The criterion should then be applied to the perturbed transfer function G (s). If we conclude that the closed-loop is stable for G (s) with very small , then the closed-loop with G(s) will also be stable and vice-versa.

6.2.2

The Nyquist criterion

Count the number #ENC of clockwise encirclements by the Nyquist plot of the point 1. To do this, we draw a ray from 1 to in any direction and add one each time the Nyquist plot crosses the ray in the clockwise direction (with respect to the origin of the ray) and

54
r
+

Joo P. Hespanha a
y G(s)

Figure 6.8. Negative feedback

subtract one each time it crosses the ray in the counter-clockwise direction. The nal count gives #ENC. Nyquist Stability Criterion. The total number of unstable (i.e., in the right-hand-side plane) closed-loop poles (#CUP) is given by #CUP = #ENC + #OUP, where #OUP denotes the number of unstable (open-loop) poles of G(s). To have a stable closed-loop one thus needs #ENC = #OUP. Figure 6.9 shows the Nyquist plot of G0 (s) = C(s)P0 (s), for the nominal process model P0 (s) := 1 , m0 s2 + b0 s + k0 m0 = 1, b0 = 1.5, k0 = 2.5, (6.5) (6.4)

(used in Figures 6.4 and 6.6) and a PID controller C(s) := 10 + 15 + 5s. s (6.6)

Since the open loop system has no unstable poles (#OUP = 0) and there are no encirclements of 1 (#ENC = 0), we conclude that the closed-loop system is stable. This means that the given PID controller C(s) stabilizes, at least, the nominal process P0 (s). It remains to check if it also stabilizes every admissible process model with multiplicative uncertainty.

6.3

Small gain condition

Consider the closed-loop system in Figure 6.7 and suppose that we are given a controller C(s) that stabilizes the nominal process P0 (s), i.e., the closed-loop is stable when m (s) = 0. Our goal is to nd out if the closed-loop remains stable for every m (j) with norm smaller than m (). Since C(s) stabilizes P0 (s), we know that the Nyquist plot of the nominal (open-loop) transfer function G0 (s) = C(s)P0 (s),

Robust Control
Nyquist Diagram 200

55
Nyquist plot for 1st controller 20

150

15

100

10

Imaginary Axis

Imaginary Axis

50

50

100

10

150

15

200 50

50

100

150 200 Real Axis

250

300

350

400

20 20

15

10

5 Real Axis

10

15

20

25

30

Figure 6.9. Nyquist plot for the (open-loop) transfer function in (6.4). The right gure shows a zoomed view of the origin. To avoid a pole over the imaginary axis, in these plots we moved the pole of the controller from 0 to .01.

has the right number of encirclements (#ENC = #OUP, where #OUP is the number of unstable poles of G0 (s). To check is the closed-loop is stable from some admissible process transfer function P (s) = P0 (s) 1 + m (s) , we need to draw the Nyquist plot of G(s) = C(s)P (s) = C(s)P0 (s) 1 + m (s) = G0 (s) + G0 (s)m (s) and verify that we still get the same number of encirclements.
|1 + G0 (j)| 1 G(j) G0 (j) |G(j) G0 (j)|
Sidebar 25. We are assuming that G(s) and G0 (s) have the same number of unstable poles #OUP and therefore stability is achieved for the same number of encirclements #ENC = #OUP. In practice this means that the uncertainty should not change the stability of any pole.

Figure 6.10. Nyquist plot derivation of the small-gain conditions

The Nyquist plots of G(s) and G0 (s) dier by |G(j) G0 (j)| = |G0 (j)m (j)| |G0 (j)| m (), A simple way to make sure that G(j) and G0 (j) have the same number of encirclements is to ask that the maximum dierence between the two be smaller than the distance from

56 G0 (j) to the point 1, i.e., |1 + G0 (j)| > |G0 (j)| m ()

Joo P. Hespanha a

|G0 (j)| 1 < |1 + G0 (j)| m ()

This leads to the so called small-gain condition:

Sidebar 26. A mnemonic to remember (6.7) is that the transfer function whose norm needs to be small is precisely the transfer function seen by the m block in Figure 6.7. This mnemonic also works for additive uncertainty. . . (p. 59) MATLAB R Hint 22. To check if (6.7) holds, draw 20 log10 1 = m () 20 log10 m () on the magnitude Bode plot of the complementary sensitivity function and see if the latter always lies below the former.

Small-gain Condition. The closed-loop system in Figure 6.7 is stable for every m (j) with norm smaller than m (), provided that C(j)P0 (j) 1 < , 1 + C(j)P0 (j) m () . (6.7)

The transfer function on the left-hand-side of (6.7) is precisely the complementary sensitivity function: T0 (s) := 1 S0 (s), S0 (s) := 1 1 + C(s)P0 (s)

for the nominal process. So (6.7) can be interpreted as requiring the norm of the nominal complementary sensitivity function to be smaller than 1/m (), . For this reason (6.7) is called a small-gain condition. Figure 6.11 shows the Bode plot of the complementary sensitivity function for the nominal process in (6.5) and the PID controller in (6.6). In the same plot we can see the 20 log10 m1 = 20 log10 m (), for the m () in Figure 6.6. Since the magnitude plot of () T0 (j) is not always below that of m1 , we conclude that the system may be unstable for () some admissible processes. However, if we redesign our controller to consist of an integrator with two lags C(s) = .005(s + 5)2 , s(s + .5)2 (6.8)

Sidebar 27. Recall that the complementary sensitivity function is also the closed-loop transfer function for the reference r to the output y.

the magnitude plot of T0 (j) is now always below that of m1 and we conclude that we have () stability for every admissible process. In this case, the price to pay was a low bandwidth. This example illustrates a common problem in the control of systems with uncertainty: it is not possible to get good reference tracking over ranges of frequencies for which there is large uncertainty.

6.4

MATLAB hints

MATLAB R Hint 20 (bode). The command bode(sys) draws the Bode plot of the system sys. To specify the system one can use: 1. sys=tf(num,den), where num is a vector with the coecients of the numerator of the systems transfer function and den a vector with the coecients of the denominator. 2s The last coecient must always be the zero-order one. E.g., to get s2 +3 one should use num=[2 0];den=[1 0 3];

Robust Control
Bode Diagram 10 5 Magnitude (dB) 0 5 10 15 20 25 30 0

57

Phase (deg)

45

90 1 10

10

10 Frequency (rad/sec)

10

Figure 6.11. Verication of the small gain condition for the nominal process (6.5), multiplicative uncertainty in Figure 6.6, and the PID controller (6.6). The solid line corresponds to the Bode plot of the complementary sensitivity function T0 (s) and the dashed line to m1 (both in dB). ()

2. sys=zpk(z,p,k), where z is a vector with the zeros of the system, p a vector with its 2s poles, and k the gain. E.g., to get (s+1)(s+3) one should use z=0;p=[1,3];k=2; 3. sys=ss(A,B,C,D), where A,B,C,D are a realization of the system. MATLAB R Hint 21 (nyquist). The command nyquist(sys) draws the Nyquist plot of the system sys. To specify the system you can use any of the commands in Matlab hint 20. Especially when there are poles very close to the imaginary axis (e.g., because they were actually on the axis and you moved them slightly to the left), the automatic scale may not be very good because it may be hard to distinguish the point 1 from the origin. In this case, you can use then zoom features of MATLAB to see what is going on near 1: Try clicking on the magnifying glass and selecting a region of interest; or try left-clicking on the mouse and selecting zoom on (-1,0) (without the magnifying glass selected.)

6.5

Exercises

Exercise 11 (Unknown parameters). Suppose one want to control the orientation of a satellite with respect to its orbital plane by applying a thrusters generated torque. The systems transfer function is give by P (s) := s2 (s2 10(bs + k) , + 11(bs + k))

where the values of the parameters b and k are not exactly known, but it is known that .09 k .4 .006 b .03.

58
Bode Diagram 40 20 0 20 40 60 0 90 Phase (deg) 180 270 360 450 2 10

Joo P. Hespanha a

Magnitude (dB)

10

10 10 Frequency (rad/sec)

10

10

Figure 6.12. Verication of the small gain condition for the nominal process (6.5), multiplicative uncertainty in Figure 6.6, and the integrator with 2 lags controller (6.8). The solid line corresponds to the Bode plot of the complementary sensitivity function T0 (s) and the dashed line to m1 (both () in dBs).

Find a nominal model and compute the corresponding additive and multiplicative uncertainty bounds. Exercise 12 (Noisy identication). The Simulink block provided corresponds to a discretetime system with transfer function H(z) = 1 z + 0 . z 2 + 1 z + 0

1. Use least-squares to estimate the 4 coecients of the transfer function. Repeat the identication experiment 25 times to obtain 25 estimates of the systems transfer functions. 2. Convert the discrete-time transfer functions so obtained to continuous-time using the Tustin transformation. Hint: Use the MATLAB command d2c. 3. Select a nominal model and compute the corresponding additive and multiplicative uncertainty bounds. Exercise 13 (Small gain). For the nominal process model and multiplicative uncertainty that you obtained in Exercise 12, use the small gain condition to verify if the following controllers achieve stability for all admissible process models: C1 (s) = .3 C2 (s) = 2 s + .1 C3 (s) = 20 s 30 C4 (s) = 5(s + 1) (s + 2)(s + 3) C5 (s) = 2(s + 1) (s + .3)2

Justify your answers with appropriate Bode plots.

Robust Control

59

Exercise 14 (Robustness vs. performance). Justify the statement: It is not possible to get good reference tracking over ranges of frequencies for which there is large uncertainty. Exercise 15 (Additive uncertainty). Derive a small-gain condition similar to (6.7) for additive uncertainty. Check if the mnemonic in Sidebar 26 still applies. Hint: With additive uncertainty, the open-loop gain is given by G(s) = C(s)P (s) = C(s) P0 (s) + a (s) = G0 (s) + C(s)a (s), which diers from G0 (s) by C(s)a (s).

60

Joo P. Hespanha a

Lecture #7

Control design by loop-shaping

Summary
The of this lecture is to show how to do control design taking into account unmodeled dynamics. Loop-shaping control design will be used for this purpose. 1. Loop-shaping control 2. Open-loop vs. closed-loop specications 3. Lead-lag compensation Contents 7.1 7.2 7.3 7.4 The loop-shaping design method . . . . Open-loop vs. closed-loop specications Open-loop gain shaping . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 62 65 66

7.1

The loop-shaping design method

d r
+ + + e

u C(s) P0 (s)

Figure 7.1. Closed-loop system

61

62

Joo P. Hespanha a

The basic idea behind control design by loop-shaping is to convert the desired specications on the closed-loop system in Figure 7.1 into constraints on the open-loop gain G0 (s) := C(s)P0 (s). The controller C(s) is then designed so that the open-loop gain G0 (s) satises the appropriate constraints. The shaping of G0 (s) to satisfy the appropriate constraints can be done using the classical methods described in [2, Chapter 6.7], such as PID, Lead, or Lag compensation. We will see later that loop-shaping can also be done using LQR/LQG.

7.2

Open-loop vs. closed-loop specications

In this section we discuss how several closed-loop specications can be converted to constraints on the open-loop gain G0 (s). Stability Assuming that the open-loop gain has no unstable poles, stability of the closedloop is guaranteed as long as the phase of the open loop gain is above 180 degrees at the cross-over frequency c , i.e., at the frequency for which |G0 (jc )| = 1. The distance between the phase of G0 (jc ) and 180 degrees is called the phase margin. Overshoot Larger phase margins generally corresponds to a smaller overshoot for the step response of the closed-loop system. The following rules of thumb work well when the open-loop gain has a pole at the original, an additional real pole, and no zeros: Phase 65 60 45 margin deg deg deg overshoot 5% 10% 15%

Reference tracking Suppose that one wants the tracking error to be at least kT 1 times smaller than the reference, over the range of frequencies [0, T ]. In the frequency domain, this can be expressed by |E(j)| kT , |R(j)| [0, T ], (7.1)

where E(s) and R(s) denote the Laplace transforms of the tracking error e := r y and the reference signal r, in the absence of noise and disturbances. For the closed-loop system in Figure 7.1, E(s) = 1 R(s), 1 + G0 (s)

Robust Control and therefore (7.1) is equivalent to 1 kT , |1 + G0 (j)| [0, T ] |1 + G0 (j)| 1 , kT [0, T ].

63

This condition is guaranteed to hold by requiring that |G0 (j)| 1 + 1, kT [0, T ]. (7.2)

Disturbance rejection Suppose that one wants measurement input disturbances to appear in the output attenuated at least kD 1 times, over the range of frequencies [0, D ]. In the frequency domain, this can be expressed by |Y (j)| kD , |D(j)| [0, D ], (7.3)

where Y (s) and D(s) denote the Laplace transforms of the output y and the input disturbance d, in the absence of reference and measurement noise. For the closed-loop system in Figure 7.1, Y (s) = and therefore (7.3) is equivalent to |P0 (j)| kD , |1 + G0 (j)| [0, D ] |1 + G0 (j)| |P0 (j)| , kD [0, D ]. P0 (s) D(s), 1 + G0 (s)

This condition is guaranteed to hold by requiring that |G0 (j)| |P0 (j)| + 1, kD [0, D ]. (7.4)

Noise rejection Suppose that one wants measurement noise to appear in the output attenuated at least kN 1 times, over the range of frequencies [N , ). In the frequency domain, this can be expressed by |Y (j)| kN , |N (j)| [N , ), (7.5)

where Y (s) and N (s) denote the Laplace transforms of the output y and the measurement noise n, in the absence of reference and disturbances. For the closed-loop system in Figure 7.1, Y (s) = G0 (s) N (s), 1 + G0 (s)

64 and therefore (7.5) is equivalent to |G0 (j)| kN , |1 + G0 (j)| 1 1 + 1, G0 (j) kN [N , ) 1+ 1 1 , G0 (j) kN

Joo P. Hespanha a

[N , ).

This condition is guaranteed to hold by requiring that [N , ) |G0 (j)| kN , 1 + kN [N , ).

Robustness with respect to multiplicative uncertainty Suppose that one wants the closed-loop to remain stable for every multiplicative uncertainty m (j) with norm smaller than m (). This can be expressed by |G0 (j)| 1 , |1 + G0 (j)| m () which is equivalent to 1+
Sidebar 28. Typically, (7.8) will hold for high-frequencies (for which |G0 (j)| is small), whereas (7.9) will hold for low-frequencies (for which |G0 (j)| is large).

(7.6)

1 m (), G0 (j)

(7.7)

This condition is guaranteed to hold if for every either one of the following conditions holds
1 1. | G0 (j) | 1 and

1 1 m () G0 (j)
1 2. | G0 (j) | 1, m () < 1 and

|G0 (j)|

1 1 + m ()

(7.8)

1 m () G0 (j)

|G0 (j)|

1 1 m ()

(7.9)

From the two conditions (7.8)(7.9), one generally needs to be mostly concerned about (7.7). This is because when m () is small, (7.7) will generally hold. However, when m () is large for (7.7) to hold we need G0 (j) to be large, which corresponds to the condition (7.8). Hopefully, m () will only be large at high frequencies, for which we do not need the (7.2) or (7.4) to hold. Table 7.1 and Figure 7.2 summarize the constraints on the open-loop gain G0 (j) discussed above. Attention! The conditions derived above for the open-loop gain G0 (j) are sucient for the original closed-loop specications to hold, but they are not necessary. When the openloop gain almost veries the conditions derived, it may be worth it to check directly if it veries the original closed-loop conditions.

Robust Control
closed-loop specication overshoot 10% ( 5%) |E(j)| kT , [0, T ] |R(j)| |Y (j)| kD , [0, D ] |D(j)| |Y (j)| kN , [N , ) |N (j)| |G0 (j)| 1 , |1 + G0 (j)| m () open-loop constraint phase margin 60 deg ( 65 deg) 1 |G0 (j)| + 1, [0, T ] kT |P0 (j)| + 1, [0, D ] |G0 (j)| kD kN |G0 (j)| , [N , ) 1 + kN 1 1 |G0 (j)| or |G0 (j)| 1 + m () 1 m ()

65

Table 7.1. Summary of the open-loop specications for the loop shaping design method

|G0 (j)|

1 + 1111111111 0000000000 k 111111111 0000000001 1111111111 0000000000 111111111 000000000 1111111111 0000000000 111111111 000000000 1111111111 0000000000 111111111 000000000 1111111111 0000000000
T

|P0 (j)| +1 kD

1 1 m () D

111111111 000000000 111111111 000000000 111111111 000000000 k

N

1 + kN

1 1 + m ()

Figure 7.2. Typical open-loop specications for the loop shaping design method

This is actually necessary for the conditions (7.8)(7.9) that arise from robustness with respect to multiplicative uncertainty, because around the crossover frequency |G0 (jc )| 1 will not satisfy either of these conditions, but it will generally satisfy the original condition (7.6) (as long as m () is not too large).

7.3

Open-loop gain shaping

C(s) = 1

In classical lead/lag compensation one starts with a basic unit-gain controller

and adds to it appropriate blocks to shape the desired open loop gain G0 (s) := C(s)P0 (s). so that it satises the appropriate open-loop constraints. This shaping can be achieved using three basic tools:

Sidebar 29. One actually does not add to the controller, to be precise, one multiplies the controller by appropriate gain, lead, and lag blocks. However, this does correspond to additions in the magnitude (in dBs) and phase Bode plots.

66

Joo P. Hespanha a 1. Proportional gain: multiplying the controller by a constant k moves the magnitude Bode plot up and down, without changing its phase. 2. Lead compensator: multiplying the controller by a lead block with transfer function Clead (s) = Ts + 1 , T s + 1 < 1,

Sidebar 30. A lead compensator also increases the cross-over frequency so it may require some trial-and-error to get the peak of the phase right at the cross-over frequency.

increases the phase margin when placed at the cross-over frequency. Figure 7.3(a) shows the Bode plot of a lead compensator. 3. Lag compensation: multiplying the controller by a lag block with transfer function Clag (s) = s/z + 1 , s/p + 1 p < z,

Sidebar 31. A lag compensator also increases the phase so it can potentially decrease the phase margin. To avoid this, one should only introduce lag compensation away from the cross-over frequency.

decreases the high-frequency gain. Figure 7.3(b) shows the Bode plot of a lag compensator.

7.4

Exercises

Exercise 16 (Loop-shape 1). Consider again the nominal process model and multiplicative uncertainty that you obtained in Exercise 12. Design a controller for this process that achieves stability for all admissible process models and that exhibits: 1. zero steady-state error to a step input, 2. Phase Margin no smaller then 60degrees, 3. steady-state error for sinusoidal inputs with frequencies < 0.1rad/sec smaller than 1/50 (34dB), and 4. Rise time no slower than .3sec.
1 1 p z p z

1 T

1 T sin max 1 = 1+

z p pz max = 1 sin max = 1 + sin max

(b) Lag

1 1 1 (a) = T max LeadTT Figure 7.3. Lead/lab compensators

Robust Control

67

Exercise 17 (Loop-shape 2). Consider the following nominal transfer function ans uncertainty bound: P0 (s) = where L(s) := 2.5 . (1 + s/20)2 1 , s(1 + s/5)(1 + s/20) m () = |L(j)|,

Use loop-shaping to design a controller that achieves stability for all admissible process models and that exhibits: 1. steady-state error to a ramp input no larger than .01, 2. Phase Margin no smaller then 45degrees, 3. steady-state error for sinusoidal inputs with frequencies < 0.2rad/sec smaller than 1/250 (50dB), and 4. attenuation of measurement noise by at least a factor of 100 (40dB) for frequencies greater than 200rad/sec.

68

Joo P. Hespanha a

LQG/LQR controller design

69

Introduction to LQG/LQR controller design

In optimal control one attempts to nd a controller that provides the best possible performance with respect to some given measure of performance. E.g., the controller that uses the least amount of control-signal energy to take the output to zero. In this case the measure of performance (also called the optimality criterion) would be the control-signal energy. In general, optimality with respect to some criterion is not the only desirable property for a controller. One would also like stability of the closed-loop system, good gain and phase margins, robustness with respect to unmodeled dynamics, etc. In this section we study controllers that are optimal with respect to energy-like criteria. These are particularly interesting because the minimization procedure automatically produces controllers that are stable and somewhat robust. In fact, the controllers obtained through this procedure are generally so good that we often use them even when we do not necessarily care about optimizing for energy. Moreover, this procedure is applicable to multiple-input/multiple-output processes for which classical designs are dicult to apply. Pre-requisites 1. Basic knowledge of state-space models (briey reviewed here) 2. Familiarity with basic vector and matrix operations. 3. Knowledge of MATLAB/Simulink.

71

72

Joo P. Hespanha a

Lecture #8

Review of State-space models

Contents 8.1 8.2 8.3 8.4 8.5 8.6 State-space models . . . . . . . . Input-output relations . . . . . . Realizations . . . . . . . . . . . . Controllability and observability Stability . . . . . . . . . . . . . . MATLAB hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 74 75 76 77 77

8.1

State-space models

Consider the system in Figure 8.1 with m inputs and k outputs. A state-space model for
u(t) Rm y(t) Rk

x = f (x, u) y = g(x, u)

Figure 8.1. System with m inputs and k outputs

this system relates the input and output of a system using the following rst-order vector ordinary dierential equation x = f (x, u), y = g(x, u). (8.1)
MATLAB R Hint 23. ss(A,B,C,D) creates a LTI state-space model with realization (8.2). . .

where x Rn is called the state of system. In this Chapter we restrict our attention to linear time-invariant (LTI) systems for which the functions f (, ) and g(, ) are linear. In this case, (8.1) has the special form x = Ax + Bu, y = Cx + Du, (8.2)

where A is a n n matrix, B a n m matrix, C a k n matrix, and D a k m matrix. 73

74

Joo P. Hespanha a

Example 1 (Aircraft roll-dynamics). Figure 8.2 shows the roll-angle dynamics of an aircraft [6, p. 381]. Dening

roll-angle

= = .875 20 = 50 + 50u

roll-rate

applied torque

Figure 8.2. Aircraft roll-angle dynamics

x := we conclude that

x = Ax + Bu with 0 1 A := 0 .875 0 0 0 20 , 50 0 B := 0 . 50

If we have both and available for control, we can dene y := with C := 1 0 0 1 0 , 0

= Cx + Du

D :=

0 . 0

8.2
Sidebar 32. See [2, Appendix A] for a review of Laplace transforms.

Input-output relations
x = Ax + Bu, y = Cx + Du, L sX(s) = AX(s) + BU (s), Y (s) = CX(s) + DU (s),

The transfer-function of this system can be found by taking Laplace transforms of (8.2):

where X(s), U (s), and Y (s) denote the Laplace transforms of x(t), u(t), and y(t). Solving for X(s), we get (sI A)X(s) = BU (s) and therefore Y (s) = C(sI A)1 BU (s) + DU (s) = C(sI A)1 B + D U (s). X(s) = (sI A)1 BU (s)

LQG/LQR controller design Dening T (s) := C(sI A)1 B + D, we conclude that Y (s) = T (s) U (s).

75
MATLAB R Hint 24. tf(num,den) creates a transfer-function with numerator and denominator specied by num, den. . .

(8.3)

To emphasize the fact that T (s) is a k m matrix, we call it the transfer-matrix of the system (8.2). The relation (8.3) between the Laplace transforms of the input and the output of the system is only valid for zero initial conditions, i.e., when x(0) = 0. The general solution to the system (8.2) in the time domain is given by
t

MATLAB R Hint 25. zpk(z,p,k) creates a transfer-function with zeros, poles, and gain specied by z, p, k. . . MATLAB R Hint 26. tf(sys ss) and zpk(sys ss) compute the transfer-function of the state-space model sys ss. . .

x(t) = eAt x(0) +

0

eA(ts) Bu(s)ds,
t

(8.4) t 0. (8.5)
MATLAB R Hint 27. expm computes the exponential of a matrix. . .

y(t) = CeAt x(0) +

0

CeA(ts) Bu(s)ds + Du(t),

Equation (8.4) is called the variation of constants formula. Example 2 (Aircraft roll-dynamics). The transfer-function for the state-space model in Example 1 is given by: T (s) =
1000 s(s+.875)(s+50) 1000 (s+.875)(s+50)

8.3

Realizations
T11 (s) T21 (s) T (s) = . . . T12 (s) T22 (s) . . . Tk2 (s) .. . T1m (s) T2m (s) . , . .

Consider a transfer-matrix

Tk1 (s)

Tkm (s)

where all the Tij (s) are given by a ratio of polynomials with the degree of the numerator smaller than or equal to the degree of the denominator. It is always possible to nd matrices A, B, C, D such that T (s) = C(sI A)1 B + D. This means that it is always possible to nd a state-space model like (8.2) whose transfermatrix is precisely T (s). The model (8.2) is called a realization of T (s).

MATLAB R Hint 28. ss(sys tf) computes a realization of the transfer-function sys tf. . .

Attention! Realizations are not unique, i.e., several state-space models may have the same transfer function.

76

Joo P. Hespanha a

8.4

Controllability and observability

The system (8.2) is said to be controllable when given any initial state xi Rn , any nal state xf Rn , and any nite time T , one can nd an input signal u(t) that takes the state of (8.2) from xi to xf in the interval of time 0 t T , i.e., when there exists an input u(t) such that
T

xf = eAT xi +
0

eA(T s) Bu(s)ds.

MATLAB R Hint 29. ctrb(sys) computes the controllability matrix of the state-space system sys. Alternatively, one can use directly ctrb(A,B). . . MATLAB R Hint 30. rank(M) computes the rank of a matrix M.

To determine if a system is controllable, one can compute the controllability matrix, which is dened by C := B AB A2 B An1 B .

The system is controllable if and only if this matrix has rank equal to the size n of the state vector. The system (8.2) is said to be observable when one can determine the initial condition x(0) by simply looking at the input and output signals u(t) and y(t) on a certain interval of time 0 t T , i.e., one can solve
t

y(t) = CeAt x(0) +

0

CeA(ts) Bu(s)ds + Du(t),

0 t T,

MATLAB R Hint 31. obsv(sys) computes the controllability matrix of the state-space system sys. Alternatively, one can use directly obsv(A,C). . .

uniquely for the unknown x(0). To determine if a system is observable, one can compute the observability matrix, which is dened by C CA 2 O := CA . . . . CAn1

The system is observable if and only if this matrix has rank equal to the size n of the state vector.

Example 3 (Aircraft roll-dynamics). The controllability and observability matrices for the state-space model in Example 1 are given by: 1 0 0 0 1 0 0 0 1000 0 1 0 . O= C = 0 1000 50875 , 20 0 .875 50 2500 125000 0 .875 20 0 .7656 1017.5 Both matrices have rank 3 so the system is both controllable and observable.

LQG/LQR controller design

77

8.5

Stability
MATLAB R Hint 32. eig(A) computes the eigenvalues of the matrix A. . .

The system (8.2) is asymptotically stable when all eigenvalues of A have negative real parts. In this case, for any bounded input u(t) the output y(t) and the state x(t) are also bounded, i.e., u(t) c1 , t 0 y(t) c2 , x(t) c3 t 0.

Moreover, if u(t) converges to zero as t , then x(t) and y(t) also converge to zero as t . Example 4 (Aircraft roll-dynamics). The eigenvalues of the matrix the A matrix for the state-space model in Example 1 are {0, .875, 50} so the system is not asymptotically stable.

8.6

MATLAB hints

MATLAB R Hint 23 (ss). The command sys ss=ss(A,B,C,D) assigns to sys ss a MATLAB LTI state-space model with realization x = Ax + Bu, y = Cx + Du.

Optionally, one can specify the names of the inputs, outputs, and state to be used in subsequent plots as follows: sys ss=ss(A,B,C,D,... InputName,{input1,input2,...},... OutputName,{output1,output2,...},... StateName,{input1,input2,...}) The number of elements in the bracketed lists must match the number of inputs,outputs, and state variables. MATLAB R Hint 24 (tf). The command sys tf=tf(num,den) assigns to sys tf a MATLAB transfer-function. num is a vector with the coecients of the numerator of the systems transfer-function and den a vector with the coecients of the denominator. The 2s last coecient must always be the zero-order one. E.g., to get s2 +3 you should use num=[2 0];den=[1 0 3]; For transfer-matrices, num and den are cell arrays. Type help tf for examples. Optionally, one can specify the names of the inputs, outputs, and state to be used in subsequent plots as follows: sys tf=tf(num,den,... InputName,{input1,input2,...},... OutputName,{output1,output2,...},... StateName,{input1,input2,...})

78

Joo P. Hespanha a

The number of elements in the bracketed lists must match the number of inputs,outputs, and state variables. MATLAB R Hint 25 (zpk). The command sys tf=zpk(z,p,k) assigns to sys tf a MATLAB transfer-function. z is a vector with the zeros of the system, p a vector with its poles, 2s and k the gain. E.g., to get (s+1)(s+3) you should use z=0;p=[1,3];k=2; For transfer matrices, z and p are cell arrays and k a regular array. Type help zpk for examples. Optionally, one can specify the names of the inputs, outputs, and state to be used in subsequent plots as follows: sys tf=zpk(z,p,k,... InputName,{input1,input2,...},... OutputName,{output1,output2,...},... StateName,{input1,input2,...}) The number of elements in the bracketed lists must match the number of inputs,outputs, and state variables. MATLAB R Hint 26 (tf). The commands tf(sys ss) and zpk(sys ss) compute the transfer-function of the state-space model sys ss specied as in Matlab Hint 23. tf(sys ss) stores (and displays) the transfer function as a ratio of polynomials on s. zpk(sys ss) stores (and displays) the polynomials factored as the product of monomials (for the real roots) and binomials (for the complex roots). This form highlights the zeros and poles of the system. MATLAB R Hint 28 (ss). The command ss(sys tf) computes the state-space model of the transfer function sys specied as in Matlab Hints 24 or 25. MATLAB R Hint 27 (expm). The command expm(M) computes the matrix exponential eM . With the symbolic toolbox, this command can be used to compute eAt symbolically as follows: syms t expm(A*t) The rst command denes t as a symbolic variable and the second computes eAt (assuming that the matrix A has been previously dened). MATLAB R Hint 29 (ctrb). The command ctrb(sys) computes the controllability matrix of the system sys. The system must be specied by a state-space model using, e.g., sys=ss(A,B,C,D), where A,B,C,D are a realization of the system. Alternatively, one can use directly ctrb(A,B). MATLAB R Hint 31 (obsv). The command obsv(sys) computes the observability matrix of the system sys. The system must be specied by a state-space model using, e.g., sys=ss(A,B,C,D), where A,B,C,D are a realization of the system. Alternatively, one can use directly obsv(A,C). MATLAB R Hint 32 (eig). The command eig(A) computes the eigenvalues of the matrix A. Alternatively, eig(sys) computes the eigenvalues of the A matrix for a state-space system sys specied by sys=ss(A,B,C,D), where A,B,C,D are a realization of the system.

Lecture #9

Linear Quadratic Regulation (LQR)

Contents 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 Feedback conguration . . . . . . Optimal Regulation . . . . . . . State-Feedback LQR . . . . . . . Stability and Robustness . . . . . Loop-shaping control using LQR MATLAB hints . . . . . . . . . . To probe further . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 80 81 82 85 87 88 89

9.1

Feedback conguration

Figure 9.1 shows the feedback conguration for the Linear quadratic regulation (LQR) problem.
u(t) Rm z(t) R y(t) Rk

controller

process

Figure 9.1. Linear quadratic regulation (LQR) feedback conguration. Note the negative feedback and the absence of a reference signal. Reference signals will be introduced in Lecture 11.

In this conguration, the state-space model of the process is of the form x = Ax + Bu, y = Cx, 79 z = Gx + Hu. (9.1)

80 and has two distinct outputs:

Joo P. Hespanha a

1. The measured output y(t) Rk corresponds to the signal(s) that can be measured and are therefore available for control. If the controller transfer-matrix is C(s), we have Y (s) = C(s)U (s), where Y (s) and U (s) denote the Laplace transforms of the process input u(t) and the measured output y(t), respectively. 2. The controlled output z(t) R corresponds to a signal that one would like to make as small as possible in the shortest possible amount of time. Sometimes z(t) = y(t), which means that our control objective is to make the whole measured output very small. However, when the output y(t) is a vector, often one simply needs to make one of the measured outputs y1 (t) small. In this case, one chooses z(t) = y1 (t). In some situations one chooses z(t) = y1 (t) , y1 (t)

which means that we want to make both the measured output y1 (t) and its derivative y1 (t) very small. Many other options are possible. The choice of z should be viewed as a design parameter. In Section 9.5 we will study the impact of this choice in the performance of the closed-loop.

9.2
Notation 4. Given an m-vector, v = v1 v2 vm ,

Optimal Regulation

The LQR problem is dened as follows: Problem 6 (Optimal LQR). Find the controller transfer-matrix C(s) that makes the following criteria as small as possible JLQR :=
0

v denotes the Euclidean norm of v, i.e., v = v v =

m X i=1 2 vi

z(t)

+ u(t) 2 dt,

(9.2)

where is a positive constant.

.

The term
0

z(t) 2 dt

corresponds to the energy of the controlled output and the term

0

u(t) 2 dt

to the energy of the control signal. In LQR one seeks a controller that minimizes both energies. However, decreasing the energy of the controlled output will require a large control signal and a small control signal will lead to large controlled outputs. The role of the constant is to establish a trade-o between these conicting goals:

LQG/LQR controller design

81

1. When we chose very large, the most eective way to decrease JLQR is to use little control, at the expense of a large controlled output. 2. When we chose very small, the most eective way to decrease JLQR is to obtain a very small controlled output, even if this is achieved at the expense of a large controlled output. Often the optimal LQR problem is dened more generally and consists of nding the controller transfer-matrix C(s) that minimizes JLQR :=
0

z(t) Qz(t) + u (t)Ru(t)dt,

(9.3)

Sidebar 33. The most general form for the quadratic criteria is Z x Qx + u Ru + 2x N u dt
0

where Q is an symmetric positive-denite matrix, R an m m symmetric positivedenite matrix, and a positive constant. Brysons rule A rst choice for the matrices Q and R in (9.3) is given by the Brysons rule [2, p. 537]: select Q and R diagonal with Qii = Rjj 1 2, maximum acceptable value of zi 1 = , maximum acceptable value of u2 j i {1, 2, . . . , } j {1, 2, . . . , m},

... Sidebar 34. A symmetric q q matrix M is positive-denite if x M x > 0, for every nonzero vector x Rq . . .

which corresponds to the following criteria JLQR :=

0 m

Qii zi (t)2 +
i=1 j=1

Rjj uj (t)2 dt.

In essence the Brysons rule scales the variables that appear in JLQR so that the maximum acceptable value for each term is one. This is especially important when the units used for the dierent components of u and z make the values for these variables numerically very dierent from each other. Although Brysons rule sometimes gives good results, often it is just the starting point to a trial-and-error iterative design procedure aimed at obtaining desirable properties for the closed-loop system. In Section 9.5 we will discuss systematic methods to chose the weights in the LQR criterion.

9.3

State-Feedback LQR

In the state-feedback version of the LQR problem (Figure 9.2), we assume that the whole state x can be measured and therefore it is available for control. Solution to the optimal state-feedback LQR Problem 6. The optimal state-feedback LQR con-MATLAB R Hint 33. computes the optimal troller for the criteria (9.3) is a simple matrix gain of the form u = Kx (9.4)
lqr

state-feedback controller gain K. . .

82
u(t) Rm z(t) R

Joo P. Hespanha a

controller

process

x(t) Rn

Figure 9.2. Linear quadratic regulation (LQR) with state feedback

where K is the m n matrix given by K = (H QH + R)1 (B P + H QG) and P is the unique positive-denite solution to the following equation A P + P A + G QG (P B + G QH)(H QH + R)1 (B P + H QG) = 0, known as the Algebraic Riccati Equation (ARE).

9.4
Sidebar 35. A system x = Ax + Bu is asymptotically stable when all eigenvalues of A have negative real parts. See Section 8.5.

Stability and Robustness

x = (A BK)x.

The state-feedback control law (9.4), results in a closed-loop system of the form

A crucial property of LQR controller design is that this closed-loop is asymptotically stable (i.e., all the eigenvalues of A BK have negative real part) as long as the following two conditions hold: 1. The system (9.1) is controllable. 2. The system (9.1) is observable when we ignore y and regard z as the sole output . Perhaps even more important is the fact that LQR controllers are inherently robust with respect to process uncertainty. To understand why, consider the open-loop transfer-matrix from the process input u to the controllers output u (Figure 9.3). The state-space model
u K u x = Ax + Bu x

Sidebar 36. The denitions and tests for controllability and observability are reviewed in Section 8.4. Attention! When selecting the measured output z, it is important to verify that the observability condition is satised.

Figure 9.3. State-feedback open-loop gain

from u to u is given by x = Ax + Bu, u = Kx,

LQG/LQR controller design which corresponds to the following open-loop negative feedback m m transfer-matrix G0 (s) = K(sI A)1 B.

83

We focus our attention in single-input processes (m = 1), for which G0 (s) is a scalar transferfunction and the following holds: Kalmans Inequality. When H G = 0, the Nyquist plot of G0 (j) does not enter a circle of radius one around 1, i.e., |1 + G0 (j)| 1, R.

Sidebar 37. LQR controllers also exhibit robustness properties for multiple-input processes. However, in this case G0 (s) is a m m transfer-matrix and one needs a multi-variable Nyquist criterion.

Kalmans Inequality is represented graphically in Figure 9.4 and has several signicant implications, which are discussed next.
Im

2 G0 (j)

Re 60o

Figure 9.4. Nyquist plot for a LQR state-feedback controller

Positive gain margin If the process gain is multiplied by a constant k > 1, its Nyquist plot simply expands radially and therefore the number of encirclements does not change. This corresponds to a positive gain margin of +. Negative gain margin If the process gain is multiplied by a constant .5 < k < 1, its Nyquist plot contracts radially but the number of encirclements still does not change. This corresponds to a negative gain margin of 20 log10 (.5) = 6dB. Phase margin If the process phase increases by [60, 60] degrees, its Nyquist plots rotates by but the number of encirclements still does not change. This corresponds to a phase margin of 60 degrees. Multiplicative uncertainty Kalmans inequality guarantees that G0 (j) 2. 1 + G0 (j) (9.5)
Sidebar 38. Why?...

84
m (s) u K
+ +

Joo P. Hespanha a
|m (j)| m (), x

x = Ax + Bu

Figure 9.5. Unity feedback conguration with multiplicative uncertainty

Since, we known that the closed-loop system in Figure 9.5 remains stable for every multiplicative uncertainty block m (j) with norm smaller than m (), as long as 1 G0 (j) , < 1 + G0 (j) m () (9.6)

we conclude that an LQR controller provides robust stability with respect to any multiplicative 1 uncertainty with magnitude smaller than 2 , because we then have 1 G0 (j) . 2< 1 + G0 (j) m () However, much larger additive uncertainties may be admissible: e.g., when G0 (j) 1, (9.6) will hold for m () almost equal to 1; and when G0 (j) 1, (9.6) will hold for m () almost equal to 1/G0 (j) 1. Attention! Kalmans inequality is only valid when H G = 0. When this is not the case, LQR controllers can be signicantly less robust. This limits to same extent the controlled outputs that can be placed in z. For example, consider the process x = Ax + Bu and suppose that we want to regulate a particular output y1 = C1 x. Choosing z = y1 = C1 x leads to G = C1 and H = 0 and therefore H G = 0, for which Kalmans inequality holds. However, choosing z=
Sidebar 39. If the transfer function from u to y1 has two more poles than zeros, then one can show that C1 B = 0 and H = 0. In this case, Kalmans inequality holds also for this choice of z.

0 C1 C1 x y1 C1 x u, x+ = = = C1 B C1 A C1 Ax + C1 Bu C1 x y1

leads to G= and therefore

H G = B C1 C1 A,

C1 , C1 A

H=

0 , C1 B

which may not be equal to zero.

LQG/LQR controller design

85

9.5

Loop-shaping control using LQR

Although Brysons rule sometimes gives good results, it may not suce to satisfy tight control specications. We will see next a few other rules that allow us to actually do loopshaping using LQR. We restrict our attention to the single-input case (m = 1) and R = 1, Q = I, which corresponds to JLQR :=
0

z(t)

+ u(t)2 dt.
MATLAB R Hint 34. sigma(sys) draws the norm-Bode plot of the system sys. . .

Low-frequency open-loop gain For the range of frequencies for which |G0 (j)| 1 (typically low frequencies), we have that |G0 (j)| where Pz (s) := G(sI A)1 B + H is the transfer function from the control signal u to the controlled output z. To understand the implications of this formula, it is instructive to consider two fairly typical cases: 1. When z = y1 , with y1 := C1 x scalar, we have |P1 (j)| . |G0 (j)| H H + where P1 (s) := C1 (sI A)1 B is the transfer function from the control input u to the output y1 . In this case, (a) the shape of the magnitude of the open-loop gain G0 (j) is determined by the magnitude of the transfer function from the control input u to the output y1 ; (b) the parameter moves the magnitude Bode plot up and down (more precisely H H + ). 2. When z = y1 y1 , we can show that |1 + j| |P1 (j)| |G0 (j)| . H H + (9.7)

Pz (j) H H +

Sidebar 40. Although the magnitude of G0 (j) mimics the magnitude of P1 (j), the phase of the open-loop gain G0 (j) always leads to a stable closed-loop with an appropriate phase margin. Sidebar 41. Why?. . .

In this case the low-frequency open-loop gain mimics the process transfer function from u to y, with an extra zero at 1/ and scaled by H 1 . Thus H+ (a) moves the magnitude Bode plot up and down (more precisely H H + ), (b) large values for lead to a low-frequency zero and generally result in a larger phase margin (above the minimum of 60 degrees) and smaller overshoot in the step response. However, this is often achieved at the expense of a slower response.

86 High-frequency open-loop gain For 1, we have that c |G0 (j)| ,

Joo P. Hespanha a

for some constant c. We thus conclude the following: 1. LQR controllers always exhibit a high-frequency magnitude decay of 20dB/decade. 2. The cross-over frequency is approximately given by c c 1 cross , cross which shows that the cross-over frequency is proportional to 1/ and generally small values for result in faster step responses. Attention! The (slow) 20dB/decade magnitude decrease is the main shortcoming of state-feedback LQR controllers because it may not be sucient to clear high-frequency upper bounds on the open-loop gain needed to reject disturbances and/or for robustness with respect to process uncertainty. We will see in Section 10.6 that this can actually be improved with output-feedback controllers. Example 5 (Aircraft roll-dynamics). Figure 9.6 shows Bode plots of the open-loop gain G0 (s) = K(sI A)1 B for several LQR controllers obtained for the aircraft roll-dynamics in Example 1. The controlled output was chosen to be z := , which corresponds to
Openloop Bode Diagrams From: u To: Out(1) 80 60 Magnitude (dB) 40 20 0 20 40 90 rho = 0.01 rho = 1 rho = 100 u to roll angle gamma = 0.01 Magnitude (dB) 80 60 40 20 0 20 40 90 gamma = 0.01 gamma = 0.1 gamma = 0.3 u to roll angle (rho = 0.01) Openloop Bode Diagrams From: u To: Out(1)

Phase (deg)

135

Phase (deg)
1 0 1 2 3

135

180 2 10

10

10 10 Frequency (rad/sec)

10

10

180 2 10

10

10 10 Frequency (rad/sec)

10

10

(a) Open-loop gain for several values of . This (b) Open-loop gain for several values of . parameter allow us to move the whole magni- Larger values for this parameter result in a tude Bode plot up and down. larger phase margin.

Figure 9.6. Bode plots for the open-loop gain of the LQR controllers in Example 5. As expected, for low frequencies the open-loop gain magnitude matches that of the process transfer function from u to (but with signicantly lower/better phase) and at high-frequencies the gains magnitude falls at 20dB/decade.

G :=

1 0 0

0 , 0

H :=

0 . 0

LQG/LQR controller design

87

The controllers were obtained with R = 1, Q = I22 , and several values for and . Figure 9.6(a) shows the open-loop gain for several values of and Figure 9.6(b) shows the open-loop gain for several values of . Figure 9.7 shows Nyquist plots of the open-loop gain G0 (s) = K(sI A)1 B for dierent choices of the controlled output z. In Figure 9.7(a) z := , which corresponds to
Openloop Nyquist Diagram (rho = 1, gamma = 0.1) From: u To: uu 3 3 Openloop Nyquist Diagram From: u To: uu

1 Imaginary Axis Imaginary Axis

2 0

2 0

/3 1

/3 1

1 Real Axis

1 Real Axis

(a) H G = 0

(b) H G = 0

Figure 9.7. Bode plots for the open-loop gain of the LQR controllers in Example 5

G :=

1 0 0 1

0 , 0

H :=

0 . 0

In this case, H G = [ 0 0 0 ] and Kalmans inequality holds as can be seen in the Nyquist plot. In Figure 9.7(b), the controlled output was chosen to be z := , which corresponds to G := 1 0 0 0 0 , 50 H := 0 . 50

In this case we have H G = [ 0 0 2500 ] and Kalmans inequality does not hold. We can see from the Nyquist plot that the phase and gain margins are very small and there is little robustness with respect to unmodeled dynamics since a small perturbation in the process can lead to an encirclement of the point 1.

9.6

MATLAB hints

MATLAB R Hint 33 (lqr). The command [K,S,E]=lqr(A,B,QQ,RR,NN) computes the optimal state-feedback LQR controller for the process x = Ax + Bu with criteria J :=
0

x(t) QQx(t) + u (t)RRu(t) + 2x (t)NNu(t)dt.

88 For the criterion in (9.2) one should select QQ = G G, and for the criterion (9.3) QQ = G QG, RR = H QH + R, RR = H H + I,

Joo P. Hespanha a

NN = G H

NN = G QH.

(cf. Sidebar 33). This command returns the optimal state-feedback matrix K, the solution P to the corresponding Algebraic Riccati Equation, and the poles E of the closed-loop system. MATLAB R Hint 34 (sigma). The command sigma(sys) draws the norm-Bode plot of the system sys. For scalar transfer functions this command plots the usual magnitude Bode plot but for vector transfer function it plots the norm of the transfer function versus the frequency.

9.7

To probe further
J :=
0

Sidebar 33 (General LQR). The most general form for a quadratic criteria is x(t) Qx(t) + u (t)Ru(t) + 2x (t)N u(t)dt. (9.8)

Since z = Gx + Hu, the criterion in (9.2) is a special form of (9.8) with Q = G G, R = H H + I, N = G H

and (9.3) is a special form of (9.8) with Q = G QG, R = H QH + R, N = G QH.

For this criteria, the optimal state-feedback LQR controller is still of the form u = Kx K = R1 (B P + N ) and P is a solution to the following Algebraic Riccati Equation (ARE) A P + P A + Q (P B + N )R1 (B P + N ) = 0. Sidebar 34. Other choices for the matrices Q and R are possible but one should always choose both matrices to be positive-denite. We recall that a symmetric q q matrix M is positive-denite if x M x > 0, for every nonzero vector x Rq . To test if a matrix is positive dene one can compute its eigenvalues. If they are all positive the matrix is positive-denite, otherwise it is not (cf. Matlab Hint 32).

but now K is given by

LQG/LQR controller design

89

Sidebar 38 (Multiplicative uncertainty). Since the Nyquist plot of G0 (j) does not enter a circle of radius one around 1, we have that |1 + G0 (j)| 1 1 G0 (j) = 1 1 1 + G0 (j) 1 + G0 (j) y , we have that G= C , CA H= 0 . CB

G0 (j) 2. 1 + G0 (j)

Sidebar 41 (Equation (9.7)). When z = y z= In this case, Pz (s) = y Cx = y CAx + CBu

Py (s) 1 = P (s), sPy (s) s y

where Py (s) := C(sI A)1 B, and therefore |G0 (j)| |1 + j| |Py (j)| 1 + 2 2 |Py (j)| = . H H + H H +

9.8

Exercises

Exercise 18. Verify using the diagram in Figure 11.1 that, for the single-input case (m = 1), the closed-loop transfer function Tu (s) from the reference r to the process input u is given by Tu (s) = 1 (KF + N ), 1 + G0

where G0 (s) = K(sI A)1 B, and the closed-loop transfer function Tz from the reference r to the controlled output z is given by Tz (s) = where Pz (s) = G(sI A)1 B + H. Exercise 19. Consider an inverted pendulum operating near the upright equilibrium position, with linearized model given by 0 = g

1 Pz (KF + N ), 1 + G0

1 b m2

0
1 m2

where T denotes an applied torque, the pendulums angle with a vertical axis pointing up, and = 1m, m = 1Kg, b = .1N/m/s, g = 9.8m/s2 . 1. Design a PD controller using LQR 2. Design a PID controller using LQR Hint: Consider an augmented process model with state , , z(t) =
t 0

(s)ds.

90

Joo P. Hespanha a

Lecture #10

LQG/LQR Output Feedback

Contents 10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8 Output Feedback . . . . . . Full-order observers . . . . LQG estimation . . . . . . . LQG/LQR output feedback Separation Principle . . . . Loop-gain recovery . . . . . MATLAB hints . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 91 93 94 95 95 97 97

10.1

Output Feedback

The state-feedback LQR formulation considered in Chapter 9.3 suered from the drawback that the optimal control law u(t) = Kx(t) (10.1)

required the whole state x of the process to be measurable. An possible approach to overcome this diculty is to estimate the state of the process based solely on the measured output y, and use u(t) = K x(t) instead of (10.1), where x(t) denotes an estimate of the process state x(t). In this chapter we consider the problem of constructing state estimates.

10.2

Full-order observers
x = Ax + Bu, 91 y = Cx, (10.2)

Consider a process with state-space model

92

Joo P. Hespanha a

where y denotes the measured output, u the control input. We assume that x cannot be measured and our goal to estimate its value based on y. Suppose we construct the estimate x by replicating the process dynamics as in x = A + Bu. x (10.3)

To see of this would generate a good estimate for x, we can dene the state estimation error e := x x and study its dynamics. From (10.2) and (10.3), we conclude that e = Ax A = Ae. x This shows that when the matrix A is asymptotically stable the error e converges to zero for any input u, which is good news because it means that x eventually converges to x as t . However, when A is not stable e is unbounded and x grow further and further apart from x as t . To avoid this, one includes a correction term in (10.3): x = A + Bu + L(y y ), x y = C x, (10.4)

where y should be viewed as an estimate of y and L a given n k matrix. When x is equal (or very close) to x, then y will be equal (or very close) to y and the correction term L(y y ) plays no role. However, when x grows away from x, this term will (hopefully!) correct the error. To see how this can be done, we re-write the estimation error dynamics now for (10.2) and (10.4): e = Ax A L(Cx C x) = (A LC)e. x Now e converges to zero as long as A LC is asymptotically stable, It turns out that, even when A is unstable, in general we will be able to select L so that A LC is asymptotically stable. The system (10.4) can be re-write as x = (A LC) + Bu + Ly, x
Sidebar 42. Full-order refers to the fact that the size of its state x is equal to the size of the process state x.

(10.5)

and is called a full-order observer for the process. Full-order observers have two inputsthe process control input u and its measured output yand a single outputthe state estimate x. Figure 10.1 shows how a full-order observer is connected to the process.
u x = Ax + Bu y z

x = (A LC) + Bu + Ly x Figure 10.1. Full-order observer

LQG/LQR controller design

93

10.3

LQG estimation

Any choice of L in (10.4) for which A LC is asymptotically stable will make x converge to x, as long at the process dynamics is given by (10.2). However, in general the output y is aected by measurement noise and the process dynamics are also aected by disturbance. In light of this, a more reasonable model for the process is x = Ax + Bu + Bd, y = Cx + n, (10.6)

where d denotes a disturbance and n measurement noise. In this we need to re-write the estimation error dynamics for (10.6) and (10.4), which leads to e = Ax + Bd A L(Cx + n C x) = (A LC)e + Bd Ln. x Because of n and d, the estimation error will generally not converge to zero, but we would still like it to remain small by appropriate choice of the matrix L. This motivates the so called Linear-quadratic Gaussian (LQG) estimation problem: Problem 7 (Optimal LQG). Find the matrix gain L that minimizes the asymptotic expected value of the estimation error: JLQG := lim E
t

e(t)

Sidebar 43. A zero-mean white noise process n has an autocorrelation of the form Rn (t1 , t2 ) := E n(t1 ) n (t2 ) = QN (t1 t2 ). Such process is wide-sense stationary in the sense that its mean is time-invariant and its autocorrelation R(t1 , t2 ) only depends on the dierence := t1 t2 . Its power spectrum is frequency-independent and given by Z Sn () := R( )ej d = QN .

where d(t) and n(t) are zero-mean Gaussian noise processes (uncorrelated from each other) with power spectrum Sd () = QN , Sn () = RN , .

Solution to the optimal LQG Problem 7. The optimal LQG estimator gain L is the n k matrix given by
1 L = P C RN

and P is the unique positive-denite solution to the following Algebraic Riccati Equation (ARE)
1 AP + P A + BQN B P C RN CP = 0.

MATLAB R Hint 35. kalman computes the optimal LQG estimator gain L. . . Sidebar 44. A symmetric q q matrix M is positive denite if x M x > 0, for every nonzero vector x Rq (cf. Sidebar 34),

When one uses the optimal gain L in (10.5), this system is called the Kalman-Bucy lter. A crucial property of this system is that A LC is asymptotically stable as long as the following two conditions hold: 1. The system (10.6) is observable. 2. The system (10.6) is controllable when we ignore u and regard d as the sole input. Dierent choices of QN and RN result in dierent estimator gains L:

94

Joo P. Hespanha a 1. When RN is very small (when compared to QN ), the measurement noise n is necessarily small so the optimal estimator interprets a large deviation of y from y as an indication that the estimate x is bad and needs to be correct. In practice, this lead to large matrices L and fast poles for A LC. 2. When RN is very large, the measurement noise n is large so the optimal estimator is much more conservative in reacting to deviations of y from y. This generally leads to smaller matrices L and slow poles for A LC.

We will return to the selection of QN and RN in Section 10.6.

10.4

LQG/LQR output feedback

x = Ax + Bu, y = Cx, z = Gx + Hu.

We now go back to the problem of designing an output-feedback controller for the process:

Suppose that we designed a state-feedback controller u = Kx that solves an LQR problem and constructed an LQG state-estimator x = (A LC) + Bu + Ly. x
MATLAB R Hint 36. reg(sys,K,L) computes the LQG/LQR positive output-feedback controller for the process sys with regulator gain K and estimator gain L. . .

(10.7)

We can obtain an output-feedback controller by using the estimated state x in (10.7), instead of the true state x. This leads to the following output-feedback controller x = (A LC) + Bu + Ly = (A LC BK) + Ly, x x with negative-feedback transfer matrix given by C(s) = K(sI A + LC + BK)1 L. This is usually known as an LQG/LQR output-feedback controller and the resulting closedloop is shown in Figure 10.2.
z x = (A LC BK) + Ly x u x = Ax + Bu z = Gx + Hu y = Cx y

u = K x,

u = Kx

Figure 10.2. LQG/LQR output-feedback

LQG/LQR controller design

95

10.5

Separation Principle

The rst question to ask about an LQG/LQR controller is whether or not the closed-loop system will be stable. To answer this question we collect all the equations that denes the closed-loop system: x = Ax + Bu, x = (A LC) + Bu + Ly, x y = Cx, u = K x. (10.8) (10.9)

To check the stability of this system it is more convenient to consider the dynamics of the estimation error e := x x instead of the the state estimate x. To this eect we replace in the above equations x by x e, which yields: x = Ax + Bu = (A BK)x + BKe, e = (A LC)e, This can be written in matrix notation as A BK x = 0 e BK A LC x , e y= C 0 x . e
Sidebar 45. Any eigenvalue of a block diagonal matrix must be an eigenvalue of one of the diagonal blocks.

y = Cx, u = K(x e).

Separation Principle. The eigenvalues of the closed-loop system (10.8) are given by those of the state-feedback regulator dynamics A BK together with those of state-estimator dynamics A LC. In case these both matrices are asymptotically stable, then so is the closedloop (10.8).

10.6

Loop-gain recovery

We saw in Sections 9.4 and 9.5 that state-feedback LQR controllers have desirable robustness properties and that we can shape the open-loop gain by appropriate choice of the LQR weighting parameter and the choice of the controlled output z. It turns out that we can, to some extent, recover the LQR open-loop gain for the LQG/LQR controller. Loop-gain recovery. Suppose that the process is single-input/single-output and has no zeros in the right half-place. Selecting B := B, RN := , > 0,

Sidebar 46. B = B corresponds to an input disturbance since the process becomes x = Ax + Bu + Bd = Ax + B(u + d).

the open-loop gain for the output-feedback LQG/LQR controller converges to the open-loop gain for the state-feedback LQR state-feedback controller over a range of frequencies [0, max ] as we make 0, i.e., C(j)P (j) 0 K(jI 1)1 B, [0, max ]

In general, the larger max is, the smaller needs to be for the gains to match.

96

Joo P. Hespanha a

Attention! 1. To achieve loop-gain recovery we need to chose RN := , even if this does not accurately describe the noise statistics. This means that the estimator may not be optimal for the actual noise. 2. One should not make smaller than necessary because we do not want to recover the (slow) 20dB/decade magnitude decrease at high frequencies. In practice we should make just small enough to get loop-recovery until just above or at cross-over. For larger values of , the output-feedback controller may actually behave much better than the state-feedback one. 3. When the process has zeros in the right half-plane, loop-gain recovery will generally only work up to the frequencies of the nonminimum-phase zeros. When the zeros are in the left half-plane but close to the axis, the closed-loop will not be very robust with respect to uncertainty in the position of the zeros. This is because the controller will attempt to cancel these zeros.

Example 6 (Aircraft roll-dynamics). Figure 10.3(a) shows Bode plots of the open-loop gain for the state-feedback LQR state-feedback controller vs. the open-loop gain for several output-feedback LQG/LQR controller obtained for the aircraft roll-dynamics in Example 1. The LQR controller was designed using the controlled output z := , = .1 and
Openloop Bode Diagrams From: u To: Out(1) 80 60 Magnitude (dB) 40 20 0 20 40 60 80 90 sigma = 0.01 sigma = 1e05 sigma = 1e08 LQR loopgain (rho = 0.01, gamma=0.1)

Step Response 1.5

Amplitude 0.5

Phase (deg)

135

180
2 1 0 1 2 3

sigma = 0.01 sigma = 1e05 sigma = 1e08 LQR loopgain (rho = 0.01, gamma=0.1)
10 10 10 Frequency (rad/sec) 10 10

10

0.2

0.4

0.6

0.8

1 Time (sec)

1.2

1.4

1.6

1.8

(a) Open-loop gain

(b) Closed-loop step response

Figure 10.3. Bode plots and closed-loop step response for the open-loop gain of the LQR controllers in Examples 6, 8.

= .01. For the LQG state-estimators we used B = B and RN = for several values of . We can see that, as decreases, the range of frequencies over which the open-loop gain of the output-feedback LQG/LQR controller matches that of the state-feedback LQR statefeedback increases. Moreover, at high frequencies the output-feedback controllers exhibit much faster (and better!) decays of the gains magnitude.

LQG/LQR controller design

97

10.7

MATLAB hints

MATLAB R Hint 35 (kalman). The command [est,L,P]=kalman(sys,QN,RN) computes the optimal LQG estimator gain for the process x = Ax + Bu + BBd, y = Cx + n,

where d(t) and n(t) are zero-mean Gaussian noise processes (uncorrelated from each other) with power spectrum Sd () = QN, Sn () = RN, .

The system sys should be a state-space model dened by sys=ss(A,[B BB],C,0). This command returns the optimal estimator gain L, the solution P to the corresponding Algebraic Riccati Equation, and a state-space model est for the estimator. The inputs to est are [u; y] and its outputs are [; x]. y MATLAB R Hint 36 (reg). The command reg(sys,K,L) computes a state-space model for a positive output-feedback LQG/LQG controller for the process with state-space model sys with regulator gain K and estimator gain L.

10.8

Exercises
on a cart operating near the upright equilib p 0 0 0 p .325 F + 1 0 .3 0

where F denotes a force applied to the cart, p the carts horizontal position, and the pendulums angle with a vertical axis pointing up. 1. Design an LQG/LQR output-feedback controller that uses only the angle and the position of the cart p. 2. Design an LQG/LQR output-feedback controller that uses the angle , the angular velocity , the position of the cart p, and its derivative using LQG/LQR (full-state feedback). Why use LQG when the state is accessible?

Exercise 20. Consider an inverted pendulum rium position, with linearized model given by p 0 1 0 p 0 0 2.94 = 0 0 0 0 0 11.76

98

Joo P. Hespanha a

Lecture #11

Set-point control
Contents 11.1 11.2 11.3 11.4 Nonzero equilibrium state and input State-feedback . . . . . . . . . . . . Output-feedback . . . . . . . . . . . To probe further . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 100 101 102

11.1

Nonzero equilibrium state and input

Sidebar 47. We will see shortly how to accomplish this.

Often one does not want to make z as small as possible, but instead make it converge as fast as possible to a given constant set-point value r. This can be achieved by making the state x and the input u of the process (9.1) converge to values x and u for which Ax + Bu = 0, r = Gx + Hu . (11.1) The right equation makes sure that z will be equal to r when x and u reach x and u , respectively. The left-equation makes sure that when these values are reached, x = 0 and therefore x will remain equal to x , i.e., x is an equilibrium state. Given the desired set-point r for x, computing x and u is straightforward because (11.1) is a system of linear equations and in general the solution to these equations is of the form x = F r, u = N r. (11.2)

Sidebar 48. When the process transfer-function has an integrator, one generally gets u = 0.

For example, when the number of inputs to the process m is equal to the number of controlled outputs , we have A B G H 0 x = r u A B x = G H u
1

0 , r

(11.3)
1

A B and F is an n matrix given by the top n rows and right-most columns of [ G H ] and A B 1 N is an m matrix given by the bottom m rows and right-most columns of [ G H ] .

Sidebar 49. The matrix A B G H is invertible unless the process transfer-function from u to z has a zero at the origin. This derivative eect will always make z converge to zero when the input converges to a constant.

99

100

Joo P. Hespanha a

Attention! When the number of process inputs m is larger than the number of controlled outputs we have an over-actuated system and the system of equations (11.1) will generally have multiple solutions. One of them is A B x = u G H

A G

B H

A B G H

0 . r

In this case, we can still express x and u as in (11.2). When the number of process inputs m is smaller than the number of controlled outputs we have an under-actuated system and the system of equations (11.1) may not have a solution. In fact, a solution will only exists for some specic references r. However, when it does exist it we can still express x and u as in (11.2).

11.2
Sidebar 50. Why?. . .

State-feedback

When one wants z to converge to a given set-point value r, the state-feedback controller should be u = K(x x ) + u = Kx + (KF + N )r, (11.4)

where x and u are given by (11.2) and K is the gain of the optimal regulation problem. The corresponding control architecture is shown in Figure 11.1. The state-space model for
u N r F x
+ +

u x = Ax + Bu

Figure 11.1. Linear quadratic set-point control with state feedback

the closed-loop system is given by x = Ax + Bu = (A BK)x + B(KF + N )r

z = Gx + Hu = (G HK)x + H(KF + N )r.

Example 7 (Aircraft roll-dynamics). Figure 11.2 shows step responses for the state-feedback LQR controllers in Example 5, whose Bode plots for the open-loop gain are shown in Figure 9.6. Figure 11.2(a) shows that smaller values of lead to faster responses and Figure 11.2(b) shows that larger values for lead to smaller overshoots (but slower responses).

LQG/LQR controller design

Step Response 1.4 1.4 Step Response

101

1.2

1.2

Amplitude

0.6

Amplitude rho = 0.01 rho = 1 rho = 100 gamma = 0.01 0 1 2 3 Time (sec) 4 5 6

0.8

0.8

0.6

0.4

0.4

0.2

0.2

gamma = 0.01 gamma = 0.1 gamma = 0.3 (rho = 0.01) 0 1 2 3 Time (sec) 4 5 6

(a)

(b)

Figure 11.2. Step responses for the closed-loop LQR controllers in Example 7

11.3

Output-feedback
Sidebar 51. Why?. . .

When one wants z to converge to a given set-point value r, the output-feedback LQG/LQR controller should be x = (A LC BK) + L(Cx y), x u = K x + u , (11.5)

where x and u are given by (11.2). The corresponding control architecture is shown in Figure 11.3. The state-space model for the closed-loop system is given by
u N r CF x
+ + +

x = (A LC BK) + Lv x u = Kx

z
x = Ax + Bu y = Cx

Sidebar 52. When z = y, we have G = C, H = 0 and in this case Cx = r. This corresponds to CF = 1 in Figure 11.3. When the process has an integrator we get N = 0 and obtain the usual unity-feedback conguration.

Figure 11.3. LQG/LQR set-point control

x Ax + B(K x + u ) A = = x (A LC BK) + L(Cx Cx) x LC z = Gx + H(K x + u ) = G HK x + HN r x

BK A LC BK

x BN + r x LCF

Example 8 (Aircraft roll-dynamics). Figure 10.3(b) shows step responses for the outputfeedback LQG/LQR controllers in Example 6, whose Bode plots for the open-loop gain are shown in Figure 10.3(a). We can see that smaller values of lead to a smaller overshoot mostly due to a larger gain margin.

102

Joo P. Hespanha a

11.4

To probe further

Sidebar 50 (Set-point control with state-feedback). To understand why (11.4) works, suppose we dene z = z r, Then x = Ax + Bu = A(x x ) + B(u u ) + Ax + Bu and we conclude that x = A + B u, x By selecting the control signal in (11.4), we are setting u = u u = K(x x ) = K x, which is the optimal state-feedback LQR controller that minimizes JLQR :=
0

x = x x ,

u = u u .

z = Gx + Hu r = G(x x ) + H(u u ) + Gx + Hu r

z = G + H u. x

(11.6)

z (t) Q(t) + u (t)R(t) dt, z u

This controller makes the system (11.6) asymptotically stable and therefore x, u, z all converge to zero as t , which means that z converges to r. Sidebar 51 (Set-point control with output-feedback). To understand why (11.5) works suppose we dene z = z r, Then x = (A LC) x = (A BK) LC x x x (11.7) (11.8) (11.9) x = x x + x.

z = G( x) + HK x r. x

1. Since A LC is asymptotically stable, we conclude from (11.7) that x 0 as t . In practice, we can view the state x of the controller as an estimate of x x. 2. Since A BK is asymptotically stable and x 0 as t , we conclude from (11.8) that x 0 as t . 3. Since x 0 and x 0 as t , we conclude from (11.9) that z r as t ! Exercise 21. Verify equations (11.7), (11.8), and (11.9).

Bibliography
[1] R. Adrain. Research concerning the probabilities of the errors which happen in making observations. The Analyst, I:93109, 1808. Article XIV. [2] G. F. Franklin, J. D. Powell, and A. Emami-Naeini. Feedback Control of Dynamic Systems. Prentice Hall, Upper Saddle River, NJ, 4th edition, 2002. [3] C. F. Gauss. Theoria motus corporum coelestium in sectionibus conicis solem ambientum (the theory of the motion of heavenly bodies moving around the sun in conic sections). Hamburg, Germany, 1809. URL http://134.76.163.65/agora_docs/137784TABLE_ OF_CONTENTS.html. [4] B. Hayes. Science on the farther shore. American Scientist, 90(6):499502, Nov. 2002. [5] L. Ljung. System Identication: Theory for the user. Information and System Sciences Series. Prentice-Hall, Upper Saddle River, NJ, 2nd edition, 1999. [6] J. V. Vegte. Feedback Control Systems. Prentice Hall, New Jersey, 3rd edition, 1994.

103