Introduction to Dynamic Systems

(Network Mathematics Graduate Programme)

Martin Corless

School of Aeronautics & Astronautics

Purdue University
West Lafayette, Indiana
[email protected]

July 14, 2011

Contents

I Representation of Dynamical Systems vii

1 Introduction 1
1.1 Ingredients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Some notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 State space representation of dynamical systems 5

2.1 Linear examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 A first example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.2 The unattached mass . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.3 Spring-mass-damper . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.4 A simple structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Nonlinear examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.1 A first nonlinear system . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 Planar pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2.3 Attitude dynamics of a rigid body . . . . . . . . . . . . . . . . . . . . 9
2.2.4 Body in central force motion . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.5 Double pendulum on cart . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.6 Two-link robotic manipulator . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Discrete-time examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 The discrete unattached mass . . . . . . . . . . . . . . . . . . . . . . 13
2.3.2 Additive increase multiplicative decrease (AIMD) algorithm for re-
source allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 General representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.1 Continuous-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.2 Discrete-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5.1 Vector spaces and IRn . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5.2 IR2 and pictures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.3 Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.4 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.6 Vector representation of dynamical systems . . . . . . . . . . . . . . . . . . . 27
2.7 Solutions and equilibrium states : continuous-time . . . . . . . . . . . . . . 30
2.7.1 Equilibrium states . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

iii
 2.7.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.7.3 Controlled equilibrium states . . . . . . . . . . . . . . . . . . . . . . 33
2.8 Solutions and equilibrium states: discrete-time . . . . . . . . . . . . . . . . . 33
2.8.1 Equilibrium states . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.8.2 Controlled equilibrium states . . . . . . . . . . . . . . . . . . . . . . 35
2.9 Numerical simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.9.1 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.9.2 Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3 Linear time-invariant (LTI) systems 41

3.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Linear time-invariant systems . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.1 Continuous-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.2 Discrete-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.3 Linearization about an equilibrium solution . . . . . . . . . . . . . . . . . . 48
3.3.1 Derivative as a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3.2 Linearization of continuous-time systems . . . . . . . . . . . . . . . . 49
3.3.3 Implicit linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.3.4 Linearization of discrete-time systems . . . . . . . . . . . . . . . . . . 55

4 Transfer functions 59
4.1 Transfer functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1.1 The Laplace transform . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1.2 Transfer functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.1.3 Poles and zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.1.4 Discrete-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Some transfer function properties . . . . . . . . . . . . . . . . . . . . . . . . 73
4.2.1 Power series expansion and Markov parameters . . . . . . . . . . . . 73
4.2.2 More properties* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.3 State space realization of transfer functions . . . . . . . . . . . . . . . . . . . 75
4.4 Realization of SISO transfer functions . . . . . . . . . . . . . . . . . . . . . . 76
4.4.1 Controllable canonical form realization . . . . . . . . . . . . . . . . . 76
4.4.2 Observable canonical form realization . . . . . . . . . . . . . . . . . . 77
4.5 Realization of MIMO systems . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.5.1 Controllable realizations . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.5.2 Observable realizations . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.5.3 Alternative realizations . . . . . . . . . . . . . . . . . . . . . . . . . 84

II System Behavior and Stability 89

5 Behavior of (LTI) systems : I 91
5.1 Initial stuff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
 5.1.1 Continuous-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.1.2 Discrete-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.2 Scalar systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2.1 Continuous-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2.2 First order complex and second order real. . . . . . . . . . . . . . . . 94
5.2.3 DT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.3 Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.3.1 Real A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.3.2 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.3.3 Companion matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.4 Behavior of continuous-time systems . . . . . . . . . . . . . . . . . . . . . . 103
5.4.1 System significance of eigenvectors and eigenvalues . . . . . . . . . . 103
5.4.2 Solutions for nondefective matrices . . . . . . . . . . . . . . . . . . . 109
5.4.3 Defective matrices and generalized eigenvectors . . . . . . . . . . . . 110
5.5 Behavior of discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . . 115
5.5.1 System significance of eigenvectors and eigenvalues . . . . . . . . . . 115
5.5.2 All solutions for nondefective matrices . . . . . . . . . . . . . . . . . 116
5.5.3 Some solutions for defective matrices . . . . . . . . . . . . . . . . . . 116
5.6 Similarity transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.7 Hotel California . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.7.1 Invariant subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.7.2 More on invariant subspaces . . . . . . . . . . . . . . . . . . . . . . 124
5.8 Diagonalizable systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.8.1 Diagonalizable matrices . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.8.2 Diagonalizable continuous-time systems . . . . . . . . . . . . . . . . . 128
5.8.3 Diagonalizable DT systems . . . . . . . . . . . . . . . . . . . . . . . . 130
5.9 Real Jordan form for real diagonalizable systems . . . . . . . . . . . . . . . 132
5.9.1 CT systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5.10 Diagonalizable second order systems . . . . . . . . . . . . . . . . . . . . . . 133
5.10.1 State plane portraits for CT systems . . . . . . . . . . . . . . . . . . 134
5.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6 eAt 141
6.1 State transition matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.1.1 Continuous time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.1.2 Discrete time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.2 Polynomials of a square matrix . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.2.1 Polynomials of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.2.2 Cayley-Hamilton Theorem . . . . . . . . . . . . . . . . . . . . . . . . 145
6.3 Functions of a square matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.3.1 The matrix exponential: eA . . . . . . . . . . . . . . . . . . . . . . . 150
6.3.2 Other matrix functions . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.4 The state transition matrix: eAt . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.5 Computation of eAt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.5.1 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
 6.5.2 Numerical simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.5.3 Jordan form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.5.4 Laplace style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.6 Sampled-data systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

7 Stability and boundedness 159

7.1 Boundedness of solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.2 Stability of equilibrium states . . . . . . . . . . . . . . . . . . . . . . . . . . 162
7.3 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.3.1 Global asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . 164
7.3.2 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.4 Exponential stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
7.5 LTI systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
7.6 Linearization and stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

8 Stability and boundedness: discrete time 173

8.1 Boundedness of solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
8.2 Stability of equilibrium states . . . . . . . . . . . . . . . . . . . . . . . . . . 174
8.3 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.3.1 Global asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . 175
8.3.2 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.4 Exponential stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
8.5 LTI systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.6 Linearization and stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

9 Basic Lyapunov theory 179

9.1 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9.1.1 Locally positive definite functions . . . . . . . . . . . . . . . . . . . . 181
9.1.2 A stability result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
9.2 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
9.3 Boundedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
9.3.1 Radially unbounded functions . . . . . . . . . . . . . . . . . . . . . . 189
9.3.2 A boundedness result . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
9.4 Global asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
9.4.1 Positive definite functions . . . . . . . . . . . . . . . . . . . . . . . . 192
9.4.2 A result on global asymptotic stability . . . . . . . . . . . . . . . . . 193
9.5 Exponential stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
9.5.1 Global exponential stability . . . . . . . . . . . . . . . . . . . . . . . 196
9.5.2 Proof of theorem ?? . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.5.3 Exponential stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.5.4 A special class of GES systems . . . . . . . . . . . . . . . . . . . . . 199
9.5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
10 Basic Lyapunov theory: discrete time* 203
10.1 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
10.2 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
10.3 Boundedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
10.4 Global asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
10.5 Exponential stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

11 Lyapunov theory for linear time-invariant systems 211

11.1 Positive and negative (semi)definite matrices . . . . . . . . . . . . . . . . . . 211
11.1.1 Definite matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
11.1.2 Semi-definite matrices* . . . . . . . . . . . . . . . . . . . . . . . . . . 212
11.2 Lyapunov theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
11.2.1 Asymptotic stability results . . . . . . . . . . . . . . . . . . . . . . . 214
11.2.2 MATLAB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
11.2.3 Stability results* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
11.3 Mechanical systems* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
11.4 Rates of convergence for linear systems* . . . . . . . . . . . . . . . . . . . . 224
11.5 Linearization and exponential stability . . . . . . . . . . . . . . . . . . . . . 225

III Input-Output Properties 227

12 Input output response 229
12.1 Response of continuous-time systems . . . . . . . . . . . . . . . . . . . . . . 229
12.1.1 State response and the convolution integral . . . . . . . . . . . . . . . 229
12.1.2 Output response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
12.2 Discretization and zero-order hold . . . . . . . . . . . . . . . . . . . . . . . . 235
12.2.1 Input-output discretization . . . . . . . . . . . . . . . . . . . . . . . . 238
12.3 Response of discrete-time systems* . . . . . . . . . . . . . . . . . . . . . . . 239
12.3.1 State response and the convolution sum . . . . . . . . . . . . . . . . 239
12.3.2 Input-output response and the pulse response matrix . . . . . . . . . 239

13 Observability 243
13.1 Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
13.2 Main result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
13.3 Development of main result . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
13.4 The unobservable subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
13.5 Unobservable modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
13.5.1 PBH Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
13.5.2 Existence of unobservable modes* . . . . . . . . . . . . . . . . . . . . 259
13.5.3 A nice transformation* . . . . . . . . . . . . . . . . . . . . . . . . . . 260
13.6 Observability grammians* . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
13.6.1 Infinite time observability grammian . . . . . . . . . . . . . . . . . . 268
13.7 Discrete time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
13.7.1 Main observability result . . . . . . . . . . . . . . . . . . . . . . . . . 270
 13.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

14 Controllability 275
14.1 Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
14.2 Main controllability result . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
14.3 The controllable subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
14.4 Proof of the controllable subspace lemma* . . . . . . . . . . . . . . . . . . . 284
14.4.1 Finite time controllability grammian . . . . . . . . . . . . . . . . . . 284
14.4.2 Proof of the controllable subspace lemma . . . . . . . . . . . . . . . . 286
14.5 Uncontrollable modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
14.5.1 Eigenvectors revisited: left eigenvectors . . . . . . . . . . . . . . . . . 287
14.5.2 Uncontrollable eigenvalues and modes . . . . . . . . . . . . . . . . . . 289
14.5.3 Existence of uncontrollable modes* . . . . . . . . . . . . . . . . . . . 289
14.5.4 A nice transformation* . . . . . . . . . . . . . . . . . . . . . . . . . . 291
14.6 PBH test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
14.6.1 PBH test for uncontrollable eigenvalues . . . . . . . . . . . . . . . . . 295
14.6.2 PBH controllability test . . . . . . . . . . . . . . . . . . . . . . . . . 297
14.7 Other characterizations of controllability* . . . . . . . . . . . . . . . . . . . 298
14.7.1 Infinite time controllability grammian . . . . . . . . . . . . . . . . . . 298
14.8 Controllability, observability, and duality . . . . . . . . . . . . . . . . . . . . 299
14.9 Discrete-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
14.9.1 Main controllability theorem . . . . . . . . . . . . . . . . . . . . . . . 300

IV Control Design 301

15 Stabilizability and state feedback 303
15.1 Stabilizability and state feedback . . . . . . . . . . . . . . . . . . . . . . . . 303
15.2 Eigenvalue placement (pole placement) by state feedback . . . . . . . . . . . 304
15.2.1 Single input systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
15.2.2 Multi-input systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
15.3 Uncontrollable eigenvalues (you cant touch them) . . . . . . . . . . . . . . . 312
15.4 Controllable canonical form . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
15.4.1 Transformation to controllable canonical form . . . . . . . . . . . . . 316
15.4.2 Eigenvalue placement by state feedback . . . . . . . . . . . . . . . . 319
15.5 Discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
15.5.1 Dead beat controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
15.6 Stabilization of nonlinear systems . . . . . . . . . . . . . . . . . . . . . . . . 322
15.6.1 Continuous-time controllers . . . . . . . . . . . . . . . . . . . . . . . 322
15.6.2 Discrete-time controllers . . . . . . . . . . . . . . . . . . . . . . . . . 322
15.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

16 Detectability and observers 329

16.1 Observers, state estimators and detectability . . . . . . . . . . . . . . . . . 329
16.2 Eigenvalue placement for estimation error dynamics . . . . . . . . . . . . . . 331
 16.3 Unobservable modes (you cant see them) . . . . . . . . . . . . . . . . . . . . 332
16.4 Observable canonical form* . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
16.5 Discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

17 Climax: output feedback controllers 337

17.1 Memoryless (static) output feedback . . . . . . . . . . . . . . . . . . . . . . 337
17.2 Dynamic output feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
17.3 Observer based controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
17.4 Discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
17.4.1 Memoryless output feedback . . . . . . . . . . . . . . . . . . . . . . . 343
17.4.2 Dynamic output feedback . . . . . . . . . . . . . . . . . . . . . . . . 343
17.4.3 Observer based controllers . . . . . . . . . . . . . . . . . . . . . . . . 344

18 Constant output tracking in the presence of constant disturbances 347

18.1 Zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
18.1.1 A state space characterization of zeros . . . . . . . . . . . . . . . . . 347
18.2 Tracking and disturbance rejection with state feedback . . . . . . . . . . . . 350
18.2.1 Nonrobust control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
18.2.2 Robust controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
18.3 Measured Output Feedback Control . . . . . . . . . . . . . . . . . . . . . . . 355
18.3.1 Observer based controllers . . . . . . . . . . . . . . . . . . . . . . . . 357

19 Lyapunov revisited* 361

19.1 Stability, observability, and controllability . . . . . . . . . . . . . . . . . . . 361
19.2 A simple stabilizing controller . . . . . . . . . . . . . . . . . . . . . . . . . . 365

20 Performance* 367
20.1 The L2 norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
20.1.1 The rms value of a signal . . . . . . . . . . . . . . . . . . . . . . . . . 367
20.1.2 The L2 norm of a time-varying matrix . . . . . . . . . . . . . . . . . 368
20.2 The H2 norm of an LTI system . . . . . . . . . . . . . . . . . . . . . . . . . 368
20.3 Computation of the H2 norm . . . . . . . . . . . . . . . . . . . . . . . . . . 369

21 Linear quadratic regulators (LQR) 373

21.1 The linear quadratic optimal control problem . . . . . . . . . . . . . . . . . 373
21.2 The algebraic Riccati equation (ARE) . . . . . . . . . . . . . . . . . . . . . 375
21.3 Linear state feedback controllers . . . . . . . . . . . . . . . . . . . . . . . . . 376
21.4 Infimal cost* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
21.4.1 Reduced cost stabilizing controllers. . . . . . . . . . . . . . . . . . . . 377
21.4.2 Cost reducing algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 377
21.4.3 Infimal cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
21.5 Optimal stabilizing controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 383
21.5.1 Existence of stabilizing solutions to the ARE . . . . . . . . . . . . . . 385
21.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
21.7 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
 21.8 Minimum energy controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
21.9 H2 optimal controllers* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
21.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
21.11Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
21.11.1 Proof of fact 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

22 LQG controllers 397

22.1 LQ observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
22.2 LQG controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
22.3 H2 Optimal controllers* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
22.3.1 LQ observers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
22.3.2 LQG controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

23 Systems with inputs and outputs: II* 401

23.1 Minimal realizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
23.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

24 Time-varying systems* 403

24.1 Linear time-varying systems (LTVs) . . . . . . . . . . . . . . . . . . . . . . . 403
24.1.1 Linearization about time-varying trajectories . . . . . . . . . . . . . . 403
24.2 The state transition matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
24.3 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
24.4 Systems with inputs and outputs . . . . . . . . . . . . . . . . . . . . . . . . 406
24.5 DT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406

25 Appendix A: Complex stuff 409

25.1 Complex numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
25.1.1 Complex functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
25.2 Complex vectors and Cn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
25.3 Complex matrices and Cmn . . . . . . . . . . . . . . . . . . . . . . . . . . . 412

26 Appendix B: Norms 415

26.1 The Euclidean norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

27 Appendix C: Some linear algebra 417

27.1 Linear equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
27.2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
27.3 Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
27.3.1 Span (not Spam) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
27.3.2 Linear independence . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
27.3.3 Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
27.4 Range and null space of a matrix . . . . . . . . . . . . . . . . . . . . . . . . 432
27.4.1 Null space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
27.4.2 Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
27.4.3 Solving linear equations revisited . . . . . . . . . . . . . . . . . . . . 437
27.5 Coordinate transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
 27.5.1 Coordinate transformations and linear maps . . . . . . . . . . . . . . 440
27.5.2 The structure of a linear map . . . . . . . . . . . . . . . . . . . . . . 442

28 Appendix D: Inner products 445

28.1 The usual scalar product on IRn and Cn . . . . . . . . . . . . . . . . . . . . 445
28.2 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
28.2.1 Orthonormal basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
28.3 Hermitian matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
28.4 Positive and negative (semi)definite matrices . . . . . . . . . . . . . . . . . . 450
28.4.1 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
28.4.2 Definite matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
28.4.3 Semi-definite matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 452
28.5 Singular value decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 453

xi
xii
Preface

These notes were developed to accompany the corresponding course in the Network Math-
ematics Graduate Programme at the Hamilton Institute. The purpose of the course is to
introduce some basic concepts and results in the analysis and control of linear and nonlin-
ear dynamical systems. There is a lot more material in these notes than can be covered in
twenty hours. However, the extra material should prove useful to the student who wants to
pursue a specific topic in more depth. Sections marked with an asterisk will not be covered.
Although proofs are given for most of the results presented here, they will not be covered in
the course.

xiii
xiv
 Part I

Representation of Dynamical Systems

xv
Chapter 1

Introduction

The purpose of this course is to introduce some basic concepts and tools which are useful
in the analysis and control of dynamical systems. The concept of a dynamical system is
very general; it refers to anything which evolves with time. A communication network is a
dynamical system. Vehicles (aircraft, spacecraft, motorcycles, cars) are dynamical systems.
Other engineering examples of dynamical systems include, metal cutting machines such as
lathes and milling machines, robots, chemical plants, and electrical circuits. Even civil
engineering structures such as bridges and skyscrapers are examples; think of a structure
subject to strong winds or an earthquake. The concept of a dynamical system is not restricted
to engineering; non-engineering examples include plants, animals, yourself, and the economy.
A system interacts with its environment via inputs and outputs. Inputs can be considered
to be exerted on the system by the environment whereas outputs are exerted by the system
on the environment. Inputs are usually divided into control inputs and disturbance inputs.
In an aircraft, the deflection of a control surface such as an elevator would be considered a
control input; a wind gust would be considered a disturbance input. An actuator is a physical
device used for the implementation of a control input. Some examples of actuators are: the
elevator on an aircraft; the throttle twistgrip on a motorcycle; a valve in a chemical plant.
Outputs are usually divided into performance outputs and measured outputs. Performance
outputs are those outputs whose behavior or performance you are interested in, for example,
heading and speed of an aircraft. The measured outputs are the outputs you actually mea-
sure, for example, speed of an aircraft. Usually, all the performance outputs are measured.
Sensors are the physical devices used to obtain the measured outputs. Some examples of
sensors are: altimeter and airspeed sensor on an aircraft; a pressure gauge in a chemical
plant.
A fundamental concept is describing the behavior of a dynamical system is the state of
the system. A precise definition will be given later. However, a main desirable property of
a system state is that the current state uniquely determines all future states, that is, if one

1
knows the current state of the system and all future inputs then, one can predict the future
state of the system.
Feedback is a fundamental concept in system control. In applying a control input to a
system the controller usually takes into account the behavior of the system; the controller
bases its control inputs on the measured outputs of the plant (the system under control).
Control based on feedback is called closed loop control. The term open loop control is usually
used when one applies a control input which is pre-specified function of time; hence no
feedback is involved.
The state space description of an input-output system usually involves a time variable t
and three sets of variables:
State variables: x1 , x2 , , xn
Input variables: u1 , u2 , , um
Output variables: y1 , y2 , , yp
In a continuous-time system, the time-variable t can be any real number. For discrete-time
systems, the time-variable only takes integer values, that is, . . . , 2, 1, 0, 1, 2, . . ..
For continuous-time systems, the description usually takes the following form

x1 = F1 (x1 , x2 , , xn , u1 , u2 , , um )
x2 = F2 (x1 , x2 , , xn , u1 , u2 , , um )
.. (1.1)
.
xn = Fn (x1 , x2 , , xn , u1 , u2 , , um )

and
y1 = H1 (x1 , x2 , , xn , u1 , u2 , , um )
y2 = H2 (x1 , x2 , , xn , u1 , u2 , , um )
.. (1.2)
.
yp = Hp (x1 , x2 , , xn , u1 , u2 , , um )
where the overhead dot indicates differentiation with respect to time. The first set of equa-
tions are called the state equations and the second set are called the output equations.
For discrete-time systems, the description usually takes the following form

x1 (k+1) = F1 (x1 (k), x2 (k), , xn (k), u1 (k), u2 (k), , um (k))

x2 (k+1) = F2 (x1 (k), x2 (k), , xn (k), u1 (k), u2 (k), , um (k))
.. (1.3)
.
xn (k+1) = Fn (x1 (k), x2 (k), , xn (k), u1 (k), u2 (k), , um (k))

and
y1 = H1 (x1 , x2 , , xn , u1 , u2 , , um )
y2 = H2 (x1 , x2 , , xn , u1 , u2 , , um )
.. (1.4)
.
yp = Hp (x1 , x2 , , xn , u1 , u2 , , um )

2
The first set of equations are called the state equations and the second set are called the
output equations.

3
1.1 Ingredients
Dynamical systems
Linear algebra
Applications
MATLAB (including Simulink)

1.2 Some notation

Sets : s S
ZZ represents the set of integers
IR represents the set of real numbers
C represents the set of complex numbers
Functions: f : S T

1.3 MATLAB
Introduce yourself to MATLAB

%matlab
>> lookfor
>> help
>> quit

Learn the representation, addition and multiplication of real and complex numbers

4
Chapter 2

State space representation of

dynamical systems

In this section, we consider a bunch of simple physical systems and demonstrate how one
can obtain a state space mathematical models of these systems. We begin with some linear
examples.

2.1 Linear examples

2.1.1 A first example

Figure 2.1: First example

Consider a small cart of mass m which is constrained to move along a horizontal line.
The cart is subject to viscous friction with damping coefficient c; it is also subject to an input
force which can be represented by a real scalar variable u(t) where the real scalar variable t
represents time. Let the real scalar variable v(t) represent the velocity of the cart at time t;
we will regard this as the output of the cart. Then, the motion of the cart can be described
by the following first order ordinary differential equation (ODE):
mv(t) = cv(t) + u
Introducing the state x := v results in
x = ax + bu
y = x
where a := c/m < 0 and b := 1/m.

5
2.1.2 The unattached mass
Consider a small cart of mass m which is constrained to move without friction along a
horizontal line. It is also subject to an input force which can be represented by a real scalar
variable u(t). Let q(t) be the horizontal displacement of the cart from a fixed point on its
line of motion; we will regard y = q as the output of the cart.

Figure 2.2: The unattached mass

Application of Newtons second law to the unattached mass illustrated in Figure 2.2 results
in
mq = u
Introducing the state variables,

x1 := q and x2 := q,

yields the following state space description:

x1 = x2
x2 = u/m
y = x1

2.1.3 Spring-mass-damper
Consider a system which can be modeled as a simple mechanical system consisting of a body
of mass m attached to a base via a linear spring of spring constant k and linear dashpot
with damping coefficient c. The base is subject to an acceleration u which we will regard
as the input to the system. As output y, we will consider the force transmitted to the mass
from the spring-damper combination. Letting q be the deflection of the spring, the motion
of the system can be described by

m(q + u) = cq kq mg

and y = kq cq where g is the gravitational acceleration constant. Introducing x1 := q

and x2 := q results in the following state space description:

x1 = x2
x2 = (k/m)x1 (c/m)x2 u g
y = kx1 cx2 .

6
 Figure 2.3: Spring-mass-damper with exciting base

Figure 2.4: A simple structure

2.1.4 A simple structure

Consider a structure consisting of two floors. The scalar variables q1 and q2 represent
the lateral displacement of the floors from their nominal positions. Application of Newtons
second law to each floor results in
m1 q1 + (c1 + c2 )q1 + (k1 + k2 )q1 c2 q2 k2 q2 = u1
m2 q2 c2 q1 k2 q1 + c2 q2 + k2 q2 = u2

Here u2 is a control input resulting from a force applied to the second floor and u1 is a
disturbance input resulting from a force applied to the first floor. We have not considered
any outputs here.

7
2.2 Nonlinear examples
2.2.1 A first nonlinear system
Recall the first example, and suppose now that the friction force on the cart is due to
Coulomb (or dry) friction with coefficient of friction > 0. Then we have

Figure 2.5: A first nonlinear system

mv = mg sgm (v) + u

where
1 if v < 0
sgm (v) := 0 if v = 0

1 if v > 0
and g is the gravitational acceleration constant of the planet on which the block resides.
With x := v we obtain

x = sgm (x) + bu
y = x

and := g and b = 1/m.

8
2.2.2 Planar pendulum
Consider a planar rigid body which is constrained to rotate about a horizontal axis which
is perpendicular to the body. Here the input u is a torque applied to the pendulum and the
output is the angle that the pendulum makes with a vertical line.

Figure 2.6: Simple pendulum

The motion of this system is governed by

J + W l sin = u
where J > 0 is the moment of inertia of the body about its axis of rotation, W > 0 is the
weight of the body and l is the distance between the mass center of the body and the axis
of rotation. Introducing state variables
x1 := and x2 :=
results in the following nonlinear state space description:
x1 = x2
x2 = a sin x1 + b2 u
y = x1
where a := W l/J > 0 and b2 = 1/J.

2.2.3 Attitude dynamics of a rigid body

The equations describing the rotational motion of a rigid body are given by Eulers equations
of motion. If we choose the axes of a body-fixed reference frame along the principal axes of
inertia of the rigid body with origin at the center of mass, Eulers equations of motion take
the simplified form
I1 1 = (I2 I3 )2 3
I2 2 = (I3 I1 )3 1
I3 3 = (I1 I2 )1 2

9
 Figure 2.7: Attitude dynamics of a rigid body

where 1 , 2 , 3 denote the components of the body angular velocity vector with respect
to the body principal axes, and the positive scalars I1 , I2 , I3 are the principal moments of
inertia of the body with respect to its mass center.

2.2.4 Body in central force motion

Figure 2.8: Body in central force motion

r r 2 + g(r) = 0
r + 2r =0
For the simplest situation in orbit mechanics ( a satellite orbiting YFHB)

g(r) = /r2 = GM

where G is the universal constant of gravitation and M is the mass of YFHB.

10
2.2.5 Double pendulum on cart
Consider the the double pendulum on a cart illustrated in Figure 2.9. The motion of this
system can be described by the cart displacement y and the two pendulum angles 1 , 2 .
The input u is a force applied to the cart. Application of your favorite laws of mechanics
results in the following equations of motion:

u
m0

P2
P1 1
2
m2 m1

Figure 2.9: Double pendulum on cart

(m0 + m1 + m2 )y m1 l1 cos 1 1 m2 l2 cos 2 2 + m1 l1 sin 1 12 + m2 l2 sin 2 22 = u

m1 l1 cos 1 y + m1 l12 1 + m1 l1 g sin 1 = 0
m2 l2 cos 2 y + m2 l22 2 + m2 l2 g sin 2 = 0

11
2.2.6 Two-link robotic manipulator

Payload

g^
q2

lc 2

lc 1
u2
q1

u1

Figure 2.10: A simplified model of a two link manipulator

The coordinates q1 and q2 denote the angular location of the first and second links relative
to the local vertical, respectively. The second link includes a payload located at its end. The
masses of the first and the second links are m1 and m2 , respectively. The moments of inertia
of the first and the second links about their centers of mass are I1 and I2 , respectively. The
locations of the center of mass of links one and two are determined by lc1 and lc2 , respectively;
l1 is the length of link 1. The equations of motion for the two arms are described by:

m11 q1 + m12 cos(q1 q2 )q2 + c1 sin(q1 q2 )q22 + g1 sin(q1 ) = u1

m21 cos(q1 q2 )q1 + m22 q2 + c2 sin(q1 q2 )q12 + g2 sin(q2 ) = u2

where
m11 = I1 + m1 lc21 + m2 l12 , m12 = m21 = m2 l1 lc2 , m22 = I2 + m2 lc22
g1 = (m1 lc1 + m2 l1 )g , g2 = m2 lc2 g
c1 = m2 l1 lc2 , c2 = m2 l1 lc2

12
2.3 Discrete-time examples
All the examples considered so far are continuous-time examples. Now we consider discrete-
time examples; here the time variable k is an integer, that is, , 2, 1, 0, 1, 2, . Some-
times, a discrete-time system results from the discretization of a continuous-time system (see
first example) and sometimes it arises naturally (see second example).

2.3.1 The discrete unattached mass

Here we consider a discretization of the unattached mass of Section 2.1.2 described by

mq = u .

Suppose that the input to this system is given by a zero order hold as follows:

u(t) = ud (k) for kT t < (k+1)T

where k is an integer, that is, k = . . . , 2, 1, 0, 1, 2 . . . and T > 0 is some specified time

interval; we call it the sampling time. Thus the input to the system is constant over each
sampling interval [kT, (k+1)T ).
Suppose we sample the state of the system at integer multiples of the sampling time and
let
x1 (k) = q(kT ) x2 (k) := q(kT )
for every integer k. Note that, for any k and any t with kT t (k+1)T we have
Z t Z t
1 tkT
q(t) = q(kT ) + q( ) d = x2 (k) + u(k) dt = x2 (k) + u(k) .
kT kT m m
Hence,
T
x2 (k+1) = q((k+1)T ) = x2 (k) + u(k)
m
and
Z (k+1)T Z (k+1)T
tkT
x1 (k+1) = q((k+1)T ) = q(kT ) + q(t) dt = x2 (k) + x2 (k) + u(k) dt
kT kT m
T2
= x1 (k) + T x2 (k) + u(k) .
2m
Thus, the sampled system is described by the two first order linear difference equations:

T2
x1 (k+1) = x1 (k) + T x2 (k) + 2m
u(k)
T
x2 (k+1) = x2 (k) + m u(k)

and the output equation:

y(k) = x1 (k)
This is a discrete-time state space description.

13
2.3.2 Additive increase multiplicative decrease (AIMD) algorithm
for resource allocation
Consider a resource such as a communications router which is serving n users and has a
w1
1 R
w1 e
2 s
o C
u
r
c
wn e
n

users

maximum capacity of c. Ideally, we would like the resource to be fully utilized at all times.
Each user determines how much capacity it will request without knowledge of the usage of
the other users. How does each user determine its use of the resource? One approach is
to use an additive increase multiplicative decrease (AIMD) algorithm. In this approach, each
user increases its use of the resource linearly with time (additive increase phase) until it is
notified that the resource has reached maximum capacity. It then (instantaneously) reduces
its usage to a fraction of its usage at notification (multiplicative decrease phase). It again
starts to increase its usage linearly with time.
To describe AIMD, let wi (t) 0 represent the i-th users share of the resource capacity at
time t. Each user increases its share linearly with time until the resource reaches maximum
capacity c. When maximum capacity is reached, we refer to this as a congestion event.
Suppose we number the congestion events consecutively with index k = 1, 2, and let tk
denote the time at which the k-th congestion event occurs. Then.
w1 (tk ) + w2 (tk ) + + wn (tk ) = c
Suppose that immediately after a congestion event, each user i decreases its share of the
resource to i times its share at congestion where
0 < i < 1 .
This is the multiplicative decrease phase of the algorithm. If t+
k represents a time immediately
after the k-th congestion event then,
wi (t+
k ) = i wi (tk )

Following the multiplicative decrease of its share after a congestion event, each user i increases
its share linearly in time at a rate i > 0 until congestion occurs again, that is,
wi (t) = i wi (tk ) + i (t tk )
for tk < t tk+1 . This is the additive increase phase of the algorithm.
Let Tk be the time between the k-th congestion event and congestion event k + 1. Then
Tk = tk+1 tk and
wi (tk+1 ) = i wi (tk ) + i Tk

14
Abusing notation and letting wi (k) = wi (tk ), the AIMD algorithm is described by

wi (k+1) = i wi (k) + i Tk for i = 1, , n (2.1)

where the inter-congestion time Tk is determined by the congestion condition

w1 (k+1) + + wn (k+1) = c

Using this, (2.1) and

w1 (k) + + wn (k) = c
results in
(11 )w1 (k) + + (1n )wn (k) = (1 + + n )Tk
Solving the above equation for Tk and substitution into (2.1) results in

wi (k+1) = i wi (k) + i (11 )w1 (k) + + i (1n )wn (k) (2.2)

for i = 1, , n. where
i
i := .
1 + + n
Note that
1 + + n = 1 . (2.3)
Not also that the above system is described by n first order difference equations.

15
2.4 General representation
2.4.1 Continuous-time
With the exception of the discrete-time examples, all of the preceding systems can be de-
scribed by a bunch of first order ordinary differential equations of the form

x1 = F1 (x1 , x2 , . . . , xn , u1 , u2 , . . . , um )
x2 = F2 (x1 , x2 , . . . , xn , u1 , u2 , . . . , um )
..
.
xn = Fn (x1 , x2 , . . . , xn , u1 , u2 , . . . , um )

Such a description of a dynamical system is called a state space description; the real scalar
variables, xi (t) are called the state variables; the real scalar variables, ui (t) are called the
input variables and the real scalar variable t is called the time variable. If the system has
outputs, they are described by

y1 = H1 (x1 , . . . , xn , u1 , . . . , um )
y2 = H2 (x1 , . . . , xn , u1 , . . . , um )
..
.
yp = Hp (x1 , . . . , xn , u1 , . . . , um )

where the real scalar variables, yi (t) are called the output variables
When a system has no inputs or the inputs are fixed at some constant values, the system
is described by
x1 = f1 (x1 , x2 , . . . , xn )
x2 = f2 (x1 , x2 , . . . , xn )
..
.
xn = fn (x1 , x2 , . . . , xn )

Limitations of the above description. The above description cannot handle

Systems with delays.
Systems described by partial differential equations.

16
Higher order ODE descriptions
Single equation. (Recall the spring-mass-damper system.) Consider a dynamical system
described by a single nth - order differential equation of the form

F (q, q, . . . , q (n) , u) = 0

dn q
where q(t) is a real scalar and q (n) := dtn
. To obtain an equivalent state space description,
we proceed as follows.

(a) First solve for the highest order derivative q (n) of q as a function of q, q, . . . , q (n1) and u
to obtain something like:
q (n) = a(q, q, . . . , q (n1) , u)
(b) Now introduce state variables,

x1 := q
x2 := q
..
.
xn := q (n1)

to obtain the following state space description:

x1 = x2
x2 = x3
..
.
xn1 = xn
xn = a(x1 , x2 , . . . , xn , u)

17
Multiple equations. (Recall the simple structure and the pendulum cart system.) Con-
sider a dynamical system described by N scalar differential equations in N scalar variables:

(n ) (n ) (n )
F1 (q1 , q1 , . . . , q1 1 , q2 , q2 , . . . , q2 2 , . . . , qN , qN , . . . , qN N , u1 , u2 , . . . , um ) = 0

(n1 ) (n2 ) (n )
F2 (q1 , q1 , . . . , q1 , q2 , q2 , . . . , q2 , . . . , qN , qN , . . . , qN N , u1 , u2 , . . . , um ) = 0

..
.
(n1 ) (n2 ) (nN )
FN (q1 , q1 , . . . , q1 , q2 , q2 , . . . , q2 , . . . , qN , qN , . . . , qN , u1 , u2 , . . . , um ) = 0

(n )
where t, q1 (t), q2 (t), . . . , qN (t) are real scalars. Note that qi i is the highest order derivative
of qi which appears in the above equations. To obtain an equivalent state space description,
we proceed as follows.

(n ) (n ) (n )
(a) Solve for the highest order derivatives, q1 1 , q2 2 , . . . , qN N of q1 , , qN which appear
in the differential equations to obtain something like:

(n1 ) (n11) (n21) (n 1)

q1 = a1 ( q1 , q1 , . . . q1 , q2 , q2 , . . . , q2 , . . . , qN , qN , . . . , qN N , u1 , u 2 , . . . , u m )
(n ) (n 1) (n 1) (n 1)
q2 2 = a2 ( q1 , q1 , . . . q1 1 , q2 , q2 , . . . , q2 2 ,
..., qN , qN , . . . , qN N ,
u1 , u 2 , . . . , u m )
..
.
(n ) (n 1) (n 1) (n 1)
qN N = aN ( q1 , q1 , . . . q1 1 , q2 , q2 , . . . , q2 2 , . . . , qN , qN , . . . , qN N , u1 , u2 , . . . , um )

If one cannot uniquely solve for the above derivatives, then one cannot obtain a unique state
space description

(b) As state variables, consider each qi variable and its derivatives up to but not including
its highest order derivative. One way to do this is as follows. Let

(n 1)
x1 := q1 x2 := q1 ... xn1 := q1 1
(n 1)
xn1 +1 := q2 , xn1 +2 := q2 ... xn1 +n2 := q2 2
..
.
(n 1)
xn1 +...+nN1 +1 := qN xn1 +...+nN1 +2 := qN ... xn := qN N

where

n := n1 + n2 + . . . + nN

18
to obtain
x1 = x2
x2 = x3
..
.
xn11 = xn1
xn1 = a1 (x1 , x2 , . . . , xn , u1 , u2 , . . . , um )
xn1 +1 = xn1 +2
..
.
xn1 +n2 = a2 (x1 , x2 , . . . , xn , u1 , u2 , . . . , um )
..
.
xn = aN (x1 , x2 , . . . , xn , u1 , u2 , . . . , um )

Example 1
q1 + q2 + 2q1 = 0
q1 + q1 + q2 + 4q2 = 0
The highest order derivatives of q1 and q2 appearing in these equations are q1 and q2 , respec-
tively.
Solving for q1 and q2 , we obtain
1
q1 = q1 + q1 + 2q2
2
1
q2 = q1 q1 2q2 .
2
Introducing state variables x1 := q1 , x2 := q1 , and x3 = q2 we obtain the following state
space description:
x1 = x2
1
x2 = x1 + x2 + 2x3
2
1
x3 = x1 x2 2x3
2

2.4.2 Discrete-time
Recall the AIMD algorithm of Section 2.3.2. The general discrete-time system is described
by a bunch of first order difference equations of the form:
x1 (k + 1) = F1 (x1 (k), x2 (k), . . . , xn (k), u1 (k), u2 (k), . . . , um (k))
x2 (k + 1) = F2 (x1 (k), x2 (k), . . . , xn (k), u1 (k), u2 (k), . . . , um (k))
..
.
xn (k + 1) = Fn (x1 (k), x2 (k), . . . , xn (k), u1 (k), u2 (k), . . . , um (k))

19
The real scalar variables, xi (k), i = 1, 2, . . . , n are called the state variables; the real scalar
variables, ui (k), i = 1, 2, . . . , m are called the input variables and the integer variable k is
called the time variable. If the system has outputs, they are described by

y1 = H1 (x1 , . . . , xn , u1 , . . . , um )
y2 = H2 (x1 , . . . , xn , u1 , . . . , um )
..
.
yp = Hp (x1 , . . . , xn , u1 , . . . , um )

where the real scalar variables, yi (k), i = 1, 2, . . . , p are called the output variables
When a system has no inputs or the inputs are fixed at some constant values, the system
is described by
x1 (k + 1) = f1 (x1 (k), x2 (k), . . . , xn (k))
x2 (k + 1) = f2 (x1 (k), x2 (k), . . . , xn (k))
..
.
xn (k + 1) = fn (x1 (k), x2 (k), . . . , xn (k))

Higher order difference equation descriptions

One converts higher order difference equation descriptions into state space descriptions in a
similar manner to that of converting higher order ODE descriptions into state space descrip-
tions; see Example 2.3.1.

20
2.4.3 Exercises
Exercise 1 Consider a system described by a single nth -order linear differential equation of
the form
q (n) + an1 q (n1) + . . . a1 q + a0 q = 0
n
where q(t) IR and q (n) := ddtnq . By appropriate definition of state variables, obtain a first
order state space description of this system.

Exercise 2 By appropriate definition of state variables, obtain a first order state space
description of the following systems.

(i) The simple structure

(ii) The body in central force motion

(iii)

q1 + q1 + 2q2 = 0
q1 + q2 + q2 = 0

where q1 , q2 are real scalars.

Exercise 3 By appropriate definition of state variables, obtain a first order state space
description of the following systems where q1 and q2 are real scalars.

(i)

2q1 + q2 + sin q1 = 0
q1 + 2q2 + sin q2 = 0

(ii)

q1 + q2 + q13 = 0
q1 + q2 + q23 = 0

Exercise 4 By appropriate definition of state variables, obtain a first order state space
description of the following systems where q1 , q2 are real scalars.

(i)

2q1 + 3q2 + q1 q2 = 0
q1 + 2q2 q2 + q1 = 0

21
 (ii)
...
q2 +2q1 + q2 q1 = 0
q2 + q1 q2 + q1 = 0

Exercise 5 By appropriate definition of state variables, obtain a first order state space
description of the following systems where q1 , q2 are real scalars.

(i)

3q1 q2 + 2q1 + 4q2 = 0

q1 + 3q2 + 2q1 4q2 = 0

(ii)

3q1 q2 + 4q2 4q1 = 0

q1 + 3q2 4q2 + 4q1 = 0

Exercise 6 Obtain a state-space description of the following single-input single-output sys-

tem with input u and output y.

q1 + q2 + q1 = 0
q1 q2 + q2 = u
y = q1

Exercise 7 Consider the second order difference equation

q(k+2) = q(k+1) + q(k)

Obtain the solution corresponding to

q(0) = 1, q(1) = 1

and k = 0, 1, . . . , 10. Obtain a state space description of this system.

Exercise 8 Obtain a state space representation of the following system:

q(k+n) + an1 q(k+n1) + . . . + a1 q(k+1) + a0 q(k) = 0

where q(k) IR.

22
2.5 Vectors
When dealing with systems containing many variables, the introduction of vectors for system
description can considerably simplify system analysis and control design.

2.5.1 Vector spaces and IRn

A scalar is a real or a complex number. The symbols IR and C represent the set of real
and complex numbers, respectively. In this section, all the definitions and results are given
for real scalars. However, they also hold for complex scalars; to get the results for complex
scalars, simply replace real with complex and IR with C.
Consider any positive integer n. A real n-vector x is an ordered n-tuple of real numbers,
x1 , x2 , . . . , xn . This is usually written as x = (x1 , x2 , , xn ) or

x1 x1
x2 x2

x = .. or x = ..
. .
xn xn
The real numbers x1 , x2 , . . . , xn are called the scalar components of x and xi is called the
i-th component. The symbol IRn represents the set of ordered n-tuples of real numbers.

Addition
The addition of any two real n-vectors x and y yields another real n-vector x + y which is
defined by:
x1 y1 x1 + y 1
x2 y2 x2 + y 2

x + y = .. + .. := ..
. . .
xn yn xn + y n
Zero element of IRn :

0
0

0 := ..
.
0
Note that we are using the same symbol, 0, for a zero scalar and a zero vector.

The negative of a vector:

x1
x2

x := ..
.
xn

23
 Properties of addition

(a) (Commutative). For each pair x, y in IRn ,

x+y =y+x

(b) (Associative). For each x, y, z in IRn ,

(x + y) + z = x + (y + z)

(c) There is an element 0 in IRn such that for every x in IRn ,

x+0=x

(d) For each x in IRn , there is an element x in IRn such that

x + (x) = 0

Scalar multiplication.
The multiplication of an element x of IRn by a real scalar yields an element of IRn and is
defined by:
x1 x1
x2 x2

x = .. := ..
. .
xn xn

Properties of scalar multiplication

(a) For each scalar and pair x, y in IRn

(x + y) = x + y

(b) For each pair of scalars , and x in IRn ,

( + )x = x + x

(c) For each pair of scalars , , and x in IRn ,

(x) = ()x

(d) For each x in IRn ,

1x = x

24
Vector space
Consider any set V equipped with an addition operation and a scalar multiplication oper-
ation. Suppose the addition operation assigns to each pair of elements x, y in V a unique
element x + y in V and it satisfies the above four properties of addition (with IRn replaced
by V). Suppose the scalar multiplication operation assigns to each scalar and element x in
V a unique element x in V and it satisfies the above four properties of scalar multiplication
(with IRn replaced by V). Then this set (along with its addition and scalar multiplication)
is called a vector space. Thus IRn equipped with its definitions of addition and scalar mul-
tiplication is a specific example of a vector space. We shall meet other examples of vectors
spaces later. An element x of a vector space is called a vector. A vector space with real
(complex) scalars is called a real (complex) vector space.
As a more abstract example of a vector space, let V be the set of continuous real-valued
functions which are defined on the interval [0, ); thus, an element x of V is a function
which maps [0, ) into IR, that is, x : [0, ) IR. Addition and scalar multiplication are
defined as follows. Suppose x and y are any two elements of V and is any scalar, then

(x + y)(t) = x(t) + y(t)

(x)(t) = x(t)

Subtraction in a vector space is defined by:

x y := x + (y)

Hence in IRn ,

x1 y1 x1 y1
x2 y2 x2 y2

xy = .. .. = ..
. . .
xn yn xn yn

2.5.2 IR2 and pictures

An element of IR2 can be represented in a plane by a point or a directed line segment.

25
2.5.3 Derivatives
Suppose x() is a function of a real variable t where x(t) is an n-vector. Then
dx
dt
1
x 1


dx2 x2
dx dt

x := := =
dt .. ..
. .


dxn
dt
xn

2.5.4 MATLAB
Representation of real and complex vectors
Addition and substraction of vectors
Multiplication of a vector by a scalar

26
2.6 Vector representation of dynamical systems
Recall the general descriptions (continuous and discrete) of dynamical systems given in
Section 2.4. We define the state (vector) x as the vector with components, x1 , x2 , , xn ,
that is,
x1
x2

x := .. .
.
xn
If the system has inputs, we define the input (vector) u as the vector with components,
u1 , u2 , , um , that is,
u1
u2

u := .. .
.
um
If the system has outputs, we define the output (vector) y as the vector with components,
y1 , y2 , , yp , that is,

y1
y2

y := .. .
.
yp
We introduce the vector valued functions F and H defined by

F1 (x1 , x2 , . . . , xn , u1 , u2 , , um )
F2 (x1 , x2 , . . . , xn , u1 , u2 , , um )

F (x, u) := ..
.
Fn (x1 , x2 , . . . , xn , u1 , u2 , , um )

and
H1 (x1 , x2 , . . . , xn , u1 , u2 , , um )
H2 (x1 , x2 , . . . , xn , u1 , u2 , , um )

H(x, u) := ..
.
Hp (x1 , x2 , . . . , xn , u1 , u2 , , um )
respectively.

Continuous-time systems. The general representation of a continuous-time dynamical

system can be compactly described by the following equations:

x = F (x, u)
(2.4)
y = H(x, u)

27
where x(t) is an n-vector, u(t) is an m-vector, y(t) is a p-vector and the real variable t is
the time variable. The first equation above is a first order vector differential equation and is
called the state equation. The second equation is called the output equation.
When a system has no inputs of the input vector vector is constant, it can be described
by
x = f (x) .
A system described by the above equations is called autonomous or time-invariant because
the right-hand sides of the equations do not depend explicitly on time t. For the first part
of the course, we will only concern ourselves with these systems.
However, one can have a system containing time-varying parameters. In this case the
system might be described by

x = F (t, x, u)
y = H(t, x, u)

that is, the right-hand sides of the differential equation and/or the output equation depend
explicitly on time. Such a system is called non-autonomous or time-varying. We will look at
them later.

Other systems described by higher order differential equations Consider a system

described by

AN y (N ) + AN 1 y (N 1) + + A0 y = BM u(M ) + BM 1 u(M 1) + + B0 u

where y(t) is a p-vector, u(t) is an m-vector, N M , and the matrix AN is invertible. When
M = 0, we can obtain a state space description with

y

x = ...
y (N1)

When M 1, we will see later how to put such a system into the standard state space form
(2.4).

Discrete-time systems. The general representation of a continuous-time dynamical sys-

tem can be compactly described by the following equations:

x(k + 1) = F (x(k), u(k))

(2.5)
y(k) = H(x(k), u(k))

where x(k) is an n-vector, u(k) is an m-vector, y(k) is a p-vector and the integer variable k
is the time variable. The first equation above is a first order vector difference equation and
is called the state equation. The second equation is called the output equation. A system
described by the above equations is called autonomous or time-invariant because the right-
hand sides of the equations do not depend explicitly on time k.

28
 However, one can have a system containing time-varying parameters. In this case the
system might be described by

x(k + 1) = F (k, x(k), u(k))

y(k) = H(k, x(k), u(k))

that is, the right-hand sides of the differential equation and/or the output equation depend
explicitly on time. Such a system is called non-autonomous or time-varying.

29
2.7 Solutions and equilibrium states : continuous-time
2.7.1 Equilibrium states
Consider a system described by
x = f (x) (2.6)
A solution or motion of this system is any continuous function x() satisfying x(t) = f (x(t))
for all t.
An equilibrium solution is the simplest type of solution; it is constant for all time, that is,
it satisfies
x(t) xe
for some fixed state vector xe . The state xe is called an equilibrium state. Since an equilibrium
solution must satisfy the above differential equation, all equilibrium states must satisfy the
equilibrium condition:
f (xe ) = 0
or, in scalar terms,
f1 (xe1 , xe2 , . . . , xen ) = 0
f2 (xe1 , xe2 , . . . , xen ) = 0
..
.
fn (xe1 , xe2 , . . . , xen ) = 0
Conversely, if a state xe satisfies the above equilibrium condition, then there is a solution
satisfying x(t) xe ; hence xe is an equilibrium state.
Example 2 Spring mass damper. With u = 0 this system is described by
x1 = x2
x2 = (k/m)x1 (c/m)x2 g .
Hence equilibrium states are given by:
xe2 = 0
(k/m)xe1 (c/m)xe2 g = 0
Hence
xe = 0 = mg/k//0
This system has a single equilibrium state.
Example 3 The unattached mass. With u = 0 this system is described by
x1 = x2
x2 = 0 .
Hence
e xe1
x =
0
where xe1 is arbitrary. This system has an infinite number of equilibrium states.

30
Example 4 Pendulum. With u = 0, this system is described by
x1 = x2
x2 = a sin x1 .
The equilibrium condition yields:
xe2 = 0 and sin(xe1 ) = 0 .
Hence, all equilibrium states are of the form

e m
x =
0
where m is an arbitrary integer. Physically, there are only two distinct equilibrium states

e 0 e
x = and x =
0 0

Higher order ODEs

F (q, q, . . . , q (n) ) = 0
An equilibrium solution is the simplest type of solution; it is constant for all time, that is, it
satisfies
q(t) q e
for some fixed scalar q e . Clearly, q e must satisfy
F (q e , 0, . . . , 0) = 0 (2.7)
For the state space description of this system introduced earlier, all equilibrium states
are given by
qe
0

xe = ..
.
0
where q e solves (2.7).

Multiple higher order ODEs

Equilibrium solutions
qi (t) qie , i = 1, 2, . . . , N
Hence
e
F1 (q1e , 0, . . . , q2e , 0, . . . , . . . , qN ,...,0 ) = 0

e
F2 (q1e , 0, . . . , q2e , 0, . . . , . . . , qN ,...,0 ) = 0
(2.8)
..
.
e e e
FN (q1 , 0, . . . , q2 , 0, . . . , . . . , qN , . . . , 0 ) = 0

31
 For the state space description of this system introduced earlier, all equilibrium states
are given by e
q1
0

..
.
e
q2

0
e
x = ..
.
.
..

qe
N
..
.
0
where q1e , q2e , . . . , qN
e
solve (2.8).

Example 5 Central force motion in inverse square gravitational field

r r 2 + /r2 = 0
r + 2r =0

Equilibrium solutions
r(t) re , (t) e
Hence,
r, r, = 0
This yields
re ( e )2 + /(re )2 = 0
0 =0
Thus there are infinite number of equilibrium solutions given by:
p
e = /(re )3

where re is arbitrary. Note that, for this state space description, an equilibrium state corre-
sponds to a circular orbit.

2.7.2 Exercises
Exercise 9 Find all equilibrium states of the following systems

(i) The first nonlinear system.

(ii) The attitude dynamics system.

(iii) The two link manipulator.

32
2.7.3 Controlled equilibrium states
Consider now a system with inputs described by
x = F (x, u) (2.9)
Suppose the input is constant and equal to ue , that is, u(t) ue . Then the resulting system
is described by
x(t) = F (x(t), ue )
The equilibrium states xe of this system are given by F (xe , ue ) = 0. This leads to the
following definition.
A state xe is a controlled equilibrium state of the system, x = F (x, u), if there is a constant
input ue such that
F (xe , ue ) = 0

Example 6 (Controlled pendulum)

x1 = x2
x2 = sin(x1 ) + u
Any state of the form
e xe1
x =
0
is a controlled equilibrium state. The corresponding constant input is ue = sin(xe1 ).

2.8 Solutions and equilibrium states: discrete-time

2.8.1 Equilibrium states
Consider a discrete-time system described by
x(k + 1) = f (x(k)) . (2.10)
A solution of (2.10) is a sequence

x(0), x(1), x(2), . . .
which satisfies (2.5).
Example 7 Simple scalar nonlinear system
x(k + 1) = x(k)2
Sample solutions are:
(0, 0, 0, . . .)
(1, 1, 1, . . .)
(2, 4, 16, . . . )

33
Equilibrium solutions and states

x(k) xe

f (xe ) = xe
In scalar terms,

f1 (xe1 , xe2 , . . . , xen ) = xe1

f2 (xe1 , xe2 , . . . , xen ) = xe2
..
.
fn (x1 , x2 , . . . , xn ) = xem
e e e

Example 8 Simple scalar nonlinear system

x(k + 1) = x(k)2

All equilibrium states are given by:

(xe )2 = xe
Solving yields
xe = 0, 1

Example 9 The discrete unattached mass

xe1 + T xe2 = xe1

xe2 = xe2
e
e x1
x =
0
where xe1 is arbitrary.

Example 10 (AIMD algoritm) Recalling (2.2) we see that an equilibrium share vector
w must satisfy

wi = i wi + i (11 )w1 + + i (1n )wn for i = 1, , n . (2.11)

Letting
(1i )w1
i =
i
above equations simplify to

i = 1 1 + + n n for i = 1, , n .

Thus every solution must satisfy

i = K for i = 1, , n .

34
for some K. Since 1 + + n = 1, it should be K can be any real number. Hence
i K
wi = for i = 1, , n .
1i
Considering w1 + + wn = c, the constant K is uniquely given by
c
K= 1 n .
11
+ + 1n

2.8.2 Controlled equilibrium states

Consider now a system with inputs described by

x(k+1) = F (x(k), u(k)) (2.12)

Suppose the input is constant and equal to ue , that is, u(k) ue . Then the resulting system
is described by
x(k+1) = F (x(k), ue )
The equilibrium states xe of this system are given by xe = F (xe , ue ). This leads to the
following definition.
A state xe is a controlled equilibrium state of the system, x(k+1) = F (x(k), u(k)), if there
is a constant input ue such that
F (xe , ue ) = xe

35
2.9 Numerical simulation
2.9.1 MATLAB
>> help ode23

ODE23 Solve differential equations, low order method.

ODE23 integrates a system of ordinary differential equations using
2nd and 3rd order Runge-Kutta formulas.
[T,Y] = ODE23(yprime, T0, Tfinal, Y0) integrates the system of
ordinary differential equations described by the M-file YPRIME.M,
over the interval T0 to Tfinal, with initial conditions Y0.
[T, Y] = ODE23(F, T0, Tfinal, Y0, TOL, 1) uses tolerance TOL
and displays status while the integration proceeds.

INPUT:
F - String containing name of user-supplied problem description.
Call: yprime = fun(t,y) where F = fun.
t - Time (scalar).
y - Solution column-vector.
yprime - Returned derivative column-vector; yprime(i) = dy(i)/dt.
t0 - Initial value of t.
tfinal- Final value of t.
y0 - Initial value column-vector.
tol - The desired accuracy. (Default: tol = 1.e-3).
trace - If nonzero, each step is printed. (Default: trace = 0).

OUTPUT:
T - Returned integration time points (column-vector).
Y - Returned solution, one solution column-vector per tout-value.

The result can be displayed by: plot(tout, yout).

See also ODE45, ODEDEMO.

36
>> help ode45

ODE45 Solve differential equations, higher order method.

ODE45 integrates a system of ordinary differential equations using
4th and 5th order Runge-Kutta formulas.
[T,Y] = ODE45(yprime, T0, Tfinal, Y0) integrates the system of
ordinary differential equations described by the M-file YPRIME.M,
over the interval T0 to Tfinal, with initial conditions Y0.
[T, Y] = ODE45(F, T0, Tfinal, Y0, TOL, 1) uses tolerance TOL
and displays status while the integration proceeds.

INPUT:
F - String containing name of user-supplied problem description.
Call: yprime = fun(t,y) where F = fun.
t - Time (scalar).
y - Solution column-vector.
yprime - Returned derivative column-vector; yprime(i) = dy(i)/dt.
t0 - Initial value of t.
tfinal- Final value of t.
y0 - Initial value column-vector.
tol - The desired accuracy. (Default: tol = 1.e-6).
trace - If nonzero, each step is printed. (Default: trace = 0).

OUTPUT:
T - Returned integration time points (column-vector).
Y - Returned solution, one solution column-vector per tout-value.

The result can be displayed by: plot(tout, yout).

See also ODE23, ODEDEMO.

2.9.2 Simulink
Simulink is part of Matlab. In Simulink, one describes a system graphically. If one has a
Simulink model of a system there are several useful Matlab operations which can be applied
to the system, for example, one may readily find the equilibrium state of the system.

37
2.10 Exercises
Exercise 10 Obtain a state-space description of the following system.
...
q 1 + q2 + q1 = 0
q1 q2 + q2 = 0
Exercise 11 Recall the two link manipulator example in the notes.
(i) By appropriate definition of state variables, obtain a first order state space description
of this system.
(ii) Find all equilibrium states.
(iii) Numerically simulate this system using MATLAB. Use the following data and initial
conditions:
m1 l1 lc1 I1 m2 l2 lc2 I2 mpayload
kg m m kg.m2 kg m m kg.m2 kg
10 1 0.5 10/12 5 1 0.5 5/12 0

q1 (0) = ; q1 (0) = 0
2

q2 (0) = ; q2 (0) = 0
4
Exercise 12 Recall the two pendulum cart example in the notes.
(a) Obtain all equilibrium configurations corresponding to u = 0.
(b) Consider the following parameter sets
m0 m1 m2 l1 l2 g
P1 2 1 1 1 1 1
P2 2 1 1 1 0.99 1
P3 2 1 0.5 1 1 1
P4 2 1 1 1 0.5 1
and initial conditions,

y 1 2 y 1 2
IC1 0 10 10 0 0 0
IC2 0 10 10 0 0 0
IC3 0 90 90 0 0 0
IC4 0 90.01 90 0 0 0
IC5 0 100 100 0 0 0
IC6 0 100.01 100 0 0 0
IC7 0 179.99 0 0 0 0

38
Simulate the system using the following combinations:

P1 : IC1, IC2, IC3, IC4, IC5, IC6, IC7

P2 : IC1, IC2, IC3, IC5.IC7
P3 : IC1, IC2, IC5, IC7
P4 : IC1, IC2, IC3, IC4, IC5, IC6, IC7

39
40
Chapter 3

Linear time-invariant (LTI) systems

To discuss linear systems, we need matrices. In this section we briefly review some properties
of matrices. All the definitions and results of this section are stated for real scalars and
matrices. However, they also hold for complex scalars and matrices; to obtain the results for
the complex case, simply replace real with complex and IR with C.

3.1 Matrices
An m n matrix A is an array of scalars consisting of m rows and n columns.

a11 a12 . . . a1n

a21 a22 . . . a2n


A= .. .. .. m rows
. . .



am1 am2 . . . amn
| {z }
n columns
The scalars aij are called the elements of A. If the scalars are real numbers, A is called a
real matrix. The set of real m n matrices is denoted by IRmn
Recall that we can do the following things with matrices.

Matrix addition: A + B

Multiplication of a matrix by a scalar: A

Zero matrices: 0

Negative of the matrix A: A

One can readily show that IRmn with the usual definitions of addition and scalar mul-
tiplication is a vector space.

Matrix multiplication: AB

41
 Some properties:

(AB)C = A(BC) (associative)

In general AB 6= BA (non-commutative)

A(B + C) = AB + AC
(B + C)A + BA + CA

A(B) = AB
(A)B = AB

Identity matrix in IRnn :

1 0 ... 0
0 1 ... 0

I := .... . . .. = diag (1, 1, . . . , 1)
. . . .
0 0 ... 1

AI = IA = A

Inverse of a square (n = m) matrix A: A1

AA1 = A1 A = I

Some properties:

1 1
(A)1 = A

(AB)1 = B 1 A1

Transpose of A: AT
Some properties:

(A + B)T = AT + B T
(A)T = AT

(AB)T = B T AT

Determinant of a square matrix A : Associated with any square matrix A is a scalar called
the determinant of A and is denoted by det A.

42
 Some properties:

det(AB) = det(A) det(B)

det(AT ) = det(A)

Note that, in general,

det(A + B) =
6 det(A) + det(B)
det(A) = 6 det(A)

The following fact provides a very useful property of determinants.

Fact: A square matrix is invertible iff its determinant is non-zero.

Exercise 13 Compute the determinant and inverse of the following matrix.

cos sin
sin cos

Exercise 14 Prove that if A is invertible then,

1
det(A1 ) =
det(A)

MATLAB
Representation of real and complex matrices
Addition and multiplication of matrices
Multiplication of a matrix by a scalar
Matrix powers
Transpose of a matrix
Inverse of a matrix

>> help zeros

ZEROS All zeros.

ZEROS(N) is an N-by-N matrix of zeros.
ZEROS(M,N) or ZEROS([M,N]) is an M-by-N matrix of zeros.
ZEROS(SIZE(A)) is the same size as A and all zeros.

>> help eye

EYE Identity matrix.

EYE(N) is the N-by-N identity matrix.
EYE(M,N) or EYE([M,N]) is an M-by-N matrix with 1s on
the diagonal and zeros elsewhere.

43
 EYE(SIZE(A)) is the same size as A.

>> help det

DET Determinant.
DET(X) is the determinant of the square matrix X.

>> help inv

INV Matrix inverse.

INV(X) is the inverse of the square matrix X.
A warning message is printed if X is badly scaled or
nearly singular.

Exercise 15 Determine which of the following matrices are invertible. Obtain the inverses
of those which are. Check your answers using MATLAB.

1 1 1 1 0 1 0 1
(a) (b) (c) (d)
0 1 1 1 0 0 1 0

Exercise 16 Determine which of the following matrices are invertible. Using MATLAB,
check your answers and determine the inverse of those that are invertible.

1 1 0 1 0 1 0 1 0
(a) 0 0 1 (b) 1 0 1 (c) 1 0 1
1 0 0 1 0 1 0 1 0

Powers and polynomials of a matrix

Suppose A is a square matrix. We can define powers of a matrix as follows:

A0 := I
A1 := A
A2 := AA
..
.
Ak+1 := AAk

We can also define polynomials of a matrix as follows. If

p(s) = a0 + a1 s + a2 s2 + . . . + am sm

where a0 , . . . , am are scalars, then,

p(A) := a0 I + a1 A + a2 A2 + . . . + am Am

44
Exercise 17 Compute p(A) for the following polynomial-matrix pairs. Check your answers
using MATLAB.

(a)
1 1 1
p(s) = s3 3s2 + 3s 1 A= 0 1 1
0 0 1

(b)
1 1 1
p(s) = s2 3s A= 1 1 1
1 1 1

(c)
1 1 0
p(s) = s3 + s2 + s + 1 A= 1 1 1
0 1 1

(d)
0 1 0
p(s) = s3 2s A= 1 0 1
0 1 0

45
3.2 Linear time-invariant systems
3.2.1 Continuous-time
A continuous-time, linear time-invariant (LTI) input-output system can be described by

x = Ax + Bu
y = Cx + Du

where the n-vector x(t) is the state vector at time t, the m-vector u(t) is the input vector at
time t, and the p-vector y(t) is the output vector at time t. The n n matrix A is sometimes
called the system matrix. The matrices B, C, and D have dimensions n m, p n, and
p m, respectively. In scalar terms, this system is described by:
x1 = a11 x1 + . . . + a1n xn + b11 u1 + . . . + b1m um
..
.
xn = an1 x1 + . . . + ann xn + bn1 u1 + . . . + bnm um
and
y1 = c11 x1 + . . . + c1n xn + d11 u1 + . . . + d1m um
..
.
yp = cp1 x1 + . . . + cpn xn + dp1 u1 + . . . + dpm um .

A system with no inputs is described by

x = Ax . (3.1)
Example 11 The unattached mass

0 1 0
A= , B= , C= 1 0 , D = 0.
0 0 1/m
Example 12 Spring-mass-damper

0 1 0
A= , B= , C= k c , D = 0.
k/m c/m 1

Other systems. It should be mentioned that the results of this section and most of these
notes only apply to finite-dimensional systems, that is, systems whose state space is finite-
dimensional. As an example of a system not included here, consider a system with a time
delay T described by
q(t) = A0 q(t) + A1 q(tT ) + Bu(T )
y(t) = Cq(t) + Du(t)
As another example, consider any system described by a partial differtial equation, for
example
2q 4
2 q
+ k = 0.
t2 4

46
3.2.2 Discrete-time
A discrete-time linear time-invariant (LTI) input-output system can be described by

x(k+1) = Ax(k) + Bu(k)

y(k) = Cx(k) + Du(k)

where the n-vector x(k) is the state vector at time k, the m-vector u(k) is the input vector
at time k, and the p-vector y(k) is the output vector at time k. The n n matrix A is
sometimes called the system matrix. The matrices B, C, and D have dimensions n m,
p n, and p m, respectively.
In scalar terms:

x1 (k+1) = a11 x1 (k) + . . . + a1n xn (k) + b11 u1 (k) + . . . + b1m um (k)

..
.
xn (k+1) = an1 x1 (k) + . . . + ann xn (k) + bn1 u1 (k) + . . . + bnm um (k)

and

y1 = c11 x1 + . . . + c1n xn + d11 u1 + . . . + d1m um

..
.
yp = cp1 x1 + . . . + cpn xn + dp1 u1 + . . . + dpm um

A system with no inputs is described by

x(k+1) = Ax(k) . (3.2)

Example 13 The discrete unattached mass

2
1 T T /2m
A= , B= , C= 1 0 , D = 0.
0 1 T /m

Example 14 AIMD This is a system with no inputs and

1 + 1 (1 1 ) 1 (1 2 ) 1 (1 n )
2 (1 1 ) 2 + 2 (1 2 ) 2 (1 n )

A= .. .. ..
. . .
n (1 1 ) n (1 2 ) n + n (1 n )

47
3.3 Linearization about an equilibrium solution
In this section we present systematic procedures for approximating a nonlinear system by a
linear system. These procedures rely heavily on derivatives.

3.3.1 Derivative as a matrix

Consider an m-vector-valued function f of an n-vector variable x. Suppose that each of the
fi
partial derivatives, x j
(x ) exist and are continuous about some x .

The derivative of f (Jacobian of f ) at x is the following m n matrix:

f1 f1 f1
x1
(x ) x 2
(x ) . . . x n
(x )

f
2
(x ) x2 (x ) . . . xn (x )
f 2 f 2
f x1

Df (x ) = (x ) := .
x .. .. ..
. . .


fm fm fm
x1
(x ) x2 (x ) . . . xn (x )
So,
fi
Df (x )ij = (x ) ,
xj
that is, the ij-th element of Df is the partial derivative of fi with respect to xj . Sometimes
Df is written as fx
.
Example 15 If
x1 x2
f (x) = cos(x1 ) .
ex2
then
x2 x1
Df (x) = sin(x1 ) 0 .
0 ex2

Some properties:

D(f + g) = Df + Dg
D(f ) = Df ( is a constant scalar)
(Constant function:) f (x) c Df (x) = 0
(Linear function:) f (x) = Ax Df (x) = A
All linearization is based on the following fundamental property of the derivative. If x
and x are any two vectors and x is close to x , then
f
f (x) f (x ) + (x )(x x )
x

48
This says that, near x , the function can be approximated by a linear function plus a constant
term.

3.3.2 Linearization of continuous-time systems

Consider a continuous-time system described by

x = F (x, u)
y = H(x, u) (3.3)

and suppose xe is an equilibrium state corresponding to a constant input ue ; hence,

F (xe , ue ) = 0 .

Let y e be the corresponding constant output, that is,

y e = H(xe , ue ) .

Sometimes we call the triple (xe , ue , y e ) a trim condition of the system. Introducing the
perturbed state x, the perturbed input u and the perturbed output y defined by

x := x xe , u := u ue , x := u ue ,

the system can also be described by

x = F (xe +x, ue +u)

y = H(xe +x, ue +u) H(xe , ue )

49
Define now the following derivative matrices:
F1 F1 F1

x1
() x2
() ... xn
()


F2
() F2
() . . . F2
()
F e e Fi x1 x2 xn

(x , u ) := () =
x xj .. .. ..
. . .


Fn Fn Fn
x1
() x2 () . . . xn ()

F1 F1 F1

u1
() u2
() ... um
()


F2
() F2
() ... F2
()
F e e Fi u1 u2 um

(x , u ) := () =
u uj .. .. ..
. . .


Fn Fn Fn
u1
() u2
() ... um
()
H1 H1 H1
x1
() x2
() ... xn
()


H2
() H2
() ... H2
()
H e e Hi x1 x2 xn

(x , u ) := () =
x xj .. .. ..
. . .


Hp Hp Hp
x1
() x2
() ... xn
()

H1 H1 H1
u1
() u2
() ... um
()


H2
() H2
() ... H2
()
H e e Hi u1 u2 um

(x , u ) := () =
u uj .. .. ..
. . .


Hp Hp Hp
u1
() u2
() ... um
()

where = (xe , ue ). When x is close to xe and u is close to ue , that is, when x and u are
small,

F e e F e e
F (xe +x, ue +u) F (xe , ue ) + (x , u )x + (x , u )u
x u
F e e F e e
= (x , u )x + (x , u )u
x u

50
and
H e e H e e
H(xe +x, ue +u) H(xe , ue ) + (x , u )x + (x , u )u
x u
Hence

F e e F e e
x (x , u )x + (x , u )u
x u
H e e H e e
y (x , u )x + (x , u )u
x u

This leads to the following definition:

The linearization of system (3.3) about (xe , ue ):

x = A x + B u
(3.4)
y = C x + D u

where

F e e F e e H e e H e e
A= (x , u ) , B= (x , u ) , C= (x , u ) , D= (x , u ) .
x u x u

Example 16 (Simple pendulum.) The linearization of this system about any (xe , ue ) is
given by (3.4) with

0 1 0
A= e B= C= 1 0 D=0
cos(x1 ) 0 1

For eqm. state

e 0
x =
0
we obtain
0 1
A=
1 0

For eqm. state

e
x =
0
we obtain
0 1
A=
1 0

51
3.3.3 Implicit linearization
This is the way to go when starting from a higher order ODE description.

It is usually easier to first linearize and then obtain a state space description than vice-versa.

For a single equation,

F (q, q, . . . , q (n) , u) = 0
linearization about an equilibrium solution q(t) q e corresponding to constant input u(t)
ue is defined by:
F F F F
()q + () q + ... + ()q (n) + ()u = 0
q q q (n) u

where () = (q e , 0, 0, . . . , 0), q = q q e and u = u ue .

For multiple equations, see next example.

Example 17 Orbit mechanics

r r 2 + /r2 = 0
r + 2r = 0

Linearization about equilibrium solutions corresponding to constant values re , e of r and

, respectively, results in:

r ( e )2 r (2re e ) (2/(re )3 )r = 0
e r + re + +2 e r + 2re = 0

Using the relationship

( e )2 = /(re )3
we obtain

r 3( e )2 r (2re e ) = 0
re + 2 e r = 0

Introducing state variables:

x1 := r, x2 := r x3 :=

we obtain a LTI system with system matrix

0 1 0
A = 3( e )2 0 2re e
0 2 e /re 0

52
Example: vehicle lateral dynamics
Consider a vehicle moving in a horizonal plane. We can describe the motion of this vehicle
with the velocity v of its center of gravity (CG) and its yaw rate . The velocity of the mass
center can be described by its magnitude v (the speed of the CG) and the vehicle sideslip
angle which is the angle between v and the vehicle. We will use these three variables to
describe the state of the vehicle. As inputs we consider the steering angle of the front
wheels and the braking and acceleration forces Ff x and Frx on the front and rear wheels of
the vehicle.
Application of Newtons second law to the vehicle mass center results in
mv = Frx cos + Fry sin + Ff x cos( ) + Ff x sin( ) (3.5a)
mv( + ) = Frx sin + Fry cos Ff x sin( ) + Fry cos( ) (3.5b)
where Ff x , Frx are the longitudinal forces on the front and rear wheels due to braking and
acceleration; Ff y , Fry are the lateral forces on the front and rear wheels. Considerations of
angular momentum yield
Jz = a(Ff y cos + Ff x sin ) bFry (3.6)
where Jz is the moment of inertia of the vehicle about the vertical line through its CG, and
a, b are the distances between the mass center and the front and rear of the vehicle.
The lateral force Fry depends on the rear sideslip angle r which is the angle between
the rear tires and the velocity of the rear of the vehicle. The lateral force Ff y depends on
the front sideslip angle f which is the angle between the front tires and the velocity of the
front of the vehicle. Kinematical considerations yield the following relationships:
b
tan r = tan (3.7a)
v cos
a
tan(f + ) = tan + (3.7b)
v cos
Assuming the lateral forces are zero for zero sideslip angles, and considering Frx = Ff x =
= 0, the vehicle has an equilibrium with = = r = f = 0 and v an arbitrary constant.
Linearization of the nonlinear equations about these equilibrium conditions results in
v = a
mv( + ) = Cf f Cr r
Jz = aCf f + bCr r
b
r =
v
a
f = +
v
where a = (Frx + Ff x )/m and
dFyf dFyr
Cf = and Cr =
df dr

53
are the front and rear axle stiffnesses.
Notice that the dynamics of v can be decoupled from that of (, ). With x = (, )
and u = , the lateral vehicle dynamics can be described by x = Ax + Bu where
Cf
(Cf +Cr ) bCr aCf
mv mv 2 1 mv

A= , B= (3.8)
bCr aCf a2 Cf +b2 Cr aCf
Jz
Jz v Jz

54
3.3.4 Linearization of discrete-time systems
Consider a discrete-time system described by

x(k+1) = F (x(k), u(k))

y(k) = H(x(k), u(k)) (3.9)

and suppose xe , ue , y e is a trim condition, that is,

F (xe , ue ) = xe
H(xe , ue ) = y e

The linearization of system (3.9) about (xe , ue ):

x(k+1) = A x(k) + B u(k)

(3.10)
y(k) = C x(k) + D u(k)

where
F e e F e e H e e H e e
A= (x , u ) , B= (x , u ) , C= (x , u ) , D= (x , u ) .
x u x u
Example 18 Simple scalar nonlinear system

x(k+1) = x(k)2

We have xe = 0, 1. For xe = 0:
x(k+1) = 0
For xe = 1:
x(k+1) = 2x(k)

55
Exercises
Exercise 18 Consider the two-degree-of-freedom spring-mass-damper system described by
mq1 + C(q1 q2 ) + K(q1 q2 ) + cq1 + kq1 = 0
mq2 + C(q2 q1 ) + K(q2 q1 ) + cq2 + kq2 = 0
Obtain an A-matrix for a state space description of this system.
Exercise 19 Linearize each of the following systems about the zero equilibrium state.
(i)
x1 = (1 + x21 )x2
x2 = x31

(ii)
x1 = sin x2
x2 = (cos x1 )x3
x3 = ex1 x2

Exercise 20
(a) Obtain all equilibrium states of the following system:
x1 = 2x2 (1 x1 ) x1
x2 = 3x1 (1 x2 ) x2

(b) Linearize the above system about the zero equilibrium state.
Exercise 21 For each of the following systems, linearize about each equilibrium solution
and obtain the system A-matrix for a state space representation of these linearized systems.
(a)
q + (q 2 1)q + q = 0 .
where q(t) is a scalar.
(b)
q + q + q q 3 = 0
where q(t) is a scalar.
(c)
(M + m)q + ml cos ml 2 sin + kq = 0
mlq cos + ml2 + mgl sin = 0
where q(t) and (t) are scalars.
(d)
q + 0.5q|q| + q = 0 .
where q(t) is a scalar.

56
Exercise 22

(a) Obtain all equilibrium solutions of the following system:

(cos q1 )q1 + (sin q1 )q2 + (sin q2 )q12 + sin q2 = 0

(sin q1 )q1 + (cos q1 )q2 + (cos q2 )q22 + sin q1 = 0

(b) Linearize the above system about its zero solution.

Exercise 23 (Simple pendulum in drag)

Figure 3.1: Pendulum in drag

Recall the simple pendulum in drag whose motion is described by

ml + V (l w sin ) + mg sin = 0

where q
SCD
V = l2 2 + w2 2lw sin() with =
2
and
g is the gravitational acceleration constant of the earth,
S is the reference area associated with the ball,
CD is the drag coefficient of the ball,
is the air density.
(a) Obtain the equilibrium values e of .
(b) Linearize the system about e .
(c) Obtain an expression for the A matrix for a state space representation of the linearized
system.
(d) Compare the behavior of the nonlinear system with that of the linearized system for the
following parameters.

l = 0.1 m m = 10 grams g = 9.81 m/s2

CD = 0.2 S = .01 m2 = 0.3809 kg/m3

57
Consider the following cases:
e w (m/s)
0 0
0 5
0 15
0 20
180 20
Illustrate your results with time histories of in degrees. Comment on your results.

58
Chapter 4

Transfer functions

4.1 Transfer functions

4.1.1 The Laplace transform
A state space description of an input-output system can be regarded as a time-domain
description. For linear time-invariant systems, transfer functions provide another way of
describing input-output systems; this description is sometimes called a frequency domain
description.
To discuss transfer functions of continuous-time systems, we need the Laplace transform.
Suppose f is a function of time t. Then the Laplace transform of f , which we denote by f or
L(f ), is a function of a complex variable s and is defined by
Z

f (s) = L(f )(s) = est f (t)dt . (4.1)
0

Note that f (t) could be a scalar, a vector or a matrix.

We will need the following very useful relationship:

L(f)(s) = sf(s) f (0) (4.2)

The following example illustrates the use of the above relationship.

Example 19 Consider f (t) = eat where a is any scalar. Since f satisfies

f = af

and f (0) = 0, we obtain that

sf(s) 1 = af(s) .
Hence, as expected, f(s) = 1/(s a).

59
4.1.2 Transfer functions
Consider now the LTI system

x(t) = Ax(t) + Bu(t)

y(t) = Cx(t) + Du(t)

along with the initial condition

x(0) = x0 .
Taking the Laplace transform of the state equation results in

sx(s) x0 = Ax(s) + B u(s)

which can be rewritten as

(sI A)x(s) = x0 + B u(s)
where x and u are the Laplace transforms of x and u, respectively, Whenever the matrix
sI A is invertible, the above equation may be uniquely solved for x yielding

x(s) = (sI A)1 x0 + (sI A)1 B u(s) .

Taking the Laplace transform of the above output equation yields

y(s) = C x(s) + Du(s)

where y is the Laplace transform of y. Using the expression for x now results in

y(s) = C(sI A)1 x0 + G(s)u(s)

where the transfer function (matrix) G(s) is defined by

G(s) = C(sI A)1 B + D

Zero initial conditions response. Suppose that x(0) = 0. Then, the relationship be-
tween the Laplace transform of the input u and the Laplace transform of the output y is
given by the simple relationship
y = Gu
Sometimes this is represented by
y G u
For a SISO system, we obtain that, for u 6= 0,
y
G = .
u

Thus, for a SISO system, the transfer function is the ratio of the Laplace transform of
the output to the Laplace transform of the input all initial conditions are zero.

60
Example 20

x(t) = x(t) + u(t)

y(t) = x(t)

Here
1
G(s) = C(sI A)1 + D =
s+1

Example 21 Unattached mass

x1 = x2
x2 = u
y = x1

Here
0 1 0
A= B= C= 1 0 D=0
0 0 1
Hence

G(s) = C(sI A)1 B

1
= 2
s

Exercise 24 Show that the transfer function of

x1 = x2
x2 = 0 x1 1 x2 + u
y = 0 x1 + 1 x2

is given by
1 s + 0
G(s) =
s2 + 1 s + 0

Exercise 25 Show that the transfer function of

x1 = 0 x2 + 0 u
x2 = x1 1 x2 + 1 u
y = x2

is given by
1 s + 0
G(s) =
s2 + 1 s + 0

61
MATLAB
>> help ss2tf

SS2TF State-space to transfer function conversion.

[NUM,DEN] = SS2TF(A,B,C,D,iu) calculates the transfer function:

NUM(s) -1
H(s) = -------- = C(sI-A) B + D
DEN(s)
of the system:
.
x = Ax + Bu
y = Cx + Du

from the iuth input. Vector DEN contains the coefficients of the
denominator in descending powers of s. The numerator coefficients
are returned in matrix NUM with as many rows as there are
outputs y.
Example 22 MIMO (Multi-input multi-output) B&B Consider the system illustrated in
Figure 4.1 which has two inputs and two outputs. Its motion can be described by

Figure 4.1: B&B

mq1 = k(q2 q1 ) + u1
mq2 = k(q2 q1 ) + u2
y1 = q1
y2 = q2
With x1 = q1 , x2 = q2 , x3 = q1 , x4 = q2 , we have

0 0 1 0 0 0
0 0 0 1 0 0
A = k/m
B =
k/m 0 0 1/m 0
k/m k/m 0 0 0 1/m

1 0 0 0
C = D = 0
0 1 0 0

62
Hence

G(s) = C(sI A)1 B + D

ms2 +k k
ms2 (ms2 +2k) ms2 (ms2 +2k)

=
k ms2 +k
ms2 (ms2 +2k) ms2 (ms2 +2k)

Note that the above calculation requires one to invert the 4 4 matrix sI A

We can also compute G(s) as follows. Taking the Laplace transform of the original second
order differential equations with zero initial conditions, we get

ms2 q1 = k(q1 q2 ) + u1
ms2 q2 = k(q1 q2 ) + u2
y1 = q1
y2 = q2

hence,

ms2 + k k q1 u1
=
k ms2 + k q2 u2

Solving, yields

q1 1 ms2 + k k u1
y = = 2
q2 (s) k ms + k u2

(s) = ms2 (ms2 + 2k)

So,
ms2 +k k
ms2 (ms2 +2k) ms2 (ms2 +2k)

G(s) =
k ms2 +k
ms2 (ms2 +2k) ms2 (ms2 +2k)

Using this second method, the matrix to be inverted was only 2 2.

MATLAB

>> help ss2tf

SS2TF State-space to transfer function conversion.

[NUM,DEN] = SS2TF(A,B,C,D,iu) calculates the transfer function:

NUM(s) -1
H(s) = -------- = C(sI-A) B + D
DEN(s)

63
 of the system:
.
x = Ax + Bu
y = Cx + Du

from the iuth input. Vector DEN contains the coefficients of the
denominator in descending powers of s. The numerator coefficients
are returned in matrix NUM with as many rows as there are
outputs y.

See also ltimodels, ss and tf.

Rational transfer functions.

A scalar valued function of a complex variable is rational if it can be expressed as the ratio
of two polynomials. It is proper if the order of the numerator polynomial is less than or
equal to the order of the denominator polynomial. It is strictly proper if the order of the
numerator polynomial is strictly less than that of the denominator polynomial. A matrix
valued function of a complex variable is rational, proper or strictly proper if each element of
the matrix function is respectively rational, proper or strictly proper.

Example 23
s
G(s) = s2 +1
rational, strictly proper

s2
G(s) = s2 +1
rational, proper

s3
G(s) = s2 +1
rational, not proper

es
G(s) = s2 +1
not rational

We now demonstrate the following result:

The transfer function of a finite-dimensional LTI system is always rational and proper.

The above result follows from the fact that every element Gij of the transfer function of
a finite dimensional LTI system can be expressed as
Nij
Gij = (4.3)

where (s) = det(sI A) is called the characteristic polynomial of A. It is a monic polynomial
of order n. A monic polynomial is one whose coefficient of its highest order term is one.
Thus
det(sI A) = sn + n1 sn1 + + 1 s + 0

64
for some scalars 0 , , n1 . The order of the polynomial Nij is at most n. It is less than
n if D is zero.

To demonstrate the above fact, recall that the inverse of an invertible n n matrix M
can be expressed as
1
M 1 = adj (M )
det(M )
where each element of the matrix adj (M ) is equal to the determinant of an (n1) (n1)
submatrix of M . Hence (sI A)1 can be expressed as

1
(sI A)1 = adj (sI A)
det(sI A)

where each element of the matrix adj (sIA) is equal to the determinant of an (n1)(n1)
submatrix of sI A. Hence, if A is n n, then each element of adj (sI A) is a polynomial
in s whose degree is at most n 1. Also, det(sI A) is a polynomial of order n.
From the above form of (sI A)1 it follows that if

G(s) = C(sI A)1 B

where A is n n, then
1
G(s) = N (s)
(s)
where is the n-th order characteristic polynomial of A , that is, (s) = det(sI A) and
each element Nij of N is a polynomial of order less than or equal to n1. So,

Nij
Gij =

that is, each element Gij of G is a rational function (ratio of two polynomials) and is strictly
proper (order of numerator strictly less than order of denominator). When each element of
G is rational and strictly proper, we say that G is rational and strictly proper.
If
G(s) = C(sI A)1 B + D
then
1
G(s) = N (s)
(s)
where each element Nij of N is a polynomial of order less than or equal to n. So, each
element of G is a rational function and is proper (order of numerator less than or equal to
order of denominator). When each element of G is rational and proper, we say that G is
rational and proper.
So, we see that if G is the transfer function of a finite-dimensional LTI system then, G
is rational and proper. Is the converse true? That is, is every proper rational G the transfer
function of some finite-dimensional system and if so, what is(are) the system(s). This is
answered in the next section.

65
Invariance of transfer function under state transformations. The transfer function
of a linear time-invariant system describes the input output behavior of the system when all
initial conditions are zero. Hence, it should be independent of the state variables used in a
state space description of the system. We now demonstrate this explicitly.
Suppose G is the transfer function of a system described by

x = Ax + Bu
y = Cx + Du

Then G(s) = C(sI A)1 B + D. Consider any state transformation

x = T

where T is square and invertible. Then, the system can also be described by

= A + Bu
y = C + Du

with
A = T 1 AT B = T 1 B
C = CT D = D
It can be readily verified that

C(sI A)1 B + D = C(sI A)1 B + D

that is, the transfer function matrix is unaffected by a state transformation.

Systems described by higher order differential equations. Consider a system de-

scribed by

Al y (l) + Al1 y (l1) + + A0 y = Bl u(l) + Bl1 u(l1) + + B0 u

where the input u(t) is an m-vector, the output y(t) is a p-vector, l 1, and the matrix Al
is invertible. How do we put such a system into state space form? By taking the Laplace
transform of the above equation and considering zero initial values for y, y, , y (l1) and
u, u, , u(l1) , we obtain

[sl Al + sl1 Al1 + + A0 ]y(s) = [sl Bl + sl1 Bl1 + + B0 ]u(s)

Hence y(s) = G(s)u(s) where

G(s) = [sl Al + sl1 Al1 + + A0 ]1 [sl Bl + sl1 Bl1 + + B0 ]

To obtain a state space description of the system under consideration, we could look for
matrices A, B, C, D such that G(s) = C(sI A)1 B + D. We consider this problem in a
later section.

66
General input-output systems. Consider any linear input-output system described by
Z t
y(t) = G(t )u( ) d
0

where G is Laplace transformable. Then,

y = Gu

where G is the Laplace transform of G.

Example 24

x(t) = x(t) + u(t 1)

y(t) = x(t)

Taking Laplace transform with x(0) = 0 yields

sx(s) = x(s) + es u(s)

y(s) = x(s)

Hence y(s) = G(s)u(s) with

es
G(s) =
s+1

Exercise 26 What is the transfer function of the following system?

x1 (t) = x2 (t)
x2 (t) = u(t 2)
y(t) = x1 (t)

Exercise 27 Obtain the transfer function of the following system

x(t) = x(t) + 2x(t h) + u(t)

y(t) = x(t)

4.1.3 Poles and zeros

Poles. Suppose G is a matrix valued function of a complex variable. A complex number
is defined to be a pole of G if for some element Gij of G,

lim Gij (s) = .

s

Example 25

67
 G(s) poles

1
s2 +3s+2
-1, -2

s+1
s2 +3s+2
-2
1 es

s s+1
0, -1, -2
s2 s+1
s+1 s+2

When G(s) = C(sI A)1 B + D it should be clear that if is a pole of G then must
be an eigenvalue of A. However, as the following example illustrates, the converse is not
always true.
Example 26 If,

0 1 0
A= B= C= 1 1 D = 0,
1 0 1
then
det(sI A) = s2 1
and
s1 1
G(s) = C(sI A)1 B = 2
= .
s 1 s+1
Here A has eigenvalues 1 and 1 whereas G only has a single pole at 1.
We say that a complex number is a pole of order m of a transfer function G if is a
pole of G and m is the smallest integer such that is not a pole of

(s )m G .

Example 27 The transfer function

s+1
G(s) =
(s 3)(s + 4)2
has two poles: 3 and 4. The order of the pole at 3 is 1 whereas the order of the pole at 4
is 2.
Suppose 1 , 2 , , l are the poles of a rational transfer function G. Then, it should be
clear that we can express G as
1
G(s) = N (s) (4.4)
d(s)
where
d(s) = (s 1 )m1 (s 2 )m2 (s l )ml ,
with mi being the order of the pole i , and N is a matrix with polynomial elements.

68
Zeroes. Here, we only define zeroes for scalar-input scalar-output transfer functions. The
definition of zeroes for MIMO systems is much more complicated. We say that a complex
number is a zero of a scalar-input scalar-output transfer function G if

G() = 0 . (4.5)

The significance of zeroes. Consider a general LTI SISO system described by

Z t
y(t) = G(t )u( ) d
0

Then, the transfer function G of this system is simply the Laplace transform of the impulse
response G, that is, Z
G(s) = est G(t) dt .
0
If we assume that the impulse response G converges to zero exponentially and s is not a pole
of G, then it can be shown that the steady-state response of this system to an input

u(t) = est

is given by the output

y(t) = G(s)est
So, we have the following conclusion:
If is a zero of G then, the steady-state response of the system to the input u(t) = et is
zero.
Suppose that G is the transfer function of a system with a real impulse response that is,
Z
G(s) = est G(t) dt
0

where G(t) is real. Using the property that est = est , the realness of G(t) implies that
est G(t) = est G(t) = est G(t); hence
Z Z Z
st
G(s) = e G(t) dt = est G(t) dt = est G(t) dt = G(s) ,
0 0 0

that is,
G(s) = G(s) . (4.6)
Let (s) be the argument of the complex number G(s), that is

G(s) = |G(s)|e(s) (4.7)

where (s) is real. We now show that the steady state response of this system to

u(t) = et sin(t) ,

69
where and are real numbers, is given by

y(t) = |G(s)|et sin(t+(s)) (4.8)

where s = + . Thus the output has the same form as the input; the non-negative real
scalar |G(s)| can be regarded as the system gain at s and (s) is the system phase shift at s.
We also obtain the following conclusion:
If = + is a zero of G then, the steady-state response of the system to the input
u(t) = et sin(t) is zero.
We also see that the steady state response to a purely sinusoidal input

u(t) = sin(t)

is given by
y(t) = |G()| sin(t+()) (4.9)

To demonstrate the above result, we first note that u(t) = et sin(t) can be expressed
as u(t) = (est est )/(2) where s = + . So, the corresponding steady state response is
given by
y(t) = [G(s)est + G(s)est ]/2 .

Since G(s) = G(s), we obtain that

st st
y(t) = [G(s)e G(s)est ]/(2) = = G(s)e .

Noting that
G(s)est = |G(s)|e(s) et et = |G(s)|et e(t+(s)) ,
it now follows that
y(t) = |G(s)|et sin(t + (s)) .

Example 28 (Tuned vibration absorber.) Consider a mechanical system subject to a

sinusoidal disturbance input:
M q + C q + Kq = w(t)
where w(t) = A sin(t).
Attaching a vibration absorber to the system results in

M q + C q + Kq k( q) = w (4.10)
m + k( q) = 0 . (4.11)

Taking the Laplace transform of the above equations yields:

(M s2 + Cs + K + k)q k = w (4.12)
(ms2 + k) k q = 0 . (4.13)

70
 Figure 4.2: Tuned vibration absorber

We see that q = Gw where

ms2 + k
G =
(s)
and
(s) = (M s2 + Cs + K + k)(ms2 + k) k 2 .
p
This transfer function has zeros at k/m. if we choose
k/m = 2
then the steady state response of q to w will be zero.

4.1.4 Discrete-time
The transfer function matrix for
x(k + 1) = Ax(k) + Bu(k)
y(k) = Cx(k) + Du(k)
is defined by
G(z) = C(zI A)1 B + D
where z C.

4.1.5 Exercises
Exercise 28 (BB in Laundromat.) Consider
m1 m2 1 + k2 (1 2 ) = b1 u

m2 m2 2 k2 (1 2 ) = b2 u

y = c1 1 + c2 2
For each
p of the following cases, obtain the system transfer function and its poles; consider
:= k/m > .

71
 (a)

b1 = 1 b2 = 0
c1 = 1 c2 = 0

(b) (Self excited.)

b1 = 1 b2 = 1
c1 = 1 c2 = 0

(c) (Mass center measurement.)

b1 = 1 b2 = 0
c1 = 1 c2 = 1

(d) (Self excited and mass center measurement.)

b1 = 1 b2 = 1
c1 = 1 c2 = 1

Comment on your results.

72
4.2 Some transfer function properties
4.2.1 Power series expansion and Markov parameters
Suppose G is a transfer function for (A, B, C, D), that is G(s) = C(sI A)1 B + D. Then
we will show that, for sufficiently large s, we have the following power series expansion:
X
1 k1 1 1 1
G(s) = D + k
CA B = D + CB + 2
CAB + 3
CA2 B + .
k=1
s s s s

To see this recall that, for sufficiently large s,

1 1 1
(sI A)1 = I + 2 A + 3 A2 + .
s s s
The result follows from this. The coefficient matrices

D, CB, CAB, CA2 B,

are called the Markov parameters of G.

4.2.2 More properties*

Recall that a transfer function G for a finite dimensional system must be rational and proper,
that is, it can be expressed as
1
G = N
d
where d is a scalar valued polynomial and N is a matrix of polynomials whose degrees are
not greater than that of d. Without loss of generality, we consider d to be monic. Then we
can express d and N as

d(s) = sn + dn1 sn1 + + d1 s + d0

N (s) = sn Nn + sn1 Nn1 + + sN1 + N0

where d0 , . . . , dn1 are scalars and N0 , . . . , Nn are constant matrices whose dimensions are
the same as those of G(s). We now show that G = d1 N is a transfer function for a system
(A, B, C, D) if and only if d satisfies

CAk d(A)B = 0 for k = 0, 1, 2, . (4.14)

and

Nn = D
Nn1 = CB + dn1 D
Nn2 = CAB + dn1 CB + dn2 D
..
.
N0 = CAn1 B + dn1 CAn2 B + + d1 CB + d0 D (4.15)

73
 To obtain the above result, we first use the power series expansion for G to obtain
1 1 1 1
N (s) = D + CB + 2 CAB + 3 CA2 B +
d(s) s s s

Multiplying both sides by d(s) yields

sn Nn + sn1 Nn1 + + sN1 + N0

1 1 1
= [sn + dn1 sn1 + + d1 s + d0 ][D + CB + 2 CAB + 3 CA2 B + ]
s s s
Since the coefficients of like powers of s on both sides of the equation must be equal, this
equation holds if and only if

Nn = D
Nn1 = CB + dn1 D
Nn2 = CAB + dn1 CB + dn2 D
..
.
N0 = CAn1 B + dn1 CAn2 B + + d1 CB + d0 D

and

0 = CAn B + dn1 CAn1 B + + d0 CB

0 = CAn+1 B + dn1 CAn B + + d0 CAB
..
.
0 = CAn+k B + dn1 CAn+k1 B + + d0 CAk B
..
.

Now note that the first set of equations above is the same as (4.15) while the second set is
equivalent to (4.14)
Suppose G is a transfer function for (A, B, C, D) and d is any polynomial for which
d(A) = 0. It follows immediately that (4.15) holds. Hence we can express G as G = d1 N
where the coefficients of N are computed from (4.14).

A proof of the Cayley Hamilton Theorem. Consider any square matrix A and let
G(s) = (sI A)1 . The function G is the transfer function for the system with B = I,
C = I and D = 0. Hence G can be expressed as G = d1 N where d is the characteristic
polynomial of A. Considering k = 1 in (4.14), it now follows that d(A) = 0.

74
4.3 State space realization of transfer functions
Here we consider the following type of problem: Given a matrix valued function G of a
complex variable s, find constant matrices A, B, C, D (or determine that none exist) so that

G(s) = C(sI A)1 B + D . (4.16)

When the above holds, we say that (A, B, C, D) is a finite-dimensional state space realization
of G.

Theorem 1 Suppose G is a matrix valued function of a complex variable. Then G has a

finite-dimensional state space realization if and only if G is rational and proper.

Proof: We have already seen the necessity of G being proper rational. We prove sufficiency
in the next two sections by demonstrating specific realizations.

Minimal realizations. It should be clear from Section 4.1 that a given transfer function
has an infinite number of state space realizations. One might expect that the dimension of
every state space realization is the same. This is not even true as the next example illustrates.
We say that a realization is a minimal realization if its dimension is less than or equal to that
of any other realization. Clearly, all minimal realizations have the same dimension. Later
on we will characterize minimal state space realizations.

Example 29 Consider a SISO system of dimension two described by

1 3 1
A= B= C= 1 0 D=0
3 1 1

Here
s4 1
G(s) = C(sI A)1 B + D = =
s2
2s 8 s+2
This also has the following one dimensional realization

A = 2 B=1 C=1 D=0

Note that this system is stable, whereas the original system is unstable.

75
4.4 Realization of SISO transfer functions
4.4.1 Controllable canonical form realization
Suppose that G is a proper rational scalar function of a complex variable s. If G 6= 0, then
for some positive integer n, the function G can be expressed as

n1 sn1 + + 1 s + 0
G(s) = + d (4.17)
sn + n1 sn1 + + 1 s + 0

Note that the denominator polynomial is monic. This is illustrated in the following example.

Example 30 Consider
4s + 8
G(s) =
2s + 10
This is a proper rational function. Also 4s + 8 = 2(2s + 10) + 28 which results in

2(2s + 10) + 28 28 14
G(s) = = 2 + = 2.
2s + 10 2s + 10 s5

When G is expressed in the form shown in (4.17) we now show that

G(s) = C(sI A)1 B + D

where
G(s) = C(sI A)1 B + D
where

0 1 0 ... 0 0 0
0 0 1 ... 0 0 0
.. .. .. ..
..
. . . . .
A = .. .. .. .. B= ..
. . . . .
(4.18)
0 0 0 ... 0 1 0
0 1 2 . . . n2 n1 1

C = 0 1 2 . . . n2 n1 D=d

It then follows that one can obtain a state space realization of any proper rational scalar
SISO transfer function.
To demonstrate the above claim, let be the denominator polynomial in the above
expression for G, that is,
(s) := 0 + 1 s + . . . + sn

76
and define
1
s

v(s) := .. .
.
sn1
Then,

s 1 0 . . . 0 0 1 0
0 s 1 . . . 0 0 s 0
.. .. . . . . .. .. ..
. .
. . . . .
(sI A)v(s) = .. .. ... ... .. .. = .. = (s)B
. . . . .

0 0 0 ... s 1 ss2 0
0 1 2 . . . n2 s + n1 sn1 (s)

Hence, whenever s is not an eigenvalue of A,

1
(sI A)1 B = v(s)
(s)

and

1
1
s

C(sI A)1 B = 0 1 . . . n1 ..
(s) .
sn1
0 + 1 s + + n1 sn1
=
0 + 1 s + + n1 sn1 + sn

So,
G(s) = C(sI A)1 B + D

We will see later why this realization is called controllable.

4.4.2 Observable canonical form realization

Suppose G(s) is a proper rational scalar function of a complex variable s. If G 6= 0, then,
for some positive integer n,

n1 sn1 + + 1 s + 0
G(s) = + d
sn + n1 sn1 + + 1 s + 0

We will show that

G(s) = C(sI A)1 B + D

77
where

0 0 ... 0 0 0
1 0 ... 0 1 1
..
..
A = 0 1 . 0 2 B= .
. . . . .. ..
.. .. . . . . . .
0 0 . . . 1 n1 n1

C = 0 0 ... 0 1 D=d

Noting that C(sI A)1 B + D is a scalar and using the results of the previous section,
we have
C(sI A)1 B + D = B T (sI AT )1 C T + D = G(s)

We will see later why this realization is called observable.

78
MATLAB
>> help tf2ss

TF2SS Transfer function to state-space conversion.

[A,B,C,D] = TF2SS(NUM,DEN) calculates the state-space
representation:
.
x = Ax + Bu
y = Cx + Du

of the system:
NUM(s)
H(s) = --------
DEN(s)

from a single input. Vector DEN must contain the coefficients of

the denominator in descending powers of s. Matrix NUM must
contain the numerator coefficients with as many rows as there are
outputs y. The A,B,C,D matrices are returned in controller
canonical form. This calculation also works for discrete systems.
To avoid confusion when using this function with discrete systems,
always use a numerator polynomial that has been padded with zeros
to make it the same length as the denominator. See the Users
guide for more details.

Example 31 Consider
s2
G(s) = 2
s +1
This can be written as
1
G(s) = +1
s2+1
hence a state space representation is given by

0 1 0
A= B= C= 1 0 D=1
1 0 1
or,

x1 = x2
x2 = x1 + u
y = x1 + u

Matlab time:

79
>> num = [1 0 0]
>> den = [1 0 1]
>> [a,b,c,d]=tf2ss(num,den)

a =
0 -1
1 0

b =
1
0

c =
0 -1

d =
1

4.5 Realization of MIMO systems

Suppose G is an p m proper rational transfer function. Because G is a proper rational
function it has a representation of the form

1
G(s) = N (s) + D (4.19)
d(s)

where D is a constant p m matrix, N is a p m matrix of polynomials and d is a scalar

valued monic polynomial.

4.5.1 Controllable realizations

Sometimes it is more convenient to express G as

G(s) = N (s)(s)1 + D (4.20)

where is a p m matrix of polynomials. Let us consider this case. Note that if G is

expressed as in (4.19) then, G can be expressed as in (4.20) with (s) = d(s)I where I is
the m m identity matrix.
Without loss of generality, we can express N and as

N (s) = sl1 Nl1 + sl2 Nl2 + + sN1 + N0

(4.21)
(s) = sl I + sl1 Dl1 + sl2 Dl2 + + sD1 + D0

Here, N0 , . . ., Nl1 are constant p m matrices while D0 , . . ., Dl1 are constant m m

matrices. With n = ml, let A, B and C be the following matrices of dimensions n n, n m

80
and p n, respectively,

0 I 0 ... 0 0 0
0 0 I ... 0 0 0
. .. .. ..
. ...
. . . .
A = . .. .. .. , B= .. , (4.22a)
.. . . . .

0 0 0 ... 0 I 0
D0 D1 D2 . . . Dl2 Dl1 I

C = N0 N1 N2 ... Nl2 Nl1 . (4.22b)

Note that in the above matrices, I and 0 represent the m m identity and zero matrices,
respectively. We claim that (A, B, C, D) is a realization of G. This is proven at the end of
this section.
1
Note that, when G(s) = d(s)
N (s) and d(s) = 0 + 1 s + + n1 sn1 + sn , we have

0 I 0 ... 0 0 0
0 0 I ... 0 0 0
.. .. .. ..
..
. . . . .
A = .. .. .. .. , B= .. , (4.23a)
. . . . .

0 0 0 ... 0 I 0
0 I 1 I 2 I . . . l2 I l1 I I

C = N0 N1 N2 ... Nl2 Nl1 (4.23b)

where I and 0 represent the m m identity and zero matrices, respectively.

Example 32 Consider
3
s2
G(s) =
s+4
s2 +2s+3

The lowest common denominator (LCD) of the two elements of G(s) is

(s 2)(s2 + 2s + 3) = s3 s 6 =: d(s)

and G(s) = N (s)d(s)1 where

2
3s + 6s + 9 2 3 6 9
N (s) = =s +s +
s2 + 2s 8 1 2 8

Hence, a controllable realization of G is given by

0 1 0 0
9 6 3
A= 0 0 1 B= 0 C= D=0
8 2 1
6 1 0 1

81
Example 33 Consider the strictly proper rational function,
1 1

s+1 s2 +3s+2
G(s) = .
1
0 s+2

Here G = d1 N where
d(s) = s2 + 3s + 2
and
s+2 1 1 0 2 1
N (s) = = s +
0 s+1 0 1 0 1
Hence a realization of G is given by

0 0 1 0 0 0
0 0 0 1 0 0 2 1 1 0 0 0
A= 2 B=

C= D=
0 3 0 1 0 0 1 0 1 0 0
0 2 0 3 0 1

Here we show that (A, B, C, D) as defined above is a realization of G. Let V be the n m

matrix of polynomials defined by

I
sI

V (s) = .. .
.
sl1 I
where I represents the m m identity matrix. Then we have

sI I 0 . . . 0 0 I 0
0 sI I . . . 0 0 sI 0
. .. . . . . .. . ..
. . . .
. . . . .
(sI A)V (s) = . . . . . . = ..
.. .. .. .. .. .. .

0 0 0 . . . sI I sl2 I 0
D0 D1 D2 . . . Dl2 sI + Dl1 sl1 I (s)

= B(s)
Hence, whenever s is not an eigenvalue of A,
(sI A)1 B = V (s)(s)1
Now note that
I
sI
..

.


CV (s) = N0 N1 Nl1 .. = N (s) .
.

sl2 I
sl1 I

82
Hence
C(sI A)1 B = CV (s)(s)1 = N (s)(s)1

and
G(s) = C(sI A)1 B + D .

4.5.2 Observable realizations

Sometimes it is more convenient to express G as

G(s) = (s)1 N (s) + D (4.24)

where is a p p matrix of polynomials. Note that if G is expressed as in (4.19) then, G

can be expressed as in (4.24) with (s) = d(s)I where I is the p p identity matrix.
Without loss of generality, we can express N and as

N (s) = sl1 Nl1 + sl2 Nl2 + + sN1 + N0

(4.25)
(s) = s I + sl1 Dl1 + sl2 Dl2 + + sD1 + D0
l

Here, N0 , . . ., Nl1 are constant p m matrices while D0 , . . ., Dl1 are constant p p

matrices With n = pl, let A, B and C be the following matrices of dimensions n n, n m
and p n, respectively,

Dl1 I 0 ...
0 0 Nn1
Dl2 0 I ...
0 0 Nn2
.. .. .. ..
...
. . . .
A = .. .. . . .. , B= .. , (4.26a)
. . . . .

D1 0 0 ... 0 I N1
D0 0 0 ... 0 0 N0

C = I 0 0 0 0 . (4.26b)

Note that in the above matrices, I and 0 represent the p p identity and zero matrices,
respectively. We claim that (A, B, C, D) is a realization of G. This can be shown as follows.
Let V be the p n matrix of polynomials defined by

V (s) = sl1 I sl2 I sI I .

83
where I represents the p p identity matrix. Then we have

sI + Dl1 I 0 . . . 0 0
D l2 sI I . . . 0 0
. . ..
l1 .. .. . ..
l2 .
V (s)(sI A) = s I s I sI I . .
.. .. . . . ...

D1 0 0 . . . 0 I
D0 0 0 . . . 0 sI

= (s) 0 0 0

= (s)C

Hence, whenever s is not an eigenvalue of A,

C(sI A)1 = (s)1 V (s)

Now note that

Nl1
Nl2

..


V (s)B = sl1 I sl2 I sI I . = N (s) .

N1
N0

Hence
C(sI A)1 B = (s)1 V (s)B = (s)1 N (s)
and
G(s) = C(sI A)1 B + D .

4.5.3 Alternative realizations

SIMO systems
Suppose G is an p 1 transfer matrix function and

g1

G = ...
gp

that is gi is the i-th element of G. Choosing a monic polynomial as the least common
denominator polynomial of g1 , . . . , gp , we can write each gi as

ni (s)
gi (s) = + di
(s)

84
where
(s) = sn + n1 sn1 + + 1 s + 0

and
i
ni (s) = n1 sn1 + + 1i s + 0i

Letting

0 1 0 ... 0 0 0
0 0 1 ... 0 0 0
.. .. .. ..
..
. . . . .
A = .. .. ... .. B= ..
. . . .

0 0 0 ... 0 1 0
0 1 2 . . . n2 n1 1

01 11 21 . . . 1
n2 1
n1 d1
.. .. .. .. ..
C = . . . . . D = ...
0p 1p 2p . . . p
n2 p
n1 dp

we obtain
C(sI A)1 B + D = G(s)

Example 34 Consider
3

s2
G(s) =
s+4
s2 +2s+3

The lowest common denominator (LCD) of the two elements of G(s) is

(s 2)(s2 + 2s + 3) = s3 s 6 =: (s)

and G(s) can be expressed as

3s2 +6s+9
(s)

G(s) =
s2 +2s8
(s)

Hence, a controllable realization of G is given by

0 1 0 0
9 6 3
A= 0 0 1 B= 0 C=
8 2 1
6 1 0 1

85
MIMO systems
Suppose G is an p m transfer matrix function and

G = G1 . . . Gm

that is Gi is the i-th column of G. Using the results of the previous section, for i = 1, . . . , m,
we can obtain matrices Ai , Bi , Ci , Di such that

Gi (s) = Ci (sI Ai )1 Bi + Di

Let
A1 . . . 0 B1 . . . 0

A = ... . . . ... B = ... . . . ...
0 . . . Am 0 . . . Bm

C= C1 . . . Cm D= D1 . . . D m
Then
C(sI A)1 B + D = G(s)
Nonminimal realization, in general.

Example 35
1 1
s s2
G(s) =
1
0 s

The first and second columns of G have realizations

1
A1 = 0 B1 = 1 C1 = D1 = 0
0

and
0 1 0 1 0
A2 = B2 = C2 = D2 = 0
0 0 1 0 1

respectively. Hence a realization of G is given by

0 0 0 1 0
1 1 0 0 0
A= 0 0 1 B= 0 0 C= D=
0 0 1 0 0
0 0 0 0 1

Note that this transfer function also has the following lower dimensional realization:

0 1 1 0 1 0 0 0
A= B= C= D=
0 0 0 1 0 1 0 0

86
Exercises
Exercise 29 Obtain a state space realization of the transfer function,
s2 + 3s + 2
G(s) = .
s2 + 5s + 6
Is your realization minimal?

Exercise 30 Obtain a state space realization of the transfer function,

1
1 s1

G(s) = .
1 1
s + 1 s2 1
Exercise 31 Obtain a state space realization of the following transfer function.
s+2 1

s+1 s+3
G(s) =
5 5s+1
s+1 s+2

Exercise 32 Obtain a state space realization of the transfer function,

1 1
s1 s1

G(s) = .
1 1

s+1 s+1
Exercise 33 Obtain a state space realization of the transfer function,

s 1
G(s) = .
s1 s+1
Is your realization minimal?

Exercise 34 Obtain a state space representation of the following transfer function.

2
s +1 2
G(s) =
s2 1 s 1
Exercise 35 Obtain a state space representation of the following input-output system.
d3 y d2 y dy d2 u du
2 3
+ 6 2
+ 4 + 2y = 6 2
+ 10 + 8u
dt dt dt dt dt
Exercise 36 (a) Obtain a state space realization of the following single-input single-output
system.
y 3y 4y = u 2u 8u

87
 (b) Is your realization minimal?

Exercise 37 Obtain a state space realization of the following input-output system.

y1 y1 + y2 = u1 + u1
y2 + y2 y1 = u2

Exercise 38 Obtain a state space realization of the following input-output system.

y1 = u1 + u2
y2 = u1 + u2

Exercise 39 Obtain a state space realization of the following input-output system.

y1 + y2 = u2 + u1
y2 + y1 = u1 + u2

Exercise 40 Consider the single-input single-output system described by

x1 = 2x2 + u
x2 = 2x1 + u
y = x1 x2

(a) Compute eAt for the matrix,

0 2
A= .
2 0

(b) Obtain the impulse response of the system.

(c) What is the transfer function of this system?

Exercise 41 Consider the single-input single-output system described by

x1 = 3x1 + x2 + w
x2 = x1 3x2 + w
y = x1 + x2

(a) Compute eAt for the matrix,

3 1
A=
1 3

(b) Obtain the impulse response of the system.

(c) What is the transfer function of this system?
(d) What are the poles and zeros of the transfer function?

88
 Part II

System Behavior and Stability

89
Chapter 5

Behavior of (LTI) systems : I

5.1 Initial stuff

5.1.1 Continuous-time
Here we look at the behavior of linear time-invariant systems described by
x = Ax (5.1)
where the state x(t) is a complex n-vector and A is a complex n n matrix. (The reason
for complexifying things will be seen shortly.) Sometimes A is called the system matrix.

Solutions. By a solution of system (5.1) we mean any continuous function x() which is
defined on some time interval [t0 , ) and which satisfies (5.1). Sometimes we are interested
in solutions which satisfy an initial condition of the form
x(t0 ) = x0 (5.2)
where t0 is called the initial time and x0 is called the initial state. As a consequence of the
linearity of the system, one can prove the following result:

For each initial condition, system (5.1) has a unique solution.

Shortly, we will study the nature of these solutions.

Since the system is time-invariant, knowledge of the solutions for zero initial time, that
is t0 = 0, yields knowledge of the solutions for any other initial time. For suppose x() is
a solution corresponding to x(0) = x0 . Then, the solution corresponding to x(t0 ) = x0 is
given by
x(t) = x(tt0 ) .
Since the system is linear, one can readily show that solutions depends linearly on the
initial state; more specifically, there is a matrix valued function (called the state transition
matrix) such that at each time t the solution due to x(0) = x0 is uniquely given by
x(t) = (t)x0
Later, we develop explicit expressions for .

91
5.1.2 Discrete-time
Here we look at the behavior of linear time-invariant systems described by

x(k+1) = Ax(k)

The state x(k) is a complex n-vector and A is a complex n n matrix. Sometimes A is

called the system matrix.

Solutions: Consider initial condition:

x(k0 ) = x0 (5.3)

Since the system is time-invariant we can, wlog (without loss of generality), consider only
zero initial time, that is, k0 = 0. For suppose x() is a solution corresponding

x(k0 ) = x0

Then
x(k) = x(kk0 )
where x() is the solution corresponding to (5.3).
For k k0 = 0, we have existence and uniqueness of solutions; specifically,

x(k) = Ak x0

Proof: (Example of a proof by induction) Suppose there is a k 0 for which the above
holds, that is,

x(k ) = Ak x0
Then

x(k +1) = Ax(k )

= AAk x0

= A(k +1) x0

So, if it holds for k it holds for k + 1. Since, x(0) = A0 x0 , it holds for k = 0. It now
follows by induction that it holds for all k 0.

Solution depends linearly on initial state; specifically,

x(k) = (k)x(0) , for k 0

where (k) = Ak .

92
5.2 Scalar systems
5.2.1 Continuous-time
Consider a scalar system described by

x = ax

where x(t) and a are scalars. All solutions are given by:

x(t) = eat x0 ; x0 := x(0)

Real a. So,

(a) If a < 0 you decay with increasing age.

(b) If a > 0 you grow with increasing age.

(c) If a = 0 you stay the same; every state is an equilibrium state.

Figure 5.1: The three types of modes for real scalar systems: = a

Example 36 (A first example.) Here

a = c/m < 0

Thus, all solutions are of the form e(c/m)t x0 ; hence they decay exponentially.

93
Complex a. Suppose a is a genuine complex number (unreal); that is,
a = +
where and are real numbers. We call the real part of a and denote this by = <(a).
Similarly, we call the imaginary part of a and denote this by = =(a).
We now show that the growth behavior of eat depends only on the real part of a. To this
end, we first note that
eat = e(+)t = e(t+t) = et et ,
and, hence, |eat | = |et | |et |. Since et is positive, |et | = et . Since,
et = cos t + sin t ,
we obtain that |et | = 1. Hence,

|eat | = et where = <(a)

Thu, |x(t)| = |eat x0 | = |eat ||x0 | = et |x0 |. So,

(a) If <(a) < 0 you decay with increasing age.

(b) If <(a) > 0 you grow with increasing age.
(c) If <(a) = 0 your magnitude remains constant.

5.2.2 First order complex and second order real.

Suppose a is a genuine complex number (unreal); that is,
a = + where = <(a), = =(a)
Since
x = 1 + 2 where 1 = <(x), 2 = =(x)
Then

1 = 1 2
=
2 = 1 + 2
| {z }
axisymmetric spacecraft

94
Exercise 42 Verify the last set of equations.

Also,

1 (t) = et cos(t)10 et sin(t)20 et cos(t) et sin(t)
= 0
2 (t) = et sin(t)10 + et cos(t)20 et sin(t) et cos(t)

where 10 = 1 (0), 20 = 2 (0)

Exercise 43 Verify the last set of equations. Hint: et = cos t + sin t

Figure 5.2: The three complex modes

95
5.2.3 DT

x(k+1) = ax(k)
where x(k) and a are scalars. All solutions are given by:

x(k) = ak x0 ; x0 := x(0)

Hence, |x(k)| = |ak x0 | = |a|k |x0 |. So,

(a) If |a| < 1 you decay with increasing age.

(b) If |a| > 1 you grow with increasing age.

(c) If |a| = 1 your magnitude remains the same.

(d) If a = 1 you stay the same; every state is an equilibrium state.

96
5.3 Eigenvalues and eigenvectors
Shortly, we shall see that the behavior of a linear time invariant system is completely deter-
mined by the eigenvalues and eigenvectors of its system matrix.
Suppose A is a complex n n matrix. (The use of the word complex also includes real.
To indicate that something is complex but not real we say genuine complex.)

DEFN. A complex number is an eigenvalue of A if there is a nonzero vector v such that

Av = v
The nonzero vector v is called an eigenvector of A corresponding to .

So, an eigenvector of A is a nonzero complex n-vector with the property that there is a
complex number such that the above relationship holds or, equivalently,

(I A)v = 0

Since v is nonzero, I A must be singular; so,

det(I A) = 0

The characteristic polynomial of A:

charpoly(A) := det(sI A)
Note that det(sI A) is an n-th order monic polynomial, that is, it has the form
det(sI A) = a0 + a1 s + . . . + sn

We conclude that a complex number is an eigenvalue of A if and only if it is a root of

the nth order characteristic polynomial of A. Hence, A has at least one eigenvalue and at
most n distinct eigenvalues. Suppose A has l distinct eigenvalues, 1 , 2 , . . . , l ; since these
are the distinct roots of the characteristic polynomial of A, we must have
l
Y
det(sI A) = (s i )mi
i=1

The integer mi is called the algebraic multiplicity of i

It should be clear that if v 1 and v 2 are any two eigenvectors corresponding to the same
eigenvalue and 1 and 2 are any two numbers then (provided it is nonzero) 1 v 1 + 2 v 2
is also an eigenvector for . The set of eigenvectors corresponding to along with the zero
vector is called the eigenspace of A associated with . This is simply the null space of I A

The geometric multiplicity of is the nullity of I A, that is, it is the maximum number
of linearly independent eigenvectors associated with .

It follows that A is invertible if and only if all its eigenvalues are nonzero.

97
Example 37 (An eigenvalue-eigenvector calculation)

3 1
A=
1 3

The characteristic polynomial of A is given by

s3 1
det(sI A) = det
1 s3

= (s 3)(s 3) 1 = s2 6s + 8
= (s 2)(s 4)

The roots of this polynomial yield two distinct real eigenvalues:

1 = 2, 2 = 4

To compute eigenvectors for 1 , we use (1 I A)v = 0 to obtain

v1 + v2 = 0
v1 v2 = 0

which is equivalent to
v1 v2 = 0
So, only one linearly independent eigenvector for 1 ; lets take

1 1
v=v =
1

In a similar fashion, 2 has one linearly independent eigenvector; we take

2 1
v =
1

To check the above calculations, note that:

1 3 1 1 2 1
Av = = =2 = 1 v 1
1 3 1 2 1

Similarly for 2 .

98
Example 38 (Another eigenvalue-eigenvector calculation)

1 1
A=
1 1

The characteristic polynomial of A is given by

s1 1
det(sI A) = det = (s 1)(s 1) + 1
1 s1
= s2 2s + 2

Computing the roots of this polynomial yields two distinct complex eigenvalues:

1 = 1 + , 2 = 1

To compute eigenvectors for 1 , we use (1 I A)v = 0 to obtain

v1 v2 = 0
v1 + v2 = 0

which is equivalent to
v1 + v2 = 0
So, only one linearly independent eigenvector; lets take

1 1
v=v =

T
(We could also take 1 .) In a similar fashion, 2 has one linearly independent eigen-
vector; we take
2 1
v =

To check the above calculations, note that

1 1 1 1 1+ 1
Av = = = (1 + ) = 1 v 1
1 1 1 +

Similarly for 2 .

Example 39
0 1
A=
0 0

Exercise 44 Compute expressions for eigenvalues and linearly independent eigenvectors of

A= , 6= 0

99
5.3.1 Real A
Recall Examples 37 and 38. They illustrate the following facts.

If A is real, its eigenvalues and eigenvectors occur in complex conjugate pairs.

To see this, suppose is an eigenvalue of A with eigenvector v. Then

Av = v

Taking complex conjugate of both sides of this equation yields

Av = v

Since A is real, A = A; hence

Av = v
that is, is an eigenvalue of A with eigenvector v

Real eigenvectors for real eigenvalues.

Genuine complex eigenvectors for (genuine) complex eigenvalues.

The real and imaginary parts of a complex eigenvector are linearly independent.

Example 40 Consider any non-zero angular velocity matrix of the form:

0 3 2
A = 3 0 1
2 1 0
q
2 2
charpoly(A) = s(s + ), := 12 + 22 + 32
Hence, A has one real eigenvalue 0 and two complex conjugate eigenvalues, , .

100
5.3.2 MATLAB
>> help eig

EIG Eigenvalues and eigenvectors.

EIG(X) is a vector containing the eigenvalues of a square
matrix X.

[V,D] = EIG(X) produces a diagonal matrix D of

eigenvalues and a full matrix V whose columns are the
corresponding eigenvectors so that X*V = V*D.

[V,D] = EIG(X,nobalance) performs the computation with

balancing disabled, which sometimes gives more accurate results
for certain problems with unusual scaling.

>> help poly

POLY Characteristic polynomial.

If A is an N by N matrix, POLY(A) is a row vector with
N+1 elements which are the coefficients of the
characteristic polynomial, DET(lambda*EYE(A) - A) .
If V is a vector, POLY(V) is a vector whose elements are
the coefficients of the polynomial whose roots are the
elements of V . For vectors, ROOTS and POLY are inverse
functions of each other, up to ordering, scaling, and
roundoff error.
ROOTS(POLY(1:20)) generates Wilkinsons famous example.

>> help roots

ROOTS Find polynomial roots.

ROOTS(C) computes the roots of the polynomial whose coefficients
are the elements of the vector C. If C has N+1 components,
the polynomial is C(1)*X^N + ... + C(N)*X + C(N+1).

101
5.3.3 Companion matrices
These matrices will be useful later in control design. We met these matrices when construct-
ing state space descriptions for systems described by a single higher order linear differential
equation.

Exercise 45 Compute the characteristic polynomial of

0 1
a0 a1

Hence, write down the characteristic polynomials for the A matrices associated with the
unattached mass and the damped linear oscillator.

Exercise 46 Compute the characteristic polynomial of

0 1 0
0 0 1
a0 a1 a2

Fact 1 The characteristic polynomial of

0 1 0 ... 0 0
0 0 1 ... 0 0
. .. ..
. ...
. . .
. .. ... ..
.. . .

0 0 0 ... 0 1
a0 a1 a2 . . . an2 an1

is given by
p(s) = a0 + a1 s + . . . + an1 sn1 + sn
Hence, given any bunch of n complex numbers, there is a companion matrix whose eigen-
values are exactly these numbers.

Exercise 47 What is the real 2 2 companion matrix with eigenvalues 1 + , 1 ?

Exercise 48 What is the companion matrix whose eigenvalues are 1, 2, and 3?

102
5.4 Behavior of continuous-time systems
5.4.1 System significance of eigenvectors and eigenvalues
In this section, we demonstrate that the eigenvalues and eigenvectors of a square matrix A
play a fundamental role in the behavior of the linear system described by

x = Ax . (5.4)

We first demonstrate the following result.

The system x = Ax has a non-zero solution of the form

x(t) = et v (5.5)

if and only if is an eigenvector of A and v is a corresponding eigenvector.

To demonstrate the above claim, note that if x(t) = et v, then

x = et v and Ax(t) = Aet v = et Av .

Hence, x(t) = et v is a non-zero solution of x = Ax if and only if v is nonzero and

et v = et Av .

Since et is nonzero, the above condition is equivalent to

Av = v .

Since v is non-zero, this means is an eigenvector of A and v is a corresponding eigenvector.

A solution of the form et v is a very special type of solution and is sometimes called a
mode of the system. Note that if x(t) = et v is a solution, then x(0) = v, that is, v is the
initial value of x. Hence, we can make the following statement.

If v is an eigenvector of A, then the solution to x = Ax with initial condition x(0) = v

is given by x(t) = et v where is the eigenvalue corresponding to v.

If v is an eigenvector then, et v is also an eigenvector for each t. Also, considering the

magnitude of et v, we obtain

||et v|| = |et | kvk = et kvk

where is the real part of . So,

(a) Once an eigenvector, always an eigenvector!

(b) If <() < 0 you decay with increasing age.

(c) If <() > 0 you grow with increasing age.

103
 (d) If <() = 0 your magnitude remains constant.

(e) If = 0 you stay put; every eigenvector corresponding to = 0 is an equilibrium state.

Example 41 The matrix A of example 37 had 2 as an eigenvalue with corresponding eigen-

vector [ 1 1 ]T . Hence with initial state,

1
x0 =
1

the system x = Ax has the following solution:

2t
e
x(t) =
e2t

What is the solution for

1
x0 = ?
1

Exercise 49 Prove the following statement. Suppose that v 1 , . . . , v m are eigenvectors of A

with corresponding eigenvalues, 1 , . . . , m . Then the solution of (5.4) with initial state

x0 = 10 v 1 + . . . + m0 v m ,

where 10 , . . . , m0 are scalars, is given by

x(t) = 10 e1 t v 1 + . . . + m0 em t v m .

Behaviour in the v subspace. Suppose v is an eigenvector of A with corresponding

eigenvalue and consider any initial state of the form x0 = 0 v. Thus, the initial state x0 is
in the 1-dimensional subspace spanned by v. We have shown that the solution corresponding
to x(0) = x0 has the form x(t) = (t)v; hence x(t) remains in this subspace. Forgetting about
= x = Ax = A(v) = v. Hence
the actual solution, we note that v

= (0) = 0

In other words, the behavior of the system in the 1-dimensional subspace spanned by v is
that of a scalar system.

Some state space pictures

104
Real systems and complex eigenvalues. Suppose we are interested in the behavior of
a real system with real state x and real system matrix A and suppose is a (genuine)
complex eigenvalue of A, that is,
Av = v
for a nonzero v. Since A is real and is complex, the eigenvector v is complex. The complex
system can start with a complex initial state but the real system cannot. So is the above
result useful? It is, just wait.
Since A is real, is also an eigenvalue of A with corresponding eigenvector v; hence the
(complex) solution corresponding to x(0) = v is given by

et v = et v

Consider now the real vector

1
u := <(v) = (v + v)
2
as an initial state. Since the solution depends linearly on initial state, the solution for this
real initial state is
1 t
(e v + et v ) = <( et v )
2
Suppose we let
= <() = =() w = =(v) .
Then
= + and v = u + w .
Hence,

et v = et (cos t + sin t)(u + w)

= [et cos(t) u et sin(t) w ] + et sin(t) u + et cos(t) w

Hence, the solution due to x(0) = u is

et cos(t) u et sin(t) w

Similarly, the solution due to initial state w is =(et v), that is,

et sin(t) u + et cos(t) w

So, if the initial state is of the form

x(0) = 10 u + 20 w

where 10 , 20 are any two real scalars (that is, the system starts in the real subspace spanned
by the real span of u, w), the resulting solution is given by

x(t) = 1 (t)u + 2 (t)w

105
where

1 (t) = et cos(t)10 + et sin(t)20 et cos(t) et sin(t)
= 0
2 (t) = et sin(t)10 + et cos(t)20 et sin(t) et cos(t)
1
2 2 2
Note that, if we let A0 = (10 + 20 ) and let 0 be the unique number in the interval [0, 2)
defined by
sin(0 ) = 10 /A0 and cos(0 ) = 20 /A0
then

1 (t) = A0 et sin(t+0 )
2 (t) = A0 et cos(t+0 )

and
x(t) = A0 et (sin(t+0 )u + cos(t+0 )w)

So, we conclude that for real systems:

(a) If the initial state is in the real 2D subspace spanned by the real and imaginary parts
of an complex eigenvector, then so is the resulting solution.

(b) If <() < 0 you decay with increasing age.

(c) If <() > 0 you grow with increasing age.

(d) If <() = 0 your magnitude remains constant and if 6= 0 you oscillate forever.

(e) If = 0 you stay put; every eigenvector corresponding to = 0 is an equilibrium state.

Example 42 Recall example 38. Considering

1 1
v=v =

we have
1 0
u= w=
0 1
So,
x = x1 u + x 2 w
Hence, all solutions are given by

x1 (t) = et cos(t)x10 + et sin(t)x20 et cos(t) et sin(t)
= x0
x2 (t) = et sin(t)x10 + et cos(t)x20 et sin(t) et cos(t)

106
Behavior in the u w subspace. We have shown that if the state starts in the 2-
dimensional subspace spanned by u, w, ( lets call it the uw subspace) it remains there,
that is, if x0 = 10 u + 20 w then x(t) = 1 (t)u + 2 (t)w. Forgetting about the actual solution,
we now show that motion in this subspace is described by:

1 = 1 + 2
2 = 1 + 2 =

To see this, we first equate real and imaginary parts of

A(u + w) = ( + )(u + w)

to obtain
Au = u w
Aw = u + w
Now

1 u + 2 w = x
= Ax
= A(1 u + 2 w)
= 1 Au + 2 Aw
= (1 + 2 )u + (1 + 2 )w

The desired result now follows by equating coefficients of u and w on both sides of the
equation.

Example 43 (Beavis and Butthead) Consider u1 = u2 = 0 in the system in Figure 5.3.

Figure 5.3: Beavis & Butthead

mq1 = k(q2 q1 )
mq2 = k(q2 q1 )

107
Introducing state variables

x1 := q1 , x2 := q2 , x3 := q1 , x4 := q2 ,

this system is described by x = Ax where A is the block partitioned matrix given by

0 I 1 1
A= , A21 = k/m .
A21 0 1 1

The characteristic polynomial of A is given by

p
p(s) = s2 (s2 + 2 ) , = 2k/m

Hence, A has three eigenvalues

1 = , 2 = , 3 = 0 .

Corresponding to each eigenvalue i , there is at most one linearly independent eigenvector

v i , for example,
1 1
1
v1 = , v 2
= v 1 , v 3
= 1 .
0
0
Consider now the subspace S 1 spanned by the real and imaginary parts of v 1 :


1 0
1 0
0 0
1 1
S = span
1
0 , = span 0 , 1 .




0 0 1

A little reflection reveals that states in this subspace correspond to those states for which
q2 = q1 and q2 = q1 . A state trajectory which remains in this subspace is characterized
by the condition that
q2 (t) q1 (t) .
We have seen that whenever a motion originates in this subspace it remains there; also
all motions in this subspace are purely oscillatory and consist of terms of the form cos(t)
and sin(t).
Consider now the subspace S 3 spanned by the eigenvector v 3 . Every state in this subspace
has the form
q1
q1

0 ,
0
that is, q1 is arbitrary, q2 = q1 , and q1 = q2 = 0. Since the eigenvalue corresponding to this
eigenspace is zero, every state in this subspace is an equilibrium state.

108
5.4.2 Solutions for nondefective matrices
Recall that the algebraic multiplicity of an eigenvalue of a matrix is the number of times
that the eigenvalue is repeated as a root of the characteristic polynomial of the matrix. Also
the geometric multiplicity of an eigenvalue is the dimension of its eigenspace, or equivalently,
the maximum number of linearly independent eigenvectors associated with that eigenvalue.
It can be proven that the geometric multiplicity of an eigenvalue is always less than or equal
to its algebraic multiplicity. An eigenvalue whose geometric multiplicity equals its algebraic
multiplicity is call a nondefective eigenvalue. Otherwise, it is called defective. If an eigenvalue
is not repeated, that is, it is not a repeated root of the matrix characteristic polynomial, then
it has algebraic multiplicity one. Since its geometric multiplicity must be greater than zero
and less than or equal to its algebraic multiplicity, it follows that its geometric multiplicity
equals its algebraic multiplicity; hence the eigenvalue is nondefective.
A matrix with the property that all its eigenvalues are nondefective is called a nondefective
matrix. Otherwise, it is called defective. When is a matrix A nondefective? Some examples
are:

(a) A has n distinct eigenvalues where A is n n.

(b) A is hermitian that is, A = A.

(c) A is unitary, that is, A A = I.

Here A is the complex conjugate transpose of A.

We have the following result.

Fact 2 An n n matrix has n linearly independent eigenvectors if and only if A is nonde-

fective.

Example 44 The following matrix is a simple example of a defective matrix.

0 1
A=
0 0

Example 45 The following matrix is another example example of a defective matrix.

0 1 0 0
2 0 1 0
A=

0 0 0 1
0 0 2 0

where is any real number.

Suppose A is a nondefective matrix and consider any initial condition: x(0) = x0 . Since
A is nondefective, it has a basis of eigenvectors v 1 , , v n . Hence, there are unique scalars
10 , 10 , . . . , n0 such that x0 can be expressed as

x0 = 10 v 1 + 20 v 2 + . . . + n0 v n .

109
From this it follows that the solution due to x(0) = x0 is given by

x(t) = e1 t 10 v 1 + e2 t 20 v 2 + . . . + en t n0 v n (5.6)

Note that each term ei t i0 v i on the right-hand-side of the above equation is a mode. This
results in the following fundamental observation.

Every motion (solution) of a linear time-invariant system with nondefective A matrix is

simply a sum of modes.

5.4.3 Defective matrices and generalized eigenvectors

Consider a square matrix A. Then we have the following fact.

Fact 3 An eigenvalue for A is defective if and only if there is a nonzero vector g such that

(A I)2 g = 0 but (A I)g 6= 0 .

The vector g is called a generalized eigenvector for A.

Example 46
0 1
A=
0 0
Consider = 0. Then
0
g=
1
is a generalized eigenvector of A.

We now demonstrate the consequence of a system having a defective eigenvalue. So,

suppose is a defective eigenvalue for a square matrix A. Let g be any corresponding
generalized eigenvector for . We now show that

x(t) = et g + tet (A I)g

is a solution of x = Ax. To see this, differentiate x to obtain

x(t) = et g + et (A I)g + tet (A I)g

= et Ag + tet (A I)g

Since (A I)2 g = 0, we have

A(A I)g = (A I)g .

Hence

x(t) = et Ag + tet A(A I)g = A(et g + tet (A I)g)

= Ax(t) .

110
 Consider the function given by tet and note that

|tet | = |t|et

where is the real part of . So, if is negative, the magnitude of the above function decays
exponentially. If is positive, the magnitude of the above function grows exponentially. If
is zero, the magnitude of the above function is given by |t| which is also unbounded. From
this discussion, we see that the defectiveness of an eigenvalue is only significant when the
eigenvalue lies on the imaginary axis of the complex plane, that is when it has zero real part.

Example 47 (Resonance) Consider a simple spring-mass system, of mass m p and spring

constant k, subject to a sinusoidal force w(t) = A sin(t + ) where = k/m is the
unforced natural frequency of the spring mass system and A and are arbitrary constants.
Letting q be the displacement of the mass from its unforced equilibrium position, the motion
of this system can be described by

mq + kq = w .

Figure 5.4: Resonance

Since w(t) = A sin(t + ) it satisfies the following differential equation:

w + 2 w = 0 .

If we introduce state variables,

x1 = q , x2 = q , x3 = w , x4 , = w

the sinusoidally forced spring-mass system can be described by x = Ax where

0 1 0 0
2 0 1/m 0
A= 0 0
.
0 1
0 0 2 0

This matrix has characteristic polynomial

p(s) = (s2 + 2 )2 .

111
Hence A has two distinct imaginary eigenvalues 1 = and 2 = , both of which are
repeated twice as roots of the characteristic polynomial. All eigenvalues for 1 and 2 are of
the form cv 1 and cv 2 , respectively, where

1 1

v1 =
0
and v2 =
0 .

0 0
Since these eigenvalues have algebraic multiplicity two and geometric multiplicity one, they
are defective. A generalized eigenvector corresponding to 1 and 2 is given by

1/2k 1/2k
0 0
g1 =
1
and g2 =
1 ,

respectively. Hence this system has solutions with terms of the form t sin(t).

Exercises
Exercise 50 (Beavis and Butthead at the laundromat)

mq1 m2 q1 + k2 (q1 q2 2l0 ) = 0

mq2 m2 q2 k2 (q1 q2 2l0 ) = 0

with p
:= k/m >

Figure 5.5: At the laundromat

(a) Obtain the equilibrium values q1e , q2e of q1 , q2 .

112
 (b) With
x1 = q1 := q1 q1e
x2 = q2 := q2 q2e
x3 = q1
x4 = q2

Obtain a state space description of the form x = Ax.

(c) Obtain expressions for the eigenvalues of A. Write down expressions for the complex
Jordan form and the real Jordan form of A?
(d) For the rest of the exercise, consider 2 = 22 . Obtain expressions for linearly inde-
pendent eigenvectors of A.
(e) For the rest of the exercise, let = 1. Numerically simulate with initial conditions
q1 (0) = 1, q2 (0) = 1, q1 (0) = 0, q2 (0) = 0
On the same graph, plot q1 (t) and q2 (t) vs t. Why is q2 (t) q1 (t)? Plot q2 (t)
vs q1 (t).
(f) Numerically simulate with initial conditions
q1 (0) = 1, q2 (0) = 1, q1 (0) = 1, q2 (0) = 1
On the same graph, plot q1 (t) and q2 (t) vs t. Why is q2 (t) q1 (t)? Why does the
solution decay to zero? Plot q2 (t) vs q1 (t).
(g) Numerically simulate with initial conditions
q1 (0) = 1, q2 (0) = 1, q1 (0) = 1, q2 (0) = 1
On the same graph, plot q1 (t) and q2 (t) vs t. Why is q2 (t) q1 (t)? Is this solution
bounded? Plot q2 (t) vs q1 (t).
(h) Numerically simulate with initial conditions
q1 (0) = 1, q2 (0) = 1, q1 (0) = , q2 (0) =
Compute ||x(t)|| and show (by plotting ||x|| vs t) that it is constant. Why is it constant?
Exercise 51 Consider the delay differential equation described by
x(t) = A1 x(t) + A2 x(t 1)
where t IR and x(t) Cn . Show that if a complex number satisfies
det(I A1 e A2 ) = 0
then the delay differential equation has a solution of the form
et v
where v Cn is nonzero.

113
Exercise 52 Consider a system described by

M q + C q + Kq = 0

where q(t) is an N -vector and M , C and K are square matrices. Suppose is a complex
number which satisfies
det(2 M + C + K) = 0 .
Show that the above system has a solution of the form

q(t) = et v

where v is a constant N -vector.

114
5.5 Behavior of discrete-time systems
5.5.1 System significance of eigenvectors and eigenvalues
Consider now a discrete time system described by

x(k+1) = Ax(k) (5.7)

We now demonstrate the following result:

The discrete-time system (5.7) has a solution of the form

x(k) = k v (5.8)

if and only if is an eigenvalue of A and v is a corresponding eigenvector.

To prove this, we first suppose that is an eigenvalue of A with v as a corresponding
eigenvector. We need to show that x(k) = k v is a solution of (5.7). Since Av = v, it
follows that x(k+1) = (k+1) v = k v = k Av = A(k v) = Ax(k), that is x(k+1) = Ax(k);
hence x() is a solution of (5.7).
Now suppose that x(k) = k v is a solution of (5.7). Then v = x(1) = Ax(0) = Av, that
is, Av = v; hence is an eigenvalue of A and v is a corresponding eigenvector.
A solution of the form k v is a very special type of solution and is sometimes called a
mode of the system. Note that if x(k) = k v is a solution, then x(0) = v, that is, v is the
initial value of x. Hence, we can make the following statement.
If v is an eigenvector of A, then the solution to x(k +1) = Ax(k) with initial condition
x(0) = v is given by x(k) = k v where is the eigenvalue corresponding to v.

If v is an eigenvector then, k v is also an eigenvector for each k. Also, considering the

magnitude of k v, we obtain

||k v|| = |k | kvk = ||k kvk

So,

(a) Once an eigenvector, always an eigenvector!

(b) If || < 1 you decay with increasing age.

(c) If || > 1 you grow with increasing age.

(d) If || = 1 and 6= 1 you oscillate forever.

If = 1 and x(0) = v, then x(k) = v for all k, that is v is an equilibrium state for the
system.
If = 0 and x(0) = v, then x(k) = 0 for all k > 0, that is the state of the system goes to
zero in one step.

115
5.5.2 All solutions for nondefective matrices
Recall that an eigenvalue whose geometric multiplicity equals its algebraic multiplicity is
called a nondefective eigenvalue. Otherwise, it is considered defective. A matrix with the
property that all its eigenvalues are nondefective is called a nondefective matrix. Otherwise,
it is considered defective. Recall also that an n n matrix has n linearly independent
eigenvectors if and only if A is nondefective.

Example 48 The following matrix is a simple example of a defective matrix.

0 1
A=
0 0

Suppose A is a nondefective matrix and consider any initial condition: x(0) = x0 . Since
A is nondefective, it has a basis of eigenvectors v 1 , , v n . Hence,

x0 = 10 v 1 + 20 v 2 + . . . + n0 v n

From this it follows that

x(k) = k1 10 v 1 + k2 20 v 2 + . . . + kn n0 v n (5.9)

Note that each term ki i0 v i on the right-hand-side of the above equation is a mode. This
results in the following fundamental observation.

Every motion (solution) of a discrete-time linear time-invariant system with nondefective

A matrix is simply a sum of modes.

5.5.3 Some solutions for defective matrices

Consider a square matrix A and recall that an eigenvalue for A is defective if and only if
there is a nonzero vector g such that

(A I)2 g = 0 but (A I)g 6= 0 .

The vector g is called a generalized eigenvector for A.

Example 49
0 1
A=
0 0
Consider = 0. Then
0
g=
1
is a generalized eigenvector of A.

116
 We now demonstrate the consequence of a system having a defective eigenvalue. So,
suppose is a defective eigenvalue for a square matrix A. Let g be any corresponding
generalized eigenvector for . We now show that

x(k) = k g + k(k1) (A I)g

is a solution of x(k +1) = Ax(k). To see this, we first note that, since (A I)2 g = 0, we
have
A(A I)g = (A I)g .
Hence

Ax(k) = k Ag + k(k1) A(A I)g

= k Ag + kk (A I)g .

Now note that

x(k+1) = (k+1) g + (k+1)k (A I)g

= (k+1) g + kk (A I)g + k Ag (k+1) g
= k Ag + kk (A I)g .

Hence x(k+1) = Ax(k)

Consider the function given by kk1 and note that

|kk1 | = |k|||k1 .

So, if || < 1, the magnitude of the above function decays exponentially. If || > 1,
the magnitude of the above function grows exponentially. If || = 1, the magnitude of the
above function is given by |k| which is also unbounded. From this discussion, we see that the
defectiveness of an eigenvalue is only significant when the eigenvalue lies on the unit circle
in the complex plane, that is when its magnitude is one.

117
5.6 Similarity transformations
Motivating example. Consider

x1 = 3x1 x2
x2 = x1 + 3x2

Suppose we define new states:

1 = (x1 + x2 )/2
2 = (x1 x2 )/2

Then,

1 = 21
2 = 42

That looks better!

Where did new states come from? First note that

x1 = 1 + 2
x2 = 1 2

and we have the following geometric interpretation. Since,

1 0
x = x1 + x2
0 1

1 1
= 1 + 2
1 1

the scalars x1 , x2 are the coordinates of the vector x wrt the usual basis for IR2 ; the scalars
1 , 2 are the coordinates of x wrt the basis

1 1
1 1

Where did these new basis vectors come from? These new basis vectors are eigenvectors of
the matrix
3 1
A=
1 3
and the system is described by x = Ax.

118
Coordinate transformations. Suppose we have n scalar variables x1 , x2 , . . . , xn and we
implicitly define new scalar variables 1 , 2 , . . . , n by

x1 1
x2 2

x = T where x = .. and = ..
. .
xn n
and T is an n n invertible matrix. Then, is explicitly given by
= T 1 x
We can obtain a geometric interpretation of this change of variables as follows. First,
observe that

x1 1 0 0
x2 0 1 0

.. = x1 .. + x2 .. + . . . + xn .. ,
. . . .
xn 0 0 1
| {z } | {z } | {z } | {z }
x e1 e2 en
that is the scalars x1 , x2 , . . . , xn are the coordinates of the vector x wrt the standard basis
e1 , e2 , . . . , en , or, x is the coordinate vector of itself wrt the standard basis. Suppose

T = t1 . . . t j . . . t n
that is, tj is the j-th column of T . Since T is invertible, its columns, t1 , t2 , . . . , tn , form a
basis for Cn . Also, x = T can be written as
x = 1 t1 + 2 t2 + . . . + n tn
From this we see that 1 , 2 , . . . , n are the coordinates of x wrt to the new basis and the
vector is the coordinate vector of the vector x wrt this new basis. So x = T defines a
coordinate transformation.

Similarity transformations. Consider now a CT (DT) system described by

x = Ax ( x(k+1) = Ax(k) )
and suppose we define new state variables by x = T where T is nonsingular; then the
behavior of is governed by
= ( (k+1) = (k) )
where
= T 1 AT
A square matrix A is said to be similar to another square matrix if there exists a
nonsingular matrix T such that = T 1 AT .
Recalling that the columns t1 , t2 , . . . , tn of T form a basis, we have now the following very
useful result.

119
Useful result. Suppose

Atj = 1j t1 + 2j t2 + . . . + nj tn

Then the matrix = T 1 AT is uniquely given by

ij = ij

that is ij is the i-th coordinate of the vector Atj wrt the basis t1 , t2 , , tn . Thus, the j-th
column of the matrix is the coordinate vector of Atj wrt the basis t1 , t2 , , tn .

Proof. Premultiply = T 1 AT by T to get:

T = AT

On the right we have

AT = A t1 . . . tj . . . tn

= At1 . . . Atj . . . Atn

And on the left,

11 . . . 1j . . . 1n
21 . . . 2j . . . 2n
.. .. ..

T = t1 t2 . . . . . . tn . . .
.. .. ..
. . .
n1 . . . nj . . . nn

. .. . .. !
11 t1 + + n1 tn .. ...... . 1j t1 + . . . + nj tn .. ...... . 1n t1 + . . . + nn tn
= | {z }
j-th column
Comparing the expressions for the j-th columns of AT and T yields

1j t1 + . . . + nj tn = Atj
= 1j t1 + . . . + nj tn

Since the bi s form a basis we must have ij = ij .

Note that
A = T T 1

Example 50 Recall example 37. Note that the eigenvectors v 1 and v 2 are linearly indepen-
dent; hence the matrix
1 2 1 1
T := v v =
1 1

120
is invertible. Since

Av 1 = 2v 1
Av 2 = 4v 2

we can use the above result and immediately write down (without computing T 1 )

1 2 0
= T AT =
0 4

Explicitly compute T 1 and then T 1 AT to check.

Example 51 Recalling example 38 we can follow the procedure of the last example and
define
1 1
T :=

to obtain (without computing T 1 )

1 1+ 0
=T AT =
0 1

You may now compute T 1 and check this.

Example 52 Suppose A is a 4 4 matrix and v 1 , v 2 , v 3 , v 4 are four linearly independent

vectors which satisfy:

Av 1 = v 2 , Av 2 = v 3 , Av 3 = v 4 , Av 4 = 3v 1 v 2 + 6v 3

If we let V be the matrix whose columns are v 1 , v 2 , v 3 , v 4 , then V is invertible and we can
immediately write down that

0 0 0 3
1 0 0 1
V 1 AV =
0 1 0

6
0 0 1 0

Example 53 B&B

Some properties:

Exercise 53 Prove the following statements:

(a) If A is similar to B, then B is similar to A

(b) If A is similar to B and B is similar to C, then A is similar to C.

Exercise 54 Show that if A is similar to then the following hold:

121
 (a) det A = det

(b) charpoly A = charpoly and, hence, A and have the same eigenvalues with the
same algebraic multiplicities.

Exercise 55 Suppose A is a 3 3 matrix, b is a 3-vector,

A3 + 2A2 + A + 1 = 0

and the matrix

T = b Ab A2 b

is invertible. What is the matrix := T 1 AT ? Specify each element of explicitly.

122
5.7 Hotel California
5.7.1 Invariant subspaces
Suppose A is a square matrix. We have already met several subspaces associated with A,
namely, its range, null space, and eigenspaces. These subspaces are all examples of invariant
subspaces for A
We say that a subspace S is an invariant subspace for A if for each vector x in S, the
vector Ax is also in S.

Figure 5.6: Invariant subspace

The significance of invariant subspaces for LTI systems lies in the following result which
is proven later.

If the state of a linear-time-invariant system starts in an invariant subspace of the system

matrix, it never leaves the subspace.

A coordinate transformation. Suppose S is an invariant subspace for a square matrix

A. If S has dimension m, let
t1 , t 2 , , t m
be a basis for S. If A is n n, extend the above basis to a basis
t1 , . . . , tm , tm+1 , , tn
for Cn (This can always be done.) Now introduce the coordinate transformation
x = T
where
T := t1 t2 . . . tn
Letting

= ,

where is an m-vector and is an (nm)-vector, we obtain
x = 1 t1 + . . . + m tm + 1 tm+1 + . . . + nm tn

123
So
x is in S if and only if =0
Let
:= T 1 AT .
Using our useful result on similarity transformations and the fact that S is invariant for A,
it follows that must have the following structure:

11 21
=
0 22

System significance of invariant subspaces. Consider a system described by

x = Ax

and suppose S is an invariant subspace for A. Introduce the above coordinate transformation
to obtain
=
or

= 11 + 12
= 22

Looking at the differential equation = 22 , we see that if (0) = 0, then (t) 0; hence,
if x(0) is in S then x(t) is always in S. In other words, if you are in S, you can check out
any time, but you can never leave.....

5.7.2 More on invariant subspaces

Suppose S1 , S2 , . . . , Sq are q invariant subspaces of a square matrix A with the property that
there is basis
t1 , . . . , t n
where
t 1 , . . . , t n1
is a basis for S1 ,
tn1 +1 , . . . , tn1 +n2
is a basis for S2 ,
tn1 ++nq1 +1 , . . . , tn
is a basis for Sq ,
x = Ax
Coordinate transformation
x = T

124
where
T = t1 t2 . . . t n
that is, ti is the i-th column of T . Consider

= T 1 AT

Claim:
= blockdiag (1 , . . . , q )

125
5.8 Diagonalizable systems
5.8.1 Diagonalizable matrices
We say that a square matrix A is diagonalizable if it is similar to a diagonal matrix, that is,
there is an invertible matrix T and a diagonal matrix such that

= T 1 AT .

In this section, we show that A is diagonalizable if and only if it has the following property:
For each distinct eigenvalue of A,

geometric multiplicity = algebraic multiplicity

Recall that the geometric multiplicity of an eigenvalue is the dimension of its eigenspace,
or equivalently, the maximum number of linearly independent eigenvectors associated with
that eigenvalue. An eigenvalue whose geometric multiplicity equals its algebraic multiplicity
is call a nondefective eigenvalue. Otherwise, it is considered defective. A matrix with the
property that all its eigenvalues are nondefective is called a nondefective matrix. Otherwise,
it is considered defective. When is a matrix A nondefective? Some examples are:

(a) A has n distinct eigenvalues where A is n n.

(b) A is hermitian that is, A = A.

We first need the following result.

Fact 4 Suppose that geometric multiplicity equals algebraic multiplicity for each distinct
eigenvalue of a square matrix A. Then A has a basis of eigenvectors:

v1, v2, . . . , vn

Example 54
0 1
A=
0 0

Theorem 2 Suppose that for each distinct eigenvalue of A, its geometric multiplicity is the
same as its algebraic multiplicity . Then A is similar to a diagonal matrix whose diagonal
elements are the eigenvalues of A. (If an eigenvalue has multiplicity m, it appears m times
on the diagonal of )

Proof. Let
T := v1 v2 . . . vn
where v 1 , . . . , v n is a basis of eigenvectors for A. Let i be the eigenvalue corresponding to
v i . Then,
Av i = i v i

126
Defining
:= T 1 AT
it follows from the useful result on similarity transformations that

= diag (1 , 2 , . . . , n )

1 0 . . . 0
0 2 . . . 0

:= .. . . . ..
.
.
0 0 . . . n
| {z }
Complex Jordan form of A
So A is similar to the diagonal matrix .

Example 55
4 2 2
A= 0 2 2
0 2 2
The characteristic polynomial of A is

p(s) = det(sI A) = (s 4)[(s 2)(s 2) 4] = s(s 4)2

So A has eigenvalues,
1 = 0, 2 = 4
with algebraic multiplicities 1 and 2.
The equation (A I)v = 0 yields:

For 1 = 0,

4 2 2 4 0 4 1 0 1
rref(A 1 I) = rref 0 2 2 = rref 0 2 2 = 0 1 1
0 2 2 0 0 0 0 0 0

Hence, 1 has geometric multiplicity 1 and its eigenspace is spanned by

1
1
v = 1
1

For 2 = 4,

0 2 2 0 2 2 0 1 1
rref(A 2 I) = rref 0 2 2 = rref 0 0 0 = 0 0 0
0 2 2 0 0 0 0 0 0

127
Hence, 2 has geometric multiplicity 2 and its eigenspace is spanned by

1 0
v2 = 0 and v 3 = 1
0 1

Since the geometric multiplicity of each eigenvalue is equal to its algebraic multiplicity,
A is diagonalizable and its Jordan form is

0 0 0
= 0 4 0
0 0 4

A quick check with MATLAB yields = T 1 AT where T = (v 1 v 2 v 3 ).

5.8.2 Diagonalizable continuous-time systems

Consider a continuous-time system
x = Ax
and suppose A is an n n diagonalizable matrix, that is, there is a nonsingular matrix T
such that
= T 1 AT
is diagonal. Hence, the diagonal elements of are the eigenvalues (including possible mul-
tiplicities) 1 , 2 , . . . , n of A. Introducing the state transformation,

x = T

we have = T 1 x and the system behavior is now equivalently described by

= .

This yields
1 = 1 1
2 = 2 2
..
.
n = n n

that is, n decoupled first order systems. Hence, for i = 1, 2, . . . , n,

i (t) = ei t i0 (5.10)

where i0 = (0). In vector form, we obtain

e1 t 0 ... 0
0 e 2 t ... 0

(t) = .. ... .. 0 (5.11)
. .
0 0 . . . e n t

128
 T
where 0 = 10 20 n0 . Since x(t) = T (t), we have
x(t) = 1 (t)v 1 + 2 (t)v 2 + . . . + n (t)v n
where v 1 , v 2 , . . . , v n are the columns of T . It now follows from (5.10) that the solution for
x(t) is given by
x(t) = e1 t 10 v 1 + e2 t 20 v 2 + . . . + en t n0 v n (5.12)
Note that each term ei t i0 v i on the right-hand-side of the above equation is a mode. This
results in the following fundamental observation.
Every motion (solution) of a diagonalizable linear time-invariant system is simply a sum
of modes.
Suppose x0 is the initial value of x at t = 0, that is, x(0) = x0 . Then, the corresponding
initial value 0 = (0) of is given by
0 = T 1 x0 .

State transition matrix. Since x(t) = T (t) and 0 = T 1 x0 , it follows from (5.11) that
x(t) = (t)x0 (5.13)
where the state transition matrix () is given by

e1 t 0 ... 0
0 e2 t ... 0
1
(t) := T .. .. .. T (5.14)
. . .
0 0 . . . en t

Example 56 Consider a CT system with A from example 37, that is,

3 1
A=
1 3
Here,
1 1
T =
1 1
and 1 = 2, 2 = 4.
Hence,

e2t 0
(t) = T T 1
0 e4t

1 e2t + e4t e2t e4t
=
2 e2t e4t e2t + e4t
and all solutions are given by
1 2t 1 2t
x1 (t) = ( e + e4t )x10 + ( e e4t )x20
2 2
1 2t 1 2t
x2 (t) = ( e e4t )x10 + ( e + e4t )x20
2 2

129
Exercise 56 Compute the state transition matrix for the CT system corresponding to the
A matrix in example 55.

5.8.3 Diagonalizable DT systems

The story here is basically the same as that for CT. Consider a DT system

x(k + 1) = Ax(k)

and suppose A is diagonalizable, that is, there is a nonsingular matrix T such that

= T 1 AT

is diagonal and the diagonal elements of are the eigenvalues 1 , . . . , n of A.

Introducing the state transformation,

x = T

we have = T 1 x and its behavior is described by:

(k + 1) = (k)

Hence,

1 (k + 1) = 1 1 (k)
2 (k + 1) = 2 2 (k)
..
.
n (k + 1) = n n (k)

If
x(0) = x0
then
(0) = 0 = T 1 x0
So,
i (k) = ki i0 , i = 1, 2, . . . , n
or,
k1 0 . . . 0
0 k2 . . . 0

(k) = .. .. . 0
. . ..
0 0 . . . kn
Since
x = 1 v 1 + 2 v 2 + . . . + n v n

130
where v 1 , v 2 , . . . , v n are the columns of T , the solution for x(k) is given by

x(k) = k1 10 v 1 + k2 20 v 2 + . . . + kn n0 v n

Or,
x(k) = (k)x0
where the DT state transition matrix () is given by

k1 0 . . . 0
0 k2 . . . 0
1
(k) := T .. .. . T
. . ..
0 0 . . . kn

Example 57 Consider a DT system with A from example 37, that is,

3 1
A=
1 3

Here,
1 1
T =
1 1
and 1 = 2, 2 = 4.
Hence,

2k 0
(k) = T T 1
0 4k

1 2k + 4k 2k 4k
=
2 2k 4k 2k + 4k

and all solutions are given by

x1 (t) = ( 2k + 4k )x10 + ( 2k 4k )x20

x2 (t) = ( 2k 4k )x10 + ( 2k + 4k )x20

131
5.9 Real Jordan form for real diagonalizable systems
Suppose A is real and has a complex eigenvalue
= +
where , are real. Then any corresponding eigenvector v is complex and can be expressed
as
v = u + w
where u, w are real and are linearly independent over the real scalars.
Since A is real, is also an eigenvector of A with eigenvector v. So we can replace each v
and v in the basis of eigenvectors with the corresponding u and w to obtain a new basis; then
we define a coordinate transformation where the columns of T are these new basis vectors.
The relationship,
Av = v
and A real implies
Au = u w
Aw = u + w

Hence, using the useful result on similarity transformations, we can choose T so that
= T 1 AT is given by

1 1 m m
= block-diag 1 , . . . , l , ,...,
1 1 m m

1
...


l

1 1
=
1 1
.
..

m m
m m
| {z }
Real Jordan form of A
where 1 , . . . , l are real eigenvalues of A and 1 + 1 , . . . , m + m are genuine complex
eigenvalues.
Example 58 Recall example 38.
Example 59
0 1 0 0
0 0 1 0
A=
0

0 0 1
1 0 0 0

132
 By inspection (recall companion matrices),

det(sI A) = s4 1

So eigenvalues are:
1 = 1 2 = 1 2 = 3 =
Since there are n = 4 distinct eigenvalues, the complex Jordan form is:

1 0 0 0
0 1 0 0

0 0 0
0 0 0

The real Jordan form is:

1 0 0 0
0 1 0 0

0 0 0 1
0 0 1 0

5.9.1 CT systems
The solutions of
x = Ax, x(0) = x0
are given by
x(t) = (t)x0
where the state transition matrix () is given by

(t) = T (t)T 1

with (t) being the block diagonal matrix:

1 t l t e1 t cos(1 t) e1 t sin(1 t) em t cos(m t) em t sin(m t)
block-diag e ,...,e , ,...,
e1 t sin(1 t) e1 t cos(1 t) em t sin(m t) em t cos(m t)

5.10 Diagonalizable second order systems

Only two cases for real Jordan form

1 0
0 2

where all the above scalars are real.

133
5.10.1 State plane portraits for CT systems
Real eigenvalues

Genuine complex eigenvalues

5.11 Exercises

134
Exercise 57 Compute the eigenvalues and eigenvectors of the matrix,

2 3 0
A = 2 3 0
3 5 1
.

Exercise 58 Compute the eigenvalues and eigenvectors of the matrix,

1 2 1
A= 1 2 1
1 2 3
.

Exercise 59 Compute the eigenvalues and eigenvectors of the following matrix.

0 1 0 0
0 0 1 0
A= 0 0 0 1

1 0 0 0

Exercise 60 Compute expressions for eigenvalues and linearly independent eigenvectors of

A= , 6= 0

Exercise 61 Find expressions for the eigenvalues and linearly independent eigenvectors of
matrices of the form:


where and are real with 0. Obtain state plane portraits for

=0 =1
= 3 =2
=3 =2

Exercise 62 Consider an LTI system described by x = Ax.

(a) Suppose that the vectors

1 1
1 1
and
1 1
1 1

135
are eigenvectors of A corresponding to eigenvalues 2 and 3, respectively. What is the
response x(t) of the system to the initial condition

1
0
x(0) = 1 ?

(b) Suppose A is a real matrix and the vector

1


1

is an eigenvector of A corresponding to the eigenvalue 2 + 3. What is the response x(t) of

the system to the initial condition

1
0
x(0) =
1 ?
0

Exercise 63 Suppose A is a square matrix and the vectors

1 1
2 and 1
3 1

are eigenvectors of A corresponding to eigenvalues 1 and 1, respectively. What is the

response x(t) of the system x = Ax to the following initial conditions.

2 2 0
(a) x(0) = 4 (b) x(0) = 2 (c) x(0) = 2 ?
6 2 4

(d) Suppose A is a real matrix and the vector

1 + 2
1
3

is an eigenvector of A corresponding to the eigenvalue 2 3. What is the response x(t) of

the system x = Ax to the initial condition

1
x(0) = 1 ?
3

136
Exercise 64 Suppose A is a real square matrix and the vectors

1 1 1
1 2

1 4 1
1 8
are eigenvectors of A corresponding to eigenvalues 1 and 2 and , respectively. What is the
response x(t) of the system x = Ax to the following initial conditions.

2 1 0 1
2 2 3 1
(a) x(0) = 2
(b) x(0) = 4
(c) x(0) = 3
(d) x(0) = 1
?

2 8 9 1
Exercise 65 Suppose A is a real square matrix with characteristic polynomial
p(s) = (s + 2)(s2 + 4)2
Suppose that all eigenvectors of A are nonzero multiples of

1 0 0
2 1+ 1

3 , or

4 0 0
0 1 1
What is the solution to x = Ax with the following initial conditions.

2 0 0
4 1 0

(a) x(0) = 6 ,
(b) x(0) =
0 , (c) x(0) =
1


8 0 0
0 1 1
(d) All all solutions of the system x = Ax bounded? Justify your answer.
Exercise 66 Consider a discrete-time LTI system described by x(k+1) = Ax(k).
(a) Suppose that the vectors

1 1
1 1
and
1 1
1 1
are eigenvectors of A corresponding to eigenvalues 2 and 3, respectively. What is the
response x(k) of the system to the initial condition

1
0
x(0) =
1 ?
0

137
(b) Suppose A is a real matrix and the vector

1


1

is an eigenvector of A corresponding to the eigenvalue 2 + 3. What is the response x(k) of

the system to the initial condition

1
0
x(0) =
1 ?
0

Exercise 67 Determine whether or not the following matrix is nondefective.

0 1 0
A= 0 0 1
1 1 1

Exercise 68 In vibration analysis, one usually encounters systems described by

M q + Kq = 0

where q(t) is an N -vector and M and K are square matrices. Suppose is a real number
which satisfies
det( 2 M K) = 0 .
Show that the above system has a solution of the form

q(t) = sin(t)v

where v is a constant N -vector.

Exercise 69 Determine whether or not the matrix,

0 1 0
A= 0 0 1 ,
0 2 3

is diagonalizable, that is, determine whether or not there exists an invertible matrix T such
that T 1 AT is diagonal. If A is diagonalizable, determine T .

Problem 1 Recall that a square matrix is said to be diagonalizable if there exists an invert-
ible matrix T so that T 1 AT is diagonal. Determine whether or not each of the following
matrices are diagonalizable. If not diagonalizable, give a reason. If diagonalizable, obtain T .

138
(a)
0 1 0
A= 0 0 1
0 1 0
(b)

0 1
A=
1 2
(c)
1 0 0
A= 0 0 1
0 1 0

139
140
Chapter 6

eAt

Recall that all solutions to the scalar system x = ax satisfy x(t) = ea(tt0 ) x(t0 ) for all t and
t0 . Recall also the power series expansion for eat :
1 1
eat = 1 + at + (at)2 + + (at)k +
2! k!
The main result of this chapter is that all solutions of x = Ax satisfy

x(t) = eA(tt0 ) x(t0 ) (6.1)

for all t and t0 where

1 1
eAt = 1 + At + (At)2 + + (At)k + (6.2)
2! k!

6.1 State transition matrix

6.1.1 Continuous time
Consider the initial value problem specified by

x = Ax and x(t0 ) = x0 . (6.3)

It can be shown that for each initial time t0 and each initial state x0 there is an unique
solution x() to the above initial value problem and this solution is defined for all time t.
Let (t; t0 , x0 ) be the value of this solution at t, that is, for any t0 and x0 , the solution to
(6.3) is given by x(t) = (t; t0 , x). Since the system x = Ax is time-invariant, we must have
(t; t0 , x0 ) = (tt0 ; x0 ) for some function . Since the system under consideration is linear,
one can show that, for any t and t0 , the solution at time t depends linearly on the initial
state x0 at time t0 ; hence there exists a matrix (tt0 ) such that (tt0 , x0 ) = (tt0 )x0 .
We call the matrix valued function () the state transition matrix for the system x = Ax.
So, we have concluded that there is a matrix valued function () such that for any t0 and
x0 , the solution x() to the above initial value problem is given by

x(t) = (tt0 )x0

141
 We now show that must satisfy the following matrix differential equation and initial
condition:
= A
(0) = I

To demonstrate this, consider any initial state x0 and let x() be the solution to (6.3)
with t0 = 0. Then x(t) = (t)x0 . Considering t = 0, we note that

x0 = x(0) = (0)x0 ;

that is (0)x0 = x0 . Since the above holds for any x0 , we must have (0) = I.
We now note that for all t,

(t)x0 = x(t) = Ax(t) = A(t)x0 ;

that is (t)x0 = A(t)x0 . Since the above must hold for any x0 , we obtain that (t) = A(t).
Note that the above two conditions on uniquely specify (Why?) The purpose of this
section is to examine and look at ways of evaluating it.

142
6.1.2 Discrete time

x(k+1) = Ax(k) and x(k0 ) = x0

We have
x(k) = (kk0 )x0
The matrix valued function () is called the state transition matrix and is given by

(k) = Ak

Note that, if A is singular, (k) is not defined for k < 0. That was easy; now for CT.

143
6.2 Polynomials of a square matrix
6.2.1 Polynomials of a matrix
We define polynomials of a matrix as follows. Consider any polynomial p of a scalar variable
s given by:
p(s) = a0 + a1 s + a2 s2 + . . . + am sm
where a0 , . . . , am are scalars. If A is a square matrix, we define p(A) as follows:
p(A) := a0 I + a1 A + a2 A2 + . . . + am Am
that is, we replace sk with Ak . (Recall that s0 = 1 and A0 = I)
The following properties can be readily deduced from the above definition.
(a) The matrices A and p(A) commute, that is,
Ap(A) = p(A)A
(b) Consider any nonsingular matrix T , and suppose A = T T 1 . Then
p(A) = T p()T 1
(c) Suppose A is diagonal, that is,

1 0 0
0 2 0

A= .. .. .
. . ..
0 0 n
then
p(1 ) 0 0
0 p(2 ) 0

p(A) = .. .. ..
. . .
0 0 p(n )
(d) Suppose A is diagonalizable, that is, for some nonsingular T ,

1 0 0
0 2 0
1
A = T .. . . .. T
. . .
0 0 n
Then
p(1 ) 0 0
0 p(2 ) 0
1
p(A) = T .. ... .. T
. .
0 0 p(n )
(e) If is an eigenvalue of A with eigenvector v, then p() is an eigenvalue of p(A) with
eigenvector v.
Properties (b)-(d) are useful for computing p(A).

144
Example 60 For
3 1
A=
1 3
we found that
1 1
T =
1 1
results in
1 2 0
=T AT =
0 4
Hence, A = T T 1 and

p(A) = T p()T 1

p(2) 0
= T T 1
0 p(4)

1 p(2) + p(4) p(2) p(4)
=
2 p(2) p(4) p(2) + p(4)

Suppose p is the characteristic polynomial of A. Since 2, 4 are the eigenvalues of A, we have

p(2) = p(4) = 0; hence
p(A) = 0

6.2.2 Cayley-Hamilton Theorem

The following result is a fundamental result in linear algebra and is very useful in systems
and control.

Theorem 3 If p is the characteristic polynomial of a square matrix A, then

p(A) = 0 .

Proof. The proof is easy for a diagonalizable matrix A. Recall that such a matrix is
similar to a diagonal matrix whose diagonal elements 1 , . . . , n are the eigenvalues of A;
hence p(1 ) = . . . = p(n ) = 0. Now use property (d) above. We will leave the proof of the
general nondiagonalizable case for another day.

Suppose the characteristic polynomial of A is

det(sI A) = a0 + a1 s + . . . + an1 sn1 + sn

Then the Cayley Hamilton theorem states that

a0 I + a1 A + . . . + an1 An1 + An = 0

145
hence,
An = a0 I a1 A . . . an1 An1
From this one can readily show that for any m n, Am can be expressed as a linear
combination of I, A, . . . , An1 . Hence, any polynomial of A can be can be expressed as a
linear combination of I, A, . . . , An1 .
Example 61 Consider a discrete-time system described by x(k +1) = Ax(k) and suppose
that all the eigenvalues of A are zero. Then the characteristic polynomial of A is given by
p(s) = sn where A is n n. It now follows from the Cayley Hamilton theorem that
An = 0
Since all solutions of the system satisfy x(k) = Ak x(0) for k = 0, 1, 2, . . ., it follows that
x(k) = 0 for k n.
Thus, all solutions go to zero in at most n steps.

Minimal polynomial of A
The minimal polynomial (minpoly) of a square matrix A is the monic polynomial q of lowest
order for which q(A) = 0.

The order of minpoly q is always less than or equal to the order of charpoly p; its roots are
precisely the eigenvalues of A and p(s) = g(s)q(s).
Example 62 The matrix
0 0
A=
0 0
has charpoly p(s) = s2 and minpoly q(s) = s. The matrix

0 1
A=
0 0
has charpoly p(s) = s2 and minimal poly q(s) = s2 .
It follows from property (d) above that for a diagonalizable matrix A, its minimal polyno-
mial is
Y l
(s i )
i=1
where i are the distinct eigenvalues of A.
Exercise 70 Without doing any matrix multiplications, compute A4 for

0 1 0 0
0 0 1 0
A= 0 0 0 1

564 1 0 0
Justify your answer.

146
6.3 Functions of a square matrix
Consider a complex valued function f of a complex variable . Suppose f is analytic in some
open disk { C : || < } of radius > 0. Then, provided || < , f () can be expressed
as the sum of a convergent power series; specifically

X
f () = ak k = a0 + a1 + a2 2 + . . .
k=0

where
1 (k) 1 dk f
ak = f (0) = (0)
k! k! dk
Suppose A is a square complex matrix
P and |i | < for every eigenvalue i of A. Then,
it can be shown that the power series { nk=0 ak Ak }
n=0 is convergent. We define f (A) to be
the sum of this power series, that is,

X
f (A) := ak Ak = a0 I + a1 A + a2 A2 + . . .
k=0

Example 63 Consider
1
f () =
1
This is analytic in the open unit disk ( = 1) and

X
f () = k = 1 + + 2 + . . .
k=0

Pn
Consider any matrix A with eigenvalues in the open unit disk. Then the series { k=0 Ak }
n=0
is convergent and we let

X
f (A) := Ak
k=0

Multiplication of both sides of the above equality by I A yields

X
(I A)f (A) = (Ak Ak+1 ) = I;
k=0

hence the matrix I A is invertible with inverse f (A) and we obtain the following power
series expansion for (I A)1 :

X
1
(I A) = Ak = I + A + A2 + . . .
k=0

147
Power series expansion for (sI A)1 . We present here an expansion which will be
useful later on. Consider any square matrix A and any complex number s which satisfies:

|s| > |i | for every eigenvalue i of A .

Then sI A is invertible and (sI A)1 = 1s (I 1s A)1 . Also, the eigenvalues of 1s A are in
the open unit disk; hence
X 1
1 1 1
(I A)1 = k
Ak = I + A + 2 A2 + . . .
s k=0
s s s

and
X
1 1 k 1 1 1
(sI A) = k+1
A = I + 2 A + 3 A2 + . . .
k=0
s s s s

Properties of f (A). The following properties can be readily deduced from the above
definition of f (A).

(a) The matrices A and f (A) commute, that is,

Af (A) = f (A)A .

(b) Consider any nonsingular matrix T and suppose A = T T 1 . Then

f (A) = T f ()T 1 .

(c) Suppose A is diagonal, that is,

1 0 0
0 2 0

A= .. . . . ..
. .
0 0 n

then
f (1 ) 0 0
0 f (2 ) 0

f (A) = .. ... ..
. .
0 0 f (n )

(d) Suppose A is diagonalizable, that is, for some nonsingular T ,

1 0 0
0 2 0
1
A = T .. . . .. T
. . .
0 0 n

148
 Then
f (1 ) 0 0
0 f (2 ) 0
1
f (A) = T .. .. .. T
. . .
0 0 f (n )

(e) If is an eigenvalue of A with eigenvector v, then f () is an eigenvalue of f (A) with

eigenvector v.

It follows from Cayley Hamilton, that f (A) is a linear combination of I, A, . . . , An1

Properties (b)-(d) are useful for calculating f (A). When A is diagonalizable, one simply has
to evaluate the scalars f (1 ), . . . , f (n ) and then compute T diag (f (1 ), . . . , f (n )) T 1 .

Example 64 For
3 1
A=
1 3
we found that
1 1
T =
1 1
results in
1 2 0
=T AT =
0 4
Hence, A = T T 1 and

f (A) = T f ()T 1

f (2) 0
= T T 1
0 f (4)

1 f (2) + f (4) f (2) f (4)
=
2 f (2) f (4) f (2) + f (4)

Exercise 71 Compute an explicit expression (see last example) for f (A) where

9 2
A=
2 6

149
6.3.1 The matrix exponential: eA
Recall the exponential function:
f () = e
This function is analytic in the whole complex plane (that is = ) and
d k e
= e
dk
for k = 1, 2, . . .; hence for any complex number ,
X
1 k 1 1
e = = 1 + + 2 + 3 + . . .
k=0
k! 2 3!

Thus the exponential of any square matrix A is defined by:

X
A 1 k 1 2
e := A = I +A+ A + ...
k=0
k! 2!

and this power series converges for every square matrix A. Some properties:
(a) For any zero matrix 0,
e0 = I

(b) Suppose t is a scalar variable.

deAt
= AeAt = eAt A
dt

The following results also hold:

(i)
eA(t1 +t2 ) = eAt1 eAt2
eA = (eA )1

(ii) If is an eigenvalue of A with eigenvector v, then e is an eigenvalue of eA with

eigenvector v. So, eA has no eigenvalue at 0 and hence:
(iii) eA is nonsingular.
(iv) The following result also holds
eA+B = eA eB if A and B commute

6.3.2 Other matrix functions

cos(A)
sin(A)

150
6.4 The state transition matrix: eAt
Using the above definitions and abusing notation (eAt instead of etA ),

X
1 1 1
eAt := (tA)k = I + tA + (tA)2 + (tA)3 + . . .
k=0
k! 2! 3!

Letting
(t) := eAt
we have

(0) = I
= A

Hence, the solution of

x = Ax x(0) = x0
is given by
x(t) = eAt x0

151
6.5 Computation of eAt
6.5.1 MATLAB

>> help expm

EXPM Matrix exponential.

EXPM(X) is the matrix exponential of X. EXPM is computed using
a scaling and squaring algorithm with a Pade approximation.
Although it is not computed this way, if X has a full set
of eigenvectors V with corresponding eigenvalues D, then
[V,D] = EIG(X) and EXPM(X) = V*diag(exp(diag(D)))/V.
See EXPM1, EXPM2 and EXPM3 for alternative methods.

EXP(X) (thats without the M) does it element-by-element.

6.5.2 Numerical simulation

Let i be the i-th column of the state transition matrix (t) = eAt ; thus

(t) = 1 (t) 2 (t) n (t) .

Let x be the unique solution to x = Ax and x(0) = x0 . Since x(t) = (t)x0 we have

x(t) = x01 1 (t) + + x0n n

where x0i is the i-th component of x0 . Hence i , is the solution to

x = Ax x(0) = ei

where ei is all zeros except for its i-th component which is 1.

6.5.3 Jordan form

eAt = T et T 1
where = T 1 AT and is the Jordan form of A. We have already seen this for diagonal-
izable systems.

6.5.4 Laplace style

x = Ax x(0) = x0

152
Suppose
X(s) = L(x)(s)
is the Laplace transform of x() evaluated at s C. Taking the Laplace transform of x = Ax
yields:
sX(s) x0 = AX(s)
Hence, except when s is an eigenvalue of A, sI A is invertible and
L(x)(s) = X(s) = (sI A)1 x0
Since x(t) = eAt x0 for all x0 , we must have

L(eAt ) = (sI A)1

eAt = L1 ((sI A)1 )

Example 65 Recall
3 1
A=
1 3
So,
s3 1
(s2)(s4) (s2)(s4)
(sI A)1 =
1 s3
(s2)(s4) (s2)(s4)

Since

s3 1 1 1
= +
(s 2)(s 4) 2 s2 s4

1 1 1 1
=
(s 2)(s 4) 2 s2 s4

and
1 1
L1 ( ) = e2t L1 ( ) = e4t
s2 s4
we have,

eAt = L1 (sI A)1

1 e2t + e4t e2t e4t
=
2 e2t e4t e2t + e4t

Exercise 72 Compute eAt at t = ln(2) for

0 1
A=
1 0
using all methods of this section.

153
6.6 Sampled-data systems
Here we discuss the process of discretizing a continuous-time system to obtain a discrete-time
system.
Consider a continuous-time LTI system described by

xc (t) = Ac xc (t) .

Suppose we only look at the behavior of this system at discrete instants of time, namely

, 2T, T, 0, T, 2T,

where T > 0 is called the sampling time.

Figure 6.1: Sampling

If we introduce the discrete-time state defined

xd (k) = xc (kT ) k = . . . , 2, 1, 0, 1, 2, . . . ,

then,

xd (k+1) = xc ( (k+1)T )
= eAc T xc (kT )
= eAc T xd (k)

Hence, this sampled data system is described by the discrete-time system

xd (k+1) = Ad xd (k)

where
Ad = eAc T

154
 The eigenvalues of Ad are
e1 T , . . . el T
where 1 , . . . , l are the eigenvalues of Ac . Note that,

|ei T | < 1 iff <(i ) < 0

|ei T | = 1 iff <(i ) = 0
|ei T | > 1 iff <(i ) > 0

e i T = 1 iff i = 0

It should be clear that all eigenvalues of Ad are nonzero, that is Ad in invertible.

Note that if we approximate eAc T by the first two terms in its power series, that is,

eAc T I + Ac T

then,
Ad I + Ac T .

6.7 Exercises
Exercise 73 Suppose A is a 3 3 matrix, b is a 3-vector and

det(sI A) = s3 + 2s2 + s + 1

(a) Express A3 b in terms of b, Ab, A2 b.

(b) Suppose that the matrix

T = b Ab A2 b

is invertible. What is the matrix := T 1 AT ? Specify each element of explicitly.

Exercise 74 Suppose A is a 3 3 matrix and

det(sI A) = s3 + 2s2 + s + 1

(a) Express A3 in terms of I, A, A2 .

(b) Express A5 in terms of I, A, A2 .

(c) Express A1 in terms of I, A, A2

Exercise 75 Suppose A is a 3 3 matrix with eigenvalues 1, 2 and 2. What are the

eigenvalues of eA .

155
Exercise 76 Consider the differential equation

x = Ax (6.4)

where A is a square matrix. Show that if A has 2 as an eigenvalue, then there is an nonzero
initial state x0 such that (6.4) has a solution x which satisfies x(1) = x(0) = x0 .

Exercise 77 Suppose A is a square matrix.

(a) Obtain expressions for
cos(At) and sin(At) .
(b) Show that

cos(At) = (eAt + eAt )/2 and sin(At) = (eAt eAt )/2

(c) Show that

d cos(At) d sin(At)
= A sin(At) and = A cos(At) .
dt dt
Exercise 78 Compute eAt for the matrix

3 5
A=
5 3

Exercise 79 Compute eAt for the matrix,

11 20
A= .
6 11

Exercise 80 Compute eAt for the matrix

0 1
A=
1 0

Exercise 81 Suppose A is a matrix with four distinct eigenvalues: 1, +1 j, j. Show that

A4 = I.

Exercise 82 Suppose
0 1 0 0
0 0 1 0
A=
0

0 0 1
0 0 0 0
What is A20 + 4A4 ?

Exercise 83 Compute eAt for the matrix,

3 2
A= .
4 3

156
Exercise 84 Compute eAt for the matrix

3 5
A=
5 3

Exercise 85 Compute eAt at t = ln(2) for

0 1
A=
1 0

using all four methods mentioned in the notes.

157
158
Chapter 7

Stability and boundedness

Consider a general nonlinear system described by

x = f (x) (7.1)

where x(t) is a real n-vector and t is a real scalar. By a solution of (7.1) we mean any
continuous function x() : [0, t1 ) IRn with t1 > 0, which satisfies x(t) = f (x(t)) for
0 t < t1 .

7.1 Boundedness of solutions

DEFN. (Boundedness) A solution x() is bounded if there exists 0 such that

||x(t)|| for all t0

A solution is unbounded if it is not bounded.

It should be clear from the above definitions that a solution x() is bounded if and only
if each component xi () of x() is bounded. Also, a solution is unbounded if and only if
at least one of its components is unbounded. So, x(t) = [ et e2t ]T is bounded while
x(t) = [ et et ]T is unbounded.

Example 66 All solutions of

x = 0
are bounded.

Example 67 All solutions of

x = x x3
are bounded.

159
Example 68 All solutions (except the zero solution) of

x = x

are unbounded.

Example 69 Consider
x = x2 , x(0) = x0
If x0 > 0, the corresponding solution has a finite escape time and is unbounded. If x0 < 0,
the corresponding solution is bounded.

Example 70 Undamped oscillator.

x1 = x2
x2 = x1

All solutions are bounded.

Boundedness and linear time-invariant systems. Consider a general LTI (linear time-
invariant) system
x = Ax (7.2)
Recall that every solution of this system has the form
l n
X X i 1

x(t) = tj ei t v ij
i=1 j=0

where 1 , . . . , l are the eigenvalues of A, ni is the index of i , and the constant vectors v ij
depend on initial state.
We say that an eigenvalue of A is non-defective if its index is one; this means that the
algebraic multiplicity and the geometric multiplicity of are the same. Otherwise we say
is defective.
Hence we conclude that all solutions of (7.2) are bounded if and only if for each eigenvalue
i of A:
(b1) <(i ) 0 and

(b2) if <(i ) = 0 then i is non-defective.

If there is an eigenvalue i of A such that either
(u1) <(i ) > 0 or

(u2) <(i ) = 0 and i is defective

then, the system has some unbounded solutions.

160
Example 71 Unattached mass

x1 = x2
x2 = 0

Here
0 1
A=
0 0
has a single eigenvalue 0. This eigenvalue has algebraic multiplicity 2 but geometric multi-
plicity 1; hence some of the solutions of the system are unbounded. One example is

t
x(t) =
1

Example 72

x1 = 0
x2 = 0

Here
0 0
A=
0 0
has a single eigenvalue 0. This eigenvalue has algebraic multiplicity 2 and geometric mul-
tiplicity 2. Hence all the solutions of the system are bounded. Actually every state is an
equilibrium state and every solution is constant.

Example 73 (Resonance) Consider a simple linear oscillator subject to a sinusoidal input

of amplitude W :
q + q = W sin(t+)
Resonance occurs when = 1. To see this, let

x1 := q , x2 := q , x3 := W sin(t+) , x4 := W cos(t+)

to yield
x = Ax
where
0 1 0 0
1 0 1 0
A=
0

0 0 1
0 0 2 0
If = 1 then, A has eigenvalues and . These eigenvalues have algebraic multiplicity two
but geometric multiplicity one; hence the system has unbounded solutions.

161
7.2 Stability of equilibrium states
Suppose xe is an equilibrium state of the system x = f (x). Then, whenever x(0) = xe , we
have (assuming uniqueness of solutions) x(t) = xe for all t 0. Roughly speaking, we say
that xe is a stable equilibrium state for the system if the following holds. If the initial state
of the system is close to xe , then the resulting solution is close to xe . The formal definition
is as follows.

DEFN. (Stability) An equilibrium state xe is stable if for each > 0 there exists > 0 such
that whenever ||x(0)xe || < one has ||x(t)xe || < for all t 0.

Figure 7.1: Stability of an equilibrium state

An equilibrium state xe is said to be unstable if it is not stable.

Example 74
x = 0
Every equilibrium state is stable. (Choose = .)

Example 75
x = x
The origin is unstable.

Example 76
x = x x3
The origin is unstable; the remaining equilibrium states 1 and 1 are stable.

Example 77 Undamped oscillator

The origin is stable.

162
Example 78 Simple pendulum

x1 = x2
x2 = sin x1

(0, 0) is stable; (, 0) is unstable.

Example 79 Van der Pol oscillator

x1 = x2
x2 = (1 x21 )x2 x1

Figure 7.2: Van der Pol oscillator

The origin is unstable. However, all solutions are bounded.

Stability of linear time-invariant systems. It can be shown that every equilibrium

state of a LTI system (7.2) is stable if and only if all eigenvalues i of A satisfy conditions
(b1) and (b2) above. Hence every equilibrium state of a LTI system is stable if and only if
all solutions are bounded.
It can also be shown that every equilibrium state is unstable if and only if there is an
eigenvalue i of A which satisfies condition (u1) or (u2) above. Hence every equilibrium
state of a LTI system is unstable if and only if there are unbounded solutions.

163
7.3 Asymptotic stability
7.3.1 Global asymptotic stability

DEFN. (Global asymptotic stability) An equilibrium state xe is globally asymptotically stable

(GAS) if

(a) It is stable

(b) Every solution x() converges to xe with increasing time, that is,

lim x(t) = xe
t

If xe is a globally asymptotically stable equilibrium state, then there are no other equi-
librium states and all solutions are bounded. In this case we say that the system x = f (x)
is globally asymptotically stable.

Example 80 The system

x = x
is GAS.

Example 81 The system

x = x3
is GAS.

7.3.2 Asymptotic stability

In asymptotic stability, we do not require that all solutions converge to the equilibrium state;
we only require that all solutions which originate in some neighborhood of the equilibrium
state converge to the equilibrium state.

DEFN. (Asymptotic stability) An equilibrium state xe is asymptotically stable (AS) if

(a) It is stable.

(b) There exists R > 0 such that whenever ||x(0)xe || < R one has

lim x(t) = xe . (7.3)

t

The region of attraction of an equilibrium state xe which is AS is the set of initial states
which result in (7.3), that is it is the set of initial states which are attracted to xe . Thus, the
region of attraction of a globally asymptotically equilibrium state is the whole state space.

164
 Figure 7.3: Asymptotic stability

Example 82
x = x

Example 83
x = x x3
The equilibrium states 1 and 1 are AS with regions of attraction (, 0) and (0, ),
respectively.

Example 84 Damped simple pendulum

x1 = x2
x2 = sin x1 x2

The zero state is AS but not GAS.

Example 85 Reverse Van der Pol oscillator

x1 = x2
x2 = (1 x21 )x2 + x1

The zero state is AS but not GAS. Also, the system has unbounded solutions.

Figure 7.4: Reverse Van der Pol oscillator

165
LTI systems. For LTI systems, it should be clear from the general form of the solution
that the zero state is AS if and only if all the eigenvalues i of A have negative real parts,
that is,
<(i ) < 0
Also AS is equivalent to GAS.

Example 86 The system

x1 = x1 + x2
x2 = x2

is GAS.

Example 87 The system

x1 = x1 + x2
x2 = x1 x2

is GAS.

7.4 Exponential stability

We now present the strongest form of stability considered in this section.

DEFN. (Global exponential stability) An equilibrium state xe is globally exponentially stable

(GES) with rate of convergence > 0 if there exists > 0 such that every solution satisfies

||x(t)xe || ||x(0)xe || exp(t) for all t0

Example 88
x = x
GES with rate = 1.

Note that global exponential stability implies global asymptotic stability, but, in general,
the converse is not true. This is illustrated in the next example. For linear time-invariant
systems, GAS and GES are equivalent.

Example 89
x = x3
Solutions satisfy
x0
x(t) = p where x0 = x(0) .
1 + 2x20 t
GAS but not GES

166
DEFN. (Exponential stability) An equilibrium state xe is exponentially stable (ES) with rate of
convergence > 0 if there exists R > 0 and > 0 such that whenever ||x(0)xe || < R one
has
||x(t)xe || ||x(0)xe || exp(t) for all t 0

Figure 7.5: Exponential stability

Note that exponential stability implies asymptotic stability, but, in general, the converse is
not true.

Example 90
x = x3
GAS but not even ES

Example 91
x
x =
1 + x2
GAS, ES, but not GES

7.5 LTI systems

Consider a LTI system
x = Ax (7.4)
Recall that every solution of this system has the form
l n
X X i 1

x(t) = tj ei t v ij
i=1 j=0

where 1 , . . . , l are the eigenvalues of A, the integer ni is the index of i , and the constant
vectors v ij depend on initial state. From this it follows that the stability properties of this
system are completely determined by the location of its eigenvalues; this is summarized in
the table below.

167
 The following table summarizes the relationhip between the stability properties of a LTI
system and the eigenproperties of its A-matrix. In the table, unless otherwise stated, a
property involving must hold for all eigenvalues of A.

Stability properties Eigenproperties

Global exponential stability <() < 0

and boundedness

Stability and <() 0

boundedness If <() = 0 then is non-defective.

Instability and some There is an eigenvalue with <() > 0 or

unbounded solutions <() = 0 and is defective.

Example 92 The system

x1 = x1 + x2
x2 = x2

is GES.

Example 93 The system

x1 = x1 + x2
x2 = x1 x2

is GES.

Example 94 The system

x1 = 0
x2 = 0

is stable about every equilibrium point.

Example 95 The system

x1 = x2
x2 = 0

is unstable about every equilibrium point.

168
Figure 7.6: The big picture. For LTI systems, the concepts in each dashed box are equivalent

169
7.6 Linearization and stability
Consider a nonlinear time-invariant system described by

x = f (x)

where x(t) is an n-vector at each time t. Suppose xe is an equilibrium state for this system,
that is, f (xe ) = 0, and consider the linearization of this system about xe :
f e
x = Ax where A = (x ) .
x
The following results can be demonstrated using nonlinear Lyapunov stability theory.
Stability. If all the eigenvalues of the A matrix of the linearized system have negative real
parts, then the nonlinear system is exponentially stable about xe .
Instability. If at least one eigenvalue of the A matrix of the linearized system has a positive
real part, then the nonlinear system is unstable about xe .
Undetermined. Suppose all the eigenvalues of the A matrix of the linearized system have
non-positive real parts and at least one eigenvalue of A has zero real part. Then, based on
the linearized system alone, one cannot predict the stability properties of the nonlinear system
about xe .
Note that the first statement above is equivalent to the following statement. If the
linearized system is exponentially stable, then the nonlinear system is exponentially stable
about xe .

Example 96 (Damped simple pendulum.) Physically, the system

x1 = x2
x2 = sin x1 x2

has two distinct equilibrium states: (0, 0) and (, 0). The A matrix for the linearization of
this system about (0, 0) is
0 1
A=
1 1
Since all the eigenvalues of this matrix have negative real parts, the nonlinear system is
exponentially stable about (0, 0). The A matrix corresponding to linearization about (, 0)
is
0 1
A=
1 1
Since this matrix has an eigenvalue with a positive real part, the nonlinear system is unstable
about (, 0)

The following example illustrates the fact that if the eigenvalues of the A matrix have non-
positive real parts and there is at least one eigenvalue with zero real part, then, one cannot
make any conclusions on the stability of the nonlinear system based on the linearization.

170
Example 97 Consider the scalar nonlinear system:

x = ax3

This origin is GAS if a < 0, unstable if a > 0 and stable if a = 0. However, the linearization
of this system about zero, given by
x = 0
is independent of a and is stable.

The following example illustrates that instability of the linearized system does not imply
instability of the nonlinear system.

Example 98 Using nonlinear techniques, one can show that the following system is GAS
about the origin.

x1 = x2
x2 = x31 x32

However, the linearization of this system, given by

x1 = x2
x2 = 0 ,

is unstable about the origin.

171
Exercises
Exercise 86 For each of the following systems, determine (from the state portrait) the
stability properties of each equilibrium state. For AS equilibrium states, give the region of
attraction.

(a)
x = x x3

(b)
x = x + x3

(c)
x = x 2x2 + x3

Exercise 87 If possible, use linearization to determine the stability properties of each of

the following systems about the zero equilibrium state.

(i)

x1 = (1 + x21 )x2
x2 = x31

(ii)

x1 = sin x2
x2 = (cos x1 )x3
x3 = ex1 x2

Exercise 88 If possible, use linearization to determine the stability properties of each equi-
librium state of the Lorenz system.

172
Chapter 8

Stability and boundedness: discrete

time

8.1 Boundedness of solutions

Consider a general discrete-time nonlinear system described by
x(k+1) = f (x(k)) (8.1)
where x(k) is an n-vector and k is an integer. By a solution of (8.1) we mean a sequence
x() = ( x(0), x(1), x(2), . . . ) which satisfies (8.1) for all k 0.

DEFN. (Boundedness of solutions) A solution x() is bounded if there exists 0 such that
||x(k)|| for all k 0.
A solution is unbounded if it is not bounded.

It should be clear from the above definitions that a solution x() is bounded if and only
if each component xi () of x() is bounded. Also, a solution is unbounded if and only if at
least one of its components is unbounded. So, x(k) = [ (0.5)k (0.5)k ]T is bounded while
x(k) = [ (0.5)k 2k ]T is unbounded.
Example 99
x(k+1) = 0
All solutions are bounded.
Example 100
x(k+1) = x(k)
All solutions are bounded.
Example 101
x(k+1) = 2x(k)
All solutions (except the zero solution) are unbounded.

173
Linear time invariant (LTI) systems. All solutions of the LTI system
x(k+1) = Ax(k) (8.2)
are bounded if and only if for each eigenvalue of A:
(b1) || 1 and
(b2) if || = 1 then is non-defective.
If there is an eigenvalue of A such that either
(u1) || > 1 or
(u2) || = 1 and is defective.
then the system has some unbounded solutions.
Example 102 Discrete unattached mass. Here

1 1
A=
0 1
has a single eigenvalue 1. This eigenvalue has algebraic multiplicity 2 but geometric mul-
tiplicity 1; hence this eigenvalue is defective. So, some of the solutions of the system
x(k+1) = Ax(k) are unbounded. One example is

k
x(k) =
1

8.2 Stability of equilibrium states

Suppose xe is an equilibrium state of the system x(k + 1) = f (x(k)). Then, whenever
x(0) = xe , we have x(k) = xe for all k 0. Roughly speaking, we say that xe is a stable
equilibrium state for the system if the following holds. If the initial state of the system is
close to xe , then the resulting solution is close to xe . The formal definition is as follows.

DEFN. (Stability) An equilibrium state xe is stable if for each > 0 there exists > 0 such
that whenever ||x(0)xe || < , one has ||x(k)xe || < for all k 0.
xe is unstable if it is not stable.

Example 103
x(k+1) = x(k) .
The origin is stable. (Consider = .)
Example 104
x(k+1) = 2x(k) .
The origin is unstable.
Example 105
x(k+1) = x(k)3 .
The single equilibrium at the origin is stable, but, the system has unbounded solutions.

174
LTI systems. Every equilibrium state of a LTI system (8.2) is stable if and only if all
eigenvalues of A satisfy conditions (b1) and (b2) above. Hence every equilibrium state of
a LTI system is stable if and only if all solutions are bounded.
Every equilibrium state is unstable if and only if there is an eigenvalue of A which
satisfies condition (u1) or (u2) above. Hence every equilibrium state of a LTI system is
unstable if and only if there are unbounded solutions.

8.3 Asymptotic stability

8.3.1 Global asymptotic stability

DEFN. (Global asymptotic stability (GAS)) An equilibrium state xe is globally asymptotically

stable (GAS) if

(a) It is stable

(b) Every solution x() converges to xe , that is,

lim x(k) = xe .
k

If xe is a globally asymptotically stable equilibrium state, then there are no other equi-
librium states. In this case we say the system (8.1) is globally asymptotically stable.

Example 106
1
x(k+1) = x(k)
2
GAS

Example 107
x(k)
x(k+1) =
2 + x(k)2

8.3.2 Asymptotic stability

DEFN. (Asymptotic stability (AS)) An equilibrium state xe is asymptotically stable (AS) if

(a) It is stable

(b) There exists R > 0 such that whenever ||x(0) xe || < R one has

lim x(k) = xe (8.3)

k

175
 The region of attraction of an equilibrium state xe which is AS is the set of initial states
which result in (8.3), that is it is the set of initial states which are attracted to xe . Thus, the
region of attraction of a globally asymptotically equilibrium state is the whole state space.

Example 108
x(k+1) = x(k)3
Origin is AS with region of attraction [1 1].

LTI systems. For LTI systems, it should be clear from the general form of the solution
that the zero state is AS if and only if all the eigenvalues i of A have magnitude less than
one, that is,
|i | < 1 .
Also AS is equivalent to GAS.

8.4 Exponential stability

We now present the strongest form of stability considered in this section.

DEFN. (Global exponential stability) An equilibrium state xe is globally exponentially stable

(GES) with there exists 0 < 1 and > 0 such that every solution satisfies

||x(k)xe || k ||x(0)xe || for all k0

Example 109
1
x(k+1) = x(k)
2
GES with = 12 .

Note that global exponential stability implies global asymptotic stability, but, in general,
the converse is not true. This is illustrated in the next example. For linear time-invariant
systems, GAS and GES are equivalent.

Lemma 1 Consider a system described by x(k + 1) = f (x(k)) and suppose that for some
scalar 0,
kf (x)k kxk
for all x. Then, every solution x() of the system satisfies

kx(k)k k kx(0)k

for all k 0. In particular, if < 1 then, the system is globally exponentially stable.

176
Example 110
x(k)
x(k+1) = p
1 + 2x(k)2
Solutions satisfy
x0
x(k) = p where x0 = x(0) .
1 + 2kx20
GAS but not GES

DEFN. (Exponential stability) An equilibrium state xe is exponentially stable (ES) if there

exists R > 0, 0 < 1 and > 0 such that whenever ||x(0)xe || < R one has

||x(k)xe || k ||x(0)xe || for all k1

Note that exponential stability implies asymptotic stability, but, in general, the converse
is not true.

Example 111
x(k)
x(k+1) = p
1 + 2x(k)2
Solutions satisfy
x0
x(k) = p where x0 = x(0) .
1 + 2kx20
GAS but not even ES

8.5 LTI systems

The following table summarizes the relationhip between the stability properties of a LTI
system and the eigenproperties of its A-matrix. In the table, unless otherwise stated, a
property involving must hold for all eigenvalues of A.

Stability properties eigenproperties

Asymptotic stability || < 1

and boundedness

Stability and || 1
boundedness If || = 1 then is non-defective

Instability and some There is an eigenvalue of A with

unbounded solutions || > 1 or
|| = 1 and is defective

177
8.6 Linearization and stability
Consider a nonlinear time-invariant system described by

x(k+1) = f (x(k))

where x(k) is an n-vector at each time t. Suppose xe is an equilibrium state for this system,
that is, f (xe ) = xe , and consider the linearization of this system about xe :

f e
x(k+1) = Ax(k) where A = (x ) .
x
The following results can be demonstrated using nonlinear Lyapunov stability theory.
Stability. If all the eigenvalues of the A matrix of the linearized system have magnitude less
than one, then the nonlinear system is exponentially stable about xe .
Instability. If at least one eigenvalue of the A matrix of the linearized system has magnitude
greater than one, then the nonlinear system is unstable about xe .
Undetermined. Suppose all the eigenvalues of the A matrix of the linearized system have
magnitude less than or equal to one and at least one eigenvalue of A has magnitude one.
Then, based on the linearized system alone, one cannot predict the stability properties of the
nonlinear system about xe .
Note that the first statement above is equivalent to the following statement. If the
linearized system is exponentially stable, then the nonlinear system is exponentially stable
about xe .

Example 112 (Newtons method) Recall that Newtons method for a scalar function
can be described by
g(x(k))
x(k+1) = x(k) 0
g (x(k))
Here
g(x)
f (x) = x
g 0 (x)
So,
g(x)g 00 (x) g(x)g 00 (x)
f 0 (x) = 1 1 + = .
g 0 (x)2 g 0 (x)2
At an equilibrium state xe , we have g(xe ) = 0; hence f 0 (xe ) = 0 and the linearization about
any equilibrium state is given by
x(k+1) = 0 .
Thus, every equilibrium state is exponentially stable.

178
Chapter 9

Basic Lyapunov theory

Suppose we are interested in the stability properties of the system

x = f (x) (9.1)
where x(t) is a real n-vector at time t. If the system is linear, we can determine its stability
properties from the properties of the eigenvalues of the system matrix. What do we do for
a nonlinear system? We could linearize about each equilibrium state and determine the
stability properties of the resulting linearizations. Under certain conditions this will tell us
something about the local stability properties of the nonlinear system about its equilibrium
states. However there are situations where linearization cannot be used to deduce even the
local stability properties of the nonlinear system. Also, linearization tells us nothing about
the global stability properties of the nonlinear system.
In general, we cannot explicitly obtain solutions for nonlinear systems. Lyapunov theory
allows to say something about the stability properties of a system without knowing the form
or structure of the solutions.
In this chapter, V is a scalar-valued function of the state, that is V : IRn IR. Suppose
V is continuously differentiable. Then, at any time t, the derivative of V along a solution x()
of system (9.1) is given by
dV
(x(t)) = DV (x(t))x(t)
dt
= DV (x(t))f (x(t))
where DV (x) is the derivative of V at x and is given by
V V V

DV (x) = x 1
(x) x 2
(x) . . . xn
(x)
Note that
V V V
DV (x)f (x) = (x)f1 (x) + (x)f2 (x) + . . . + (x)fn (x)
x1 x2 xn

In what follows, if a condition involves DV , then it is implicitly assumed that V is continu-

ously differentiable. Sometimes DV is denoted by
V
.
x

179
Also, when the system under consideration is fixed, DV f is sometimes denoted by

V .

This a slight but convenient abuse of notation.

180
9.1 Stability
9.1.1 Locally positive definite functions

DEFN. (Locally positive definite function) A function V is locally positive definite (lpd) about
a point xe if
V (xe ) = 0
and there is a scalar R > 0 such that

V (x) > 0 whenever x 6= xe and ||x xe || < R

Basically, a function is lpd about a point xe if it is zero at xe has a strict local minimum
at xe .

Figure 9.1: A locally positive definite function

Example 113 (Scalar x) The following functions are lpd about zero.

V (x) = x2
2
V (x) = 1 ex
V (x) = 1 cos x
V (x) = x2 x4

Example 114

V (x) = ||x||2
= x21 + x22 + . . . + x2n

Lpd about the origin.

Quadratic forms. Suppose P is a real n n symmetric matrix and is positive definite.

Consider the quadratic form defined by
n X
X n
T
V (x) = x P x = Pij xi xj .
i=1 j=1

181
Clearly V (0) = 0. Recalling the definition of a positive definite matrix, it follows that
V (x) = xT P x > 0 for all nonzero x. Hence V is locally positive definite about the origin.

The second derivative of V at x is the square symmetric matrix given by:

2V 2V 2V
2 x1
(x) x1 x2
(x) ... x1 xn
(x)


2V 2V 2V
(x) (x) . . . (x)
x2 x1 2 x2 x2 xn
D V (x) :=
2



.. .. ..
. . .


2V 2V 2V
xn x1
(x) xn x2
(x) . . . 2 xn
(x)
that is,
2V
D2 V (x)ij = (x)
xi xj
2V
Sometimes D2 V (x) is written as x2
(x) and is referred to as the Hessian of V .

The following lemma is sometimes useful in demonstrating that a function is lpd.

Lemma 2 Suppose V is twice continuously differentiable and

V (xe ) = 0
DV (xe ) = 0
D2 V (xe ) > 0

Then V is a locally positive definite about xe .

Example 115 Consider V (x) = x2 x4 where x is a scalar. Since V (0) = DV (0) = 0 and
D2 V (0) = 2 > 0, V is lpd about zero.

Example 116 Consider

1
V (x) = 1 cos x1 + x22
2
Clearly
V (0) = 0
Since
DV (x) = sin x1 x2
we have
DV (0) = 0
Also,
2 cos x1 0
D V (x) =
0 1

182
Hence,
2 1 0
D V (0) = >0
0 1
Since V satisfies the hypotheses of the previous lemma with xe = 0, it is lpd about zero.

9.1.2 A stability result

If the equilibrium state of a nonlinear system is stable but not asymptotically stable, then
one cannot deduce the stability properties of the equilibrium state of the nonlinear system
from the linearization of the nonlinear system about that equilibrium state

Theorem 4 (Stability) Suppose there exists a function V and a scalar R > 0 such that V is
locally positive definite about xe and

DV (x)f (x) 0 for ||x xe || < R

Then xe is a stable equilibrium state.

If V satisfies the hypotheses of the above theorem, then V is said to be a Lyapunov

function which guarantees the stability of xe .

Example 117
x = 0
Consider
V (x) = x2
as a candidate Lyapunov function. Then V is lpd about 0 and

DV (x)f (x) = 0

Hence (it follows from Theorem 4 that) the origin is stable.

Example 118 (Undamped linear oscillator.)

x1 = x2
x2 = kx1 k>0

Consider the total energy,

1 1
V (x) = kx21 + x22
2 2
as a candidate Lyapunov function. Then V is lpd about the origin and

DV (x)f (x) = 0

Hence the origin is stable.

183
Example 119 (Simple pendulum.)
x1 = x2
x2 = sin x1
Consider the total energy,
1
V (x) = 1 cos x1 + x22
2
as a candidate Lyapunov function. Then V is lpd about the origin and
DV (x)f (x) = 0
Hence the origin is stable
Example 120 (Stability of origin for attitude dynamics system.) Recall
(I2 I3 )
x1 = x2 x3
I1
(I3 I1 )
x2 = x3 x1
I2
(I1 I2 )
x3 = x1 x2
I3
where
I1 , I2 , I3 > 0
Consider the kinetic energy
1 2
V (x) = I1 x1 + I2 x22 + I3 x23
2
as a candidate Lyapunov function. Then V is lpd about the origin and
DV (x)f (x) = 0
Hence the origin is stable.
Example 121 (Undamped Duffing system)
x1 = x2
x2 = x1 x31
As a candidate Lyapunov function for the equilibrium state xe = [ 1 0 ]T consider the
total energy
1 1 1 1
V (x) = x41 x21 + x22 + .
4 2 2 4
Since 2
3 2 3x1 1 0
DV (x) = x1 x1 x2 and D V (x) =
0 1
we have V (xe ) = 0, DV (xe ) = 0 and D2 V (xe ) > 0, and it follows that V is lpd about xe .
One can readily verify that DV (x)f (x) = 0. Hence, xe is stable.

184
 Phase portrait of ddot y = y y3
3

1
ydot

3
3 2 1 0 1 2 3
y

Figure 9.2: Duffing system

185
Exercises
Exercise 89 Determine whether or not the following functions are lpd. (a)

V (x) = x21 x41 + x22

(b)
V (x) = x1 + x22
(c)
V (x) = 2x21 x31 + x1 x2 + x22

Exercise 90 (Simple pendulum with Coulomb damping.) By appropriate choice of Lya-

punov function, show that the origin is a stable equilibrium state for

x1 = x2
x2 = sin x1 c sgm (x2 )

where c > 0.

Exercise 91 By appropriate choice of Lyapunov function, show that the origin is a stable
equilibrium state for

x1 = x2
x2 = x31

Note that the linearization of this system about the origin is unstable.

Exercise 92 By appropriate choice of Lyapunov function, show that the origin is a stable
equilibrium state for

x1 = x2
x2 = x1 + x31

186
9.2 Asymptotic stability
The following result presents conditions which guarantee that an equilibrium state is asymp-
totically stable.

Theorem 5 (Asymptotic stability) Suppose there exists a function V and a scalar R > 0
such that V is locally positive definite about xe and

DV (x)f (x) < 0 for x 6= xe and ||x xe || < R

Then xe is an asymptotically stable equilibrium state for x = f (x).

Example 122
x = x3
Consider
V (x) = x2
Then V is lpd about zero and

DV (x)f (x) = 2x4 < 0 for x 6= 0

Hence the origin is AS.

Example 123
x = x + x3
Consider
V (x) = x2
Then V is lpd about zero and

DV (x)f (x) = 2x2 (1 x2 ) < 0 for |x| < 1, x 6= 0

Hence the origin is AS. Although the origin is AS, there are solutions which go unbounded
in a finite time.

Example 124
x = sin x
Consider
V (x) = x2
Then V is lpd about zero and

DV (x)f (x) = 2x sin(x) < 0 for |x| < , x 6= 0

Hence the origin is AS.

187
Example 125 (Simple pendulum with viscous damping.) Intuitively, we expect the origin
to be an asymptotically stable equilibrium state for the damped simple pendulum:
x1 = x2
x2 = sin x1 cx2
where c > 0 is the damping coefficient. If we consider the total mechanical energy
1
V (x) = 1 cos x1 + x22
2
as a candidate Lyapunov function, we obtain
DV (x)f (x) = cx22 .
Since DV (x)f (x) 0 for all x, we have stability of the origin. Since DV (x)f (x) = 0
whenever x2 = 0, it follows that DV (x)f (x) = 0 for points arbitrarily close to the origin;
hence V does not satisfy the requirements of the above theorem for asymptotic stability.
Suppose we modify V to
1 1
V (x) = c2 x21 + cx1 x2 + x22 + 1 cos x1
2 2
where is any scalar with 0 < < 1. Letting

1 c2 c
P =
2 c 1
note that P > 0 and
V (x) = xT P x + 1 cos x1
xT P x
Hence V is lpd about zero and we obtain
DV (x)f (x) = cx1 sin x1 (1 )cx22 < 0 for ||x|| < , x 6= 0
to satisfy the requirements of above theorem; hence the origin is AS.

Exercises
Exercise 93 By appropriate choice of Lyapunov function, show that the origin is an asymp-
totically stable equilibrium state for
x1 = x2
x2 = x51 x2
Exercise 94 By appropriate choice of Lyapunov function, show that the origin is a asymp-
totically stable equilibrium state for
x1 = x2
x2 = x1 + x31 x2

188
9.3 Boundedness
9.3.1 Radially unbounded functions

DEFN. A scalar valued function V of the state x is said to be radially unbounded if

lim V (x) =
x

Example 126
V (x) = x2 yes
2
V (x) = 1 ex no
V (x) = x2 x yes
V (x) = x4 x2 yes
V (x) = x sin x no
V (x) = x2 (1 cos x) no

Example 127 Suppose P is a real n n matrix and is positive definite symmetric and
consider the quadratic form defined by

V (x) = xT P x

Since P is real symmetric,

xT P x min (P )||x||2
for all x, where min (P ) is the minimum eigenvalue of P . Since P is positive definite,
min (P ) > 0. From this it should be clear that V is radially unbounded.

The following lemma can be useful for guaranteeing radial unboundedness.

Lemma 3 Suppose V is twice continuously differentiable, and there is a positive definite

symmetric matrix P and a scalar R 0 such that

D2 V (x) P for ||x|| R

Then, V is radially unbounded.

9.3.2 A boundedness result

Theorem 6 Suppose there exists a radially unbounded function V and a scalar R 0 such
that
DV (x)f (x) 0 for ||x|| R
Then all solutions of x = f (x) are bounded.

Note that, in the above theorem, V does not have to be positive away from the origin; it
only has to be radially unbounded.

189
Example 128 Recall
x = x x3 .
Consider
V (x) = x2
Since V is radially unbounded and
DV (x)f (x) = 2x2 (x2 1)
the hypotheses of the above theorem are satisfied with R = 1; hence all solutions are bounded.
Note that the origin is unstable.
Example 129 Duffings equation
x1 = x2
x2 = x1 x31
Consider
1 1 1
V (x) = x22 x21 + x41
2 2 4
It should be clear that V is radially unbounded; also
DV (x)f (x) = 0 0 for all x
So, the hypotheses of the above theorem are satisfied with any R; hence all solutions are
bounded.

Exercises
Exercise 95 Determine whether or not the the following function is radially unbounded.
V (x) = x1 x31 + x41 x22 + x42
Exercise 96 (Forced Duffings equation with damping.) Show that all solutions of the
system
x1 = x2
x2 = x1 x31 cx2 + 1 c>0
are bounded.
Hint: Consider
1 1 1 1
V (x) = c2 x21 + cx1 x2 + x22 x21 + x41
2 2 2 4
where 0 < < 1. Letting
1 c2 c
P =
2 c 1
note that P > 0 and
1 1
V (x) = xT P x x21 + x41
2 4
1
xT P x
4

190
Exercise 97 Recall the Lorenz system

x1 = (x2 x1 )
x2 = rx1 x2 x1 x3
x3 = bx3 + x1 x2

with b > 0. Prove that all solutions of this system are bounded. (Hint: Consider V (x) =
rx21 + x22 + (x3 2r)2 .)

191
9.4 Global asymptotic stability
9.4.1 Positive definite functions

DEFN. (Positive definite function.) A function V is positive definite (pd) if

V (0) = 0
V (x) > 0 for all x 6= 0
lim V (x) =
x

In the above definition, note the requirement that V be radially unbounded.

Example 130 Scalar x

V (x) = x2 pd
2
V (x) = 1 ex lpd but not pd
V (x) = 1 cos x lpd but not pd

Example 131
V (x) = x41 + x22

Example 132

V (x) = ||x||2
= x21 + x22 + . . . + x2n

Example 133 Suppose P is a real n n matrix and is positive definite symmetric and
consider the quadratic form defined by

V (x) = xT P x

V is a positive definite function.

Lemma 4 Suppose V is twice continuously differentiable and

V (0) = 0
DV (0) = 0
D2 V (x) > 0 for all x

Then V (x) > 0 for all x.

If V satisfies the hypotheses of the above lemma, it is not guaranteed to be radially un-
bounded, hence it is not guaranteed to be positive definite. Lemma 3 can be useful for
guaranteeing radial unboundedness. We also have the following lemma.

192
Lemma 5 Suppose V is twice continuously differentiable,

V (0) = 0
DV (0) = 0

and there is a positive definite symmetric matrix P such that

D2 V (x) P for all x

Then
1
V (x) xT P x
2
for all x.

9.4.2 A result on global asymptotic stability

Theorem 7 (Global asymptotic stability) Suppose there exists a positive definite function V
such that
DV (x)f (x) < 0 for all x 6= 0
Then the origin is a globally asymptotically stable equilibrium state for x = f (x).

Example 134
x = x3
V (x) = x2

DV (x)f (x) = 2x4

< 0 for all x 6= 0

We have GAS. Note that linearization of this system about the origin cannot be used to
deduce the asymptotic stability of this system.

Example 135 The first nonlinear system

x = sgm (x) .

This system is not linearizable about its unique equilibrium state at the origin. Considering

V (x) = x2

we obtain

DV (x)f (x) = 2|x|

< 0 for all x 6= 0 .

Hence, we have GAS.

193
Example 136 (Linear globally stabilizing controller for inverted pendulum.)

x1 = x2
x2 = sin x1 + u

Consider
u = k1 x1 k2 x2 with k1 > 1, k2 > 0
Closed loop system:

x1 = x2
x2 = k1 x1 + sin x1 k2 x2

Consider
1 1 1
V (x) = k22 x21 + k2 x1 x2 + x22 + k1 x21 + cos x1 1
2 2 2
where 0 < < 1. Then V is pd (apply lemma 5) and

DV (x)f (x) = k2 (k1 x21 + x1 sin x1 ) (1 )k2 x22

Since
| sin x1 | |x1 | for all x1
it follows that
x1 sin x1 x21 for all x1
hence
DV (x)f (x) k2 (k1 1)x21 (1 )k2 x22 <0 for all x 6= 0
The closed loop system is GAS.

Example 137 (Stabilization of origin for attitude dynamics system.)

(I2 I3 ) u1
x1 = x2 x3 +
I1 I1
(I3 I1 ) u2
x2 = x3 x1 +
I2 I2
(I1 I2 ) u3
x3 = x1 x2 +
I3 I3

where
I1 , I2 , I3 > 0
Consider any linear controller of the form

ui = kxi i = 1, 2, 3, k>0

194
Closed loop system:

I2 I3 kx1
x1 = x2 x3
I1 I1

I3 I1 kx2
x2 = x3 x1
I2 I2

I1 I2 kx3
x3 = x1 x2
I3 I3
Consider the kinetic energy
1 2
V (x) = I1 x1 + I2 x22 + I3 x23
2
as a candidate Lyapunov function. Then V is pd and

DV (x)f (x) = k(x21 + x22 + x23 )

< 0 for all x 6= 0

Hence the origin is GAS for the closed loop system.

Exercises
Exercise 98 Determine whether or not the following function is positive definite.

V (x) = x41 x21 x2 + x22

Exercise 99 Consider any scalar system described by

x = g(x)

where g has the following properties:

g(x) > 0 for x > 0

g(x) < 0 for x < 0

Show that this system is GAS.

Exercise 100 Recall the Lorenz system

x1 = (x2 x1 )
x2 = rx1 x2 x1 x3
x3 = bx3 + x1 x2

Prove that if
b>0 and 0 r < 1,
then this system is GAS about the origin. (Hint: Consider V (x) = x21 + x22 + x23 .)

195
Exercise 101 (Stabilization of the Duffing system.) Consider the Duffing system with a
scalar control input u(t):

x1 = x2
x2 = x1 x31 + u

Obtain a linear controller of the form

u = k1 x1 k2 x2

which results in a closed loop system which is GAS about the origin. Numerically simulate
the open loop system (u = 0) and the closed loop system for several initial conditions.

9.5 Exponential stability

9.5.1 Global exponential stability
Theorem 8 (Global exponential stability) Suppose there exists a function V and positive
scalars , 1 , 2 such that for all x,

1 ||x||2 V (x) 2 ||x||2

and
DV (x)f (x) 2V (x) .
Then, for the system x = f (x), the origin is a globally exponentially stable equilibrium state
with rate of convergence . In particular, all solutions x() of the system satisfy
p
kx(t)k 2 /1 kx(0)k et for all t 0.

Proof. See below.

Example 138
x = x
Considering
V (x) = x2
we have

DV (x)f (x) = 2x2

= 2V (x)

Hence, we have GES with rate of convergence 1.

196
Example 139
x = x x3
Considering
V (x) = x2
we have

DV (x)f (x) = 2x2 2x4

2V (x)

Hence, we have GES with rate of convergence 1.

9.5.2 Proof of theorem 8

Consider any solution x() and let v(t) = V (x(t)). Then

v 2v .

9.5.3 Exponential stability

Theorem 9 (Exponential stability) Suppose there exists a function V and positive scalars R,
, 1 , 2 such that, whenever kxk R, one has

1 ||x||2 V (x) 2 ||x||2

and
DV (x)f (x) 2V (x)
Then, for system x = f (x), the state xe is an exponentially stable equilibrium state with rate
of convergence .

197
Exercise 102 Consider the scalar system

x = x + x3

As a candidate Lyapunov function for exponential stability, consider V (x) = x2 . Clearly,

the condition on V is satisfied with 1 = 2 = 1. Noting that

V = 2x2 + 2x4 = 2(1 x2 )x2 ,

and considering R = 1/2, we obtain that whenever |x| R, we have V 2V where

= 3/4. Hence, we have ES.

198
9.5.4 A special class of GES systems
Consider a system described by
x = f (x) (9.2)
and suppose that there exist two positive definite symmetric matrices P and Q such that

xT P f (x) xT Qx .

We will show that the origin is GES with rate

:= min (P 1 Q)

where min (P 1 Q) > 0 is the smallest eigenvalue of P 1 Q.

As a candidate Lyapunov function, consider
V (x) = xT P x .
Then
min (P )||x||2 V (x) max (P )||x||2
that is
1 ||x||2 V (x) 2 ||x||2
with
1 = min (P ) > 0 and 2 = max (P ) > 0 .
Now note that
DV (x)f (x) = 2xT P f (x)
2xT Qx
For any two positive-definite matrices P and Q, one can show that all the eigenvalues of
P 1 Q are real positive and
xT Qx min (P 1 Q) xT P x
where min (P 1 Q) > 0 is the smallest eigenvalue of P 1 Q. Thus,
DV (x)f (x) 2min (P 1 Q)xT P x
= 2V (x)
Hence GES with rate .
Example 140 Recall example 137.
Closed loop system:

I2 I3 kx1
x1 = x2 x3
I1 I1

I3 I1 kx2
x2 = x3 x1
I2 I2

I1 I2 kx3
x3 = x1 x2
I3 I3

199
 Considering
I1 0 0
P = 0 I2 0
0 0 I3
we have
xT P f (x) = kxT x
that is, Q = kI. Hence, GES with rate

= min (P 1 Q)
= min (kP 1 )
= kmin (P 1 )
k
=
max (P )
k
=
max{I1 , I2 , I3 }

200
9.5.5 Summary
The following table summarizes the results of this chapter for stability about the origin.

V V stability properties

lpd 0 for ||x|| R S

lpd < 0 for ||x|| R, x 6= 0 AS

ru 0 for ||x|| R B

pd < 0 for x 6= 0 GAS

1 ||x||2 V (x) 2 ||x||2 2V (x) GES

1 ||x||2 V (x) 2 ||x||2 for ||x|| R 2V (x) for ||x|| R ES

Figure 9.3: Lyapunov table

201
202
Chapter 10

Basic Lyapunov theory: discrete

time*

Suppose we are interested in the stability properties of the system,

x(k + 1) = f (x(k)) (10.1)

where x(k) IRn and k IR. If the system is linear, we can determine its stability properties
from the properties of the eigenvalues of the system matrix. What do we for a nonlinear
system? We could linearize about each equilibrium state and determine the stability prop-
erties of the resulting linearizations. Under certain conditions (see later) this will tell us
something about the local stability properties of the nonlinear system about its equilibrium
states. However there are situations where linearization cannot be used to deduce even the
local stability properties of the nonlinear system. Also, linearization tells us nothing about
the global stability properties of the nonlinear system.
In general, we cannot explicitly obtain solutions for nonlinear systems. Lyapunov theory
allows to say something about the stability properties of a system without knowing the form
or structure of the solutions.

In this chapter, V is a scalar-valued function of the state, i.e. V : IRn IR. At any time
k, the one step change in V along a solution x() of system (10.1) is given by

V (x(k + 1)) V (x(k)) = V (x(k))

where
V (x) := V (f (x)) V (x)

203
10.1 Stability
Theorem 10 (Stability) Suppose there exists a locally positive definite function V and a
scalar R > 0 such that
V (x) 0 for ||x|| < R
Then the origin is a stable equilibrium state.

If V satisfies the hypotheses of the above theorem, then V is said to be a Lyapunov function
which guarantees the stability of origin.

Example 141
x(k + 1) = x(k)
Consider
V (x) = x2
as a candidate Lyapunov function. Then V is a lpdf and

V (x) = 0

Hence (it follows from theorem 10 that) the origin is stable.

204
10.2 Asymptotic stability

Theorem 11 (Asymptotic stability) Suppose there exists a locally positive definite function
V and a scalar R > 0 such that

V (x) < 0 for x 6= 0 and ||x|| < R

Then the origin is an asymptotically stable equilibrium state.

Example 142
1
x(k + 1) = x(k)
2
Consider
V (x) = x2
Then V is a lpdf and
1
V (x) = ( x)2 x2
2
3
= x2
4
< 0 for x 6= 0

Hence the origin is AS.

Example 143
1
x(k + 1) = x(k) + x(k)2
2
Consider
V (x) = x2
Then V is a lpdf and
1
V (x) = ( x + x2 )2 x2
2
3 1
= x2 ( + x)( x)
2 2
1
< 0 for |x| < , x 6= 0
2
Hence the origin is AS.

205
10.3 Boundedness

Theorem 12 Suppose there exists a radially unbounded function V and a scalar R 0 such
that
V (x) 0 for ||x|| R
Then all solutions of (10.1) are bounded.

Note that, in the above theorem, V does not have to positive away from the origin; it only
has to be radially unbounded.

Example 144
2x(k)
x(k + 1) =
1 + x(k)2
Consider
V (x) = x2
Since V is radially unbounded and
2
2x
V (x) = x2
1 + x2
x2 (x2 + 3)(x2 1)
=
(x2 + 1)2

0 for |x| 1

the hypotheses of the above theorem are satisfied with R = 1; hence all solutions are bounded.
Note that the origin is unstable.

206
10.4 Global asymptotic stability
Theorem 13 (Global asymptotic stability) Suppose there exists a positive definite function
V such that
V (x) < 0 for all x 6= 0
Then the origin is a globally asymptotically stable equilibrium state.

Example 145
1
x(k + 1) = x(k)
2
Example 146
x(k)
x(k + 1) =
1 + x(k)2
Consider
V (x) = x2
Then
2
x
V (x) = x2
1 + x2
2x4 + x6
=
(1 + x2 )2

< 0 for all x 6= 0

Hence this system is GAS about zero.

Exercise 103 Consider any scalar system described by

x(k + 1) = g(x(k))

where g has the following properties:

|g(x)| < |x| for x 6= 0

Show that this system is GAS.

207
10.5 Exponential stability

Theorem 14 (Global exponential stability.) Suppose there exists a function V and scalars
, 1 , 2 such that for all x,

1 ||x||2 V (x) 2 ||x||2 1 , 2 > 0

and

V (f (x)) 2 V (x) 0<1

Then, every solution satisfies

s
2 k
||x(k)|| ||x(0)|| for k0
1

Hence, the origin is a globally exponentially stable equilibrium state with rate of convergence
.

Proof.

Example 147
1
x(k + 1) = x(k)
2

208
Considering
V (x) = x2
we have
1 2
V (f (x)) = x
4
1
= ( )2 V (x)
2
Hence, we have GES with rate of convergence = 21 .

Example 148
1
x(k + 1) = sin(x(k))
2
Considering
V (x) = x2
we have
1
V (f (x)) = ( sin x)2
2
1
= ( )2 | sin x|2
2
1
( )2 |x|2
2
1
= ( )2 V (x)
2
1
Hence, we have GES with rate of convergence = 2

209
210
Chapter 11

Lyapunov theory for linear

time-invariant systems

The main result of this section is contained in Theorem 15.

11.1 Positive and negative (semi)definite matrices

11.1.1 Definite matrices
For any square matrix n n matrix P we can define an associated quadratic form by
n X
X n

x Px = pij xi xj
i=1 j=1

Recall that a square complex matrix P is hermitian if P = P where P is the complex

conjugate of P . If P is real then hermitian is equivalent to symmetric, that is, P T = P .
Fact 5 Every hermitian matrix P has the following properties.
(a) All eigenvalues of P are real.

(b) The scalar x P x is real for all x Cn . and satisfies

min (P )||x||2 x P x max (P )||x||2

where min (P ) and max (P ) denote the minimum and maximum eigenvalues of P .

DEFN. A hermitian matrix P is positive definite (pd) if

x P x > 0

for all nonzero x. We denote this by P > 0. The matrix P is negative definite (nd) if P is
positive definite; we denote this by P < 0.

211
Example 149 For
1 1
P =
1 2
we have (note the completion of squares trick)
x P x = x1 x1 x1 x2 x2 x1 + 2x2 x2
= (x1 x2 ) (x1 x2 ) + x2 x2
= |x1 x2 |2 + |x2 |2
Clearly, x P x 0 for all x. If x P x = 0, then x1 x2 = 0 and x2 = 0; hence x = 0. So,
P > 0.

Fact 6 The following statements are equivalent for any hermitian matrix P .
(a) P is positive definite.
(b) All the eigenvalues of P are positive.
(c) All the leading principal minors of P are positive.

Example 150 Consider

1 1
P =
1 2
Since p11 = 1 > 0 and det(P ) = 1 > 0, we must have P > 0.

>> eig(P)
ans =
0.3820
2.6180

Note the positive eigenvalues.

Exercise 104 Suppose P is hermitian and T is invertible. Show that P > 0 iff T P T > 0.

Exercise 105 Suppose P and Q are two hermitian matrices with P > 0. Show that P +
Q > 0 for all real sufficiently small; i.e., there exists > 0 such that whenever || < ,
one has P + Q > 0.

11.1.2 Semi-definite matrices*

DEFN. A hermitian matrix P is positive semi-definite (psd) if

x P x 0
for all non-zero x. We denote this by P 0
P is negative semi-definite (nsd) if P is positive semi-definite; we denote this by P 0

212
Fact 7 The following statements are equivalent for any hermitian matrix P .

(a) P is positive semi-definite.

(b) All the eigenvalues of P are non-negative.

(c) All the leading minors of P are positive.

Example 151 This example illustrates that non-negativity of the leading principal minors
of P is not sufficient for P 0.
0 0
P =
0 1
We have p11 = 0 and det(P ) = 0. However,

x P x = |x2 |2

hence, P is not psd. Actually, P is nsd.

Fact 8 Consider any m n complex matrix M and let P = M M . Then

(a) P is hermitian and P 0

(b) P > 0 iff rank M = n.

Example 152
1 1
P =
1 1
Since
1
P = 1 1
1
and
rank 1 1 =1
P 0 but P is not pd.

213
11.2 Lyapunov theory
11.2.1 Asymptotic stability results
The scalar system
x = ax
is asymptotically stable if and only if a+a < 0. Consider now a general linear time-invariant
system described by
x = Ax (11.1)
The generalization of a + a < 0 to this system is A + A < 0. We will see shortly that this
condition is sufficient for asymptotic stability; however, as the following example illustrates,
it is not necessary.

Example 153 The matrix

1 3
A=
0 1
is asymptotically stable. However, the matrix

2 3
A+A =
3 2

is not negative definite.

Since stability is invariant under a similarity transformation T , i.e., the stability of A and
1
T AT are equivalent for any nonsingular T , we should consider the more general condition

T 1 AT + T A T < 0

Introducing the hermitian matrix P := T T 1 , and pre- and post-multiplying the above
inequality by T and T 1 , respectively, yields

P > 0 (11.2a)

PA + A P < 0 (11.2b)

We now show that the existence of a hermitian matrix P satisfying these conditions guar-
antees asymptotic stability.

Lemma 6 Suppose there is a hermitian matrix P satisfying (11.2). Then system (11.1) is
asymptotically stable

Proof. Suppose there exists a hermitian matrix P which satisfies inequalities (11.2).
Consider any eigenvalue of A. Let v 6= 0 be an eigenvector corresponding to , i.e.,

Av = v

Then
v P Av = v P v ; v A P v = v P v

214
Pre- and post-multiplying inequality (11.2b) by v and v, respectively, yields

v P v + v P v < 0

i.e.,
2Re()v P v < 0
Since P > 0, we must have v P v > 0; hence Re() < 0. Since the above holds for every
eigenvalue of A, system (11.1) is asymptotically stable.

Remark 1 A hermitian matrix P which satisfies inequalities (11.2) will be referred to as a

Lyapunov matrix for (11.1) or A.

Remark 2 Conditions (11.2) are referred to as linear matrix inequalities (LMIs). In recent
years, efficient numerical algorithms have been developed to solve LMIs or determine that
a solution does not exist.

Lyapunov functions. To obtain an alternative interpretation of inequalities (11.2), con-

sider the quadratic function V of the state defined by

V (x) := x P x

Since P > 0, this function has a strict global minimum at zero. Using inequality (11.2b), it
follows that along any non-zero solution x() of (11.1),

dV (x(t))
= x P x + x P x
dt
= x (A P + P A) x
< 0

i.e., V (x()) is strictly decreasing along any non-zero solution. Intuitively, one then expects
every solution to asymptotically approach the origin. The function V is called a Lyapunov
function for the system. The concept of a Lyapunov function readily generalizes to nonlinear
systems and has proved a very useful tool in the analysis of the stability properties of
nonlinear systems.

Remark 3 Defining the hermitian matrix S := P 1 , inequalities (11.2) become

S > 0 (11.3a)
AS + SA < 0 (11.3b)

Hence the above lemma can be stated with S replacing P and the preceding inequalities
replacing (11.2).

So far we have shown that if a LTI system has a Lyapunov matrix, then it is AS. Is the
converse true? That is, does every AS LTI system have a Lyapunov matrix? And if this is

215
true how does one find a Lyapunov matrix? To answer this question note that satisfaction
of inequality (11.2b) is equivalent to

P A + A P + Q = 0 (11.4)

where Q is a hermitian positive definite matrix. This linear matrix equation is known as the
Lyapunov equation. So one approach to looking for Lyapunov matrices could be to choose a
pd hermitian Q and determine whether the Lyapunov equation has a pd hermitian solution
for P .
We first show that if the system x = Ax is asymptotically stable and the Lyapunov
equation (11.4) has a solution then, the solution is unique. Suppose P1 and P2 are two
solutions to (11.4). Then,

(P2 P1 )A + A (P2 P1 ) = 0 .

Hence,

eA t (P2 P1 )AeAt + eA t A (P2 P1 )eAt = 0
that is,
d eA t (P2 P1 )eAt
=0
dt
This implies that, for all t,

eA t (P2 P1 )eAt = eA 0 (P2 P1 )eA0 = P2 P1 .

Since x = Ax is asymptotically stable, limt eAt = 0. Hence

P2 P1 = lim eA t (P2 P1 )eAt = 0 .
t

From this it follows that P2 = P1 .

The following lemma tells us that every AS LTI system has a Lyapunov matrix and a
Lyapunov matrix can be obtained by solving the Lyapunov equation with any pd hermitian
Q.

Lemma 7 Suppose system (11.1) is asymptotically stable. Then for every matrix Q, the
Lyapunov equation (11.4) has a unique solution for P . If Q is positive definite hermitian
then this solution is positive-definite hermitian.

Proof. Suppose system (11.1) is asymptotically stable and consider any matrix Q. Let
Z

P := eA t QeAt dt
0

This integral exists because each element of eAt is exponentially decreasing.

To show that P satisfies the Lyapunov equation (11.4), use the following properties of
At
e ,

At deAt A t deA t
e A= Ae =
dt dt

216
to obtain
Z

A t At
PA + A P = e Qe A + A eA t QeAt dt
0
Z
deAt deA t At
A t
= e Q + Qe dt
0 dt dt
Z A t At
d e Qe
= dt
0 dt
Z t A t At
de Qe
= lim dt
t 0 dt

= lim eA t QeAt Q
t

= Q

We have already demonstrated uniqueness of solutions to (11.4),

Suppose Q is pd hermitian. Then it should be clear that P is pd hermitian. .

Using the above two lemmas, we can now state the main result of this section.

Theorem 15 The following statements are equivalent.

(a) The system x = Ax is asymptotically stable.

(b) There exist positive definite hermitian matrices P and Q satisfying the Lyapunov equa-
tion (11.4).

(c) For every positive definite hermitian matrix Q, the Lyapunov equation (11.4) has a
unique solution for P and this solution is hermitian positive-definite.

Proof. The first lemma yields (b) = (a). The second lemma says that (a) = (c).
Hence, (b) = (c).
To see that (c) = (b), pick any positive definite hermitian Q. So, (b) is equivalent to
(c). Also, (c) = (a); hence (a) and (c) are equivalent.
.

Example 154

x1 = x2
x2 = x1 + cx2

Here
0 1
A=
1 c

217
 Choosing
1 0
Q=
0 1
and letting
p11 p12
P =
p12 p22
(note we have taken p21 = p12 because we are looking for symmetric solutions) the Lyapunov
equation results in

2p12 + 1 = 0
p11 + cp12 p22 = 0
2p12 + 2cp22 + 1 = 0

We consider three cases:

Case i) c = 0. No solution; hence no GAS.
Case ii) c = 1. A single solution; this solution is pd; hence GAS.
Case iii) c = 1. A single solution; this solution is not pd; hence no GAS.

Remark 4 One can state the above theorem replacing P with S and replacing (11.4) with

AS + SA + Q = 0 (11.5)

11.2.2 MATLAB.
>> help lyap

LYAP Lyapunov equation.

X = LYAP(A,C) solves the special form of the Lyapunov matrix
equation:

A*X + X*A = -C

X = LYAP(A,B,C) solves the general form of the Lyapunov matrix

equation:

A*X + X*B = -C

See also DLYAP.

Note. In MATLAB, A is the complex conjugate transpose of A (i.e. A ). Hence, to solve

(11.4) one must use P= lyap(A, Q).

218
>> q=eye(2);

>> a=[0 1; -1 -1];

>> p=lyap(a, q)
p =
1.5000 0.5000
0.5000 1.0000

>> p*a + a*p

ans =
-1.0000 -0.0000
0 -1.0000

>> det(p)
ans =
1.2500

>> eig(p)
ans =
1.8090
0.6910

Exercise 106 Consider the Lyapunov equation

P A + A P + 2P + Q = 0

where A is a square matrix, Q is some positive-definite hermitian matrix and is some

positive scalar. Show that this equation has a unique solution for P and this solution is
positive-definite hermitian if and only if for every eigenvalue of A,

Re() <

11.2.3 Stability results*

For stability, the following lemma is the analog of Lemma 11.2.

Lemma 8 Suppose there is a hermitian matrix P satisfying

P > 0 (11.6a)

PA + A P 0 (11.6b)

Then system (11.1) is stable.

We do not have the exact analog of lemma 7. However the following can be shown.

219
Lemma 9 Suppose system (11.1) is stable. Then there exist a positive definite hermitian
matrix P and a positive semi-definite hermitian matrix Q satisfying the Lyapunov equation
(11.4).

The above lemma does not state that for every positive semi-definite hermitian Q the
Lyapunov equation has a solution for P . Also, when the Lyapunov equation has a solution,
it is not unique. This is illustrated in the following example.

Example 155
0 1
A=
1 0
With
1 0
P =
0 1
we obtain
P A + A P = 0
In this example the Lyapunov equation has a pd solution with Q = 0; this solution is
not unique; any matrix of the form

p 0
P =
0 p

(where p is arbitrary) is also a solution.

If we consider the psd matrix
1 0
Q=
0 0
the Lyapunov equation has no solution.

Combining the above two lemmas results in the following theorem.

Theorem 16 The following statements are equivalent.

(a) The system x = Ax is stable.

(b) There exist a positive definite hermitian matrix P and a positive semi-definite matrix
Q satisfying the Lyapunov equation (11.4).

220
11.3 Mechanical systems*
Example 156 A simple structure

Figure 11.1: A simple structure

Recall
m1 q1 + (c1 + c2 )q1 c2 q2 + (k1 + k2 )q1 k2 q2 = 0
m2 q2 c2 q1 + c2 q2 k 2 q1 + k 2 q2 = 0

Letting
q1
q=
q2
this system can be described by the following second order vector differential equation:

M q + C q + Kq = 0

where the symmetric matrices M, C, K are given by

m1 0 c1 + c2 c2 k1 + k2 k2
M= C= K=
0 m2 c2 c2 k2 k2

Since m1 , m2 > 0, we have M > 0.

If k1 , k2 > 0 then K > 0 (why?) and if c1 , c2 > 0 then C > 0.

Note also that

1
kinetic energy = m q 2
2 1 1
+ 12 m2 q22 = 1 T
2
q M q

1
potential energy = k q2
2 1 1
+ 12 k2 (q2 q1 )2 = 1 T
2
q Kq

221
 Consider now a general mechanical system described by
M q + C q + Kq = 0
where q(t) is a real N -vector of generalized coordinates which describes the configuraton of
the system. The real matrix M is the inertia matrix and satisfies
MT = M > 0
We call K and C the stiffness matrix and damping matrix respectively. The kinetic energy
of the system is given by
1 T
q M q
2
T
With x = q T , q T , this system has a state space description of the form x = Ax with

0 I
A=
M 1 K M 1 C

A first Lyapunov matrix. Suppose

KT = K > 0 and CT = C 0 .
The potential energy of the system is given by
1 T
q Kq
2
Consider the following candidate Lyapunov matrix

1 K 0
P =
2 0 M
Then P = P > 0 and
P A + A P + Q = 0
where
0 0
Q=
0 C
It should be clear that
P A + A P 0
iff
C0
Hence,

K T = K > 0 and C T = C 0 imply stability.

Note that
1 1
V (x) = xT P x = q T M q + q T Kq = total energy
2 2
T
V (x) = q C q

222
A second Lyapunov matrix.

1 K + C M 1 K 0 C M
P = = +
2 M M 2 0 M 2 M 0

For sufficiently small , the matrix P is positive definite.

Now,
P A + A P + Q = 0
where
K 0
Q=
0 C M
For sufficiently small > 0, the matrix C M is pd and, hence, Q is pd. So,

K T = K > 0 and C T = C > 0 imply asymptotic stability.

223
11.4 Rates of convergence for linear systems*
Theorem 17 Consider an asymptotically stable linear system described by x = Ax and let
be any real number satisfying
0 < <
where
:= max{<() : is an eigenvalue of A } .
Then x = Ax is GES with rate .

Proof: Consider any satisfying 0 < < . As a consequence of the definition of , all
the eigenvalues of the matrix A+I have negative parts. Hence, the Lyapunov equation

P (A+ I) + (A+I)0 P + I = 0 .

has a unique solution for P and P = P T > 0. As a candidate Lyapunov for the system
x = Ax, consider V (x) = xT P x. Then,

V = xT P x + xT P x
= xT P Ax + (Ax)T P x
= xT (P A + AT P )x .

From the above Lyapunov matrix equation, we obtain that

P A + AT P = 2P I ;

hence,

V = 2xT P x xT x
2V (x) .

Hence the system is globally exponentially stable with rate of convergence .

224
11.5 Linearization and exponential stability
Consider a nonlinear system
x = f (x) (11.7)
and suppose that xe is an equilibrium state for this system. We assume that f is differential
at xe and let
f1 e f1 e f1 e
x1 (x ) x2 (x ) xn (x )



f2 e f2 e f2 e
(x ) (x ) (x )
x1 x 2 x n
Df (x ) =
e



. . .
.. .. ..



fn e fn e fn e
(x ) (x ) (x )
x1 x2 xn
where x = (x1 , x2 , , xn ). Then the linearization of x = f (x) about xe is the linear system
defined by
x = Ax where A = Df (xe ) . (11.8)

Theorem 18 Suppose xe is an equilibrium state of a nonlinear system of the form x = f (x)

and the corresponding linearization is exponentially stable. Then the nonlinear system is
exponentially stable about xe .

Proof. Suppose the linearization is exponentially stable and let

:= max{<() : is an eigenvalue of A } .

Since all the eigenvalues have negative real parts, we have > 0. Consider now any
satisfying 0 < < . As a consequence of the definition of , all the eigenvalues of the
matrix A+I have negative parts. Hence, the Lyapunov equation

P (A+ I) + (A+I)0 P + I = 0 .

has a unique solution for P and P = P T > 0. As a candidate Lyapunov function for the
nonlinear system, consider
V (x) = (xxe )0 P (xxe ) .
Recall that
f (x) = f (xe ) + Df (xe )(xxe ) + o(x)
where the remainder term has the following property:
o(x)
lim = 0.
x0, x6=0 kxk

Hence
f (x) = A(xxe ) + o(x)

225
and

V = 2xT P x
= xT P f (x)
= 2xT P Ax + 2xT o(x)
xT (P A + AT P )x + 2kxko(x) .

From the above Lyapunov matrix equation, we obtain that

P A + AT P = 2P I ;

hence,

V = 2xT P x xT x + 2kxko(x)
= 2V (x) kxk2 + o(x)kxk .

As a consequence of the properties of o, there is a scalar R > 0 such that

o(x) kxk when kxk R .

Hence, whenever kxk R we obtain that

V 2V (x) .

Hence the nonlinear system is exponentially stable about xe with rate of convergence .

226
 Part III

Input-Output Properties

227
Chapter 12

Input output response

outputs inputs
system

12.1 Response of continuous-time systems

In general, a linear time-invariant continuous-time system with inputs and outputs is de-
scribed by

y u
x = Ax + Bu

y = Cx + Du

where A, B, C and D are matrices of dimensions n n, n m, p n and p m, respectively.

12.1.1 State response and the convolution integral

The solution to
x(t) = Ax(t) + Bu(t) with x(t0 ) = x0 (12.1)
is unique and is given by
Z t
A(tt0 )
x(t) = e x0 + eA(t ) Bu( )d (12.2)
t0

for all t.

Proof. Consider any time t. Considering any time in the interval bounded by t0 and t,
we have
x( ) = Ax( ) + Bu( )
Pre-multiply both sides by eA and rearrange to get:

eA x( ) eA Ax( ) = eA Bu( )

229
Hence
d A
(e x( )) = eA Bu( )
d
Integrate with respect to to from t0 to t and use x(t0 ) = x0 :
Z t
At At0
e x(t) e x0 = eA Bu( )d
t0

Now premultiply by eAt and use eAt eAt = I and eAt eAt0 = eA(tt0 ) to obtain
Z t
A(tt0 )
x(t) e x0 = eA(t ) Bu( )d
t0

and, hence, the desired result.

Note that the solution depends linearly on the pair (x0 , u()).

Suppose that for some input function u(), the function xp () is a particular solution to
(12.1) with xp (t0 ) = xp0 . Then all other solutions are given by

x(t) = eA(tt0 ) (x0 xp0 ) + xp (t)

(Why?)

Impulse response

Suppose t0 = 0 and
u(t) = (t)v

where () is the Dirac delta function or the unit impulse function and v is some constant
vector. (Dont confuse with the used in linearization.) Recall that if zero is in the interval
bounded by t0 and t and g is any continuous function then,
Z t
g(t)(t)dt = g(0)
t0

Hence, the solution to (12.1) is given by

x(t) = eAt x0 + eAt Bv

From this we see that the zero initial state response (that is, x0 = 0) to the above impulse
input is given by
x(t) = eAt Bv

This is the same as the system response to zero input and initial state, x0 = Bv.

230
Response to exponentials and sinusoids
Suppose t0 = 0 and
u(t) = et v
where v is a constant m-vector and is not an eigenvalue of A. Then the solution to (12.1)
is
x(t) = eAt (x0 xp0 ) + et xp0
| {z } | {z }
transient part steady state part
where
xp0 = (I A)1 Bv
We need only prove the result for x0 = xp0 . So considering

x(t) = et (I A)1 Bv

we have
x(t) = et (I A)1 Bv = x(t)
and
(I A)x(t) = et Bv = Bu(t)
Hence,
x(t) = Ax(t) + Bu(t)
and
x(t) = x(t) = Ax(t) + Bu(t)

If A is asymptotically stable, the transient part goes to zero as t goes to infinity.

12.1.2 Output response

The output response of the system

x(t) = Ax(t) + Bu(t)

y(t) = Cx(t) + Du(t)

subject to initial condition x(t0 ) = x0 is unique and is given by

Z t
A(tt0 )
y(t) = Ce x0 + G(t )u( )d (12.3)
t0

where the impulse response matrix G is defined by

G(t) = CeAt B + (t)D

where () is the Dirac delta function or the unit impulse function. (Dont confuse with the
used in linearization.)

231
 To see this, recall that if zero is in the interval bounded by t0 and t and g is any continuous
function then,
Z t
(t)g(t)dt = g(0)
t0

Recalling that
Z t
A(tt0 )
x(t) = e x0 + eA(t ) Bu( )d
t0

we see that

y(t) = Cx(t) + Du(t)

Z t
A(tt0 )
= Ce x0 + C eA(t ) Bu( )d + Du(t)
t
Z t 0
= CeA(tt0 ) x0 + CeA(t ) Bu( ) + (t )Du( )d
t
Z 0t
= CeA(tt0 ) x0 + G(t )u( ) d
t0

Zero initial conditions response: x(0) = 0.

Z t
y(t) = G(t )u( )d (12.4)
0

Sometimes this is represented by

y G u
This defines a linear map from the space of input functions to the space of output functions.

Example 157

x = x + u
y = x

Here

G(t) = CeAt B + (t)D

= et

So, for zero initial state,

Z t
y(t) = e(t ) u( ) d
0

232
Example 158 Unattached mass

x1 = x2
x2 = u
y = x1

Here
0 1 0
A= B= C= 1 0 D=0
0 0 1
Hence
At 1 t
e =
0 1
and

G(t) = CeAt B
= t

So, Z t
y(t) = (t )u( ) d
0

Remark 5 Description (12.4) is a very general representation of a linear input-output system.

It can also be used for infinite-dimensional systems such as distributed parameter systems and
systems with delays.

Example 159 Consider the system with input delay described by

x(t) = x(t) + u(t 1)

y(t) = x(t) .

Since Z t
y(t) = e(t ) u( 1) d
0

(why?) we have (upon changing variable of integration)

Z t1
y(t) = e(t1 ) u( ) d
1

Considering only inputs for which

u(t) = 0 for t < 0

and defining
0 for t < 1
G(t) = (t1)
e for t 1
the input-output response of this system is described by (12.4).

233
Impulse response
Consider first a SISO (single-input single-output) system with impulse response function g
and subject to zero initial conditions, x(0) = 0. Suppose such a system is subject to a unit
impulse input, that is, u(t) = (t). Then, using (12.3), its output is given by
Z t
y(t) = g(t )( ) = g(t) .
0

Thus, as its name suggests, the impulse response function g is the output response of the
system to zero initial conditions and a unit impulse input.
Consider now a MIMO (multi-input multi-output) system with impulse response function
G and subject to zero initial conditions. Suppose this system is subject to an input of the
form
u(t) = (t)v
where v is constant m-vector. Then, from (12.3),
Z t
y(t) = G(t )v( ) = G(t)v .
0

By considering v = ej , where each component of ej is zero except for its j-th component
which equals one, we see that Gij is the zero initial state response of yi when uj = and all
other inputs are zero, that is,

u1 (t) = 0

y1 (t) = G1j (t)
..

..
.

.
uj (t) = (t) = yi (t) = Gij (t)
..

..
.

.



um (t) = 0 yp (t) = Gpj (t)

Impulse response and transfer function

Recall that the transfer function (matrix) is given by
G(s) = C(sI A)1 B + D
Also, the impulse response (matrix) is given by
G(t) = CeAt B + (t)D
Using the following Laplace transform results:
L(eAt )(s) = (sI A)1 , L() = 1
we obtain
G = L(G)
that is,

the transfer function G is the Laplace transform of the impulse response G.

234
Response to exponentials and sinusoids

Suppose t0 = 0 and
u(t) = et v

where v is a constant m vector and is not an eigenvalue of A. Then

y(t) = CeAt (x0 xp0 ) + et G()v

| {z } | {z }
transient part steady state part

where
xp0 = (I A)1 Bv

and
G() := C(I A)1 B + D

If A is asymptotically stable, then the transient part goes to zero as t goes to infinity.

12.2 Discretization and zero-order hold

Here we consider the discretization of a continuous-time system described by

xc (t) = Ac xc (t) + Bc uc (t) (12.5)

Let T > 0 be the sampling time for discretization.

Zero-order hold. Suppose that the input to the continuous-time system comes from a
zero-order hold with sampling time T , that is,

uc (t) = ud (k) for kT t < (k + 1)T

Thus the input uc is constant over each sampling interval, [kT, (k + 1)T ).

Suppose also that we sample the state of the continuous-time system at multiples of the
sampling time and let
xd (k) := xc (kT )

235
 Then
xd (k + 1) = xc ((k + 1)T )
Z (k+1)T
Ac T
= e xc (kT ) + eAc ((k+1)T t) Bc uc (t) dt
kT
Z (k+1)T
= eAc T xd (k) + eAc ((k+1)T t) Bc ud (k) dt
kT
Z (k+1)T
= eAc T xd (k) + eAc ((k+1)T t) dt Bc ud (k)
kT

Changing the variable of integration from t to := (k + 1)T t yields

Z (k+1)T Z 0 Z T
Ac ((k+1)T t) Ac
e dt = e d = eAc d
kT T 0

Hence, we obtain the following discretization of the continuous-time system (12.5):

xd (k + 1) = Ad xd (k) + Bd ud (k) (12.6)
where
Ad = eAc T
RT
Bd = 0 eAc d Bc

Recall that
1 22 1 33
eAc t = I + Ac t + A t + Ac t +
2! c 3!
Thus, Z T
T2 T2 2 3
eAc d = T I +
Ac + A T + .
0 2! 3! c
and we have the following power series for Ad and Bd :
T2 2 T3 3
Ad = I + T Ac + A + A +
2! c 3! c
T2 T3 2
Bd = T Bc + Ac Bc + A Bc + .
2! 3! c
If Ac is invertible, then
Z T
eAc d = (eAc T I)A1 1
c = (Ad I)Ac
0
and
Bd = (Ad I)A1
c Bc (12.7)
If we approximate Ad and Bd by considering only terms up to first order in T in their
power series, we obtain
Ad I + T Ac and Bd T B c .

236
Example 160 (Discretization of forced undamped oscillator)

x1 = x2
x2 = x1 + u

0 1 0
Ac = Bc =
1 0 1
Here,
Ac t cos t sin t
e =
sin t cos t
Hence,
Z T
Ac T cos T sin T Ac 1 cos T
Ad = e = Bd = e d Bc =
sin T cos T 0 sin T

Consider T = 2. Then,

1 0 0
Ad = Bd =
0 1 0

So,

x1d (k + 1) = x1d (k)

x2d (k + 1) = x2d (k)

Where did the input go?

What about T = /2?

0 1 1
Ad = Bd =
1 0 1

>> help c2d

C2D Conversion of state space models from continuous to discrete time.

[Phi, Gamma] = C2D(A,B,T) converts the continuous-time system:
.
x = Ax + Bu

237
 to the discrete-time state-space system:

x[n+1] = Phi * x[n] + Gamma * u[n]

assuming a zero-order hold on the inputs and sample time T.

See also: C2DM, and D2C.

>> a=[0 1 ; -1 0] ; b=[0; 1] ; T= pi/2;

>> [phi, gamma]= c2d(a,b,T)

phi =
-0.0000 1.0000
-1.0000 0.0000

gamma =
1.0000
1.0000

12.2.1 Input-output discretization

Consider a continuous-time system:

xc (t) = Ac xc (t) + Bc uc (t)

yc (t) = Cc xc (t) + Dc uc (t)

Suppose T > 0 is the sampling time.

Zero order hold:

uc (t) = ud (k) for kT t < (k + 1)T
Letting

xd (k) := xc (kT ) yd (k) := y(kT )

we obtain

xd (k + 1) = Ad xd (k) + Bd ud (k)
yd (k) = Cd xd (k) + Dd ud (k)

where
Z T
Ac T
Ad = e Bd = eAc d Bc Cd = Cc Dd = Dc
0

238
12.3 Response of discrete-time systems*
A general linear time-invariant discrete-time system is described by

x(k + 1) = Ax(k) + Bu(k)

y(k) = Cx(k) + Du(k)

where k IR is time, x(k) IRn in the state, u(k) IRm the input, and y(k) IRp is the
output.

12.3.1 State response and the convolution sum

The solution to
x(k + 1) = Ax(k) + Bu(k) x(k0 ) = x0 (12.8)
is unique and is given by

k1
X
x(k) = A(kk0 ) x0 + A(k1j) Bu(j) (12.9)
j=k0

Proof. Use induction.

12.3.2 Input-output response and the pulse response matrix

The solution to

x(k + 1) = Ax(k) + Bu(k)

y(k) = Cx(k) + Du(k)

subject to initial condition

x(k0 ) = x0
is unique and is given by
k
X
(kk0 )
y(k) = CA x0 + G(k j)u(j)
j=k0

where the pulse response matrix G is defined by

G(0) = D
G(k) = CAk1 B for k = 1, 2, . . .

Clearly, the above holds for k = k0 . For k > k0 , recall that

k1
X
(kk0 )
x(k) = A x0 + A(k1j) Bu(j)
j=k0

239
Hence,
k1
X
(kk0 )
y(k) = CA x0 + CA(k1j) Bu(j) + Du(k)
j=k0
k1
X
= CA(kk0 ) x0 + G(k j)u(j) + G(0)u(k)
j=k0
k
X
(kk0 )
= CA x0 + G(k j)u(j)
j=k0

Zero initial conditions response: x(0) = 0.

k
X
y(k) = G(k j)u(j)
j=0

Sometimes this is represented by

y G u
Pulse response: Suppose x(0) = 0 and

u(0) = p(k)v for k = 0, 1, . . .

where v is constant m vector and p is the unit pulse function defined by

p(0) = 1
p(k) = 0 for k = 1, 2, . . .

Then,
y(k) = G(k)v for k = 0, 1, . . .
From this we can see that Gij is the zero initial condition response of output component yi
when input component uj is a unit pulse and all other inputs are zero, that is,

u1 (k) = 0

y1 (k) = G1j (k)
..

..
.

.
uj (k) = p(k) = yi (k) = Gij (k)
..

..
.

.



um (k) = 0 yp (k) = Gpj (k)

So, for a SISO (single-input single-output) system, G is the output response due a unit
pulse input and zero initial conditions.

240
Exercises
Exercise 107 Obtain (by hand) the response of each of the following systems due to a unit
impulse input and the zero initial conditions. For each case, determine whether the response
contains all the system modes.

a)

x1 = 5x1 + 2x2 + u
x2 = 12x1 + 5x2 + u

b)

x1 = 5x1 + 2x2 + u
x2 = 12x1 + 5x2 + 2u

c)

x1 = 5x1 + 2x2 + u
x2 = 12x1 + 5x2 + 3u

Exercise 108 Consider the system with disturbance input w and performance output z
described by

x1 = x1 + x2 + w
x2 = x1 + 4x2 + 2w
z = x1 .

Using an appropriate Lyapunov equation, determine

Z
kz(t)k2 dt
0

for each of the following situations.

(a)
w=0 and x(0) = (1, 0) .
(b)
w(t) = (t) and x(0) = 0 .

241
242
Chapter 13

Observability

13.1 Observability
We are concerned here with the problem of determining the state of a system based on
measurements of its inputs and outputs. Consider the system
x = Ax + Bu
(13.1)
y = Cx + Du
with initial condition
x(0) = x0
where the n-vector x(t) is the state, the m-vector u(t) is the input and the p-vector y(t) is
a vector of measured outputs; we call y the measured output (vector).
Consider any time interval [0, T ] with T > 0. The basic observability problem is as
follows. Suppose we have knowledge of the input u(t) and the output y(t) over the interval
[0, T ]; can we uniquely determine the initial state x0 ? If we can do this for all input histories
and initial states x0 we say that the system is observable. So, for observability, we require
that for each input history, different initial states produce different output histories, or,
equivalently, if two output histories are identical, then the corresponding initial states must
be the same. A formal definition of observability follows the next example. Note that
knowledge of the initial state and the input history over the interval [0, T ] allows one to
compute the state x(t) for 0 t T . This follows from the relationship
Z t
At
x(t) = e x0 + eA(t ) Bu( ) d .
0

Example 161 Two unattached masses

x1 = 0
x2 = 0
y = x1

Clearly, this system is unobservable.

243
DEFN. System (13.1) is observable over an interval [0, T ] if the following holds for each
input history u(). Suppose y a () and y b () are any two output histories of system (13.1) due
to initial states xa0 and xb0 , respectively, and y a (t) = y b (t) for 0 t T ; then xa0 = xb0 .

13.2 Main result

To describe the main results associated with observability of LTI systems, we introduce the
observability matrix Qo associated with system (13.1). This is the following pn n matrix

C
CA

Qo = ..
.
CAn1

The following result is the main observability result for continuous-time LTI systems.

Theorem 19 (Main observability result) For any T > 0, system (13.1) is observable
over the interval [0, T ] if and only if the its observability matrix has maximum rank, that is,

C
CA

rank .. =n
.
CAn1

where n is the number of state variables.

An immediate consequence of the above theorem is that observability does not depend
on the interval [0, T ]. So, from now on we drop reference to the interval. Since observability
only depends on the matrix pair (C, A), we say that this pair is observable if system (13.1)
is observable.

SO systems. For scalar output (p = 1) systems, Qo is a square n n matrix; hence it has

rank n if and only if its determinant is nonzero. So, the above theorem has the following
corollary.

Corollary 1 A scalar output system of the form (13.1) is observable if and only if its ob-
servability matrix Qo has non-zero determinant.

Example 162 (Two unattached masses.) Recall example 161. Here n = 2,

0 0
A= C= 1 0
0 0

244
Hence,

C 1 0
Qo = =
CA 0 0

Since rank Qo = 1 < n, this system is unobservable.

Example 163 (The unattached mass)

x1 = x2
x2 = 0

We consider two cases:

(a) (Velocity measurement.) Suppose y = x2 . Then

0 1
A= C= 0 1
0 0

So,
C 0 1
Qo = =
CA 0 0
Since rank Q0 = 1 < n, we have unobservability.

(b) (Position measurement.) Suppose y = x1 . Then

C 1 0
Qo = =
CA 0 1

Since rank Q0 = 2 = n, we have observability.

Example 164 (Beavis and Butthead: mass center measurement)

Figure 13.1: Example 164

mq1 = k(q2 q1 )
mq2 = k(q2 q1 )
y = q1 + q2

Actually, y is not the location of the mass center of the system, however there is a one-to-one
correspondence between y and the mass center location.

245
 With
x1 = q 1 x2 = q 2 x3 = q1 x4 = q2
we have
0 0 1 0
0 0 0 1
A=
k/m
C= 1 1 0 0
k/m 0 0
k/m k/m 0 0
Since,
CA = 0 0 1 1
CA2 = (CA)A = 0 0 0 0
CA3 = (CA2 )A = 0 0 0 0
the observability matrix for this system is

C 1 1 0 0
CA 0 0 1 1
Qo =
CA2 =

0 0 0 0
CA3 0 0 0 0

Since rank Qo = 2 < n, this system is not observable.

Physically, one can see why this system is not observable. The system can oscillate with
the mass center fixed, that is, a non-zero state motion results in zero output history.

Example 165 (Beavis and Butthead: observable) Here we take the system from the
last example and consider the measured output to be the position of the first mass.

mq1 = k(q2 q1 )
mq2 = k(q2 q1 )
y = q1

With
x1 = q 1 x2 = q 2 x3 = q1 x4 = q2
we have
0 0 1 0
0 0 0 1
A=
k/m
C= 1 0 0 0
k/m 0 0
k/m k/m 0 0
Hence,
CA = 0 0 1 0

CA2 = (CA)A = k/m k/m 0 0

CA3 = (CA2 )A = 0 0 k/m k/m

246
So,

C 1 0 0 0
CA 0 0 1 0
Qo =
CA2 = k/m k/m
.

0 0
CA3 0 0 k/m k/m

Since rank Qo = 4 = n; this system is observable.

Useful facts for checking observability. In checking observability, the following facts
are useful.

(a) Suppose there exists n such that

C
C
CA
.. CA
N . = N ..
.
CAn 1 1
CAn
CAn

or, equivalently,

C
C
CA
.. CA
rank . = rank ..
.
CAn 1 1
CAn
CAn

Then

C
CA

N (Qo ) = N ..
.
1
CAn

and

C
CA

rank (Qo ) = rank ..
.
1
CAn

247
 (b) Suppose V 1 , . . . , V k is a sequence of matrices which satisfy
N (V 1 ) = N (C)

V1 V1
.. ..

N . j1 = N . j1 for j = 2, . . . , k
V V
Vj V j1 A

Then
C V1
.. ..
.
N . k = N k
CA V
CAk+1 V kA
and
C V1
.. ..

rank . k = rank . k
CA V
CAk+1 V kA
See next two examples.
Example 166 (Beavis and Butthead: mass center measurement revisited)
mq1 = k(q2 q1 )
mq2 = k(q2 q1 )
y = q1 + q2
With
x1 = q 1 x2 = q 2 x3 = q1 x4 = q2
we have
0 0 1 0
0 0 0 1
A=
k/m
C= 1 1 0 0
k/m 0 0
k/m k/m 0 0
Hence,
CA = 0 0 1 1

CA2 = 0 0 0 0
So, using useful fact (b) above,

C 1 1 0 0
rank (Qo ) = rank = rank =2
CA 0 0 1 1
Since rank (Qo ) < n, this system is not observable.

248
Example 167 (Beavis and Butthead: observable)

mq1 = k(q2 q1 )
mq2 = k(q2 q1 )
y = q1

With
x1 = q 1 x2 = q 2 x3 = q1 x4 = q2
we have
0 0 1 0
0 0 0 1
A=
k/m
C= 1 0 0 0
k/m 0 0
k/m k/m 0 0
Hence,

CA = 0 0 1 0

CA2 = k/m k/m 0 0
So,

C 1 0 0 0
rank CA = rank 0 0 1 0
CA2 0 1 0 0
and
C 1 0 0 0
CA 0 0 1 0
rank (Qo ) = rank
CA2 = rank

0
= 4
1 0 0
CA3 0 0 0 1
Since rank Qo = n; this system is observable.

Example 168 State feedback can destroy observability.

0 1
A= C= 1 0
0 0

(C, A) is observable. Consider

0 1
BK =
0 0
Then
A + BK = 0
and (C, A + BK) is unobservable.

249
13.3 Development of main result
To develop the main result presented in the last section, we first demonstrate the following
result:

System (13.1) is observable over an interval [0, T ] if and only if the uncontrolled system
(u(t) 0)
x = Ax
(13.2)
y = Cx
has the property that the zero state is the only initial state which results in an output history
which is identically zero; that is, it has the following property:

y(t) = 0 for 0tT implies x(0) = 0

To see that the above property is necessary for observability, simply note that the zero
initial state produces a zero output history; hence if some nonzero initial state x0 produces
a zero output history, then there are two different initial states producing the same output
history and the system is not observable.
To see that the above property is sufficient, consider any input history u() and suppose
y () and y b () are two output histories due to initial states xa0 and xb0 , repectively, and y a () =
a

y b (). Let xa () and xb () be the state histories corresponding the xa0 and xb0 , respectively,
that is,

xa = Axa + Bu , xa (0) = xa0

y a = Cxa + Du

and

xb = Axb + Bu , xb (0) = xb0

y b = Cxb + Du

Then, letting

x(t) := xb (t) xa (t)

y(t) := y b (t) y a (t)

we obtain

x = Ax , x(0) = xb0 xa0

y = Cx

Since y(t) 0 we must have xb0 xa0 = 0; hence xb0 = xa0 . So, system (13.1) is observable.

250
Example 169 (The unattached mass)

x1 = x2
x2 = 0

If y = x2 , the system is unobservable: consider nonzero initial state x0 = [1 0]T ; the

corresponding output history satisfies y(t) 0. If y = x1 , the system is observable. To see
this note that x = [y, y]T . If y(t) 0 over a nonzero interval, then y(0) = 0 and hence,
x(0) = 0.

We have just shown that to investigate the observability of system (13.1), we need only
determine the initial states which result in an identically zero output for the system with
zero input, that is system (13.2). So, let us look at the consequences of a zero output for the
above system. So, suppose x0 is an initial state which, for some T > 0, results in y(t) = 0
for 0 t T , that is,
Cx(t) = 0 for 0 t T (13.3)
where
x(t) = Ax(t), x(0) = x0 .
Differentiating the equation in (13.3) with respect to t results in

0 = C x(t) = CAx(t) = 0 for 0 t T .

Repeated differentiation of (13.3) yields:

CAk x(t) = 0 for 0 t T and k = 0, 1, 2, . . .

Letting t = 0 and using the initial condition x(0) = x0 results in

CAk x0 = 0 for k = 0, 1, 2, . . . . (13.4)

So, we have shown that if an initial state x0 results in a zero output over some nonzero
interval then, (13.4) holds.
Let us now demonstrate the converse, that is, if (13.4) holds, then the resulting output
is zero over any nonzero interval. So, suppose (13.4) holds for some initial state x0 . Recall
that x(t) = eAt x0 ; hence
y(t) = CeAt x0
P
Since eAt can be expressed as eAt = k
k=0 k (t)A , we have

X
At
y(t) = Ce x0 = (t)CAk x0 = 0 .
k=0

So, we now conclude that an initial state x0 results in a zero output over some nonzero
interval if and only if (13.4) holds. Since, for each k = 0, 1, 2 . . ., the matrix Ak can be

251
expressed as a linear combination of I, A, A2 , . . . , An1 , it follows that (13.4) holds if and
only if
CAk x0 = 0 for k = 0, 1, 2, n1 . (13.5)
Thus an initial state x0 results in a zero output over some nonzero interval if and only if
(13.5) holds. Recalling the definition of the observability matrix Qo associated with system
(13.1) we obtain the following result:

An initial state x0 results in zero output over some nonzero interval if and only if Qo x0 = 0

Hence, system (13.1) is observable if and only if the only vector satisfying Qo x0 = 0 is
the zero vector. This immediately leads to the following result on observability. Recall that
the nullity of a matrix is the dimension of its null space.

For any T > 0, system (13.1) is observable over the interval [0, T ] if and only if the
nullity of its observability matrix is zero.

Since the rank and nullity of Qo must sum to n which is the maximum of Qo , we now
obtain main result, that is Theorem 19.

252
13.4 The unobservable subspace
Example 170 At first sight, the system

x1 = x1
x2 = x2
y = x1 + x2

might look observable. However, it is not. To see this, introduce new state variables

xo := x1 + x2 and xuo := x1 x2

Then

xo = xo
xu = xu
y = xo

Hence, xo (0) = 0 implies xo (t) = 0 and, hence, y(t) = 0 for all t. In other words if the initial
state x(0) is in the set
{x IR2 : x1 + x2 = 0}
The resulting output history is zero.

Figure 13.2: Example 170

Recall that observability matrix Qo associated with (C, A) is defined to be the following
pn n matrix
C
CA

Qo = ..
.
n1
CA
Recall also that an initial state x0 results in a zero output if and only if Qo x0 = 0, that is
xo is in the null space of Qo . So, we define the unobservable subspace Xuo associated with
system (13.1) or (C, A) to be the null space of the observability matrix, that is,

Xuo = N (Qo )

and we have the following result.

253
Lemma 10 (Unobservable subspace lemma) For any nonzero time interval [0, T ], the
following statements are equivalent for system (13.2).

(i) y(t) = 0 for 0 t T .

(ii) Qo x(0) = 0, or, equivalently, x(0) is in the unobservable subspace Xuo .

Example 171 Recalling the last example we have n = 2 and

1 0
A= C= 1 1
0 1

Hence,

C 1 1
Qo = =
CA 1 1

Clearly,
Xuo = N (Qo ) = N 1 1 = N (C)
Hence
N (Qo ) = {x IR2 : x1 + x2 = 0}

254
13.5 Unobservable modes
Suppose we are interested in the observability of the the input-output system
x = Ax + Bu
y = Cx + Du
Since observability is independent of u, we only need to consider the corresponding zero
input system:
x = Ax
y = Cx (13.6)
Recall that a mode of this system is a special solution of the form x(t) = et v. Such a
solution results when x(0) = v and v is an eigenvector of A with eigenvalue . We say that
et v is an unobservable mode of the above system if, the above system has a solution of the
form x(t) = et v and the corresponding output is zero, that is,

x(t) et v and y(t) 0

The eigenvalue is called an unobservable eigenvalue of the system. We also refer to as an
unobservable eigenvalue of (C, A).
Clearly, if a system has an unobservable mode then, the system is not observable. In
the next section, we show that if an LTI system is not observable, then it must have an
unobservable mode. Thus:

An LTI system is observable if and only if it has no unobservable modes.

This is illustrated in the following example.

Example 172
x1 = 1 x1 + b1 u
x2 = 2 x2 + b2 u
y = c1 x1 + c2 x2 + du
Since
C c1 c2
Qo = =
CA 1 c1 2 c2
This system is observable if and only if
c1 6= 0, c2 6= 0 , 1 6= 2
Note that if c1 = 0 then considering u(t) 0 this system has a solution

1 t 1
x(t) = e
0
and the corresponding output satisfies
y(t) 0

255
13.5.1 PBH Test
The next lemma provides a useful characterization of unobservable eigenvalues.

Lemma 11 A complex number is an unobservable eigenvalue of the pair (C, A) if and

only if the matrix
A I
(13.7)
C
does not have maximum rank (that is n, where n is the number of columns in A.)

Proof. Recall that is an unobservable eigenvalue of (C, A) if the system x = Ax

has a non-zero solution of the form x(t) = et v which satisfies Cx(t) 0. We have already
seen that the existence of a nonzero solution of the form et v is equivalent to v being an
eigenvector of A with eigenvalue , that is, v 6= 0 and Av = v; this is equivalent to

(A I)v = 0

and v 6= 0. Since x(t) = et v and et 6= 0, the statement that Cx(t) 0 is equivalent to

Cv = 0 .

So, we have shown that is an unobservable eigenvalue of (C, A) if and only if there is a
nonzero vector satisfying
A I
v = 0.
C
The existence of a nonzero v satisfying the above equality is equivalent to saying that the
columns of the matrix in the equality are linearly dependent, that is, the rank of the matrix
is less than the number of its columns. Since the number of columns in the above matrix is
less than or equal to the number of its rows, the maximum rank of this matrix equals the
number of its columns. Hence, being an unobservable eigenvalue of (C, A) is equivalent to
the matrix in (13.7) not having maximum rank.

Example 173 Consider a system with

0 1 0
A= B= C= 1 1 D = 0.
1 0 1
Here, the observability matrix is

C 1 1
Qo = = .
CA 1 1

Since Qo has zero determinant, this system is not observable. Hence, the system has some
unobservable modes. The eigenvalues of A are 1 and 1. To determine the unobservable
eigenvalues, we carry out the PBH test:

1
A I
= 1
C
1 1

256
For = 1, the above matrix is rank deficient; hence 1 is an unobservable eigenvalue. For
= 1, the PBH matrix has maximum rank. Hence, this eigenvalue is observable.
Note that the transfer function for this system is given by
s1 1
G(s) = = .
s2 1 s+1
So, the unobservable eigenvalue is not a pole of the transfer function. This transfer function
is also the transfer function of the stable system with with A = 1 and B = C = 1 and
D = 0.
Since we have already stated that an LTI system is observable if and only if it has no
unobservable eigenvalues, we obtain the following corollary to Lemma 11. This new result
provides another useful test for observability.

Corollary 1 (PBH observability test.) The pair (C, A) is observable if and only if the matrix

A I
C

has maximum rank for every eigenvalue of A.

Example 174 (Unobservable modes)

A
-2 2 1 2
-1 1 1 2
-3 2 2 2
-1 0 -1 1

B =

2
0
0
2

C
1 0 0 -1

Qo= obsv(A, C)
1 0 0 -1
-1 2 2 1
-7 4 4 7
-9 -2 -2 9

rank(Qo)
2 %System is unobservable

257
 eig(A)
1.0000 + 2.0000i
1.0000 - 2.0000i
-1.0000
1.0000

I=eye(4);

rank([A-(-1)*I; C]) %PBH Time

3

rank([A-(1)*I; C])
3

rank([A-(1+2i)*I; C])
4

rank([A-(1-2i)*I; C])
4
So the eigenvalues 1 and +1 are unobservable. The eigenvalues 1 + 2 and 1 2 are not
unobservable.

[num den] = ss2tf(A,B,C,0)

num = 1.0e-013 *
0 -0.0400 0.0933 0.0044 -0.1954

den =
1.0000 -2.0000 4.0000 2.0000 -5.0000

%Actually, the transfer function is ZERO! Why?

Here we observe that unobservable eigenvalues are invariant under a state transformation.
To see this consider any state transformation of the form x = T x; then the new PBH
observability matrix is given by
1 1
A I T AT I T 0 A I
= = T
C CT 0 I C

258
Since T is invertible, we see that the new and the old PBH matrices have the same rank.

13.5.2 Existence of unobservable modes*

Here we will show that, if a system is unobservable then, is must have an unobservable mode.
So, consider a linear time-invariant input-output system described

x = Ax + Bu (13.8a)
y = Cx + Du (13.8b)

where x(t) is an n-vector, u(t) is an m-vector, and y(t) is a p-vector. Suppose (C, A) is not
observable. In the following section, we show that there is a state transformation x = T x so
that the corresponding transformed system has the following special structure:

xo = Aoo xo + Bo u
xu = Auo xo + Auu xu + Bu u
y = Co xo + Du

with
xo
= x .
xu
Also, the pair (Co , Aoo ) is observable. Roughly speaking, this transformation splits the state
into an observable part xo and an unobservable part xu ; the subsystem

xo = Aoo xo + Bo u
y = Co xo + Du

is observable; one can obtain xo from knowledge of the histories of u and y.

Notice that, in the transformed system description, there is no connection from the state
xu to the output y. We also note that, when u(t) 0 and xo (0) = 0 we must have xo (t) 0.
Also,

xu = Auu xu
y = 0

From this it should be clear that, if is any eigenvalue of Auu then the transformed system
(with zero input) has a solution which satisfies

x(t) = et v and y(t) 0

for some nonzero v, that is is an unobservable eigenvalue of the transformed system.

Hence, the system
x = Ax , y = Cx
has a nonzero solution x which satisfies

x(t) et v and y(t) 0

259
 Since the matrices A and
Aoo 0
A =
Auo Auu
are similar, they have the same characteristic polynomial. Moreover the characteristic poly-
nomial of A is given by

det(sI A) = det(sI Aoo ) det(sI Auu )

Hence the characteristic polynomial of A is the product of the characteristic polynomials of

Aoo and Auu . Thus, the eigenvalues of A are the union of those of Aoo and Auu . Hence, the
eigenvalues of Auu are the unobservable eigenvalues of (C, A). The corresponding modes are
the unobservable modes.

Transfer function considerations. Consider the reduced order observable system:

xo = Aoo xo + Bo u
y = Co xo + Du

One can readily show that the transfer function of this system is exactly the same as that
of the original system, that is,

C(sI A)1 B + D = Co (sI Ao )1 Bo + D

13.5.3 A nice transformation*

Consider a LTI input-output system described by

x = Ax + Bu
y = Cx + Du

where x(t) is an n-vector, u(t) is an m-vector, and y(t) is a p-vector. Suppose (C, A) is not
observable; then, its unobservable subspace Xuo is nontrivial. The next lemma states that
Xuo is an invariant subspace of A.

Lemma 12 For any n n matrix A and any p n matrix, C, the subspace

C
CA

Xuo = N ..
.
n1
CA

is an invariant subspace for A.

Proof. Consider any element x of Xuo . It satisfies

CAk x = 0 for k = 0, , n 1

260
Since
An = 0 I 1 A n1 An1
we obtain
CAn x = 0
from which it follows that

CAk (Ax) = 0 for k = 0, , n 1

hence Ax is contained in Xuo . Since this holds for every element x of Xuo , it follows that Xuo
is A-invariant.

Transformation. Suppose nu is the dimension of the unobservable subspace Xuo , that is,
nu is the nullity of the observability matrix, and let

no := n nu .

Choose any basis

tno +1 , . . . , tn
for the unobservable subspace, that is, the null space of the observability matrix Qo . Extend
it to a basis
t1 , . . . , tno , tno +1 , . . . , tn
for the whole state space IRn and let

T := t1 t2 . . . tn

(One could do this by choosing T = V where U SV is a singular value decomposition of

Qo ). Introduce the state transformation

x = T x

If we let
xo
= x
xu
where xo is an no -vector and xu is an nu -vector, then

x is in Xuo if and only if xo = 0

The transformed system description is given by

x = Ax + Bu
y = C x + Du

where
A = T 1 AT B = T 1 B C = CT

261
Since Xuo is an invariant subspace of A, the matrix A has the following structure:

Aoo 0
A =
Auo Auu

where Aoo is no no . Since the vectors tno +1 , . . . , tn are in the null space of C, the matrix C
has the following structure
C = Co 0
where Co is p no . If we partition B as

Bo
B =
Bu

where Bo is no m. then the transformed system is described by

xo = Aoo xo +Bo u
xu = Auo xo + Auu xu +Bu u
y = C o xo +Du

The pair (Co , Aoo ) is observable.

Example 175 (Unobservable modes)

A
-2 2 1 2
-1 1 1 2
-3 2 2 2
-1 0 -1 1

C
1 0 0 -1

Qo= obsv(A, C)
1 0 0 -1
-1 2 2 1
-7 4 4 7
-9 -2 -2 9

rank(Qo)
2 %System is unobservable

eig(A)
1.0000 + 2.0000i

262
 1.0000 - 2.0000i
-1.0000
1.0000

I=eye(4);

rank([A-(-1)*I; C]) %PBH Time

3

rank([A-(1)*I; C])
3

rank([A-(1+2i)*I; C])
4

rank([A-(1-2i)*I; C])
4

So the eigenvalues 1 and +1 are unobservable. The eigenvalues 1 + 2 and 1 2 are not
unobservable.

rref(Qo)
1 0 0 -1
0 1 1 0
0 0 0 0
0 0 0 0

N=[1 0 0 1; 0 1 -1 0] %Basis for unobservable subspace

1 0
0 1
0 -1
1 0

Qo*N
0 0 %Just checking!
0 0
0 0
0 0

rref([N I])
1 0 0 0 0 1
0 1 0 0 -1 0
0 0 1 0 0 -1

263
 0 0 0 1 1 0

T=[E(:,1:2) N ] % A nice transformation

1 0 1 0
0 1 0 1
0 0 0 -1
0 0 1 0

Anew = inv(T)*A*T
-1 2 0 0
-4 3 0 0
-1 0 0 1
3 -2 1 0

Cnew = C*T
1 0 0 0

Aoo= Anew(1:2,1:2)
-1 2
-4 3

Auu= Anew(3:4,3:4)
0 1
1 0

eig(Auu)
-1.0000 %Unobservable eigenvalues
1.0000

eig(Aoo)
1.0000 + 2.0000i %The remaining eigenvalues
1.0000 - 2.0000i
B
2
0
0
2

[num den] = ss2tf(A,B,C,0) num = 1.0e-013 *

0 -0.0400 0.0933 0.0044 -0.1954

den =
1.0000 -2.0000 4.0000 2.0000 -5.0000

264
 %Actually, the transfer function is ZERO! Why?

13.6 Observability grammians*

For each T > 0, we define the finite time observability grammian associated with (C, A):
Z T

Wo (T ) = eA t C CeAt dt
0

To obtain an interpretation of Wo (T ), consider the output y due to zero input and any
initial state x0 , that is,

x = Ax x(0) = x0
y = Cx

Then y(t) = CeAt x0 , and

Z T Z T Z T
2 At 2
At
||y(t)|| dt = ||Ce x0 || dt = CeAt x0 Ce x0 dt
0 0 0
Z T Z T
A t At A t At
= x0 e C Ce x0 dt = x0 e C Ce dt x0
0 0
= x0 Wo (T )x0 ;

thus Z T
||y(t)||2 dt = x0 Wo (T )x0 .
0

Remark 6 Note that Wo (T ) is the solution at t = T to the following initial value problem:

Wo = Wo A + A Wo + C C and Wo (0) = 0

Proof. With s = t , we have = t s and

Z t Z t
A A (ts)
Wc (t) = e C Ce d = eA(ts) C CeA ds
0 0

Hence, Wc (0) = 0 and

Wc = AWc + Wc A + C C

Lemma 13 For each T > 0, the matrices Wo (T ) and Qo have the same null space.

265
Proof. We need to show that N (Qo ) = N (Wo (T )). Since

C
CA

Qo = ..
.
(n1)
CA

it follows that x is in N (Qo ) if and only if

CAk x = 0 for k = 0, 1, . . . , n1 (13.9)

We first show that N (Qo ) N (Wo (T )): Consider any x in N (Qo ). Then (13.9) holds.
Recalling the application of the Cayley-Hamilton theorem to functions of a matrix, it follows
that for 0 t T , there are scalars 0 (t), . . . , n1 (t) such that
n1
X
eAt = i (t)Ai
i=0

Hence,
n1
X
At
Ce x = i (t)CAi x = 0
i=0

and
Z T

Wo (T )x = eA t C CeAt dt x
0
Z T

= eA t C CeAt x dt
0

= 0

that is, x is in N (Wo (T )). Since the above holds for any x N (Qo ), we must have N (Qo )
N (Wo (T )).
We now show that N (Wo (T )) N (Qo ) and hence N (Qo ) = N (Wo (T )). To this end,
consider any x in N (Wo (T )). Then,

0 = x Wo (T )x
Z T
A t At
= x e C Ce dt x
0
Z T

= x eA t C CeAt x dt
0
Z T
At At
= Ce x Ce x dt
0
Z T
= ||CeAt x||2 dt .
0

266
Since the last integrand is non-negative for all t and the integral is zero, the integrand must
be zero for all t, that is,
CeAt x 0 .
If a function is zero for all t, all its derivatives must be zero for all t; hence

CAk eAt x 0 for k = 0, 1, . . . , n1 .

Considering t = 0 yields

CAk x = 0 for k = 0, 1, . . . , n1 ,

that is, x is in N (Qo ). Since the above holds for any x in N (Wo (T )), we must have
N (Wo (T )) N (Qo ).

Example 176 (The unattached mass)

x1 = x2
x2 = 0

Here
0 1 At 1 t
A= e =
0 0 0 1

(a) (Position measurement.) y = x1 . Here

C= 1 0

Hence, Z
T
A t At T T 2 /2
Wo (T ) = e C Ce dt = 2
0 T /2 T 3 /3
and Wo (T ) and Qo have the same null space; that is, {0}.

(b) (Velocity measurement.) y = x2 . Here

C= 0 1

Hence, Z
T
A t At 0 0
Wo (T ) = e C Ce dt =
0 0 T
and Wo (T ) and Qo have the same null space.

The above lemma has the following consequence.

The pair (C, A) is observable if and only if for every T > 0 the corresponding observability
grammian is invertible.

267
Computation of initial state. One reason for introducing the observability grammian
is as follows: Suppose (C, A) is observable; then from the above lemma, Wo (T ) is invertible.
The output for

x = Ax + Bu x(0) = x0
y = Cx + Du

is given by Z t
At
y(t) = Ce x0 + CeA(t ) Bu( ) d + Du(t)
0
Letting Z t
y(t) = y(t) CeA(t ) Bu( ) d Du(t)
0
At
we have y(t) = Ce x0 and
Z T Z T
A t
e C y(t) dt = eA t C CeAt x0 dt
0 0

= Wo (T )x0 .

Since Wo (T ) is invertible, we obtain the following explicit expression for the initial state in
terms of the input and output histories:
RT
x0 = Wo (T )1 0
eA t C y(t) dt

The above expression explicitly shows us that if a system is observable, we can obtain the
initial state from measurements of the input and output over any time interval. However,
this formula is not usually used to obtain the state. One usually uses asymptotic observers
or state estimators to asymptotically obtain the state; these are covered in a later chapter.

13.6.1 Infinite time observability grammian

Suppose A is asymptotically stable, that is, all its eigenvalues have negative real parts, and
consider the linear matrix equation

Wo A + A Wo + C C = 0

This is a Lyapunov equation of the form P A + A P + Q = 0 with Q = C C. Since A is

asymptotically stable the above equation has a unique solution for Wo and this solution is
given by Z

Wo = eA t C CeAt dt
0
We call this matrix the (infinite-time) observability grammian associated with (C, A). One
may show that Wo and the observability matrix Qo have the same null-space. Since Wo is a
symmetric positive definite matrix, it follows that the null space of Wo is {0} if and only if
Wo is positive definite. Hence, we have the following result.

268
Lemma 14 Suppose all the eigenvalues of A have negative real parts. Then (C, A) is ob-
servable if and only if Wo is positive definite.

Note that if y is the output of the system

x = Ax
y = Cx

with initial condition x(0) = x0 then

Z
ky(t)k2 dt = x0 W0 x0
0

This follows from y(t) = CeAt x0 and

Z Z
2
ky(t)k dt = y(t) y(t) dt
0 0
Z
= (CeAt x0 ) CeAt x0 dt
0
Z

= x0 eA t C CeAt dt x0
0
= x0 W0 x0 .

269
13.7 Discrete time
Consider the discrete-time system described by
x(k + 1) = Ax(k) + Bu(k)
(13.10)
y(k) = Cx(k) + Du(k)
where k ZZ is time, x(k) IRn is the state, u(k) IRm is the input, and y(k) IRp is the
measured output.
Consider any fixed time N > 0. The basic observability problem is as follows. Sup-
pose we have knowledge of the input sequence { u(0), u(1), , u(N ) } and output sequence
{ y(0), y(1), , y(N ) } over the interval [0, N ]. Can we uniquely determine the initial state
x0 = x(0)
If we can do this for input histories and all initial states we say the system is observable.

DEFN. System (13.10) is observable over the interval [0, N ] if the following holds for each
input history u(). Suppose y a () and y b () are any two output histories of system (13.1) due
to initial states xa0 and xb0 , respectively, and y a (k) = y b (k) for 0 k N ; then xa0 = xb0 .

Our first observation is that system (13.10) is observable over [0, N ] if and only if the
system
x(k + 1) = Ax(k)
(13.11)
y(k) = Cx(k)
has the property that the zero state is the only initial state resulting in an output history
which is identically zero; that is, it has the the following property:
y(k) = 0 for 0 k N = x(0) = 0

13.7.1 Main observability result

Theorem 20 (Main observability theorem) For each N n 1 , system (13.10) is
observable over [0, N ] if and only if

C
CA

rank .. =n
.
n1
CA

Proof. The proof can be obtained by showing that, for system (13.11) with x0 = x(0),

y(0) Cx0
y(1) CAx0

.. = .. = Q 0 x0
. .
n1
y(n1) CA x0
Also use Cayley Hamilton.

270
13.8 Exercises
Exercise 109 Determine whether or not each of the following systems are observable.

x1 = x1 x1 = x1 x1 = x1 x1 = x2
x2 = x2 + u x2 = x2 + u x2 = x2 + u x2 = 4x1 + u
y = x1 + x2 y = x2 y = x1 + x2 y = 2x1 + x2

Exercise 110 Consider the system with input u, output y, and state variables x1 , x2 de-
scribed by

x1 = 3x1 + 2x2 + u
x2 = 4x1 + 3x2 + 2u
y = 2x1 + x2

(a) Is this system observable?

(b) If the system is unobservable, determine the unobservable eigenvalues.

Exercise 111 Determine (by hand) whether or not the following system is observable.

x1 = 5x1 x2 2x3
x2 = x1 + 3x2 2x3
x3 = x1 x2 + 4x3
y1 = x1 + x2
y2 = x2 + x3

If the system is unobservable, compute the unobservable eigenvalues.

Exercise 112 Determine the unobservable eigenvalues for each of the systems of Exercise
109.

Exercise 113 For each system in Exercise 109 which is not observable, obtain a reduced
order system which is observable and has the same transfer function as the original system.

Exercise 114 (Damped linear oscillator.) Recall

x1 = x2
x2 = (k/m) x1 (d/m) x2

Show the following:

(a) (Position measurement.) If y = x1 , we have observability.

(b) (Velocity measurement.) If y = x2 , we have observability if and only if k 6= 0.

(c) (Acceleration measurement.) If y = x2 , we have observability if and only if k 6= 0.

271
Exercise 115 (BB in laundromat) Obtain a state space representation of the following sys-
tem.
mq1 m2 q1 + k(q1 q2 ) = 0

mq2 m2 q2 k(q1 q2 ) = 0

y = q1
Determine whether or not your state space representation is observable.

Figure 13.3: Beavis and Butthead in the laundromat

Exercise 116 (BB in laundromat: mass center observations.) Obtain a state space represen-
tation of the following system.

mq1 m2 q1 + k(q1 q2 ) = 0

mq2 m2 q2 k(q1 q2 ) = 0

1
y = (q
2 1
+ q2 )

(a) Obtain a basis for its unobservable subspace.

p
(b) Determine the unobservable eigenvalues. Consider := k/2m > .

Exercise 117 (BB in laundromat: internal observations.) Obtain a state representation of

the following system.

mq1 m2 q1 + k(q1 q2 ) = 0

mq2 m2 q2 k(q1 q2 ) = 0

y = q2 q1

(a) Obtain a basis for its unobservable subspace.

(b) By using a nice transformation, obtain a reduced order observable system which has
the same transfer function as the original system.

272
 p
(c) Determine the unobservable eigenvalues. Consider := k/2m > .

Exercise 118 Consider a system described by

x1 = 1 x1 + b1 u
x2 = 2 x2 + b2 u
..
.
xn = n xn + bn u
y = c 1 x1 + c 2 x2 + + c n xn

where all quantities are scalar. Obtain conditions on the numbers 1 , , n and c1 , , cn
which are necessary and sufficient for the observability of this system.

Exercise 119 Compute the observability grammian for the damped linear oscillator with
position measurement and d, k > 0.

Exercise 120 Carry out the following for linearizations L1-L8 of the two pendulum cart
system.

(a) Determine which linearizations are observable?

(b) Compute the singular values of the observability matrix.

(b) Determine the unobservable eigenvalues for the unobservable linearizations.

273
274
Chapter 14

Controllability

14.1 Controllability
Consider a system in which x(t) is the system state at time t and u(t) is the control input at
time t. Consider now any non-trivial time interval [0, T ]; by non-trivial we mean that T > 0.
We say that the system is controllable over the interval if it can be driven from any one
state to any other state over the time interval by appropriate choice of control input over
the interval. A formal definition follows the next two examples.

Example 177 (Two unattached masses)

x1 = u
x2 = 0

Since the behavior of x2 is completely unaffected by the control input u, this system is clearly
uncontrollable over any time interval.

Example 178 (Integrator) Consider the scalar system described by

x = u

Consider any initial state x0 and any desired final state xf . For any any T > 0, consider the
constant input:
u(t) = (xf x0 )/T for 0tT
Then the state trajectory x() of this system with x(0) = x0 satisfies x(T ) = xf . We consider
this system to be controllable over any time interval [0, T ].

DEFN.[Controllability] A system is controllable over a non-trivial time interval [0, T ] if, for
every pair of states x0 and xf , there is a continuous control function u() defined on the
interval such that the solution x() of the system with x(0) = x0 satisfies x(T ) = xf .

Note that the definition says nothing about keeping the state at xf after T .

275
 Figure 14.1: Controllability

14.2 Main controllability result

The concept of controllability also applies to linear and nonlinear systems. However, we are
concerned here with linear time-invariant systems described by

x = Ax + Bu (14.1)

where the n-vector x(t) is the system state at time t and the m-vector u(t) is the control
input at time t. In this section, we introduce a simple algebraic condition which is necessary
and sufficient for the controllability of a linear time invariant system over any time interval.
This condition involves the controllability matrix Qc associated with the above system or the
pair (A, B). This matrix is defined to be the following n nm partitioned matrix:

Qc = B AB An1 B (14.2)

Theorem 21 (Main controllability result) For any T > 0, system (14.1) is controllable
over the interval [0, T ] if and only if its controllability has maximum rank, that is,

rank B AB An1 B =n

Proof. A proof is given at the end of Section 14.3.

An immediate consequence of the above theorem is that controllability does not depend
on the interval [0, T ]. So, from now on we drop reference to the interval. Since controllability
only depends on the matrix pair (A, B), we say that this pair is controllable if system (14.1)
is controllable.
For scalar input (m = 1) systems, the controllability matrix is a square matrix; hence it
has maximum rank if and only if its determinant is nonzero. So, the above theorem has the
following corollary.

Corollary 2 A scalar input system of the form (14.1) is controllable if and only if its con-
trollability matrix has non-zero determinant.

276
Example 179 (Two unattached masses.) This is a scalar input system with

0 0 1
A= and B= .
0 0 0
Hence,
1 0
Qc = B AB =
0 0
Clearly Qc has zero determinant, hence this system is not controllable.
Example 180
x1 = x3 + u1
x2 = x3 + u1
x3 = u2
Since there is a control input entering every state equation, this system looks controllable.
Is it?
Here n = 3 and
0 0 1 1 0
A= 0 0 1 B= 1 0
0 0 0 0 1
So,
1 0 0 1 0 0
Qc = B AB A2 B = 1 0 0 1 0 0
0 1 0 0 0 0
and

1 0
rank (Qc ) = rank 1 0 = 2
0 1
Since rank Qc = 2, we have rank Qc 6= n; hence this system is not controllable.

MATLAB
>> help ctrb

CTRB Form controllability matrix.

CTRB(A,B) returns the controllability matrix
Co = [B AB A^2B ...]

>> A=[0 0 1; 0 0 1; 0 0 0];

>> B=[1 0; 1 0; 0 1];

>> ctrb(A,B)
ans =
1 0 0 1 0 0
1 0 0 1 0 0
0 1 0 0 0 0

277
Useful fact for computing rank Qc . In computing rank Qc , the following fact is useful.
Suppose there exists a positive integer k such that

rank B AB . . . Ak B = rank B AB . . . Ak1 B
Then
rank Qc = rank B AB . . . Ak1 B
The next example illustrates the above fact.

Example 181 (Beavis and Butthead: self-controlled) Consider the situation in which
the system generates a pair of control forces which are equal in magnitude and opposite in
direction.

Figure 14.2: B&B: self-controlled

The motion of this system can be described by

mq1 = k(q2 q1 ) u
mq2 = k(q2 q1 ) + u

Based on simple mechanical considerations, we can see that this system is not controllable.
Now lets do the controllability test. With

x1 = q 1 x2 = q 2 x3 = q1 x4 = q2

we have
0 0 1 0 0
0 0 0 1 0
A=
k/m
B=
k/m 0 0 1/m
k/m k/m 0 0 1/m
Here
0 1/m
0 1/m
B AB =
1/m

0
1/m 0
and
0 1/m 0
0 1/m 0
B AB A2 B =
1/m

0 2k/m2
1/m 0 2k/m2

278
Clearly,
rank B AB A2 B = rank B AB
Hence
rank Qc = rank B AB =2

MATLAB time.
>> a = [0 0 1 0
0 0 0 1
-1 1 0 0
1 -1 0 0];
>> b = [0; 0; -1; 1];

>> rank(ctrb(a,b))
ans = 2

Example 182 (Beavis and Butthead with external control) Here we consider the sys-
tem of the previous example and replace the pair of internal forces with a single external
control force u. This system is described by

Figure 14.3: B&B: external control

mq1 = k(q2 q1 )
mq2 = k(q2 q1 ) +u
With
x1 = q 1 x2 = q 2 x3 = q1 x4 = q2
we have
0 0 1 0 0
0 0 0 1 0
A=
k/m
B=
k/m 0 0 0
k/m k/m 0 0 1/m
Hence,

0 0 0 k/m2
0 1/m 0 k/m2
Qc = B AB A2 B A3 B =
2


0 0 k/m 0
2
1/m 0 k/m 0
Clearly, rank Qc = 4 = n; hence this system is controllable.

279
Exercise 121 Suppose A is n n, B is n m, T is n n and nonsingular and let

A = T 1 AT B = T 1 B

Then (A, B) is controllable if and only if (A, B) is controllable.

14.3 The controllable subspace

Let us now consider the situation when the system x = Ax + Bu is not controllable.

Example 183 At first sight the system

x1 = x1 + u
x2 = x2 + u

might look controllable. However, it is not. To see this, consider the following change of
state variables:

x c = x1 + x2 and xu = x1 x2

These variables satisfy

xc = xc + 2u
xu = xu

Hence, xu (0) = 0 implies xu (t) = 0 for all t. In other words if the initial x(0) is in the set

{(x1 , x2 ) IR2 : x1 x2 = 0}

then, regardless of the control input, the state will always stay in this set; it cannot leave
this set. So, the system is not controllable.

Figure 14.4: Controllable subspace in Example 183

280
 Recall that the controllability matrix associated with the pair (A, B) is defined to be the
following n nm matrix
Qc = B AB . . . An1 B

The controllable subspace is defined as the range of the controllability matrix, that is,

R(Qc )

So, according to the Main Controllability Theorem (Theorem 21), the system x = Ax+Bu
is controllable if and only if its controllable subspace is the whole state space. It should be
clear that a vector x is in the controllable subspace associated with (A, B) if and only if it
can be expressed as
n1
X
x= Ai B i
i=0

where 0 , . . . , n1 are m-vectors.

The following lemma states that when the state of system (14.1) starts at zero, then
regardless of the control input, the state always stays in the controllable subspace; that is,
it cannot leave this subspace.

Lemma 15 If x(0) = 0 then, for every continuous control input function u(), the solution
x() of (14.1) lies in the controllable subspace, that is, x(t) is in the range of Qc for all t 0.

Proof. For any time t > 0 we have

Z t
x(t) = eA(t ) Bu( ) d
0

Recalling the application of the Cayley-Hamilton theorem to functions of a matrix, it follows

that for 0 t, there are scalars 0 ( ), . . . , n1 ( ) such that
n1
X
eA(t ) = i ( )Ai
i=0

Hence,
n1
X
A(t )
e Bu( ) = Ai Bi ( ) i ( ) := i ( )u( )
i=0

From this it should be clear that for each R, the n vector eA(t ) Bu( ) is in R(Qc ), the range
t
of Qc . From this one obtains that x(t) = 0 eA(t ) Bu( ) d is in R(Qc ).

Example 184 For the system of example 183, we have

1 1
Qc = B AB =
1 1

281
Clearly,
1
R(Qc ) = R = R (B)
1
If a vector x is in this controllable subspace, then x = c [ 1 1 ]T for some scalar c. This is
equivalent to the requirement that x1 x2 = 0.

Actually, if the state starts at any point in R(Qc ) then, regardless of the control input,
the state always stays in R(Qc ); see next exercise. In other words, the control input cannot
drive the state out of the controllable subspace.

Figure 14.5: Invariance of the controllable subspace

Exercise 122 Prove the following statement. If x(0) is in R(Qc ), then for every continuous
control input function u(), the solution of (14.1) satisfies x(t) R(Qc ) for all t 0.

The next result states that over any time interval, system (14.1) can be driven between
any two states in the controllable subspace R(Qc ) by appropriate choice of control input.

Lemma 16 [Controllable subspace lemma] Consider any pair of states x0 , xf in the con-
trollable subspace R(Qc ). Then for each T > 0, there is a continuous control function
u() : [0, T ] IRm such that the solution x() of (14.1) with x(0) = x0 satisfies x(T ) = xf

Proof. A proof is contained in section 14.4.

Example 185

x1 = x1 + x2 + u
x2 = x1 + x2 + u

Here,
1 1 1
A= B=
1 1 1
Hence
1 0
Qc = B AB =
1 0

282
and
1
R(Qc ) = R
1
To illustrate the controllable subspace lemma, introduce new states

xc = x1 xu = x1 + x2

Then
1 0
x = xc + xu
1 1
and

xc = xu + u
xu = 0

It should be clear that x is in the controllable subspace if and only if xu = 0. In this case x
is completely specified by the scalar xc and

xc = u

Recalling example 178, we see that given any xc0 , xcf and any T > 0, one can always find
a control input which drives xc from xc0 to xcf over the interval [0, T ]. Hence, the state x
can be driven between any two points in the controllable subspace.

Proof of main controllability result. First note that rank Qc = n if and only if R(Qc ) =
IRn , that is, the controllability matrix has rank n if and only if the controllable subspace is
the whole state space. If rank Qc = n, then Lemma 16 implies controllability. If rank Qc < n,
then the controllable subspace is not the whole state space and from Lemma 15 any state
which is not in the controllable subspace cannot be reached from the origin.

283
14.4 Proof of the controllable subspace lemma*
14.4.1 Finite time controllability grammian
Before proving the controllable subspace lemma, we first establish a preliminary lemma. For
each T > 0, we define the finite time controllability grammian associated with (A, B):
Z T

Wc (T ) := eAt BB eA t dt (14.3)
0

Note that this matrix is hermitian and positive semi-definite.

Remark 7 Note that Wc (T ) is the solution at t = T of the following initial value problem:

Wc = AWc + Wc A + BB and Wc (0) = 0

Proof. Z t Z t
A A (ts)
Wc (t) = e BB e d = eA(ts) BB eA ds
0 0

Hence, Wc (0) = 0 and

Wc = AWc + Wc A + BB

Lemma 17 For each T > 0,

R(Wc (T )) = R(Qc )

Proof. We will show that N (Qc ) = N (Wc (T )). From this it will follow that R(Qc ) =
R(Wc (T ) ) = R(Wc (T )).
Since
B
B A

Qc = ..
.
(n1)
B A
it follows that x is in N (Qc ) if and only if

B Ak x = 0 for k = 0, 1, . . . , n1 (14.4)

We first show that N (Qc ) N (Wc (T )): Consider any x N (Qc ). Then (14.4) holds.
Recalling the application of the Cayley-Hamilton theorem to functions of a matrix, it follows
that for 0 t T , there are scalars 0 (t), . . . , n1 (t) such that
n1
X
A t
e = i (t)Ai
i=0

284
Hence,
n1
X
A t
B e x = i (t)B Ai x = 0
i=0

and
Z T

Wc (T )x = eAt BB eA t dt x
0
Z T

= eAt BB eA t x dt
0

= 0
that is, x N (Wc (T )). Since the above holds for any x N (Qc ), we must have N (Qc )
N (Wc (T )).
We now show that N (Wc (T )) N (Qc ) and hence N (Qc ) = N (Wc (T )): Consider any
x N (Wc (T )). Then,
0 = x Wc (T )x
Z T
At A t
= x e BB e dt x
0
Z T

= x eAt BB eA t x dt
0
Z T
A t A t
= B e x B e x dt
0
Z T

= ||B eA t x||2 dt
0

Since the integrand is non-negative for all t and the integral is zero, the integrand must be
zero for all t, that is,

B eA t x 0
If a function is zero for all t, all its derivatives must be zero for all t; hence

B Ak eA t x 0 for k = 0, 1, . . . , n1
Considering t = 0 yields
B Ak x = 0 for k = 0, 1, . . . , n1
that is x N (Qc ). Since the above holds for any x N (Wc (T )), we must have N (Wc (T ))
N (Qc ).
From our main controllability result and the above lemma, we see that (A, B) is control-
lable if and only if , for any T > 0, its finite time controllability grammian Wc (T ) has full
rank. Since Wc (T ) is square, it has full rank if and only if it is invertible. Moreover, since
Wc (T ) is symmetric and positive semidefinite, it has full rank if and only if it is positive
definite. This yields the following result.

285
Corollary 2 The system x = Ax + Bu is controllable if and only if its controllability gram-
mian over any non-trivial interval is positive definite.

14.4.2 Proof of the controllable subspace lemma

Consider any T > 0 and any pair of states x0 , xf in R(Qc ). The solution of (14.1) at time T
is: Z T
AT
x(T ) = e x0 + eA(T ) Bu( ) d
0

Hence, a control input drives the state from x0 to xf if and only if

Z T
eA(T ) Bu( ) d = z := xf eAT x0
0

We first show that

z R(Qc )
Since x0 R(Qc ),
n1
X
x0 = Ai B i
i=0

hence
n1
X
eAT x0 = eAT Ai B i
i=0

By application of Cayley Hamilton to functions of a matrix

n1
X
AT i
e A = ik Ak
k=0

so,
n1 X
X n1 n1
X
AT k i
e x0 = ik A B = Ak B k
i=0 k=0 k=0
P
where k := n1 i
i=0 ik . From this it should be clear that e
AT
x0 is in R(Qc ). Since xf is
also in R(Qc ) we have z R(Qc ).
Since z is in R(Qc ), it follows from the previous lemma that z is also in R(Wc (T )). Hence
there exists an n-vector y such that

z = Wc (T )y

Considering the control input

(T )
u( ) = B eA y

286
we obtain
Z T Z T
A(T ) (T )
e Bu( ) d = eA(T ) BB eA y d
0 0
Z T
(T )
= eA(T ) BB eA d y
0
Z T

= eAt BB eA t dt y
0
= Wc (T )y
= z

hence, this control input drives the state from x0 to xf .

Remark 8 Suppose (A, B) is controllable. Then rank Wc (T ) = rank Qc = n. Hence Wc (T )

is invertible and the control which drives x0 to xf over the interval [0, T ] is given by
(T t)
u(t) = B eA x x = Wc (T )1 (xf eAT x0 )

14.5 Uncontrollable modes

We shall see that if a system is uncontrollable, it has modes which cannot be affected by the
input. To define exactly what we mean by an uncontrollable mode, we need to look at the
left eigenvectors of the system A matrix.

14.5.1 Eigenvectors revisited: left eigenvectors

Suppose that is an eigenvalue of a square matrix A. Then an eigenvector corresponding
to is any nonzero vector satisfying Av = v. We say that a nonzero vector w is a left
eigenvector of A corresponding to if it satisfies

w0 A = w0 (14.5)

To see that left eigenvectors exist, note that the above equation is equivalent to A0 w = w,
that is, is an eigenvalue of A0 and w is an eigenvector of A0 corresponding to . We now
recall that is an eigenvalue of A if and only if is an eigenvalue of A0 . Thus, we can
say that is an eigenvalue of A if and only if it satisfies (14.5) for some non-zero vector
w. Moreover, we can also say that every eigenvalue of A has left eigenvectors and these
vectors are simply the eigenvectors of A0 corresponding to .

Consider now any two eigenvectors 1 and 2 of A with 1 6= 2 . Suppose that v1 is an

eigenvector for 1 and w2 is a left eigenvector for 2 . Then

w20 v1 = 0 .

287
To see this, we first use the eigenvector definitions to obtain
Av1 = 1 v1 and w20 A = 2 w20 .
Pre-multiplying the first equation by w20 we obtain that w20 Av1 = 1 w20 v1 . Post-multiplying
the section equation by v1 we obtain that w20 Av1 = 2 w20 v1 . This implies that 1 w20 v1 =
2 w20 v1 , that is (1 2 )w20 v1 = 0. Since 1 2 6= 0, we must have w20 v1 = 0.

System significance of left eigenvectors. Consider now a system described by

x = Ax . (14.6)
Recall that is an eigenvalue of A if and only if the above system has a solution of the form
x(t) = et v where v is a nonzero constant vector. When this occurs, the vector v can be any
eigenvector of A corresponding to and the above special solution is described as a mode
of the system. We now demonstrate the following result:
A scalar is an eigenvalue of A if and only if there is a nonzero vector w such that every
solution x() of system (14.6) satisfies

w0 x(t) = et w0 x(0) (14.7)

When this occurs, the vector w can be any left eigenvector of A corresponding to .
In particular, if x(0) = w/||w||2 then, w0 x(t) = et . If w0 x(0) = 0 then w0 x(t) = 0 for all
t.
To gain further insight into the above result, introduce the scalar variable (t) = w0 x(t).
Then (0) = w0 x(0) and (14.7) is equivalent to
(t) = et (0) ,
that is, the behavior of is governed by
= .
To demonstrate the above result, we first suppose that is an eigenvalue of A. Then there
is a nonzero vector w such that (14.5) holds; also w is a left eigenvector of A corresponding
to . Consider now any solution x() of system (14.6). Since it satisfies x = Ax, it follows
from (14.5) that
d(w0 x)
= w0 x = w0 Ax = (w0 x)
dt
It now follows that (14.7) holds for all t.
To demonstrate the converse, suppose that there is a nonzero vector w so that (14.7)
holds for every solution x(). This implies that
d(w0 x)
w0 Ax(t) = w0 x(t) = = et w0 x(0) = w0 x(t) ;
dt
hence,
w0 (A I)x(t) = 0 .
Since the above holds for any x(t), we must have w0 (A I) = 0. Thus, w0 A = w0 . This
implies that is an eigenvalue of A with left eigenvector w.

288
14.5.2 Uncontrollable eigenvalues and modes
Consider now a system with input u described by

x = Ax + Bu (14.8)

We say that is an uncontrollable eigenvalue of this system or is an uncontrollable eigen-

value of (A, B) if there is a nonzero vector w such that for every input u(), every solution
x() of the system satisfies
w0 x(t) = et w0 x(0) (14.9)

To gain further insight into the above concept, introduce the scalar variable (t) = w0 x(t).
Then, as we have shown above, (14.9) is equivalent to the statement that the behavior of
is governed by
= .
Thus the behavior of is completely unaffected by the input u. So, the mode et is completely
unaffected by the input u. We say that this is an uncontrollable mode.
When system (14.8) has an uncontrollable mode, then the system is not controllable. To
see this choose any initial state x(0) = x0 such that w0 x0 6= 0. Then, regardless of the control
input, we have w0 x(t) = et w0 x0 6= 0 for all t. Hence, regardless of the control input, x(t) 6= 0
for all t. This means that over any time interval [0, T ] we cannot obtain a control history
which drives the system from x0 to 0. Hence the system is not controllable. In Section
14.5.3, we demonstrate the following result.

If a system is not controllable then, it must have some uncontrollable modes.

14.5.3 Existence of uncontrollable modes*

In this section, we show that if a LTI system is not controllable, then it has uncontrollable
modes. Consider a linear time-invariant input-output system described

x = Ax + Bu
y = Cx + Du (14.10)

where x(t) is an n-vector, u(t) is an m-vector, and y(t) is a p-vector. Suppose (A, B) is
not controllable; then, its controllable subspace Xc is not equal to the whole state space.
In the following section, we show that there is a state transformation x = T x so that the
corresponding transformed system is described by

xc = Acc xc + Acu xu + Bc u
xu = Auu xu
y = Cc xc + Cu xu + Du (14.11)

with
xc
= x .
xu

289
 Figure 14.6: Controllable/uncontrollable decomposition

Also, the pair (Acc , Bc ) is controllable.

Clearly, the state xu of the subsystem

xu = Auu xu (14.12)

is completely unaffected by the input u. Also, if xu (0) = 0 then xu (t) = 0 for all t and the
input-output system in (14.11) can be described by the controllable subsystem:

xc = Acc xc + Bc u
y = Cc xc + Du (14.13)

Hence, when the initial state x(0) of the input-output system (14.10) is zero, its behavior
can be described by the lower order system (14.13).
Since the matrices A and
Acc Acu
A =
0 Auu
are similar, they have the same characteristic polynomial. In particular, the characteristic
polynomial of A is given by

det(sI A) = det(sI Acc ) det(sI Auu )

Hence the characteristic polynomial of A is the product of the characteristic polynomials of

Acc and Auu . Thus, the eigenvalues of A are the union of those of Acc and Auu . We now
make the following claim:
The eigenvalues of Auu are the uncontrollable eigenvalues of A.
Proof. To prove this claim, let be any eigenvalue of Auu . We will show that is an
uncontrollable eigenvalue of (A, B). Since is an eigenvalue of Auu there is a nonzero vector
wu such that for every solution xu () of (14.12), we have

wu0 xu (t) = et wu0 xu (0) .

Let w0 = [0 wu0 ]. Then every solution x of (14.11) satisfies

w0 x(t) = et w0 x(0)

With w = T w, we now obtain that every solution of (14.10) satisfies w0 x(t) = et w0 x(0) .
Thus, is an uncontrollable eigenvalue of A.

290
 We now prove the converse, that is, if is an uncontrollable eigenvalue of (A, B), then
must be an eigenvalue of Auu . So, suppose that is an uncontrollable eigenvalue of
(A, B). Then (A, B) is not controllable and there is a state transformation x = T [x0c x0u ]0
such that the system can be represented as in (14.11) with a nontrivial xu and with (Acc , Bc )
controllable. Suppose, on the contrary, that is not an eigenvalue of Auu . Then must be
an eigenvalue of Acc . Let w be such that (14.9) holds and let [wu0 wc0 ]0 = T 1 w. Then we
must have
wc0 xc (t) + wu0 xu (t) = et [wc0 xc (0) + wu0 xu (0)]
for every input u and every solution x() of (14.11). Considering those solutions with xu (0) =
0, we obtain that xu (t) = 0 for all t and

xc = Ac xc + Bc u . (14.14)

Thus wc0 xc (t) = et wc0 xc (0) for every input and every solution xc () of (14.14); hence is an
uncontrollable eigenvalue of (Ac , Bc ). This implies that (Ac , Bc ) is not controllable and we
have a contradiction. So, must be an eigenvalue of Auu

Transfer function considerations. Consider the reduced order controllable subsystem:

xc = Acc xc + Bc u
y = Cc xc + Du

One can readily show that the transfer function of this system is exactly the same as that
of the original system, that is

C(sI A)1 B + D = Cc (sI Ac )1 Bc + D .

An immediate consequence of this result is that a minimal state space realization of a transfer
function must be controllable.

14.5.4 A nice transformation*

We first show that the controllability subspace Xc is an invariant subspace for A, that is, is
x is in Xc , then Ax is in Xc .

Lemma 18 For any pair of matrices A IRnn and B IRnm , the controllable subspace

Xc = R B AB An1 B

is an invariant subspace for A.

Proof. Consider any element x of Xc . It can be expressed as

x = B1 + AB2 + + An1 Bn

Hence
Ax = AB1 + A2 B2 + + An Bn

291
Since
An = 0 I 1 A n1 An1
we obtain

Ax = B(0 n ) + AB(1 1 n ) + + An1 B(n1 n1 n )

hence Ax is contained in Xc . Since this holds for every element x of Xc , it follows that Xc is
A-invariant.

A nice transformation. Choose any basis

t1 , . . . , tnc

for the controllable subspace Xc of (A, B). Extend it to a basis

t1 , . . . , tnc , tnc +1 , . . . , tn

for IRn and let

T := t1 t2 . . . tn
One could do this by choosing T = U where U SV is a singular value decomposition of the
controllability matrix, that is

U SV = B AB An1 B

Introduce the state transformation

x = T x
Let
xc
= x
xu
where xc is a nc -vector and xu is a (nnc )-vector. Then

x is in Xc if and only if xu = 0

The transformed system description is given by

x = Ax + Bu
y = C x + Du

where
A = T 1 AT B = T 1 B C = CT
Since Xc is an invariant subspace of A, the matrix A has the following structure:

Acc Acu
A =
0 Auu

292
where Acc is nc nc . Since the columns of B are contained in the Xc , the matrix B has the
following structure
Bc
B =
0
where Bc is nc m. If we partition C as

C = Cc Cu

where Cc is p nc , then the transformed system is described by

xc = Acc xc + Acu xu + Bc u
xu = Auu xu
y = Cc xc + Cu xu + Du

The pair (Acc , Bc ) is controllable.

Example 186 Consider

x1 = x2 u
x2 = x1 + u
y = x2
Here,
0 1 1
A= B= C= 0 1 D=0
1 0 1
and the system transfer function is given by
1
G(s) =
s+1
Since the controllability matrix

1 1
Qc = B AB =
1 1
has rank less than two, this system is not controllable. The controllable subspace is spanned
by the vector
1
=B
1
Introducing the state transformation

xc 1 1
x = T x x = T =
xu 1 1
we obtain an equivalent system description with

1 0 1
A = B = C = 1 1 D = 0
0 1 0

293
that is

xc = xc + u
xu = xu
y = xc + xu

The transfer function of the reduced order controllable system

xc = xc + u
y = xc

is clearly the same as that of the original system.

294
14.6 PBH test
14.6.1 PBH test for uncontrollable eigenvalues
The next lemma provides a useful characterization of uncontrollable eigenvalues.

Lemma 19 A complex number is an uncontrollable eigenvalue of the pair (A, B) if and

only if
rank (A I B) < n
where n is the number of rows in A.

Proof. When the above rank condition holds, there exists a non-zero n-vector w such that

w0 (A I B) = 0 ,

that is
w0 A = w0 and w0 B = 0 .
So, if x() is any solution of the system x = Ax + Bu corresponding to any input u(), we
obtain that
d(w0 x)
= w0 Ax + w0 Bu = (w0 x) .
dt
Hence,
w0 x(t) = et w0 x(0) . (14.15)
So is an uncontrollable eigenvalue.
To prove the converse, suppose now that is an uncontrollable eigenvalue of (A, B).
Then, (14.15) holds for any solution x() of the system x = Ax + Bu corresponding to any
input u(). Since (14.15) implies that

w0 x(t) = et w0 x(0) = w0 x(t), ,

it follows that
w0 Ax(t) + w0 Bu(t) = w0 x(t) ;
hence
0
x(t)
w A I B =0
u(t)
Since x(t) and u(t) can be arbitrary chosen, we must have

w0 (A I B) = 0 ,

Since w is non zero, it follows that the rank of (A I B) is less than n.

Alternate Proof*. We first show that when the above rank condition holds, the pair
(A, B) is not controllable. When this rank condition holds, there exists a non-zero n vector
x such that
x (A I B) = 0

295
hence,
x A = x , x B = 0
It now follows that
x AB = x B = 0
and, by induction, one can readily prove that
x A k B = 0 for k = 0, , n 1
Hence
x Q c = 0
where Qc is the controllability matrix associated with (A, B). This last condition implies
that Qc has rank less than n, hence, (A, B) is not controllable.
So, if either (A, B) has an uncontrollable eigenvalue or the above rank condition holds
then, (A, B) is not controllable and there exists a nonsingular matrix T such that the matrices
A = T 1 AT , B = T 1 B have the following structure;

Acc Acu Bc
A = B =
0 Auu 0
with (Acc , Bc ) controllable. Noting that

T 0
A I B = T 1 (A I B)
0 I
it follows that
rank A I B = rank (A I B)
Since (Acc , Bc ) is controllable,

rank Acc I Bc = nc
where nc is the number of rows of Ac ; hence

Acc I Acu Bc
rank A I B = rank
0 Auu I 0

Acc I Bc Acu
= rank
0 0 Auu I

Acc I Bc 0
= rank
0 0 Auu I

= nc + rank [Auu I]
It now follows that
rank (A I B) < n
if and only if
rank [Auu I] < nu
where nu = n nc is the number of rows of Auu . This last condition is equivalent to being
an uncontrollable eigenvalue of (A, B).

296
14.6.2 PBH controllability test
Since a system is controllable if and only if it has no uncontrollable eigenvalues, the PBH
test for uncontrollable eigenvalues gives us the following test for controllability.

Corollary 3 (PBH controllability test) The pair (A, B) is controllable if and only if
rank (A I B) = n for all C

Example 187 Two unattached masses. There,

0 0
(A I B) =
0 1
The above rank test fails for = 0.

Example 188 Recall example 186. There

1 1
(A I B) =
1 1
The above rank test fails for = 1
Exercise 123 If is any complex number, prove the following result. The pair (A + I, B)
is controllable if and only if (A, B) is controllable.

Example 189 (Beavis and Butthead: self-controlled) Recall the system of Example
181.

Figure 14.7: B&B: self-controlled

Here
0 0 1 0 0
0 0 0 1 0
A=
k/m
B=
k/m 0 0 1/m
k/m k/m 0 0 1/m
So, the PBH controllability matrix for the zero eigenvalue is

0 0 1 0 0
0 0 0 1 0
(A B) = k/m

k/m 0 0 1/m
k/m k/m 0 0 1/m
Since the last two rows of this matrix are not linearly independent, this matrix does not
have the maximum rank of 4; hence zero is an uncontrollable eigenvalue. This eigenvalue
corresponds to the rigid body mode of this system.

297
14.7 Other characterizations of controllability*
14.7.1 Infinite time controllability grammian
Suppose A is asymptotically stable, that is all its eigenvalues have negative real parts, and
consider the linear matrix equation

AWc + Wc A + BB = 0

This is a Lyapunov equation of the form AS + SA + Q = 0 with Q = BB . Since A is

asymptotically stable the above equation has a unique solution for Wc and this solution is
given by Z

Wc = eAt BB eA t dt .
0

We call this matrix the (infinite-time) controllability grammian associated with (A, B). For
asymptotically stable systems, this matrix provides another controllability test.

Lemma 20 Suppose all the eigenvalues of A have negative real parts. Then (A, B) is con-
trollable if and only if Wc > 0

Proof. When A is asymptotically stable, one can readily generalize Lemma 17 to include
the case T = ; hence
R(Wc ) = R(Qc )
Since Wc > 0 if and only if it has rank n, the lemma now follows from the main controllability
theorem.

MATLAB

>> help gram

GRAM Controllability and observability gramians.
GRAM(A,B) returns the controllability gramian:

Gc = integral {exp(tA)BBexp(tA)} dt

GRAM(A,C) returns the observability gramian:

Go = integral {exp(tA)CCexp(tA)} dt

See also DGRAM, CTRB and OBSV.

298
14.8 Controllability, observability, and duality
Recall that for any matrix M , we have rank M = rank M . It now follows from the main
controllability theorem that a pair (A, B) is controllable if and only if the rank of Qc is n,
that is,
B
B A

rank .. =n
.
(n1)
B A
Note that the above matrix is the observability matrix associated with the pair (B , A ).
Recalling the main observability theorem, we obtain the following result:

(A, B) is controllable if and only if (B , A ) is observable.

The above statement is equivalent to

(C, A) is observable if and only if (A , C ) is controllable.

This is what is meant by the duality between controllability and observability.

299
14.9 Discrete-time
With one exception, all the results we have developed for continuous time also hold in an
analogous fashion for discrete time. In discrete-time we need to consider time intervals [0, N ]
where N is bigger then n, the number state variables. Consider the system described by

x(k + 1) = Ax(k) + Bu(k) (14.16)

where k ZZ is time, x(k) IRn is the state, and u(k) IRm is the control input.
Consider any fixed time N > 0. We say that this system is controllable if it can be
driven from any state to any other state over the time interval [0, N ] by appropriate choice
of control input.

DEFN. System (14.16) is controllable over the interval [0, N ] if for every pair of states
x0 , xf IRn , there is a control function u() : {0, 1 . . . , N } IRm such that the solution x()
of
x(k + 1) = Ax(k) + Bu(k) x(0) = x0 (14.17)
satisfies x(N ) = xf .

14.9.1 Main controllability theorem

Theorem 22 (Main controllability theorem.) For each N n , system (14.16) is control-
lable over [0, N ] if and only if

rank B AB . . . An1 B =n

300
 Part IV

Control Design

301
Chapter 15

Stabilizability and state feedback

In this chapter we look at the problem of changing the dynamics of a system using state
feedback. In particular, we look at the problem of stabilization via state feedback.

15.1 Stabilizability and state feedback

Consider a linear time-invariant system with control input described by

x = Ax + Bu . (15.1)

For now, we consider linear static state feedback controllers, that is, at each instant t of time
the current control input u(t) depends linearly on the current state x(t), that is, we consider
the control input to be given by
u(t) = Kx(t) (15.2)
where K is a constant m n matrix, sometimes called a state feedback gain matrix. When
system (15.1) is subject to such a controller, its behavior is governed by

x = (A + BK)x (15.3)

We call this the closed loop system resulting from controller (15.2). The following result is

Figure 15.1: State feedback

easily shown using the PBH test for controllability.

Fact 9 The pair (A + BK, B) is controllable if and only if (A, B) is controllable.

303
 Suppose the open loop system, that is, the system with zero feedback, or, x = Ax is
unstable or at most marginally stable. A natural question is the following. Can we choose
the gain matrix K so that A + BK is asymptotically stable? If yes, we say that system
(15.1) is stabilizable.

DEFN.[Stabilizability] System (15.1) is stabilizable if there exists a matrix K such that A+BK
is asymptotically stable.

We sometimes say that the pair (A, B) is stabilizable if system (15.1) is stabilizable.
The big question is: under what conditions is a given pair (A, B) stabilizable? We shall
see in Section 15.4.2 that controllability is a sufficient condition for stabilizability; that is,
if a system is controllable it is stabilizable. However, controllability is not necessary for
stabilizability, that is, it is possible for a system to be stabilizable but not controllable. To
see this, consider a system with A asymptotically stable and B equal to zero; see also the
next two examples.

Example 190 (Not stabilizable and not controllable)

x1 = x1
x2 = u
For any gain matrix K, the matrix

1 0
A + BK =
k1 k2
has eigenvalues 1 and k2 . Hence, A + BK is unstable for all K. So, this system is not
stabilizable.

Example 191 (Stabilizable but not controllable.)

x1 = x1
x2 = u
For any gain matrix K, the matrix

1 0
A + BK =
k1 k2
has eigenvalues 1 and k2 . Hence, A + BK is asymptotically stable provided k2 < 0. So,
this system is stabilizable. However, it is not controllable.

15.2 Eigenvalue placement (pole placement) by state

feedback
Recall the stabilizability problem for the system
x = Ax + Bu

304
and suppose A and B are real matrices. Here we consider a more general question: Can
we arbitrarily assign the eigenvalues of A + BK by choice of the gain matrix K? The main
result of this section is that the answer is yes if and only if the pair (A, B) is controllable.
We consider first single input systems.

15.2.1 Single input systems

Here we consider scalar input (m = 1) systems. We first demonstrate that the characteristic
polynomial of A + BK depends in an affine linear fashion on the gain matrix K. To achieve
this, we need to use the following result whose proof is given in the appendix at the end of
this chapter.

Fact 10 Suppose M and N are any two matrices of dimensions nm and mn, respectively.
Then,
det(I + M N ) = det(I + N M ) .
In particular, if m = 1, we have

det(I + M N ) = 1 + N M .

Whenever s is not an eigenvalue of A, we have sI ABK = (sI A)(I (sI A)1 BK).
Thus, using Fact 10, we obtain that

det(sI A BK) = det(sI A) det(I (sI A)1 BK)

= det(sI A) det(I K(sI A)1 B)
= det(sI A)(1 K(sI A)1 B) ,

that is, the characteristic polynomial of A + BK satisfies

det(sI A BK) = det(sI A)(1 K(sI A)1 B) . (15.4)

From this expression, we can make the following statement:

For a single input system, the characteristic polynomial of A + BK depends in an affine

linear fashion on the gain matrix K.

We now show that we can arbitrarily assign the eigenvalues of A + BK by choice of the
gain matrix K if (A, B) is controllable. Consider any bunch of complex numbers 1 , . . . , n
with the property that if is in the bunch then so is and suppose we wish to choose a gain
matrix K such that the eigenvalues of A + BK are precisely these complex numbers. Let
n
Y
=
d(s) (s i )
i=1

that is, d is the unique monic polynomial whose roots are the desired closed loop eigenvalues,

1 , . . . , n . We need to show that there is a gain matrix K so that det(sI A BK) = d(s).

305
Let d be the characteristic polynomial of A, that is, d(s) = det(sI A). Then, recalling
(15.4) we need to show that there exists a gain matrix K such that
= d(s)(1 K(sI A)1 B)
d(s)
that is,
.
d(s)K(sI A)1 B = d(s) d(s) (15.5)
Let
d(s) = a0 + a1 s + + an1 sn1 + sn

d(s) = a0 + a1 s + + an1 sn1 + sn .
Recall the power series expansion for (sI A)1 :
1 1 1
(sI A)1 = I + 2 A + + k Ak1 +
s s s
Substituting this expression into (15.5) and equating the coefficients of like powers of s we
obtain that (15.5) holds if and only if
s0 : a1 KB + a2 KAB + + an1 KAn2 B + KAn1 B = a0 a0
s1 : a2 KB + + an1 KAn3 B + KAn2 B = a1 a1
..
.
sn2 an1 KB + KAB = an2 an2
sn1 : KB = an1 an1
that is,

KB KAB KAn1 B = a0 a0 a1 a1 an1 an1 (15.6)
where is the invertible matrix given by

a1 a2 an2 an1 1
a2 a3 an1 1 0

a3 a4 1 0 0

= .. .. .. .. ... (15.7)
. . . . .

an1 1 0 0 0
1 0 ... 0 0 0
Let Qc be the controllability matrix associated with (A, B), that is,

Qc = B AB An1 B .
Then condition (15.6) on K reduces to

KQc = a0 a0 a1 a1 an1 an1 .
Because the pair (A, B) is controllable, Qc is invertible. This readily implies that the gain
matrix K is uniquely given by

K = a0 a0 a1 a1 an1 an1 1 Q1 c . (15.8)
We have just demonstrated the following result.

306
Theorem 23 (Pole placement theorem: SI case) Suppose the real matrix pair (A, B)
is a single input controllable pair with state dimension n and 1 , . . . , n is any bunch of n
complex numbers with the property that if is in the bunch then so is . Then there exists a
real matrix K such that 1 , . . . , n are the eigenvalues (multiplicities included) of A + BK.

MATLAB

>> help acker

ACKER Pole placement gain selection using Ackermanns formula.

K = ACKER(A,B,P) calculates the feedback gain matrix K such that
the single input system
.
x = Ax + Bu

with a feedback law of u = -Kx has closed loop poles at the

values specified in vector P, i.e., P = eig(A-B*K).

See also PLACE.

Note: This algorithm uses Ackermanns formula. This method

is NOT numerically reliable and starts to break down rapidly
for problems of order greater than 10, or for weakly controllable
systems. A warning message is printed if the nonzero closed loop
poles are greater than 10% from the desired locations specified
in P.

Example 192 Consider

3 2 2
A= and B=
4 3 3

Here the controllability matrix

2 0
Qc = B AB =
3 1

is
full rank;
hence (A, B) is controllable. Considering any state feedback gain matrix K =
k1 k2 , we have

3 + 2k1 2 + 2k2
A + BK =
4 + 3k1 3 + 3k2
and
det(sI A BK) = s2 (2k1 + 3k2 )s 1 + k2 .

307
Suppose we desire the eigenvalues of A + BK to be 2 and 3. Then the desired charac-
teristic polynomial d of A + BK is given by
= (s + 2)(s + 3) = s2 + 5s + 6 .
d(s)

By equating the coefficients of like powers of s, we obtain

1 + k2 = 6
2k1 3k2 = 5

Solving these equations yields (uniquely) k1 = 13 and k2 = 7. Hence the gain matrix is
given by
K = 13 7 .
Checking our answer in Matlab, we obtain

K = -acker(A,B,[-2 -3])
K =
-13 7

Example 193 (Beavis and Butthead with external control) Considering m = k =

Figure 15.2: B&B: external control

1, a state space description of this system is given by

0 0 1 0 0
0 0 0 1 0
A= 1

and B=
0 .
1 0 0
1 1 0 0 1

Here,
det(sI A) = s4 + 2s2
Let
= s4 + a3 s3 + a2 s2 + a1 s + a0
d(s)
be the desired characteristic polynomial of A + BK.
Since this is a scalar input system, the coefficients of the characteristic polynomial of
A + BK depend affinely on K and we can use the following solution approach to find K.
Let
K = k1 k2 k3 k4

308
Then

s 0 1 0
0 s 0 1
det(sI A BK) = det


1 1 s 0
1 k1 1 k2 k3 s k4

= s4 k4 s3 + (2 k2 )s2 (k3 + k4 )s (k1 + k2 ) .

Note the affine dependence of the coefficients of the above polynomial on K. Letting the
above polynomial equal the desired polynomial d and equating like coefficients results in the
following four linear equations in the four unknowns components of K:

k4 = a3
2 k2 = a2
k3 k4 = a1
k1 k2 = a0

Solving for k1 , . . . , k4 uniquely yields

k1 = 2 a0 + a2
k2 = 2 a2
k3 = a1 + a3
k4 = a3

To illustrate MATLAB, consider desired closed loop eigenvalues: 1, 2 3, 4.

>> poles=[-1 -2 -3 -4];

>> poly(poles)
ans =
1 10 35 50 24

Hence
a0 = 24 a1 = 50 a2 = 35 a3 = 10
and the above expressions for the gain matrix yield

k1 = 9 k2 = 33 k3 = 40 k4 = 10

Using MATLAB pole placement commands, we obtain:

>> k= place(a,b,poles)
place: ndigits= 18

k =

309
 -9.0000 33.0000 40.0000 10.0000

>> k= acker(a,b,poles)

k =
-9 33 40 10

Remembering that MATLAB yields K, these results agree with our hand calculated
results. Lets check to see if we get the desired closed loop eigenvalues.

>> eig(a-b*k)

ans =
-1.0000
-2.0000
-3.0000
-4.0000

YEES!

15.2.2 Multi-input systems

The following theorem (which we do not prove here) states that the eigenvalue placement
result also holds in the general multi-input case.

Theorem 24 (Pole placement theorem ) Suppose the real matrix pair (A, B) is a con-
trollable pair with state dimension n and 1 , . . . , n is any bunch of n complex numbers with
the property that if is in the bunch then so is . Then there exists a real matrix K such
that 1 , . . . , n are the eigenvalues (multiplicities included) of A + BK.

It follows from the above theorem that

controllability = stabilizability

MATLAB

>> help place

PLACE K = place(A,B,P) computes the state feedback matrix K such that

the eigenvalues of A-B*K are those specified in vector P.
The complex eigenvalues in the vector P must appear in consecu-
tive complex conjugate pairs. No eigenvalue may be placed with
multiplicity greater than the number of inputs.

310
The displayed "ndigits" is an estimate of how well the
eigenvalues were placed. The value seems to give an estimate
of how many decimal digits in the eigenvalues of A-B*K match
the specified numbers given in the array P.

A warning message is printed if the nonzero closed loop poles

are greater than 10% from the desired locations specified in P.

See also: LQR and RLOCUS.

311
15.3 Uncontrollable eigenvalues (you cant touch them)
In the previous section, we saw that, if the pair (A, B) is controllable, then one could
arbitrarily place the eigenvalues of A + BK by appropriate choice of the gain matrix K;
hence controllability is a sufficient condition for arbitrary eigenvalue placement. In this
section, we show that controllability is also necessary for arbitrary eigenvalue placement.
In particular, we show that if is an uncontrollable eigenvalue of (A, B), then is an
eigenvalue of A + BK for every K. Recall Examples 190 and 191 in which the matrix A
had an eigenvalue with the property that for every gain matrix K, it is an eigenvalue of
A + BK. We now demonstrate the following result:

If is an uncontrollable eigenvalue of (A, B), then is an eigenvalue of A + BK for every

gain matrix K.

To see this, recall that is an uncontrollable eigenvalue of (A, B) if and only if

rank A I B < n (15.9)

where A is n n. Hence, there exists a nonzero n-vector w such that

w0 A I B = 0 .

that is
w0 (A I) = 0 and w0 B = 0
So, for any gain matrix K, we have

w0 (A + BK) = w0

that is, is an eigenvalue of A + BK with left eigenvector w. So, regardless of the feed-
back gain matrix, is always an eigenvalue of A + BK. We cannot alter this eigenvalue
by feedback. In particular, if has a non-negative real part, then the pair (A, B) is not
stabilizable.

Another way to see the above result is to consider the nice transformation introduced
earlier, namely,
xc
x = T x and x =
xu
which results in the equivalent system description,

x = Ax + Bu

where
1 Acc Acu 1 Bc
A = T AT = and B = T B=
0 Auu 0

312
and the
pair (Acc , Bc ) is controllable. If u = Kx, then u = K x where K = KT . Letting
K = Kc Ku , we obtain

Acc + Bc Kc Acu + Bc Ku
A = B K =
0 Auu

Since
A + B K = T 1 [A + BK]T
the eigenvalues of A + BK are exactly the same as those of A + B K which are equal to the
union of the eigenvalues of Acc + Bc Kc and Auu . From this we see that, regardless of K, the
eigenvalues of Auu are always eigenvalues of A + BK. Now recall that the eigenvalues of Auu
are the uncontrollable eigenvalues of (A, B).
Furthermore, since the pair (Acc , Bc ) is controllable, the eigenvalues of Acc + Bc Kc can
be arbitrarily placed by appropriate choice of Kc . Hence, except for the eigenvalues of Auu ,
all eigenvalues of A + BK can be arbitrarily placed. In particular, if all the eigenvalues of
Auu have negative real parts, then K can be chosen so that all the eigenvalues of A + BK
have negative real parts. So, we have the following conclusion.

A pair (A, B) is stabilizable if and only if all the uncontrollable eigenvalues of (A, B) have
negative real parts.

Recalling the PBH test for uncontrollable eigenvalues, we now have the following PBH
result on stabilizability.

Theorem 25 (PBH stabilizability theorem) A pair (A, B) is stabilizable if and only

if
rank (A I B) = n
for every eigenvalue of A with nonnegative real part.

Example 194
A =
0 -1 0
1 -1 -1
-1 0 0

B =
1
-1
1

rank(ctrb(A,B))
2 %Uncontrollable

eig(A)

313
 0 + 1.0000i
0 - 1.0000i
-1.0000

rank([ A-i*eye(3) B])

3 %Stabilizable

rank([ A+eye(3) B])

2 %-1 is an uncontrollable eigenvalue

Considering K = k1 k2 k3 we obtain that

det(sI A BK) = s3 + (1 k1 + k2 k3 )s2 + (1 2k1 )s + (1 k1 k2 + k3 ) .

As expected, we can write this as

det(sI A BK) = (s + 1) s2 + (k1 + k2 k3 )s + (1 k1 k2 + k3 )

Suppose that the desired eigenvalues of A + BK are 1 2, 3. Then

= (s + 1)(s + 2)(s + 3) = s3 + 6s2 + 11s + 6 .

d(s)

Equating coefficients, we obtain

1 k1 k2 + k3 = 6
1 2k1 = 11
1 k1 + k2 k3 = 6

Solving yields

k1 = 5
k2 = k3
k3 is arbitrary

We choose

K= 5 0 0 .

eig(A+B*K)
-3.0000
-2.0000 %Just checking!
-1.0000

314
Remark 9 If system (15.1) is not stabilizable via a linear static state controller of the form
u = Kx then, it is not stabilizable with any controller. This can be seen as follows. The
fact that the system is not stabilizable via a linear controller implies that the system has an
uncontrollable eigenvalue with non-negative real part. Hence there exists a nonzero vector
w such that, regardless of the input u, we have

w0 x(t) = et w0 x(0)

for all t. Considering any initial state x(0) for which w0 x(0) 6= 0 (for example, let x(0) be
the real part of w), we see that, regardless of the input u, the resulting solution x(t) does
not go to zero as t goes to infinity.

15.4 Controllable canonical form

Consider a scalar input system of the form x = Ax + Bu where A and B have the following
structure:

0 1 0 ... 0 0 0
0 0 1 ... 0 0 0
. .. .. ..
. ..
. . . . .
A= . .. .. .. and B= .. .
.. . . . .

0 0 0 ... 0 1 0
a0 a1 a2 . . . an2 an1 1

We say that the pair (A, B) is in controllable canonical form. We saw this structure when we
constructed realizations for scalar transfer functions. Note that x = Ax + Bu looks like:

x1 = x2
..
.
xn1 = xn
xn = a0 x1 + a1 x2 + . . . + an1 xn + u

Also,
det(sI A) = a0 + a1 s + + an1 sn1 + sn

Example 195 (Controlled unattached mass)

x1 = x2
x2 = u

Here,
0 1 0
A= and B= .
0 0 1

315
Fact 11 If the pair (A, B) is in controllable canonical form, then it is controllable.

To prove the above result, let K = a0 a1 an1 . Then

0 1 0 ... 0 0
0 0 1 ... 0 0
.. .. ..
...
. . .
A + BK = .. .. . . . ..
. . .

0 0 0 ... 0 1
0 0 0 ... 0 0

The controllability matrix Qc for the pair (A + BK, B) is simply given by

0 0 0 1
0 0 1 0

.... . .
Qc = . . . 0 0

0 1 0 0
1 0 0 0

This matrix has rank n; hence (A + BK, B) is controllable. It now follows that (A, B) is
controllable.

15.4.1 Transformation to controllable canonical form

Here we show that any controllable single-input system

x = Ax + Bu

can be transformed via a state transformation to a system which is in controllable canonical

form.

State transformations. Suppose T is a nonsingular matrix and consider the state trans-
formation
x = T x
Then the evolution of x is governed by

x = Ax + Bu

where A = T 1 AT and B = T 1 B.
Note that controllability is invariant under a state transformation, that is, (A, B) is
controllable if and only if (T 1 AT, T 1 B) is controllable. This can readily seen by noting
the controllability matrix Qc for the pair (T 1 AT, T 1 B) is given by Qc = T 1 Qc where Qc
is the controllability matrix for (A, B); hence Qc and Qc have the same rank.

316
Transformation to controllable canonical form. Suppose (A, B) is controllable and
m = 1. Then the following algorithm yields a state transformation which transforms (A, B)
into controllable canonical form.

Algorithm. Let
a0 + . . . + an1 sn1 + sn = det(sI A)
Recursively define the following sequence of n-vectors:

tn = B
tj = Atj+1 + aj B , j = n 1, . . . , 1

and let
T = t1 . . . tn
Fact 12 If (A, B) is controllable then, the matrix T is generated by the above algorithm is
nonsingular and the pair (T 1 AT, T 1 B) is in controllable canonical form.
Proof. We first need to prove that T is nonsingular. By applying the algorithm, we obtain
tn = B
tn1 = AB + an1 B
tn2 = A(AB + an1 B) + an2 B = A2 B + an1 AB + an2 B
One may readily show by induction that for j = n 1, n 2 , 1, one has
tj = Anj B + an1 Anj1 B + + aj+1 AB + aj B (15.10)
From this it should be clear that for any j = n, n 1, 1,

rank tn tn1 tj = rank B AB Anj B
Hence

rank T = rank tn tn1 t1 = rank B AB An B = rank Qc
Since (A, B) is controllable, Qc has rank n. Hence T has rank n and is invertible.
We now show that T 1 AT and T 1 B have the required structure. It follows from (15.10)
that
t1 = An1 B + an1 An2 B + + a2 AB + a1 B
Using the Cayley Hamilton Theorem, we obtain
At1 = [An + an1 An1 + + a2 A2 + a1 A]B = a0 B ,
that is At1 = a0 tn . Also from the algorithm, we have
Atj = tj1 aj1 tn for j = 2, 3, , n
Recalling a useful fact on similarity transformations, it follows that T 1 AT has the required
structure. Since B = tn , it follows that T 1 B also has the required structure.

We can now state the following result.

317
Theorem 26 A single input system (A, B) is controllable if and only if there is a nonsingular
matrix T such that (T 1 AT, T 1 B) is in controllable canonical form.

Example 196 (Beavis and Butthead with external control.) Consider m = k = 1. Then,

0 0 1 0 0
0 0 0 1 0
A= 1
and B=
0 .
1 0 0
1 1 0 0 1

Hence,
det(sI A) = s4 + 2s2
and
t4 = B
0
1
t3 = At4 + a3 B = At4 =
0
0

0
0
t2 = At3 + a2 B = At3 + 2B =
1
1

1
1
t1 = At2 + a1 B = At2 =
0
0
So, the transformation matrix is given by:

1 0 0 0
1 0 1 0
T = t1 t2 t3 t4 =
0

1 0 0
0 1 0 1

>> inv(t)
ans =
1 0 0 0
0 0 1 0
-1 1 0 0
0 0 -1 1

>> inv(t)*a*t
ans =
0 1 0 0
0 0 1 0

318
 0 0 0 1
0 0 -2 0

15.4.2 Eigenvalue placement by state feedback

Recall that if (A, B) is controllable then, we arbitrarily assign the eigenvalues of A + BK
by choice of the gain matrix K? We can demonstrate this result for single input systems
using the controllable canonical form. Consider any bunch of complex numbers 1 , . . . , n
with the property that if is in the bunch then so is and suppose we wish to choose a
gain matrix K such that the eigenvalues of A + BK are precisely these complex numbers.
Let
n
Y
n
d(s) = s + an1 sn1
+ . . . + a0 = (s i )
i=1

that is, the real numbers, a0 , . . . , an1 , are the coefficients of the unique monic polynomial d
whose roots are the desired closed loop eigenvalues, 1 , . . . , n . If (A, B) is controllable and
m = 1, then there exists a nonsingular matrix T such that the pair (A, B) is in controllable
canonical form, where A = T 1 AT and B = T 1 B, that is,

0 1 0 ... 0 0 0
0 0 1 ... 0 0 0
.. .. .. ..
...
. . . .
A = .. .. .. .. , B = ..
. . . . .

0 0 0 ... 0 1 0
a0 a1 a2 . . . an2 an1 1

where
sn + an1 sn1 + . . . + a0 = det(sI A) .
Considering the 1 n real gain matrix

K = a0 a0 a1 a1 . . . an1 an1

we obtain
0 1 0 ... 0 0
0 0 1 ... 0 0
.. .. ..
...
. . .
A + B K = .. .. .. .. .
. . . .

0 0 0 ... 0 1
a0 a1 a2 . . . an2 an1
Let
Clearly, the matrix A + B K has the desired characteristic polynomial d.

K = KT 1 .

319
Then

A + BK = T AT 1 + T B KT 1
= T [A + B K]T 1

and hence

det(sI A BK) = det(sI A B K) = d(s)
in other words, the eigenvalues of A + BK are as desired.

Remark 10 For scalar input systems, the coefficients of the characteristic polynomial of
A + BK depend affinely on K. Since K = KT 1 , it follows that the coefficients of the
characteristic polynomial of A + BK depend affinely on K.

Example 197 (Beavis and Butthead with external control) Here

0 0 1 0 0
0 0 0 1 0
A= 1
and B = .
1 0 0 0
1 1 0 0 1

We have already shown that this system is controllable and it can be transformed to con-
trollable canonical form with the matrix

1 0 0 0
1 0 1 0
T = 0 1 0 0

0 1 0 1

Also,
det(sI A) = s4 + 2s2
Let
p(s) = s4 + a3 s3 + a2 s2 + a1 s + a0
be the desired characteristic polynomial of A + BK. Then

K = a0 a1 2a2 a3

and

1 0 0 0
0 0 1 0
K = = KT 1 = a0 a1 2a2 a3 1

1 0 0
0 0 1 1

= 2a0 +a2 2a2 a1 + a3 a3

This is the same as the result we previously obtained for this example.

320
15.5 Discrete-time systems
Basically, everything we have said in this chapter so far also holds for discrete time systems,
except that in talking about asymptotic stability of discrete-time systems, we need the
eigenvalues to have magnitude strictly less than one.

15.5.1 Dead beat controllers

One cannot drive the state of a LTI continuous-time system to zero in a finite time using a
static linear state-feedback controller. However, this is possible for a controllable discrete-
time system. To see this, consider

x(k+1) = Ax(k) + Bu(k)

where x(k) is an n-vector and u(k) is an m-vector and suppose that the pair (A, B) is con-
trollable. Then there exists a gain matrix K so that all the eigenvalues of the corresponding
closed loop system matrix Acl := A + BK are all zero. Hence the characteristic polynomial
of Acl is p(s) = sn . From the Cayley-Hamilton Theorem, we obtain that

Ancl = 0 .

Since the solutions x() of the closed lop system

x(k+1) = Acl x(k)

satisfy x(k) = Akcl x(0) for all k 0, we see that

x(k) = 0 for k n.

Thus, all solutions go to zero in at most n steps.

Application to continuous-time systems. One could use the above result to obtain a
controller which drives the state of a continuous-time system to zero. Simply, pick a sampling
time T and discretize the continuous-time system. Now design a deadbeat controller for the
discrete-time system. Using a zero order hold, apply this controller to the continuous-time
system. Note that the resulting controller is not a static controller for the continuous-time
system.

321
15.6 Stabilization of nonlinear systems
Consider a nonlinear system described by

x = F (x, u)

and suppose we wish to stabilize this system about some controlled equilibrium state xe .
Let ue be a constant input that achieves the desired controlled equilibrium state xe , that is,
F (xe , ue ) = 0. Let
x = Ax + Bu (15.11)
be the linearization of the nonlinear system about (xe , ue ). Thus,

F e e F e e
A= (x , u ) and B= (x , u ) .
x u

15.6.1 Continuous-time controllers

If the pair (A, B) is stabilizable, the following procedure yields a controller which stabilizes
the nonlinear system about xe .
Choose a gain matrix K such that all eigenvalues of A + BK have negative real part and
let
u = ue + K(x xe ) (15.12)

We now show that the above controller results in the closed loop nonlinear system being
asymptotically about xe . To this end, note that the closed loop system is described by

x = f (x) = F (x, Kx + ue Kxe )

Note that
f (xe ) = F (xe , ue ) = 0 ;
hence xe is an equilibrium state for the closed loop system. Linearizing the closed loop
system about xe , we obtain
x = (A + BK)x
Since all the eigenvalues of A+BK have negative real parts, the closed loop nonlinear system
is asymptotically stable about xe .

15.6.2 Discrete-time controllers

Linearize the continuous time nonlinear system to obtain the LTI continuous-time system
(15.11).
Choose a sample time T > 0 and discretize (15.11) to obtain

xd (k+1) = Ad xd (k) + Bd ud (k) (15.13)

322
If the pair (Ad , Bd ) is stabilizable, choose matrix K so that all the eigenvalues of Ad + Bd K
have magnitude less than one.
Let
ud (k) = K(xd (k) xe ) + ue = K[x(kT ) xe ] + ue . (15.14)
The continuous control input u is obtained form ud via a zero order hold.

Exercises
Exercise 124 Consider the system with input u, output y, and state variables x1 , x2 , x3 , x4
described by

x1 = x1
x2 = x1 2x2
x3 = x1 3x3 + u
x4 = x1 + x2 + x3 + x4
y = x3

(a) Is this system controllable?

(b) Is this system stabilizable using state feedback? Justify your answers.

Exercise 125 (BB in laundromat: external excitation.) Obtain a state space representation
of the following system.

m1 m2 1 + k2 (1 2 ) = 0

m2 m2 2 k2 (1 2 ) = u

Show that this system is controllable.

Considering
= 1, m = 1, k = 2
obtain a linear state feedback controller which results in a closed loop system with eigenvalues

1, 2, 3, 4

Use the following methods.

(a) Method based on computing the transformation matrix which transforms to control-
lable canonical form.

(b) Obtaining an expression for the closed loop characteristic polynomial in terms of the
components of the gain matrix.

(c) acker

(d) place

323
Numerically simulate the closed loop system with the same initial conditions you used in to
simulate the open-loop system.

Exercise 126 Consider the system described by

x1 = x31 + sin x2 1 + u
x2 = ex1 + (cos x2 )u

Obtain a state feedback controller which results in a closed loop system which is asymptoti-
cally stable about the zero state.

Exercise 127 Stabilization of 2-link manipulator with linear state feedback.

[m1 lc21 + m2 l12 + I1 ]q1 + [m2 l1 lc2 cos(q1 q2 )]q2 + m2 l1 lc2 sin(q1 q2 )q22
[m1 lc1 + m2 l1 ]g sin(q1 ) = u

[m2 l1 lc2 cos(q1 q2 )]q1 + [m2 lc22 + I2 ]q2 m2 l1 lc2 sin(q1 q2 )q12 m2 glc2 sin(q2 ) = 0

Linearize this system about

q1 = q2 = u = 0
and obtain a state space description.
For rest of exercise, use MATLAB and the following data: Is the linearization control-

m1 l1 lc1 I1 m2 l2 lc2 I2 mpayload

kg m m kg.m2 kg m m kg.m2 kg
10 1 0.5 10/12 5 1 0.5 5/12 0

lable? Design a state feedback controller which stabilizes this linearized system . Apply the
controller to the nonlinear system and simulate the nonlinear closed loop system for various
initial conditions.

Exercise 128 Consider the system described by

x1 = x2 + u
x2 = x1 + u

(a) Is this system stabilizable ?

(b) Does there exist a linear state feedback controller which results in closed loop eigen-
values 1, 2?

(b) Does there exist a linear state feedback controller which results in closed loop eigen-
values 2, 3?
In parts (b) and (c): If no controller exists, explain why; if one does exist, give an example
of one.

324
Exercise 129 Consider the system described by
x1 = x2 + u
x2 = x1 u
where all quantities are scalars.
(a) Is this system stabilizable via state feedback?
(b) Does there exist a linear state feedback controller which results is closed loop eigenval-
ues 1, 4?
(c) Does there exist a linear state feedback controller which results is closed loop eigenval-
ues 2, 4?
In parts (b) and (c): If no controller exists, explain why; if one does exist, give an example
of one.
Exercise 130 Consider the system
x1 = x2
x2 = x1 + u
(a) Design a continuous-time linear state feedback controller which stabilizes this system
about the zero state. Illustrate the effectiveness of your controller with a simulation.
(b) For various sampling times, discretize the controller found in part (a) and simulate the
corresponding closed loop. Is the closed loop system stable for any sampling time?
(c) Design a stabilizing discrete-time controller based on a discretization of the system.
Consider sampling times T = 0.1, 1, 2, 10. Illustrate the effectiveness of your controller
with simulations of the continuous-time system subject to the discrete-time controller.
(d) Design a dead-beat controller based on a discretization of the system. Consider sam-
pling times T = 0.1, 1, 10. Illustrate the effectiveness of your controller with simula-
tions.
Exercise 131 (BB in laundromat: external excitation.) Obtain a state space representation
of the following system.
m1 m2 1 + k2 (1 2 ) = 0

m2 m2 2 k2 (1 2 ) = u
(a) Show that this system is controllable.
(b) Considering
= 1, m = 1, k = 2
obtain a linear state feedback controller which results in a closed loop system with eigenvalues
1, 2, 3, 4
Use the following methods.

325
 (i) Obtain (by hand) an expression for the closed loop characteristic polynomial in terms
of the components of the gain matrix.

(ii) acker

(iii) place

Illustrate the effectiveness of your controllers with numerical simulations.

Exercise 132 (Stabilization of cart pendulum system via state feedback.) Carry out
the following for parameter sets P2 and P4 and equilibriums E1 and E2. Illustrate the ef-
fectiveness of your controllers with numerical simulations.
Using eigenvalue placement techniques, obtain a state feedback controller which stabilizes
the nonlinear system about the equilibrium.
What is the largest value of (in degrees) for which your controller guarantees conver-
gence of the closed loop system to the equilibrium for initial condition

(y, 1 , 2 , y, 1 , 2 )(0) = (0, 1e , 2e +, 0, 0, 0)

where 1e and 2e are the equilibrium values of 1 and 2 .

Exercise 133 (Discrete-time stabilization of cart pendulum system via state feedback.)
Carry out the following for parameter set P4 and equilibriums E1 and E2. Illustrate the
effectiveness of your controllers with numerical simulations.
Using eigenvalue placement techniques, obtain a discrete-time state feedback controller
which stabilizes the nonlinear system about the equilibrium.
What is the largest value of (in degrees) for which your controller guarantees conver-
gence of the closed loop system to the equilibrium for initial condition

(y, 1 , 2 , y, 1 , 2 )(0) = (0, 1e , 2e +, 0, 0, 0)

where 1e and 2e are the equilibrium values of 1 and 2 .

15.7 Appendix
Proof of Fact 10. We need to prove that for any two matrices M and N of dimensions
m n and n m, respectively, we have

det(I + M N ) = det(I + N M ) .

Consider the following matrix equation:

I M I O I + MN M
= (15.15)
N I N I O I

Since the matrix on the right of the equality is block diagonal, its determinant is det(I) det(I+
M N ) = det(I + M N ). Hence the product of the determinants of the two matrices on the

326
left of the equation is det(I + M N ). Reversing the order of the two matrices on the left of
(15.15), we obtain
I O I M I M
=
N I N I O I + NM
From this we can conclude that the product of the determinants of the two matrices on the left
of equation (15.15) is det(I + N M ). Hence we obtain the desired result that det(I + M N ) =
det(I + N M ).

327
328
Chapter 16

Detectability and observers

16.1 Observers, state estimators and detectability

Consider the system (we will call it the plant)
x = Ax + Bu
(16.1)
y = Cx + Du
Suppose that at each instant time t we can measure the plant output y(t) and the plant
input u(t) and we wish to estimate the plant state x(t). In this section we demonstrate how
to obtain an estimate x(t) of the plant state x(t) with the property that as t the state
estimation error x(t) x(t) goes to zero. We do this by constructing an observer or state
estimator.

Observer or state estimator. An observer or state estimator for a plant is a system

whose inputs consist of the plant input and plant output while its output is an estimate of
the plant state; see Figure 16.1.

Figure 16.1: Observer

We consider here observers which have the following structure.

x = Ax + Bu + L(y y)
x(0) = x0 (16.2)
y = C x + Du

where the n-vector x(t) is the estimated state; the matrix L is called the observer gain matrix
and is yet to be determined. The initial state x0 for the observer is arbitrary. One can regard
x0 as an initial guess of the initial plant state x0 .

329
 Note that the observer consists of a copy of the plant plus the correction term, L(y y).

If we rewrite the observer description as

x = (A + LC)x + (B + LD)u Ly ,

it should be clear that we can regard the observer as a linear system whose inputs are the
plant input u and plant output y. We can regard the estimated state x as the observer
output.
To discuss the behavior of the above estimator, we introduce the state estimation error

x := x x

Using (16.1) and (16.2) one can readily show that the evolution of the estimation error is
simply governed by
x = (A + LC)x
Hence, if one can choose L such that all the eigenvalues of the matrix A + LC have negative
real parts then, we obtain the desired result that

lim x(t) = 0 .
t

In other words, the state estimation error goes to zero as t goes to infinity. When this
happens, we call the observer an asymptotic observer. This leads to the next definition.

DEFN. System (16.1) or the pair (C, A) is detectable if there exists a matrix L such that
A + LC is asymptotically stable.

Note that the matrix A + LC is asymptotically stable if and only if the matrix

(A + LC) = A + C L

is asymptotically stable. Thus, the problem of choosing a matrix L so that A + LC is

asymptotically stable is equivalent to the stabilizability problem of choosing K so that A +
C K is asymptotically stable. If one solves the stabilizability problem for K, then L = K
solves the original problem.
From the above observations, we obtain the following duality between stabilizability and
detectability.

(C, A) is detectable if and only if (A , C ) is stabilizable.

(A, B) is stabilizable if and only if (B , A ) is detectable.

Also, it follows that

observability = detectability

330
16.2 Eigenvalue placement for estimation error dynam-
ics
Suppose (C, A) is a real observable pair. Then (A , C ) is a real controllable pair. Now
consider any bunch of complex numbers
{ 1 , 2 , , n }
with the property that if is in the bunch then so is . Since (A , C ) is real and controllable,
there is a real gain matrix K such that { 1 , 2 , , n } are precisely the eigenvalues of
A + C K. Letting L = K and recalling that a matrix and its transpose have the same
eigenvalues, we obtain that the eigenvalues of A + LC are precisely { 1 , 2 , , n }. We
have just demonstrated the following result.

If (C, A) is observable, one can arbitrarily place the eigenvalues of A + LC by appropriate

choice of the observer gain matrix L.

Example 198 (The unattached mass with position measurement and an input.)
x1 = x2
x2 = u
y = x1
Here
0 1
A= C= 1 0
0 0
Since this system is observable, it is detectable. If

l1
L=
l2
we have
l1 1
A + LC =
l2 0
Hence,
det(sI A LC) = s2 l1 s l2
and A + LC is asymptotically stable if
l1 < 0 l2 < 0
An asymptotic observer is then given by
x 1 = x2 + l1 (x1 y)
x 2 = u + l2 (x1 y)
and the estimation error x = x x satisfies
x 1 = l1 x1 + x2
x 2 = l2 x1

331
16.3 Unobservable modes (you cant see them)
Suppose system (16.1) is unobservable. Then, as we have already seen, this system will have
at least one unobservable eigenvalue . This means that, when u(t) 0, the system has a
nonzero solution of the form xuo (t) = et v which produces zero output, y(t) 0. Thus one
cannot distinguish this state motion from the zero motion, x(t) 0. If <() < 0 then xuo (t)
asymptotically goes to zero. However, if <() 0 then, xuo (t) does not asymptotically go
to zero. Hence, even using nonlinear observers, there is no way that one can asymptotically
estimate the state of this system. For linear observers, we have the following result.

If an unobservable eigenvalue of (C, A), then is an eigenvalue of A + LC for every

observer gain matrix L .

To see this, recall that is an unobservable eigenvalue of (C, A) if and only if

A I
rank <n
C

where A is n n. Hence, there is a nonzero n-vector v such that

A I
v=0
C

or, equivalently,

Av = v
Cv = 0

Hence, for any matrix L

(A + LC)v = v
Since v 6= 0, it follows that is an eigenvalue of A + LC.
The next result states that a necessary and sufficient condition for stabilizability is that
all unobservable eigenvalues are asymptotically stable.

Theorem 27 (PBH detectability theorem) The pair (C, A) is detectable if and only if

A I
rank =n
C

for every eigenvalue of A with nonnegative real part.

Proof. Use duality and the corresponding result for stabilizability.

Example 199 (The unattached mass with velocity measurement.)

x1 = x2
x2 = 0
y = x2

332
 Here
0 1
A= C= 0 1
0 0
Hence
1
A I
= 0
C
0 1
The above matrix has rank 1 < n for = 0; hence = 0 is an unobservable eigenvalue and
this system in not detectable.
Note that for any observer gain matrix

l1
L=
l2

we have
0 1 + l1
A + LC =
0 l2
Hence,
det(sI A LC) = s(s l2 )
So, regardless of choice of L, the matrix A + LC always has an eigenvalue at 0.

16.4 Observable canonical form*

Suppose A and C have the following structure:

0 0 0 a0
1 0 0 a1

0 a2
A= 0 1 and C= 0 0 0 1 .
.. .. ..
. . .
0 0 1 an1
Then we say that the pair (C, A) is in observable canonical form. Note that (C, A) is in
observable canonical form if and only if (A , C ) is in controllable canonical form. If (C, A)
is in observable canonical form, then x = Ax and y = Cx look like

x1 = a0 xn
x2 = a1 xn + x1
..
.
xn = an1 xn + xn1

y = xn

Also,
det(sI A) = sn + an1 sn1 + + a0 .

333
If we reverse the ordering of the states, then

x1 = an1 x1 + x2
..
.
xn1 = a1 x1 + xn
xn = a0 x1

y = x1

and the corresponding A and C matrices have the following structure:

an1 1 0 0
an2 0 1 0

.. .. ..
A= . . . and C = 1 0 0 0

a1 1 0 1
a0 0 0 0

16.5 Discrete-time systems

The results for discrete-time systems are basically the same as those for continuous-time
systems. Consider a discrete-time system described by

x(k+1) = Ax(k) + Bu(k)

(16.3)
y(k) = Cx(k) + Du(k)

Observer or state estimator. Recalling the observer structure in the continuous-time

case, we consider here observers which have the following structure.

x(k+1) = Ax(k) + Bu(k) + L(y(k) y(k))

x(0) = x0 (16.4)
y(k) = C x(k) + Du(k)

where the n-vector x(k) is the estimated state; the matrix L is called the observer gain matrix.

Introducing the state estimation error

x := x x

and using (16.3) and (16.4) one can readily show that the evolution of the estimation error
is simply governed by
x(k+1) = (A + LC)x(k)
Hence, if one can choose L such that all the eigenvalues of the matrix A+LC have magnitude
less than one then, we obtain the desired result that

lim x(k) = 0 .
t

334
Dead beat observer. If (C, A) is observable, choose L so that all the eigenvalues of A+LC
are zero. Then, the state estimation error goes to zero in at most n steps.

Exercises
Exercise 134 BB in laundromat: Obtain a state space representation of the following
system.
m1 m2 1 + k2 (1 2 ) = 0

m2 m2 2 k2 (1 2 ) = u

y = 2
Show that this system is observable. Considering

= 1, m = 1, k = 2

obtain a state estimator (observer) which results in an estimation error system with eigen-
values
1, 2, 3, 4
Use the following methods.

(a) Obtaining an expression for the characteristic polynomial of the error system in terms
of the components of the observer gain matrix.

(b) acker

(c) place

Illustrate the effectiveness of your observer(s) with numerical simulations.

Exercise 135 Consider the system described by

x1 = x2 + u
x2 = x1 u
y = x1 x2

where all quantities are scalars.

(a) Is this system observable?

(b) Is this system detectable?

(c) Does there exist an asymptotic state estimator for this system? If an estimator does
not exist, explain why; if one does exist, give an example of one.

335
336
Chapter 17

Climax: output feedback controllers

Plant. (The object of your control desires.) Consider a system described by

x = Ax + Bu
(17.1)
y = Cx
where the state x(t) is an n-vector, the control input u(t) is an m-vector, and the measured
output y(t) is a p-vector. Initially, we consider D = 0 for simplicity of presentation.

17.1 Memoryless (static) output feedback

The simplest type of controller is a memoryless (or static) linear output feedback controller;
this is of the form
u = Ky
where K is a real m p matrix, sometimes called a gain matrix. This controller results in
the following closed loop system:
x = (A + BKC)x
If the open loop system x = Ax is not asymptotically stable, a natural question is whether
one can choose K so that the closed loop system is asymptotically stable. For full state
feedback (C = I), we have seen that it is possible to do this if (A, B) is controllable or, less
restrictively, if (A, B) is stabilizable. If (C, A) is observable or detectable, we might expect
to be able to stabilize the plant with static output feedback. This is not the case as the
following example illustrates.
Example 200 Unattached mass with position measurement.
x1 = x2
x2 = u
y = x1
This system is both controllable and observable. All linear static output feedback controllers
are given by
u = ky

337
which results in

x1 = x2
x2 = kx1

Such a system is never asymptotically stable.

Note that if the plant has some damping in it, that is,

x1 = x2
x2 = dx2 + u
y = x1

where d > 0, the closed loop system is given by

x1 = x2
x2 = kx1 dx2

This is asymptotically stable provided k < 0.

Example 201 Consider

x1 = x2
x2 = x1 + u
y = 2x1 + x2

For a SISO system, one could use root locus techniques to determine whether or not
the system is stabilizable via static output feedback. For a general MIMO system, there
are currently are no easily verifiable conditions which are both necessary and sufficient for
stabilizability via static output feedback. It is a topic of current research. So where do we
go now?

17.2 Dynamic output feedback

A dynamic output feedback controller is a dynamic input-output system whose input is the
measured output of the plant and whose output is the control input to the plant. Thus, a
linear, time-invariant, dynamic, output-feedback controller is described by

xc = Ac xc + Bc y
(17.2)
u = Cc xc + Dc y

338
where the nc -vector xc (t) is called the controller state and nc is the order of the controller. We
regard a memoryless controller as a controller of order zero.
Using transfer functions, the above controller can be described by

u(s) = Gc (s)y(s)

where
Gc (s) = Cc (sI Ac )1 Bc + Dc

Closed loop system. Application of the dynamic controller (17.2) to the plant (17.1)
yields the following closed loop system:

x = (A + BDc C)x + BCc xc

(17.3)
xc = Bc Cx + A c xc

This is an LTI system whose state is

x
xc
and whose A-matrix is given by

A + BDc C BCc
A=
Bc C Ac

So the order of the closed loop system is n + nc .

Example 202 Consider a SISO system and recall that a PI (proportional integral) controller
can be described by Z t
u(t) = kP y(t) kI y( ) dt
0
Letting Z t
xc (t) = y( ) dt
0
it can readily be seen that this controller is a first order dynamic system described by

xc = y
u = kI xc kP y

with initial condition xc (0) = 0.

Exercise 136 Show that the unattached mass with position feedback can be stabilized with
a first order dynamic output feedback controller. What controller parameters place all the
eigenvalues of the closed loop system at 1?

We now demonstrate the following result:

If a complex number is an unobservable eigenvalue of (C, A) then is an eigenvalue of
A.

339
 To see this, suppose is an unobservable eigenvalue of (C, A). Then

A I
rank <n
C

Since the above matrix has n columns, it must have non-zero nullity; this means there exists
a nonzero n-vector v with
A I
v=0
C
or, equivalently,

Av = v
Cv = 0

Letting be the n + p vector

v
=
0
it should be clear that
A =
Since 6= 0, it follows that is an eigenvalue of A

Exercise 137 Prove the following statement. If is an uncontrollable eigenvalue of (A, B),
then is an eigenvalue of A.

We can now state the following intuitively appealing result.

Lemma 21 If either (A, B) is not stabilizable or (C, A) is not detectable, then plant (17.1)
is not stabilizable by a LTI dynamic output feedback controller of any order.

The last lemma states that stabilizability of (A, B) and and detectability of (C, A) is
necessary for stabilizability via linear dynamic output feedback control. In the next section,
we demonstrate that if these conditions are satisfied then closed loop asymptotic stability
can be achieved with a controller of order no more than the plant order.

17.3 Observer based controllers

Consider a general LTI plant described by

x = Ax + Bu
(17.4)
y = Cx + Du

An observer based controller has the following structure:

x = Ax + Bu + L(C x + Du y)
(17.5)
u = K x

340
Note that this controller is completely specified by specifying the gain matrices K and L;
also this controller can be written as

x = [A + BK + L(C + DK)]x Ly
u = K x

This is a dynamic output feedback controller with state xc = x (hence nc = n, that is, the
controller has same order as plant) and Ac = A + BK + L(C + DK), Bc = L, Cc = K,
and Dc = 0.

Closed loop system. Combining the plant description (17.1) with the controller description
(17.5), the closed loop system can be described by

x = Ax + BK x
x = LCx + (A + BK + LC)x

Let x be the state estimation error, that is,

x = x x . (17.6)

Then the closed loop system is described by

x = (A + BK)x + BK x
(17.7)
x = (A + LC)x

This is an LTI system with A matrix

A + BK BK
A=
0 A + LC

Noting that
det(sI A) = det(sI A BK) det(sI A LC) (17.8)
it follows that the set of eigenvalues of the closed loop system are simply the union of those
of A + BK and those of A + LC. So, if both A + BK and A + LC are asymptotically stable,
the closed loop system is asymptotically stable. If (A, B) is stabilizable, one can choose K
so that A + BK is asymptotically stable. If (C, A) is detectable, one can choose L so that
A + LC is asymptotically stable. Combining these observations with Lemma 21 leads to the
following result.

Theorem 28 The following statements are equivalent.

(a) (A, B) is stabilizable and (C, A) is detectable.

(b) Plant (17.1) is stabilizable via a LTI dynamic output feedback controller.

(c) Plant (17.1) is stabilizable via a LTI dynamic output feedback controller whose order
is less than or equal to that of the plant.

341
Example 203 The unattached mass with position measurement

x1 = x2
x2 = u
y = x1

Observer based controllers are given by

x 1 = x2 + l1 (x1 y)
x2 = u l2 (x1 y)
u = k1 x1 + k2 x2

which is the same as

x 1 = l1 x1 + x2 l1 y
x2 = (k1 + l2 )x1 + k2 x2 l2 y
The closed loop system is described by

x1 = x2
x2 = k1 x1 + k2 x2 + k1 x1 + k2 x2
x 1 = l1 x1 + x2
x 1 = l2 x1

where xi = xi xi , i = 1, 2 are the estimation error state variables. The characteristic

polynomial of the closed loop system is

p(s) = (s2 k2 s k1 )(s2 l1 s l2 )

Hence, if
k1 , k 2 , l 1 , l 2 < 0
the closed loop system is asymptotically stable.

342
17.4 Discrete-time systems
Basically, everything said above for continuous-time systems also holds for discrete-time
systems. The only differences are the way systems are described are the characterization of
stability, stabilizability, and detectability in terms of eigenvalues.
Plant.

x(k + 1) = Ax(k) + Bu(k) (17.9a)

y(k) = Cx(k) (17.9b)

with state x(k) IRn , control input u(k) IRm , and measured output y(k) IRp .

17.4.1 Memoryless output feedback

A memoryless (or static) linear output feedback controller is of the form

u(k) = Ky(k)

where K is a real m p matrix, sometimes called a gain matrix. This controller results in
the following closed loop system:

x(k + 1) = (A + BKC)x(k)

17.4.2 Dynamic output feedback

In general, a linear dynamic output feedback controller is described by

xc (k + 1) = Ac xc (k) + Bc y(k)
(17.10)
u(k) = Cc xc (k) + Dc y(k)

where xc (k) IRnc is called the controller state; nc is the order of the controller.

Closed loop system.

x(k + 1) = (A + BDc C)x(k) + BCc xc (k)

(17.11)
xc (k + 1) = Bc Cx(k) + Ac xc (k)

x
This is an lti system with state and A-matrix
xc

A + BDc C BCc
A=
Bc C Ac

343
17.4.3 Observer based controllers
Consider

x(k + 1) = Ax(k) + Bu(k) (17.12a)

y(k) = Cx(k) + Du(k) (17.12b)

An observer based controller has the following structure:

x(k + 1) = Ax(k) + Bu + L [C x(k) + Du(k) y(k)]

(17.13)
u(k) = K x(k)

Closed loop system. Let x be the estimation error, that is, x = x x. Then the closed loop
system is described by

x(k + 1) = (A + BK)x(k) + BK x(k)

(17.14)
x(k + 1) = (A + LC)x(k)

This is an lti system with A matrix

A + BK BK
A=
0 A + LC

Dead-beat controllers. Place all eigenvalues of A + BK and A + LC at zero.

344
Exercises
Exercise 138 Consider the system,

x1 = x2 + u
x2 = x1
y = x1

where all quantities are scalar. Obtain an output feedback controller which results in an
asymptotically stable closed loop system.

Exercise 139 Consider the system with input u, output y, and state variables x1 , x2 , x3 , x4
described by

x1 = x1
x2 = x1 2x2
x3 = x1 3x3 + u
x4 = x1 + x2 + x3 + x4
y = x3

(a) Is this system observable? (b) Is this system stabilizable using output feedback? Justify
your answers.

Exercise 140 Consider the system,

x1 = x2
x2 = x1 + u
y = x1

where all quantities are scalar. Obtain an output feedback controller which results in an
asymptotically stable closed loop system.

Exercise 141 Consider the system

x1 = x1 + x3
x2 = u
x3 = x2
y = x3

with scalar control input u and scalar measured output y.

(a) Obtain an observer-based output feedback controller which results in an asymptotically
stable closed loop system.

(b) Can all the eigenvalues of the closed loop system be arbitrarily placed?

345
Exercise 142 Consider the system,

x1 = x2 + u1
x2 = u2
y = x1

where all quantities are scalar. Obtain an output feedback controller which results in an
asymptotically stable closed loop system.

Exercise 143 Stabilization of 2-link manipulator with dynamic output feedback.

[m1 lc21 + m2 l12 + I1 ]q1 + [m2 l1 lc2 cos(q1 q2 )]q2 + m2 l1 lc2 sin(q1 q2 )q22
[m1 lc1 + m2 l1 ]g sin(q1 ) = u

[m2 l1 lc2 cos(q1 q2 )]q1 + [m2 lc22 + I2 ]q2 m2 l1 lc2 sin(q1 q2 )q12 m2 glc2 sin(q2 ) = 0
y = q2

Recall the linearizization of this system about

q1 = q2 = u = 0

and recall its state space description.

Use parameter values previously given.
Is the linearization observable? Design a observer based dynamic output feedback con-
troller which asymptotically stabilizes this linearized system. Apply the controller to the
nonlinear system and simulate the nonlinear closed loop system for various initial condi-
tions. (Use zero initial conditions for the observer.)

346
Chapter 18

Constant output tracking in the

presence of constant disturbances

18.1 Zeros
Consider a SISO system described by
x = Ax + Bu (18.1a)
y = Cx + Du (18.1b)
where u(t) and y(t) are scalars while x(t) is an n-vector. The transfer function g associated
with this system is given by
g(s) = C(sI A)1 B + D . (18.2)
Recall that a scalar is a zero of g if
g() = 0 .
Since every rational scalar transfer function can be written as g = n/d where n and d
are polynomials with no common zeros, it follows that the zeros of g are the zeros of n while
the poles of g are the zeros of d. Hence, a scalar cannot be simultaneously a pole and a zero
of g. Also, the eigenvalues of A which are neither uncontrollable nor unobservable are the
poles of g.

18.1.1 A state space characterization of zeros

For each scalar we define T to be the (n+1) (n+1) matrix given by

A I B
T = . (18.3)
C D
This matrix yields the following state space characterization of zeros.

Theorem 29 Consider a SISO system described by (18.1) and let T be the matrix defined
by (18.3). Then det T = 0 if and only if at least one of the following conditions hold.

347
 (a) is an uncontrollable eigenvalue of (A, B).

(b) is an unobservable eigenvalue of (C, A).

(c) is a zero of g.

Proof. We first demonstrate that if at least one of conditions (a), (b), or (c) hold, then
det T = 0. When is an uncontrollable eigenvalue of (A, B), we have

rank A I B < n .

This implies that rank T < n + 1; hence det T = 0. In a similar fashion, if is an

unobservable eigenvalue of (C, A), then

A I
rank < n.
C

From this it also follows that det T = 0. Now consider the case in which is a zero of g but
is neither an uncontrollable eigenvalue of (A, B) nor an unobservable eigenvalue of (C, A).
Since is a zero of g it follows that is not a pole of g. Thus is not an eigenvalue of A.
Hence I A is invertible. Letting x0 = (I A)1 B, we obtain that

(A I)x0 + B = 0

and
Cx0 + D = C(I A)1 B + D = g() = 0 .
This implies that
A I B x0
= 0.
C D 1
Hence, det T = 0.
We now show that if det T = 0, then at least one of conditions (a), (b), or (c) must hold.
To this end suppose that det T = 0 and (a) and (b) do not hold. Then we need to show
that (c) holds. Since det T = 0, there is an n-vector x0 and a scalar u0 , such that

A I B x0
= 0. (18.4)
C D u0

and u0 and x0 cannot both be zero. We now claim that u0 cannot be zero. If u0 = 0, then
x0 6= 0 and
A I
x0 = 0 .
C
This contradicts the assumption that is not an unobservable eigenvalue of (C, A). Hence
u0 is non-zero and, by scaling x0 , we can consider u0 = 1. It follows from (18.4) that

(A I)x0 + B = 0 (18.5a)
Cx0 + D = 0 (18.5b)

348
Hence, B = (A I)x0 , that is B is a linear combination of the columns of A I. This
implies that

rank A I B = rank(A I) .

Using this conclusion and the fact that is not an uncontrollable eigenvalue of (A, B), we
obtain that

n = rank A I B = rank(A I) .

Since rank(A I) = n, it follows that is not an eigenvalue of A. It now follows from

(18.5a) that x0 = (I A)1 B; hence, using (18.5a), we obtain

0 = Cx0 + D = C(I A)1 B + D = g() ,

that is, is a zero of g.

Corollary 4 Suppose {A, B, C, D} is a minimal realization of a SISO transfer function g.

Then a scalar is a zero of g if and only if

A I B
det = 0.
C D

Proof. If {A, B, C, D} is a minimal realization of a SISO transfer function then, (A, B) is

controllable and (C, A) is observable. The result now follows from Theorem 29.

Example 204 Consider the transfer function

s2 1
g(s) = .
s2 + 1

The zeros of this transfer function are 1 and 1. A minimal realization of this transfer
function is given by

0 1 0
A= , B= , C= 2 0 , D = 1.
1 0 1

For this realization,

1 0
A I B
det T = det = 1 1 = 2 1 .
C D
2 0 1

Clearly, the zeros of g are given by det T = 0.

349
 w z

u x

Figure 18.1: A block diagram

18.2 Tracking and disturbance rejection with state feed-

back
Consider a system described by

x = Ax + B1 w + B2 u
(18.6)
z = Cx + D1 w + D2 u

where the n-vector x(t) is the state; the m1 -vector w is an unknown, constant, disturbance
input; the m2 -vector u(t) is the control input; and the p-vector z(t) is a performance output.
Initially, we assume that the state x and the performance output z can be measured.
The control problem we consider here is the following: Suppose r is any constant reference
output; we wish to design a feedback controller such that for all initial conditions x(0) = x0 ,
we have
lim z(t) = r
t

and the state x(t) is bounded for all t 0.

Assumptions. We will assume that the pair (A, B2 ) is stabilizable. We will also need the
following condition:

A B2
rank =n+p (18.7)
C D2

that is, the the rank of the above matrix must equal the number of its rows.
For a system with scalar control input u and scalar output z, the above condition is
equivalent to
A B2
det 6= 0 .
C D2
By Theorem 29, this is equivalent to the requirement that 0 is neither an uncontrollable
eigenvalue of (A, B2 ), an unobservable eigenvalue of (C, A), nor a zero of the transfer function
Tzu (s) = C(sI A)1 B2 + D2 .
Why do we need the condition in (18.7)? If it does not hold then, there is a reference
output r0 such that one cannot achieve limt z(t) = r0 with a bounded state history. To
see this, suppose the above condition does not hold. Then there is a nonzero vector

v1
v2

350
with v1 Rn and v2 IRp such that

v1 A B2
= 0,
v2 C D2

that is,
v1 A + v2 C = 0 , v1 B2 + v2 D2 = 0
Since (A, B2 ) is stabilizable, we must have v2 6= 0. (Why?) Consider w = 0, premultiply
the first and second equations in the system description by v1 and v2 , respectively, and add
to obtain
v1 x + v2 z = 0 ,
that is,
dv1 x
= v2 z
dt
Suppose
lim z(t) = r0 := v2
t

Then
dv1 x
= ||v2 ||2 v2 (z r0 );
dt
Since v2 6= 0 and limt (z(t) r0 ) = 0, it follows that there exists a time T and a positive
number such that
||v2 ||2 v2 (z(t) r0 ) for all t T
Hence, for t T ,
v1 x(t) v1 x(T ) + (t T )
This implies that v1 x is unbounded, So, x is unbounded.

Example 205 Consider

x1 = x2
x2 = x1 x2 + u
z = x2
Here
0 1 0
A B2
= 1 1 1
C D2
0 1 0
Clearly rank condition (18.7) does not hold. It should be clear from the first differential
equation describing this system that, in order to keep z = x2 equal to a nonzero constant,
x1 must be unbounded. Note that the transfer function from u to z, given by
s
,
s2 +s+1
has a zero at 0.

351
 Since the rank of a matrix is always less than or equal to the number of its columns, it
follows that if rank condition (18.7) holds then,

p m2 ,

that is, the number of performance output variables is less than or equal to the number of
control input variables.

18.2.1 Nonrobust control

Consider first the question of the existence of steady state values of x and u such that z has
a steady state value of r. Letting x(t) xe , u(t) ue , and z(t) r in system description
(18.6) results in e
A B2 x B1 w
=
C D2 ue D1 w + r

A B2
Since the matrix has full row rank, there exists a solution (xe , ue ) to the above
C D2
equation. Consider now the controller

u = ue + K[x xe ]

Letting
x := x xe
the closed loop system is described by

x = [A + B2 K]x
z = Cx + r

Choosing K so that A+B2 K is asymptotically stable results in limt x(t) = 0 and, hence,
limt z(t) = r.
In addition to requiring knowledge of w, this approach also requires exact knowledge of
the matrices A, B2 , C, D2 . Inexact knowledge of these matrices yields inexact values of the
required steady values xe and ue . This results in the steady state value of z being offset from
the desired value r.

18.2.2 Robust controller

Integration of output error. Introduce a new state variable xI given by

xI = z r

where xI (0) = xI0 and xI0 is arbitrary. Roughly speaking, this state is the integral of the
output tracking error, z r; specifically, it is given by
Z t
xI (t) = xI0 + (z( ) r) d .
0

352
Introducing new state, disturbance input and performance output,

x w
x := , w := , z := z r ,
xI r
respectively, we obtain the augmented plant,

x = Ax + B1 w + B2 u
z = C x + D1 w + D2 u
where

A 0 B1 0 B2
A = B1 = B2 =
C 0 D1 I D2

C = C 0 D1 = D1 I D2 = D2

We have now the following result.

Lemma 22 The pair (A, B2 ) is controllable (stabilizable) if and only if

(a) the pair (A, B2 ) is controllable (stabilizable), and
(b) rank condition (18.7) holds.

Proof. The pair (A, B2 ) is controllable (stabilizable) iff

rank A I B2 = n + p

for all complex numbers (for all complex with <() 0). Also,

A I 0 B2
A I B2 =
C I D2
Considering = 0, we obtain

A 0 B2 A B2
rank A I B2 = rank = rank . (18.8)
C 0 D2 C D2
Considering 6= 0, we obtain

A I 0 B2
rank A I B2 = rank
C I D2

A I B2 0
= rank
0 0 I

= rank A I B2 +p (18.9)

Since the pair (A, B2 ) is controllable (stabilizable) iff rank [ A I B2 ] = n for all complex
numbers (for all complex with <() 0), the desired result now follows from (18.8) and
(18.9).

353
Controller design. Suppose (A, B2 ) is stabilizable and rank condition (18.7) holds. Then
(A, B2 ) is also stabilizable. Choose any matrix K such that A+ B2 K is asymptotically stable.
Let
u = K x
If we partition K as
K = KP KI
where KP is given by the first n columns of K and KI is given by the last p columns of K,
then the controller is given by

xI = z r
u = K P x + K I xI

Note that this controller can be written as

Z
u = KP x + KI (z r)

This is a generalization of the classical PI (Proportional Integral) Controller.

Closed loop system. The closed loop system is given by

x = [A + B2 K]x + B1 w
z = [C + D2 K]x + D1 w

Since the matrix A + B2 K is asymptotically stable and w is constant, all solutions x are
bounded for t 0. Thus, x(t) is bounded for all t 0.
Letting = x and noting that w is constant, we obtain that

= [A + B2 K] .

Since the matrix A + B2 K is asymptotically stable, we must have

lim x = lim (t) = 0 .

t t

In particular, we have
lim xI = 0 ;
t

hence
lim z(t) = r .
t

Thus, the proposed controller achieves the desired behavior.

Example 206 Consider the constantly disturbed and controlled harmonic oscillator de-
scribed by

x1 = x2
x2 = x1 + w + u
z = x1

354
Here

0 1 0
A= , B 1 = B2 = , C= 1 0 , D 1 = D2 = 0
1 0 1
The pair (A, B2 ) is controllable and

0 1 0
A B2
rank = rank 1 0 1 = 3 = n + p
C D2
1 0 0
Hence

0 1 0 0
A 0 B2
A = = 1 0 0 , and B2 = = 1
C 0 D2
1 0 0 0
Letting
K = k1 k2 kI
we obtain
0 1 0
A + B2 K = 1 + k1 k2 kI
1 0 0
The characteristic polynomial p of A + B2 K is given by
p(s) = s3 k2 s2 + (1 k1 )s kI
Choosing k1 , k2 , kI so that this polynomial has roots with negative real parts, a controller
yielding constant output tracking, rejecting constant disturbances and yielding bounded
states is given by
xI = x1 r
u = k1 x1 + k2 x2 + kI xI

18.3 Measured Output Feedback Control

Consider a system described by
x = Ax + B1 w + B2 u
z = C1 x + D11 w + D12 u
y = C2 x + D21 w
where the n-vector x(t) is the state; the m1 -vector w is an unknown, constant, disturbance
input; the m2 -vector u(t) is the control input; and the p1 -vector z(t) is a performance output.
We will assume z(t) can be measured and that instead of x we can only measure the measured
output y(t) which is a p2 -vector.
Suppose the p1 -vector r is any constant reference output and we wish to design a controller
such that for any initial condition x(0) = x0 ,
lim z(t) = r
t

and the state x(t) is bounded for all t 0. To achieve this objective, we need the following
assumptions.

355
Assumptions.
(a) The pair (A, B2 ) is stabilizable.
(b) The pair (C2 , A) is detectable.
(c)
A B2
rank = n + p1
C1 D12

As before, we introduce a new state variable xI given by

xI = z r
where xI (0) is arbitrary. Letting

y
x w
x := , w := , z := z r , y := xI
xI r
r
we obtain the augmented plant
x = Ax + B1 w + B2 u
z = C1 x + D11 w + D12 u (18.10)
y = C2 x + D21 w
with

A 0 B1 0 B2
A = B1 = B2 =
C1 0 D11 I D12

C1 = C1 0 D11 = D11 I D12 = D12

C2 0 D21 0
C2 = 0 I D21 = 0 0
0 0 0 I
Consider now any output feedback controller of the form
xc = Ac xc + Bc y
(18.11)
u = Cc xc + Dc y
The resulting closed loop system is described by

x A + B2 Dc C2 B2 Cc x B1 + B2 Dc D21
= + w (18.12)
xc Bc C2 Ac xc Dc D21
| {z }
Acl

From the lemma of the previous section, assumptions (a) and (c) imply that the pair (A, B2 )
is stabilizable. Also, detectability of the pair (C2 , A) is equivalent to detectability of the pair
(C2 , A). Hence, one can always choose matrices Ac , Bc , Cc , Dc so that so that the closed loop
system matrix Acl is asymptotically stable.

356
Controller design. Choose any matrices Ac , Bc , Cc , Dc so that the closed loop system
matrix Acl in (18.12) is asymptotically stable. Then the controller is given by (18.11).
If we let

Bc = (BcP BcI Bcr )

Dc = (BcP BcI Bcr )

where the above partitions correspond to the partitions of y, we can express this controller
as

xI = z r
xc = Ac xc + BcI xI + BcP y + Bcr r
u = Cc xc + DcI xI + DcP y + Dcr r

Note that Acl does not depend on Bcr and Dcr ; hence these two matrices can be arbitrarily
chosen.

Closed loop system. The closed loop system is described by (18.12). Since the closed
loop system matrix Acl is asymptotically stable and w is constant, it follows that x and xc
are bounded. Hence x is bounded.
Letting
x
=
xc
and noting that w is constant, we obtain that

= Acl

Since the matrix Acl is asymptotically stable, we must have

x
lim = lim (t) = 0
t xc t

In particular, we have
lim xI = 0 ;
t

hence
lim z(t) = r
t

18.3.1 Observer based controllers

As we have seen in a previous chapter, one approach to the design of stabilizing output
feedback controllers is to combine an asymptotic state estimator with a stabilizing state
feedback gain matrix. Since there is no need to estimate the state xI of the augmented plant
(18.10), we design the observer to estimate only x. This yields the following design.
Choose gain matrix K so that the A + B2 K is asymptotically stable and let

K = KP KI

357
where KP is given by the first n columns of K and KI is given by the last p1 columns of K.
Choose observer gain matrix L so that A + LC2 is asymptotically stable.
An observer based controller is given by

xI = z r integrator
xc = Axc + B2 u + L(C2 xc y) observer
u = KP xc + KI xI control input

where xc (0) and xI (0) are arbitrary.

If we let
e := xc x
the closed loop system due to this controller is described by

x A + B2 K x
= + w
e 0 A + LC2 e

Since the eigenvalues of the closed loop system matrix

A + B2 K
0 A + LC2

are the union of those of A + B2 K and A + LC2 , it follows that this matrix is asympotically
stable. Hence, this controller yields the desired behavior.

Exercises
Exercise 144 Consider a system described by

z(s) = Tzw (s)w(s) + Tzu (s)u(s)

with
1 s2
Tzw (s) = and Tzu (s) = 2
s2
1 s 1
where z, w, and u are the Laplace transforms of the performance output z, disturbance
input w, and control input u, respectively. Let r be a desired constant output and assume
the disturbance input w is an unknown constant. Design an output (z) feedback controller
which guarantees that for all r and w,

lim z(t) = r
t

and all signals are bounded. Illustrate your results with simulations.

358
Exercise 145 Consider a system described by

x = Ax + B1 w + B2 u
y = Cx + D1 w + D2 u

where x(t), u(t), and w, and y(t) IRp are as above. Suppose that we can only measure the
control input u and the measured output y. It is desired to obtain an asymptotic observer
for x and w; that is, we wish to design an observer producing estimates x and w of x and w
with the property that for any initial value x0 of x and any w, one has

lim [x(t) x(t)] = 0 , lim [w(t) w] = 0

t t

(a) State the least restrictive conditions on the system which allows one to construct an
asymptotic estimator.

(b) Give a procedure for constructing an asymptotic estimator.

(c) Illustrate your results with example(s) and simulation results.

359
360
Chapter 19

Lyapunov revisited*

19.1 Stability, observability, and controllability

Recall the damped linear oscillator described by

x1 = x2
k d (19.1)
x2 = m x1 x
m 2

with m, d and k positive. This system is asymptotically stable. The energy of this system is
given by
k 2 m 2
V (x) = x + x2 = x P x
2 1 2
where
k
2
0
P = .
m
0 2
Along any solution, the rate of change of the energy is given by
dV
= dx22 = x C Cx
dt
where
1
C= 0 d2
From this it should be clear that

P > 0

PA + A P + C C = 0

The matrix Q = C C is hermitian positive semi-definite, but it is not positive definite.

Hence, using this P matrix, our existing Lyapunov results only guarantee that the system is
stable; they do not guarantee asymptotic stability. Noting that the pair (C, A) is observable,
we present a result in this section which allows us to use the above P matrix to guarantee
asymptotic stability of (19.1).

361
 Consider a system described by
x = Ax (19.2)
where x(t) is an n-vector and A is a constant n n matrix. We have the following result.

Lemma 23 Suppose there is a hermitian matrix P satisfying

P > 0
(19.3)
PA + A P + C C 0

where (C, A) is observable. Then system (19.2) is asymptotically stable.

Proof. Suppose there exists a hermitian matrix P which satisfies inequalities (19.3). Con-
sider any eigenvalue of A; we will show that <() < 0. Let v 6= 0 be an eigenvector
corresponding to , that is,
Av = v . (19.4)
Hence,
v P Av = v P v ;
and taking the complex conjugate transpose of both sides of this last equation yields

v A P v = v P v

Pre- and post-multiplying the second inequality in (19.3) by v and v, respectively, yields

v P Av + v AP v + v C Cv 0

which implies
v P v + v P v + ||Cv||2 0
that is,
2<()v P v ||Cv||2 .
Since P > 0, we must have v P v > 0.
We now show that we must also have ||Cv|| > 0. Suppose ||Cv|| = 0. Then Cv = 0.
Recalling (19.4) results in

A I
v=0
C
Since (C, A) is observable, the above matrix has rank n, and hence v = 0; this contradicts
v 6= 0. So, we must have ||Cv|| > 0.
Hence,
<() = ||Cv||2 /2v P v < 0
Since <() < 0 for every eigenvalue of A, system (19.2) is asymptotically stable.

362
Example 207 Returning to the damped linear oscillator, we see that inequalities (19.3) are
satisfied by k
0 1
P = 2
m C = 0 d2
0 2
Since the oscillator is described by (19.2) with

0 1
A= k
m md
we have " 1
#
C 0 d2
= 1 3
CA d 2 k
dm2
m
Since this observability matrix is invertible, (C, A) is observable. Using Lemma 23, one can
use the above matrix P to guarantee asymptotic stability of the damped oscillator.
Exercise 146 Prove the following result. Suppose there is a hermitian matrix P satisfying
P 0

PA + A P + C C 0
where (C, A) is detectable. Then the system (19.2) is asymptotically stable.
Recalling that controllability of (A, B) is equivalent to observability of (B , A ), and that
asymptotic stability of A and A are equivalent, we obtain the following result
Lemma 24 Suppose there is a hermitian matrix S satisfying
S > 0
(19.5)
AS + SA + BB 0

where (A, B) is controllable. Then system (19.2) is asymptotically stable

Exercise 147 Suppose there is a hermitian matrix S satisfying
S 0
AS + SA + BB 0

where (A, B) is stabilizable. Then system (19.2) is asymptotically stable

Suppose system (19.2) is asymptotically stable, Then for any matrix C (of appropriate
dimensions) the Lyapunov equation
P A + A P + C C = 0 (19.6)
has the unique solution Z

P = eA t C CeAt dt
0
Recall that this is the infinite time observability grammian associated with (C, A), that is,
P = Wo ; it is always hermitian and positive semidefinite. In addition, as we have seen
earlier, Wo is positive definite if and only if (C, A) is observable. These observations and
Lemma 23 lead to the following result.

363
Theorem 30 The following statements are equivalent.

(a) The system x = Ax is asymptotically stable.

(b) There exist a positive definite hermitian matrix P and a matrix C with (C, A) observ-
able which satisfy the Lyapunov equation

P A + A P + C C = 0 (19.7)

(c) For every matrix C with (C, A) observable, the Lyapunov equation (19.7) has a unique
solution for P and this solution is hermitian positive-definite.

Recall the original Lyapunov equation

P A + A P + Q = 0

If Q is hermitian positive semi-definite, it can always be expressed as Q = C C by appropriate

choice of C and one can readily show that (Q, A) is observable iff (C, A) is observable. Also
rank C = n if and only if Q is positive definite. If C has rank n, the pair (C, A) is observable
for any A. So, if Q is positive definite hermitian, the original Lyapunov equation is of the
form (19.7) with (C, A) observable.
Recalling that controllability of (A, B) is equivalent to observability of (B , A ), and that
asymptotic stability of A and A are equivalent, we can obtain the following result.

Theorem 31 The following statements are equivalent.

(a) The system x = Ax is asymptotically stable.

(b) There exist a positive definite hermitian matrix S and a matrix B with (A, B) control-
lable which satisfy the Lyapunov equation

AS + SA + BB = 0 (19.8)

(c) For every matrix B with (A, B) controllable, the Lyapunov equation (19.8) has a unique
solution for S and this solution is hermitian positive-definite.

Note that if A is asymptotically stable, the solution S to Lyapunov equation (19.8) is

the infinite-time controllability grammian associated with (A, B), that is S = Wc . Also,
Z

S= eAt BB eA t dt
0

364
19.2 A simple stabilizing controller
Consider a linear time-invariant system described by

x = Ax + Bu

where x(t) is an n-vector and u(t) is an m-vector. Consider any positive T > 0 and let
Z 0

W = Wc (T ) := eAt BB eA t dt
T

The above matrix can be regarded as the finite time controllability grammian (over the
interval [T, 0]) associated with (A, B). If (A, B) is controllable, W is invertible.
We now show that the controller

u = B W 1 x
yields an asymptotically stable closed loop system.

Using the following properties of eAt ,

t
At deAt A t deA
Ae = ; e A =
dt dt
we obtain
Z 0


AW + W A = AeAt BB eA t + eAt BB eA t A dt
T
Z 0 t
deAt deA
= BB eA t + eAt BB dt
T dt dt
Z
0
d eAt BB eA t
= dt
T dt
T
= BB eAT BB eA ,

that is,
T
AW + W A BB = eAT BB eA
Considering the closed loop system

x = (A BB W 1 )x

we have

(A BB W 1 )W + W (A BB W 1 ) = AW + W A 2BB

= BB eAT BB eA T
BB ,

365
that is,
(A BB W 1 )W + W (A BB W 1 ) + BB 0 .
Controllability of (A, B) implies controllability of (A BB W 1 , B). This fact, along
with W > 0 and the previous inequality imply that the closed-loop system is asymptotically
stable.

So, the above yields another proof of

controllability = stabilizability

366
Chapter 20

Performance*

20.1 The L2 norm

20.1.1 The rms value of a signal
Consider any scalar signal s which is defined for 0 t < . One measure of the size of the
signal is given by the rms value of the signal; we denote this by ksk2 and define it by
Z 21
2
ksk2 = |s(t)| dt .
0

This is also called the L2 norm of the signal. One can readily show that the three required
properties of a norm are satisfied. This norm is simply the generalization of the usual
Euclidean norm for n-vectors to signals or functions. Clearly, a signal has a finite L2 norm
if and only if it is square integrable.
Example 208 Consider the exponential signal s(t) = et where = + j and and
are both real with > 0. For any T > 0,
Z T Z T
2 1
|s(t)| dt = e2t dt = (e2T 1) .
0 0 2
Thus, Z Z
T
2 1 1
|s(t)| dt = lim |s(t)|2 dt = lim (e2T 1) = .
0 T 0 T 2 2

Hence ksk2 = 1/ 2.
Consider now a signal s where s(t) is an n-vector for 0 t < . We define the L2 norm
of the signal s by
Z 12
ksk2 = |s(t)|2 dt
0
where ks(t)k is the usual Euclidean norm of the n-vector s(t) as given by
ks(t)k2 = |s1 (t)|2 + |s2 (t)|2 + + |sn (t)|2 .
One can readily show that the three required properties of a norm are satisfied.

367
20.1.2 The L2 norm of a time-varying matrix
Consider now a function G defined on [0, ) where G(t) is a matrix for 0 t < . We
define the L2 norm of G by
Z 21

kGk2 = trace (G (t)G(t)) dt .
0

Suppose G(t) is mn and the m-vectors g1 (t), g2 (t), , gn (t) are the columns of G(t). Then

trace (G(t) G(t)) = g1 (t) g1 (t) + g2 (t) g2 (t) + + gn (t) gn (t)

= kg1 (t)k2 + kg2 (t)k2 + + kgn (t)k2 .

So we conclude that n Z n
X X
kGk22 = 2 2
kgi (t) k dt = kgi k22 (20.1)
i=1 0 i=1

where the gi is the i-th column of G.

20.2 The H2 norm of an LTI system

Consider an input-output system with disturbance input w and performance output z de-
scribed by
x = Ax + Bw
(20.2)
z = Cx
and suppose A is asymptotically stable. Recall that zero initial condition input-output
behavior of this system can be completely characterized by its impulse response

G(t) = CeAt B . (20.3)

Consider first a SISO system with impulse response g. When A is asymptotically stable,
it follows that g is square integrable and the L2 norm kgk2 of g is finite where
Z 12
2
kgk2 = |g(t)| dt . (20.4)
0

We refer to kgk2 as the H2 norm of the system. Thus the H2 norm of an asymptotically stable
SISO system is the rms value of its impulse response. This norm is a a performance measure
for the system; it can be regarded as a measure of the ability of the system to mitigate the
effect of the disturbance w on the performance output z.
Consider now a general linear time-invariant system described by (20.2) with inpulse
response given by (20.3). When A is asymptotically stable, if follows that each element of
G is square integrable, and we define the L2 norm of the impulse response by
Z 21

kGk2 = trace (G (t)G(t)) dt . (20.5)
0

368
We refer to kGk2 as the H2 norm of the system.
We now obtain a frequency domain interpretation of kGk2 . Recall that the transfer
function of this system is given by G(s) = C(sI A)1 B. When A is asymptotically stable,
we define the 2-norm of the transfer function by
Z 12
1
kGk2 = trace (G() G()) d . (20.6)
2

For a SISO system with transfer function g, we have

Z
2 1
kgk2 = |g()|2 d .
2

Since G is the Laplace transform of G, it follows from Parsevals Theorem that

kGk2 = kGk2 , (20.7)

that is, the 2-norm of the transfer function equals the L2 norm of the impulse response.

20.3 Computation of the H2 norm

We now show how the H2 -norm can be computed using Lyapunov equations. Suppose A is
asymptotically
R Astable and let Wo be the observability grammian associated with (C, A), that
is, Wo = 0 e C CeAt dt. Recall that Wo is the unique solution to the Lyapunov equation
t

Wo A + A Wo + C C = 0 .

We now obtain that

Z Z Z
A t At
G (t)G(t) dt = B e C Ce B dt = B eA t C CeAt dt B = B Wo B ,
0 0 0

that is, Z
G (t)G(t) dt = B Wo B . (20.8)
0

Using the linearity of the trace operation, it now follows that

Z Z
2
kG()k2 = trace (G (t)G(t)) dt = trace G (t)G(t) dt = trace (B Wo B) ;

0 0

hence
kGk22 = trace (B Wo B) . (20.9)
We can also compute the H2 -norm using the controllability grammian. Since

trace (G (t)G(t)) = trace (G(t)G (t)) ;

369
it follows that the H2 -norm is also given by
Z 21

kGk2 = trace (G(t)G (t)) dt . (20.10)
0
R
Let Wc be the controllability grammian associated with (A, B), that is, Wo = 0
eAt BB eA t dt.
Recall that Wc is the unique solution to the Lyapunov equation

AWc + Wc A + BB = 0 .

Hence,
Z Z Z
At A t
G(t)G(t) dt = Ce BB e C dt = C eAt BB eA t dt C = CWc C ,
0 0 0

that is, Z
G(t)G (t) dt = CWc C . (20.11)
0
Using the linearity of the trace operation, it now follows that
Z Z
2
kG()k2 = trace (G(t)G (t)) dt = trace G(t)G (t) dt = trace (CWc C ) ;

0 0

hence
kGk22 = trace (CWc C ) . (20.12)

Example 209 Consider the scalar system described by

x = x + u
y = x
1
with > 0. This system has transfer function g(s) = s+ . Hence,
Z Z Z
2 1 2 1 1 1 1
kgk2 = |g| d = 2 2
d = d ( = /)
2 2 + 2 1 + 2
1
= tan1 ()
2
1
= .
2
The impulse response for this system is given by g(t) = et . Hence
Z Z
2 2 1 2t
kgk2 = g(t) dt = e2t dt = |
0 e
0 0 2
1
= .
2
1
This clearly illustrates the fact that kgk2 = kgk2 and the H2 norm of this system is (1/2) 2 .

370
 The observability grammian Wo is given by the solution to the Lyapunov equation
Wo Wo + 1 = 0 .
This yields Wo = 1/2. Since B = 1, we readily see that the H2 norm is given by
1
trace (B Wo B) 2 . The controllability grammian Wc is given by the solution to the Lyapunov
equation
Wc Wc + 1 = 0 .
This yields Wc = 1/2. Since C = 1, we readily see that the H2 norm is given by
1
trace (CWc C ) 2 .

MATLAB

>> help normh2

NORMH2 Continuous H2 norm.

[NMH2] = NORMH2(A,B,C,D) or
[NMH2] = NORMH2(SS_) computes the H2 norm of the given state-space
realization. If the system not strictly proper, INF is returned.
Otherwise, the H2 norm is computed as

NMH2 = trace[CPC]^0.5 = trace[BQB]^0.5

where P is the controllability grammian, and Q is the observability

grammian.

>> help norm2

nh2 = norm2(sys)

Computes the H2 norm of a stable and strictly proper LTI

system
-1
SYS = C (sE-A) B

This norm is given by

______________
/
V Trace(C*P*C)

where P solves the Lyapunov equation: A*P+P*A+B*B = 0.

See also LTISYS.

371
Exercises
Exercise 148 Consider the system described by

x1 = x2
x2 = 2x1 3x2 + w
y = 4x1 + 2x2

Obtain the H2 norm of this system using the following methods.

(a) The definition in (20.4).
(b) Equation (20.9).
(c) Equation (20.11).
(d) MATLAB.

Exercise 149 Without carrying out any integration, obtain the H2 norm of the system
whose impulse response g is given by

g(t) = et + e2t .

372
Chapter 21

Linear quadratic regulators (LQR)

21.1 The linear quadratic optimal control problem

In this chapter we learn another method of designing stabilizing state feedback controllers.
These controllers result from solving the linear quadratic (LQ) optimal control problem and
are called linear quadratic regulators (LQR). This problem is characterized by a linear system
and a quadratic cost.
Consider a linear time-invariant system (plant)

x = Ax + Bu (21.1)

with initial condition

x(0) = x0 (21.2)
where the real scalar t is the time variable, the n-vector x(t) is the plant state, and the
m-vector u(t) is the control input. When we refer to a control history, control signal, or
open loop control we will mean a continuous control input function u() which is defined for
0 t < .
Suppose Q and R are hermitian matrices with

R > 0 and Q 0

We call them weighting matrices. For each initial state x0 and each control history u(), we
define the corresponding quadratic cost by
Z
J(x0 , u()) := x(t) Qx(t) + u(t) Ru(t) dt
0

where x() is the unique solution of (21.1) with initial condition (21.2). We wish to minimize
this cost by appropriate choice of the control history. Although the above cost can be infinite,
it follows from the assumptions on Q and R that it is always non-negative, that is,

0 J(x0 , u()) .

373
 In addition to minimizing the above cost, we will demand that the control input drives
the state to zero, that is, we require that
lim x(t) = 0 .
t

So, for a fixed initial state x0 , we say that a control history u() is admissible at x0 if it results
in limt x(t) = 0. We say that a control history uopt () is optimal at x0 if it is admissible
at x0 and
J(x0 , uopt ()) J(x0 , u()) (21.3)
for every admissible control history u(). We refer to
J opt (x0 ) := J(x0 , uopt ())
as the optimal cost at x0 .
In general, the optimal control history and the optimal cost depend on the initial state.
Also, for a fixed initial state, the optimal cost is unique whereas the optimal control history
need not be. In some cases an optimal control history might not exist. Since the cost is
bounded below by zero, there is always an infimal cost J inf (x0 ). That is, one cannot achieve
better than this cost but can come arbitrarily close to it; more precisely, for every admissible
control history u(), one has
J inf (x0 ) J(x0 , u())
and for every > 0 there is an admissible control history u () such that
J(x0 , u ()) J inf (x0 ) +
When an optimal control history exists, we must have J inf (x0 ) = J opt (x0 ). If there exists at
least one admissible control history yielding a bounded cost, then J inf (x0 ) < .
Example 210 Consider the scalar integrator plant described by
x = u
with initial condition x(0) = 1. Suppose we wish to drive the state of this plant to zero while
minimizing the cost Z
u(t)2 dt .
0
Consider any positive > 0 and let
u(t) = et .
Then, the resulting state history, x(t) = et , has the desired behavior and the corresponding
cost is Z
2t
2 e2t dt = |
0 e = .
0 2 2
By choosing > 0 arbitrarily small, we can make the cost arbitrarily close to zero. Since
the cost is bounded below by zero, it follows that theR infimal cost is zero. However, we

cannot make the cost zero. If the cost is zero, that is 0 u(t)2 dt = 0, then we must have
u(t) 0; hence x(t) 1. This control history does not drive the state to zero and, hence,
is not admissible. So there is no optimal control history for this problem. Note that in this
problem, (Q, A) = (0, 0) has an imaginary axis unobservable eigenvalue.

374
Optimal feedback controllers. Suppose one has a feedback controller C with the prop-
erty that for any initial state x0 the control histories generated by the controller are admissi-
ble. We say that C is an optimal feedback controller if for each initial state x0 the corresponding
control histories generated by C are optimal at x0 . We will shortly demonstrate the following
fact for linear quadratic optimal control problems:

If there is an optimal control history for every initial state then, for every initial state, the
optimal control history is unique and is given by a linear memoryless state feedback controller.

Since we are going to look controllers which minimize the cost and result in

lim x(t) = 0
t

we need to make the following assumption for the rest of the chapter:

The pair (A, B) is stabilizable.

21.2 The algebraic Riccati equation (ARE)

We will see that, when an optimal feedback controller exists, its determination involves the
following matrix equation which is known as the Algebraic Riccati Equation (ARE):

P A + A P P BR1 B P + Q = 0 (21.4)

The matrices A, B, Q, R are determined by the LQR problem data. Solving the LQR problem
involves solving the ARE for a hermitian matrix P . Since the ARE depends quadratically
on P , it is a nonlinear matrix equation and, in general, has more than one solution.
A solution P is said to be a stabilizing solution if the matrix A BR1 B P is Hurwitz,
that is, all its eigenvalues have negative real parts. This is the solution we will be looking
for. The significance of this property is that the linear state feedback controller

u = R1 B P x

results in the asymptotically stable closed loop system:

x = (A BR1 B P )x

It shown in Section 21.4 that if there is an optimal control history for every initial state
then, the ARE must have a stabilizing solution. Hence, if the ARE does not have a stabilizing
solution then, there is not an optimal control history for every initial state.
It is also shown in Section 21.4 that if the pair (Q, A) has no imaginary unobservable
eigenvalues then, the ARE must have a stabilizing solution.
Section 21.4 also shows that a stabilizing solution is maximal in the sense that if P is a
stabilizing and P is any other solution then, P P . Hence there can only be one stabilizing
solution.

375
21.3 Linear state feedback controllers
Suppose
u(t) = Kx(t)
where K is a constant state feedback gain matrix. The resulting closed loop system is
described by
x = (A + BK)x
and for each initial state x0 the controller results in the cost
Z
0 , K) :=
J(x0 , u()) = J(x x(t) [K RK + Q]x(t) dt
0

This feedback controller is an optimal feedback controller if and only if it is a stabilizing

controller (that is, A + BK is asymptotically stable) and for each initial state x0 we have
0 , K) J(x0 , u())
J(x

for every admissible control history u().

If K is stabilizing then, recalling a result from Lyapunov theory,
0 , K) = x P x0
J(x (21.5)
0

where P is the unique solution to the following Lyapunov equation

P (A + BK) + (A + BK) P + K RK + Q = 0 (21.6)

Since A + BK is asymptotically stable and K RK + Q is hermitian positive semi-definite,

we have P = P 0.

Example 211 Consider the simple real scalar system

x = x + u

with cost Z
u(t)2 dt
0
If we do not require limt x(t) = 0 then regardless of initial state x0 , this cost is uniquely
minimized by
u(t) 0
and the resulting controlled system is described by

x = x

This system is not asymptotically stable for 0. So, in looking for an optimal control we
restrict consideration to control inputs that result in limt x(t) = 0
Now consider any linear controller u = kx. This generates a closed loop system

x = ( + k)x

376
which is asymptotically stable if and only if + k < 0. Utilizing (21.5), (21.6), the cost
corresponding to a stabilizing linear controller is P x20 where

k2
P =
2( + k)

If = 0, P = k/2. Since we require asymptotic stability of the closed loop system, this
cost can be made arbitrarily small but nonzero. So for = 0, we have an infimum cost (that
is, zero) but no minimum cost. The infimum cannot be achieved by controls which drive
the state to zero. Note that in this case, (Q, A) = (0, 0) has an imaginary axis unobservable
eigenvalue.

21.4 Infimal cost*

21.4.1 Reduced cost stabilizing controllers.
Suppose one has a stabilizing gain matrix K and wishes to obtain another stabilizing gain
matrix which results in reduced cost. We will show that this is achieved by the gain matrix

K = R1 B P (21.7)

where P solves the Lyapunov equation (21.6).

Fact 13 Suppose A + BK is asymptotically stable and K is given by (21.7) and (21.6).
Then,
(i) the matrix A + B K is asymptotically stable;

(ii) for every initial state x0 ,

0 , K) J(x
J(x 0 , K)

Proof. The Appendix contains a proof.

21.4.2 Cost reducing algorithm

The above result suggests the following algorithm to iteratively reduce the cost.

Jack Benny algorithm.

(0) Choose any gain matrix K0 so that A + BK0 is asymptotically stable.

(1) For k = 0, 1, 2, , let Pk be the unique solution to the Lyapunov equation

Pk (A + BKk ) + (A + BKk ) Pk + Kk RKk + Q = 0 (21.8)

and let
Kk+1 = R1 B Pk (21.9)

377
Fact 14 Suppose (A, B) is stabilizable, R = R > 0 and Q = Q 0. Consider any
sequence of matrices {Pk } generated by the Jack Benny algorithm. Then this sequence has a
limit P , that is,
lim Pk = P .
k

Furthermore, P has the following properties.

(a)
P = P 0

(b) P = P solves

P A + A P P BR1 B P + Q = 0 (21.10)

(c) Every eigenvalue of the matrix A BR1 B P has real part less than or equal to zero.

(d) If (Q, A) has no imaginary axis unobservable eigenvalues, then every eigenvalue of the
matrix A BR1 B P has negative real part.

Proof. (a) Since Pk solves (21.8) we have Pk 0. From consequence (ii) of fact 13, we
have that Pk+1 Pk . Hence,
0 Pk+1 Pk
From this one can show that there is a matrix P such that

P := lim Pk
k

and P 0.
(b) In the limit, and using (21.9),

K := lim Kk = lim Kk+1 = R1 B P

k k

hence, it follows from (21.8) that

P (A + B K) + (A + B K) P + K RK + Q = 0 (21.11)

Substituting
K = R1 B P
results in (21.10).
(c) For each k, <() < 0 for all eigenvalues of A + B K. Since K = limk Kk and
the eigenvalues of a matrix depend continuously on its coefficients, we must have <() 0
for all eigenvalues of A + B K.
(d) Now suppose A+B K has an eigenvalue with <() = 0. Letting v be an eigenvector
0 to and pre- and post-multiplying (21.11) by v and v yields

v K RKv + v Qv = 0

378
Hence,
Kv = 0 Qv = 0
and
v = (A + B K)v = Av
So,
A I
v=0
Q
that is, is an unobservable eigenvalue of (Q, A). So, if (Q, A) has no imaginary axis
unobservable eigenvalues, A + B K has no imaginary axis eigenvalues. So all the eigenvalues
of A + B K have negative real parts.

Example 212 Recall first example. Here the algorithm yields:

Kk2
Pk =
2( + Kk )
Kk+1 = Pk

Hence,

Pk2
Pk+1 =
2( + Pk )

1. ( < 0): Here P = limk Pk = 0 and A BR1 B P = < 0.

2. ( > 0): Here P = limk Pk = 2 and A BR1 B P = < 0.

3. ( = 0): Here Pk+1 = Pk /2 for all k. Hence, P = limk Pk = 0 and

A BR1 B P = 0.
Note that this is the only case in which (Q, A) has an imaginary axis unobservable
eigenvalue.

Remark 11 The matrix equation (21.10) is called an algebraic Riccati equation (ARE). Since
it depends quadratically on P , it is a nonlinear matrix equation and, in general, has more
than one solution.

Remark 12 We say that a hermitian matrix P is a stabilizing solution to ARE (21.10) if all
the eigenvalues of the matrix A BR1 B P have negative real parts. The significance of
this is that the controller
u = R1 B P x
results in the asymptotically stable closed loop system:

x = (A BR1 B P )x

379
21.4.3 Infimal cost
The following result provides a lower bound on the infimal cost J inf (x0 ) in terms of a specific
solution to the ARE.

Fact 15 (Completion of squares lemma.) Suppose R = R > 0 and P is any hermitian

solution to ARE (21.10). Then for every initial condition x0 and every continuous control
input history u() which guarantees

lim x(t) = 0
t

we have
x0 P x0 J(x0 , u()) ;
hence
x0 P x0 J inf (x0 ) .

Proof. Suppose P is any hermitian solution P to the ARE (21.10), let x0 be any initial
state, and consider any control history u() which results in limt x(t) = 0. Consider the
function V (x()) given by
V (x(t)) = x (t)P x(t)
Its derivative is given by

dV (x(t))
= 2x (t)P x(t)
dt
= 2x (t)P Ax + 2x (t)P Bu(t)

Utilizing the ARE (21.10) we obtain

2x P Ax = x (P A + A P )x
= x P BR1 B P x x Qx

Hence (omitting the argument t for clarity),

dV (x(t))
= x P BR1 B P x + 2x P Bu x Qx
dt
= x P BR1 B P x + 2x P Bu + u Ru x Qx u Ru
= (u + R1 B P x) R(u + R1 B P x) x Qx u Ru

Hence
dV (x)
x Qx + u Ru = + (u + R1 B P x) R(u + R1 B P x)
dt
Considering any t 0 and integrating over the interval [0, t] yields
Z t Z t

(x Qx + u Ru) dt = V (x(t)) + V (x(0)) + (u + R1 B P x) R(u + R1 B P x) dt
0 0

380
 Recall V (x) = x P x, x(0) = x0 , and limt x(t) = 0; so, considering the limit as t
and recalling the definition of the cost, we have
Z
J(x0 , u()) = x0 P x0 + (u + R1 B P x) R(u + R1 B P x) dt (21.12)
0

Since R > 0, we immediately have

J(x0 , u()) x0 P x0 (21.13)

The next result states that for any initial state x0 , the infimal cost is x0 P x0 where P is
the limit of any sequence {Pk } generated by Jack Benny algorithm.

Lemma 25 (Infimal cost lemma.) Suppose (A, B) is stabilizable, R = R > 0 and Q = Q

0. Consider any sequence of matrices {Pk }
k=0 generated by the Jack Benny algorithm. Then
this sequence has a limit P , that is,

lim Pk = P
k

and P has the following properties.

(a) If P is any hermitian solution to ARE (21.10), then

P P

(b) For every initial state x0 ,

J inf (x0 ) = x0 P x0

(c) If there exists an optimal control history for every initial state x0 then P is a stabilizing
solution to ARE (21.10).

Proof. Consider any sequence of matrices {Pk } generated by the Jack Benny algorithm.
Then, from fact (14), this sequence has a limit P , that is,

lim Pk = P
k

and it satisfies ARE (21.10).

Consider any initial state x0 and any k = 1, 2, . . . . The control history given by

uk (t) = R1 B Pk1 x(t)

results in the cost

J(x0 , uk ()) = x0 Pk x0
Hence
J inf (x0 ) x0 Pk x0

381
Since P is a solution to the ARE, it follows from the completion of squares lemma that

x0 P x0 J inf (x0 )

Hence
x0 P x0 J inf (x0 ) x0 Pk x0
Considering the limit as k we must have

x0 P x0 J inf (x0 ) x0 P x0 ;

so
J inf (x0 ) = x0 P x0
If P is any hermitian solution to ARE (21.10), we must have

x0 P x0 J(x0 , uk ()) = x0 Pk x0

Taking the limit as k , we obtain

x0 P x0 x0 P x0

Since this is true for all x0 we have P P .

Suppose an optimal control history uopt exists for every x0 . Then

J(x0 , uopt ) = J opt (x0 ) = J inf (x0 ) = x0 P x0

Recalling the proof of the completion of squares lemma, we have

Z
opt
J(x0 , u ()) = x0 P x0 + (uopt + R1 B P x) R(uopt + R1 B P x) dt
0

Hence Z
(uopt + R1 B P x) R(uopt + R1 B P x) dt = 0
0
Since R > 0 we must have
uopt = R1 B P x
Since uopt () is admissable, it results in limt . Since the above is true for any initial state
x0 , it follows that the system
x = (A BR1 B )x
must be asymptotically stable. Hence P is a stabilizing solution to ARE.

Remark 13 A solution to the ARE which satisfies condition (a) above is called the maximal
solution to the ARE. Clearly, it must be unique.

382
21.5 Optimal stabilizing controllers
The solution to the LQR problem is based on the following matrix equation which is known
as the Algebraic Riccati Equation:

P A + A P P BR1 B P + Q = 0 (21.14)

Since this is a nonlinear equation in P it can have more that one solution. We will be
interested in a stabilizing solution which is defined as a hermitian matrix P with the property
that all the eigenvalues of the matrix A BR1 B P have negative real parts.

Theorem 32 Suppose R = R > 0 and the ARE (21.14) has a stabilizing solution P . Then
for every initial state x0 , there is a unique optimal control history uopt ; it is given by

uopt (t) = R1 B P x(t)

and the optimal cost is

J opt (x0 ) = x0 P x0 .

Proof. Let x0 be any initial state, and consider any control history u() which results in
limt x(t) = 0. Define the function V given by

V (t) = x (t)P x(t)

The derivative of V is given by

dV
= 2x (t)P x(t)
dt
= 2x (t)P Ax(t) + 2x (t)P Bu(t)

Utilizing the ARE (21.14) we obtain

2x P Ax = x (P A + A P )x
= x P BR1 B P x x Qx

Hence (omitting the argument t for clarity),

dV
= x P BR1 B P x + 2x P Bu x Qx
dt
= x P BR1 B P x + 2x P Bu + u Ru x Qx u Ru
= (u + R1 B P x) R(u + R1 B P x) x Qx u Ru

Hence
dV
x Qx + u Ru = + (u + R1 B P x) R(u + R1 B P x)
dt

383
Considering any t 0 and integrating over the interval [0, t] yields
Z t Z t

(x Qx + u Ru) dt = V (t) + V (0) + (u + R1 B P x) R(u + R1 B P x) dt
0 0

Recall V (t) = x (t)P x(t), x(0) = x0 , and limt x(t) = 0; so, considering the limit as
t and recalling the definition of the cost, we have
Z
J(x0 , u()) = x0 P x0 + (u + R1 B P x) R(u + R1 B P x) dt (21.15)
0

Since P is a stabilizing solution to ARE (21.10) the control history given by u(t) =
u(t) := R1 B P x(t) results in limt x(t) = 0; hence

J(x0 , u()) = x0 P x0

Consider any other admissible control history u(). Since R > 0, we have

J(x0 , u()) x0 P x0

This implies that u is optimal at x0 and the optimal cost is x0 P x0 .

Also, (21.15) implies that if any other control u yields the optimal cost x0 P x0 , then the
u(t) = R1 B P x(t); hence u(t) = u(t). So, u is a unique optimal control history.

Example 213 Returning to Example 211 we have A = , B = 1, Q = 0, and R = 1. Hence

the ARE is
2P P 2 = 0
This has two solutions
P = P1 := 0 P = P2 := 2
Hence
A BR1 B P1 = A BR1 B P2 =
We have three cases.
Case 1. ( < 0): P1 = 0 is the stabilizing solution and optimal controller is u = 0.
Case 2. ( > 0): P1 = 2 is the stabilizing solution and optimal controller is u = 2x.
Case 3. ( = 0): No stabilizing solution.

Remark 14 If P is a stabilizing solution to ARE (21.10) then, it can be shown that P must
be the maximal solution P . This follows from

x0 P x0 = J opt (x0 ) = J inf (x0 ) = x0 P x0 .

Hence there can be at most one stabilizing solution.

384
21.5.1 Existence of stabilizing solutions to the ARE
Lemma 26 Suppose (A, B) is stabilizable, R = R > 0 and Q = Q. Then ARE (21.10) has
a stabilizing solution if and only if (Q, A) has no imaginary axis unobservable eigenvalues.

Proof. From Fact 14 it follows that if (Q, A) has no imaginary unobservable eigenvalues
then the ARE has a stabilizing solution.
We now prove the converse by contradiction. Suppose P is a stabilizing solution to ARE
and is an imaginary unobservable eigenvalue of (Q, A). Then, there is a nonzero vector v
such that
Av = v and Qv = 0
Pre- and postmultiplying the ARE by v and v yields

2<()v P v v P BR1 B P v = 0

Hence, R1 B P v = 0 and

(A BR1 B P )v = Av BR1 B P v
= v

that is, is an eigenvalue of the matrix A BR1 B P . This contradicts the asymptotic
stability of this matrix.

Lemma 27 Suppose (A, B) is stabilizable, R = R > 0 and Q = Q 0. If (Q, A) is

detectable, then

(a) The ARE has a stabilizing solution.

(b) P is a stabilizing solution if and only if P 0

Proof. Part (a) follows from previous lemma.

(b) We have already seen that if P is stabilizing then P 0. So, all we have to show is
that if P solves the ARE and P 0 then P must be stabilizing. To prove this we write the
ARE as
P (A + BK) + (A + BK) P + K RK + Q = 0
where K = R1 B P . Hence,

P (A + BK) + (A + BK) P + Q 0

Since P 0 and (Q, A) is detectable, it follows from a previous Lyapunov result that A+BK
is asymptotically stable; hence P is stabilizing.

385
21.6 Summary
The main results of this chapter are summarized in the following theorem.

Theorem 33 Suppose (A, B) is stabilizable, R = R > 0 and Q = Q 0. Then the

following statements are equivalent.

(a) For every initial state x0 , there is an optimal control history.

(b) (Q, A) has no imaginary axis unobservable eigenvalues.

(c) ARE (21.10) has a stabilizing solution.

(d) ARE (21.10) has a unique stabilizing solution P ; also P is positive semi-definite and
is the maximal solution.

Suppose any of conditions (a)-(d) hold. Then for every initial state x0 , the optimal control
history is unique and is generated by the linear state feedback controller

u(t) = R1 B P x(t)

where P is the stabilizing solution to ARE (21.10). Also, the optimal cost is given by

J opt (x0 ) = x0 P x0

Example 214 Consider the problem of finding an optimal stabilizing controller in the prob-
lem of minimizing the cost, Z
x22 + u2 dt
0

for the system,

x1 = x1 + x2
x2 = u

This is an LQR (linear quadratic regulator) problem with

1 0 0 0
A= B= Q= R=1
0 0 1 0 1

The pair (Q, A) has a single unobservable eigenvalue . Hence an optimal stabilizing
controller exists if and only if 6= 0.
Considering
p11 p12
P =
p12 p22

386
(we are looking for a symmetric P ) the ARE yields the following three scalar equations

2p11 p212 = 0
p11 + p12 p12 p22 = 0 (21.16)
2p12 p222 + 1 = 0

We consider three cases:

Case 1 ( < 0.) The first equation in (21.16) yields p11 = p212 /. Since we require p11 0
we must have
p11 = p12 = 0
The second equation in (21.16) is now automatically satisfied and (requiring p22 0) the
third equation yields
p22 = 1
Since
0 0
P =
0 1
is the only positive semi-definite solution to the ARE; it must be the stabilizing one. The
fact that it is stabilizing is evident from

1 1
Acl = A BR B P =
0 1

So, the optimal stabilizing controller for this case is given by

u = x2

Case 2 ( > 0.) We obtain three positive semi-definite solutions:

2
i /2 i
Pi = 1 1 = 2( + 1), 2 = 2( 1), 3 = 0
i (1 + 2i ) 2

Since (P1 )11 (P2 )11 , (P3 )11 , and the stabilizing solution is maximal, the matrix P1 must
be the stabilizing solution. Let us check that P1 is indeed a stabilizing solution. The
corresponding closed loop matrix,

1 1
Acl = A BR B P1 = 1
1 (1 + 21 ) 2

has characteristic polynomial

q(s) = s2 + a1 s + a0
with 1 1
a1 = (1 + 21 ) 2 = (42 + 4 + 1) 2 > 0
1 1
2
a0 = 1 (1 + 21 ) 2 = 2( + 1) (4 + 4 + 1) 2 > 0

387
Since all coefficients of this second order polynomial are positive, all roots of this polynomial
have negative real part; hence Acl is asymptotically stable. So, the optimal stabilizing
controller for this case is given by
1
u = 2( + 1)x1 (42 + 4 + 1) 2 x2

Case 3 ( = 0.) We already know that there is no optimal stabilizing controller for this case.
However, to see this explicitly note that the only positive semi-definite solution to the ARE
is
0 0
P =
0 1
Since
1 0 1
Acl = A BR B P = ,
0 1
this solution is not stabilizing; hence there there is no stabilizing solution and no optimal
stabilizing controller.

388
21.7 MATLAB
>> help lqr

LQR Linear quadratic regulator design for continuous systems.

[K,S,E] = LQR(A,B,Q,R) calculates the optimal feedback gain
matrix K such that the feedback law u = -Kx minimizes the cost
function:
J = Integral {xQx + uRu} dt

subject to the constraint equation:

.
x = Ax + Bu

Also returned is S, the steady-state solution to the associated

algebraic Riccati equation and the closed loop eigenvalues E:
-1
0 = SA + AS - SBR BS + Q E = EIG(A-B*K)

[K,S,E] = LQR(A,B,Q,R,N) includes the cross-term N that relates

u to x in the cost function.

J = Integral {xQx + uRu + 2*xNu}

The controller can be formed with REG.

See also: LQRY, LQR2, and REG.

>> help lqr2

LQR2 Linear-quadratic regulator design for continuous-time systems.

[K,S] = LQR2(A,B,Q,R) calculates the optimal feedback gain matrix
K such that the feedback law u = -Kx minimizes the cost function

J = Integral {xQx + uRu} dt

.
subject to the constraint equation: x = Ax + Bu

Also returned is S, the steady-state solution to the associated

algebraic Riccati equation:
-1
0 = SA + AS - SBR BS + Q

[K,S] = LQR2(A,B,Q,R,N) includes the cross-term 2xNu that

relates u to x in the cost functional.

389
 The controller can be formed with REG.

LQR2 uses the SCHUR algorithm of [1] and is more numerically

reliable than LQR, which uses eigenvector decomposition.

See also: ARE, LQR, and LQE2.

>> help are

ARE Algebraic Riccati Equation solution.

X = ARE(A, B, C) returns the stablizing solution (if it
exists) to the continuous-time Riccati equation:

A*X + X*A - X*B*X + C = 0

assuming B is symmetric and nonnegative definite and C is

symmetric.

See also: RIC.

21.8 Minimum energy controllers

Suppose we are interested in stabilizing a system with the minimum amount of control effort.
Then, we might consider the problem of minimizing
Z
ku(t)k2 dt
0

subject to

x = Ax + Bu and x(0) = x0 and lim x(t) = 0 . (21.17)

t

This is an LQR problem with Q = 0 and R = I. The corresponding ARE is

P A + A P P BR1 B P = 0 . (21.18)

Since Q = 0, it follows from our main LQR result that that an optimal stabilizing controller
exists if and only if A has no imaginary eigenvalues.

The optimal controllers for this problem result in a closed loop system having the following
interesting property.

If is an eigenvalue of A := A + BK = A BR1 B P , then either or is an

eigenvalue of A.

To demonstrate the above property, rewrite ARE as

P A + A P = 0 .

390
Suppose is an eigenvalue of A. Then there is a nonzero vector v such that Av = v.
Post-multiplying the ARE by v yields

P v + A P v = 0

6 0, then P v is an eigenvector of A with eigenvalue . Hence , is an eigenvalue

If P v =
of A. If P v = 0, then Av = Av BB P v = Av; hence Av = v and is an eigenvalue of A.

Example 215 Suppose

R we wish to obtain a stabilizing state feedback controller which al-
ways minimizes 0 u(t)2 dt for the scalar system

x = ax + u .

Here
A = a, B = 1, R = 1, Q = 0.
So the problem has a solution provided a is not on the imaginary axis. The ARE corre-
sponding to this problem,
(a + a)P P 2 = 0 ,
has two solutions
P = 0 and P = a + a .
The two corresponding values of A := A BR1 B P are

a and a

respectively.
From this we can explicitly that, when a is on the imaginary axis, there is no stabilizing
P.
When a has negative real part, P = 0 is stabilizing, the optimal controller is zero and
A = A.
When a has positive real part, P = a + a is stabilizing, the optimal controller is u =
(a + a)x and A = a.

391
21.9 H2 optimal controllers*
Consider a system described by

x = Ax + B1 w + B2 u
z = Cx + Du

where the m1 -vector w(t) is a disturbance input; the m2 -vector u(t) is a control input; and
the p-vector z performance output. We assume that
(i) rankD = m2 .

(ii) D C = 0.
With the above assumptions, the energy kzk2 associated with the output is given by
Z Z Z
2 2 2
kzk2 = kz(t)k dt = kCx(t) + Du(t)k dt = (x(t) Qx(t) + u(t) Ru(t)) dt
0 0 0

with
Q = C C and R = D D .
Since D has full column rank, it follows that R is positive definite.
A linear state feedback controller
u = Kx
results in the closed loop system

x = (A + B2 K)x + B1 w (21.19a)
z = (C + DK)x (21.19b)

Suppose that A + B2 K is asymptotically stable and recall the definition of the H2 -norm of
a linear system. If GK is the impulse response matrix from w to z then the H2 norm of the
R 1
above system is given by kGK k2 = trace 0 GK (t)GK (t) dt 2 . Noting that

(C + DK) (C + DK) = C C + K D DK

it follows that
kGK k22 = trace B1 PK B1
where PK is the unique solution to

PK (A + B2 K) + (A + B2 K) PK + K D DK + C C = 0 .

From this one can readily show that if one considers the LQ problem with

B = B2 , R = D D , Q = C C

then an optimal LQ controller minimizes the H2 norm of the closed loop system.
The observability condition on (Q, A) is equivalent to:

392
 The pair (C, A) has no imaginary axis unobservable eigenvalues.

So provided this condition holds the controller which minimizes kGK k2 is given by

u = (D D)1 B2 P

where P is the stabilizing solution to

P A + A P P B1 (D D)1 B1 P + C C = 0

If A + B2 K is asymptotically stable, it follows from an identity in Lyapunov revisited that

kGK k22 = trace (C + DK) X(C + DK)

where X is the unique solution to

(A + B2 K)X + X(A + B2 K) + B1 B1 = 0

The hermitian positive semi-definite matrix X is sometimes referred to as the state covariance
matrix associated with the above system.

393
21.10 Exercises
Exercise 150 Consider the system
x = 2x + u ,
where x and u are real scalars. Obtain an optimal stabilizing feedback controller which mini-
mizes Z
u(t)2 dt .
0

Exercise 151 Consider the system

x1 = x1 + x2
x2 = x2 + u ,
where x1 , x2 and u are real scalars. Obtain an optimal stabilizing feedback controller which
minimizes Z
u(t)2 dt .
0

Exercise 152 Consider the system

x1 = x2 + u
x2 = x1 u
where all quantities are scalar. Obtain a optimal stabilizing feedback controller which mini-
mizes Z
u(t)2 dt .
0

Exercise 153 For the system

x1 = x1 + x2
x2 = x2 + u
Obtain a optimal feedback controller which minimizes
Z
u(t)2 dt
0

You may have to distinguish between different cases of .

Exercise 154 2-link manipulator Recall the two link manipulator with measured output
y = q2 . Recall the linearization of this system about
q1 = q2 = u = 0
and recall its state space description.
Obtain an LQG controller for this linearized system. Apply the controller to the nonlinear
system and simulate the nonlinear closed loop system for various initial conditions. (Use zero
initial conditions for the observer.)

394
21.11 Appendix
21.11.1 Proof of fact 1
Asymptotic Stability

P (A + B K) + (A + B K) P = P (A BR1 B P ) + (A BR1 B P ) P
= P A + A P 2P BR1 B P
= P BK K B P K RK 2P BR1 B P Q
= P BR1 B P Q (K K) R(K K)

Hence

P (A + B K) + (A + B K) P + P BR1 B P + Q + (K K) R(K K) = 0 (21.20)

Consider any eigenvalue of A + B K and let v 6= 0 be a corresponding eigenvector. Then

2<()v P v + v (K K) R(K K)v 0

First note that if

v (K K) R(K K)v = 0
we must have
Kv = Kv
and hence
(A + B K)v = (A + BK)v
that is, is an eigenvalue of A + BK. Since K is stabilizing, <() < 0. So suppose
v (K K)R(K K)v > 0. Then

2<()v P v < 0

Since v P v 0 we must have <() < 0. Since <() < 0 for every eigenvalue of A + B K,
we have asymptotic stability of A + B K.

Reduced cost: The cost associated with K is given by

0 , K) = x P x0
J(x 0

where P is the unique solution to

P (A + B K) + (A + B K)T P + K T RK + Q = 0

Substition for K yields

P (A + B K) + (A + B K) P + P BR1 B P + Q = 0 (21.21)

Combining (21.21) and (21.20) yields

(P P )(A + B K) + (A + B K) (P P ) 0

395
In other words, for some hermitian positive semi-definite matrix Q,

(P P )(A + B K) + (A + B K) (P P ) + Q = 0

Since A + B K is asymptotically stable,

Z

P P = e(A+B K) t Qe(A+B K)t dt
0

Hence P P 0, or
P P

396
Chapter 22

LQG controllers

In the previous chapter, we saw how to obtain stabilizing state feedback gain matrices K
by solving a Ricatti equation. Anytime one has a technique for obtaining stabilizing state
feedback gain matrices, one also has a technique for obtaining stabilizing observer gain
matrices L. This simply follows from the fact the A + LC is asymptotically stable if and
only if the same is true for A + C K with L = K . In this chapter, we use the results of the
last chapter to obtain an observer design method which is based on a Ricatti equation which
we call the observer Ricatti equation. We then combine these results with the results of
the previous chapter to obtain a methodolology for the design of stabilizing output feedback
controllers via the solution of two uncoupled Ricatti equations.

22.1 LQ observer
Plant

x = Ax + Bu
y = Cx

where t is the time variable, the n-vector x(t) is the plant state, the m-vector u(t) is the
control input and the p-vector y(t) is the measured output.
Observer

xc = Axc + Bu + L(Cxc y)

Error dynamics : x = xc x

x = (A + LC)x

In what follows V is a p p matrix and W is a n n matrix.

Lemma 28 Suppose (C, A) is detectable, V = V > 0, and W = W 0. Then the

following are equivalent.

397
 (i) The pair (A, W ) has no imaginary axis uncontrollable eigenvalues.
(ii) The Riccati equation
AS + SA SC V 1 CS + W = 0 (22.1)
has a hermitian solution S with A SC V 1 C asymptotically stable.

Proof. By duality.

We will refer to ARE (22.1) as an observer Riccati equation associated with the plant.

We say that S is a stabilizing solution to (22.1) if A SC V 1 C is asymptotically stable.

The significance of this is that if one chooses the observer gain L as

L = SC V 1 (22.2)
then the error dynamics are described by the asymptotically stable system
x = (A SC V 1 C)x

22.2 LQG controllers

Plant
x = Ax + Bu
y = Cx
Basically we combine an LQR regulator with an LQ observer.

So suppose the the plant satisfies the following minimal requirements for stabilization via
output feedback:
(i) (A, B) is stabilizable.
(ii) (C, A) is detectable.
Choose weighting matrices
Q R W V
(of appropriate dimensions) which satisfy the following conditions:
(i) R = R > 0 and Q = Q 0, and (Q, A) has no imaginary axis unobservable
eigenvalues.
(ii) V = V > 0 and W = W 0, and (A, W ) has no imaginary axis uncontrollable
eigenvalues.
Let P and S be the stabilizing solutions to the Riccati equations
P A + A P P BR1 B P + Q = 0

AS + SA SC V 1 CS + W = 0

398
LQG Controller The corresponding LQG controller is given by

xc = (A BR1 B P )xc SC V 1 (Cxc y)

u = R1 B P xc

Note that this can be written as

xc = (A BR1 B P SC V 1 C)xc + SC V 1 y

u = R1 B P xc

Letting x = xc x, the closed loop system is described by

x = (A BR1 B P )x BR1 B P x
x = (A SC V 1 C)x

As we have seen before the eigenvalues of this closed loop system are simply the union of
those of the asymptotically matrices A BR1 B P and A SC V 1 C. Hence, the closed
loop system is asymptotically stable.

22.3 H2 Optimal controllers*

22.3.1 LQ observers
Consider

x = Ax + B1 w + B2 u
y = Cx + Dw

where the measurement vector y(t) is a p-vector. We assume that

(i) rankD = p

(ii) DB1 = 0

With an observer
xc = Axc + B2 u + L(Cxc y) ,
the corresponding observer error dynamics are described by

x = (A + LC)x + (B1 LD)w .

Note that
(B1 LD)(B1 LD) = B1 B1 + LDD L
Consider any performance variable

z = M x

399
Suppose A + LC is asymptotically stable and GL is the impulse response matrix from w to
z. Then, the H2 norm of GL is given by
Z
2
kGL k2 = trace (G (t)G(t)) dt = M SL M
0

where SL uniquely solves

(A + LC)SL + SL (A + LC) + LDD L + B1 B1 = 0

With
V = DD W = B1 B1
the corresponding LQ observer minimizes the H2 norm of GL .
The controllability condition on (A, W ) is equivalent to:

The pair (A, B1 ) has no imaginary axis uncontrollable eigenvalues.

So provided this condition holds the observer gain which minimizes the H2 norm of GL
is given by
L = SC (DD )1
where S is the stabilizing solution to

AS + SA SC (DD )1 CS + B1 B1 = 0

22.3.2 LQG controllers

Consider
x = Ax + B1 w + B2 u
z = C1 x + D12 u
y = C2 x + D21 w

400
Chapter 23

Systems with inputs and outputs: II*

Recall

x = Ax + Bu
(23.1)
y = Cx + Du

23.1 Minimal realizations

Recall that (A, B, C, D) is a realization of a transfer matrix G if
G(s) = C(sI A)1 B + D.

A realization is minimal if the dimension of its state-space is less than or equal to the state
space dimension of any other realization.

SISO systems. Consider a SISO system described by

n(s)
G(s) =
d(s)
where n and d are polynomials. If n and d are coprime (i.e., they have no common zeros),
then the order of a minimal realization is simply the degree of d.

Equivalent systems. System

x = Ax + Bu
y = Cx + Du
is equivalent to system
x = Ax + Bu
y = C x + Du
if there is a nonsingular matrix T such that
A = T 1 AT B = T 1 B
C = CT D = D

401
Theorem 34

(i) (A, B, C, D) is a minimal realization of G(s) iff (A, B) is controllable and (C, A) is
observable.

(ii) All minimal realizations are equivalent.

23.2 Exercises
Exercise 155 Obtain a state space realization of the transfer function,

1 1
s1 s1

G(s) = .
1 1

s+1 s+1
Is your realization minimal?

Exercise 156 Obtain a state space realization of the transfer function,

s 1
G(s) = .
s1 s+1

Is your realization minimal?

Problem 2 Obtain a minimal state space realization of the transfer function

s2 3s + 2
G(s) = .
s2 + s 2

402
Chapter 24

Time-varying systems*

24.1 Linear time-varying systems (LTVs)

Here we are concerned with linear time-varying systems (LTVs) systems described by

x(t) = A(t)x(t) + B(t)u(t)

(24.1)
y(t) = C(t)x(t) + D(t)u(t)

where t is the time variable, the state x(t) is an n-vector, the input u(t) is an m-vector while
the output y(t) is a p-vector. We suppose that the time-varying matrices A(t), B(t), C(t), D(t)
depend continuously on t.

24.1.1 Linearization about time-varying trajectories

Consider a nonlinear system described by

x = F (x, u) (24.2a)
y = H(x, u) . (24.2b)

Suppose u() is a specific time-varying input history to the nonlinear system and x() is any
corresponding state trajectory. Then

x(t) = F (x(t), u(t)) .

Let y() be the corresponding output history, that is,

y(t) = F (y(t), u(t)) .

Then, introducing the perturbed variables

x = x x and u = u u and y = y y ,

the linearization of system (24.2) about (x, u) is defined by

x = A(t)x + B(t)u (24.3a)

y = C(t)x + D(t)u (24.3b)

403
where
F F
A(t) = (x(t), u(t)) and B(t) = (u(t), u(t))
x x
H H
C(t) = (x(t), u(t)) and D(t) = (x(t), u(t)) .
x u
Note that, although the original nonlinear system is time-invariant, its linearization about a
time-varying trajectory can be time-varying.

24.2 The state transition matrix

Consider a linear time-varying system described by

x = A(t)x (24.4)

where t is the time variable, x(t) is an n-vector and the time-varying matrix A(t) depends
continuously on t.

Fact 16 We have existence and uniqueness of solutions, that is, for each initial condition,

x(t0 ) = x0 , (24.5)

there is a unique solution to the differential equation (24.4) which satisfies the initial condi-
tion. Also, this solution is defined for all time.

Fact 17 Solutions depend linearly on initial state. More specifically, for each pair of times
t, t0 , there is a matrix (t, t0 ) such that for every initial state x0 , the solution x(t) at time t
due to initial condition (24.5) is given by

x(t) = (t, t0 )x0

The matrix (t, t0 ) is called the state transition matrix associated with system (24.4).
Note that it is uniquely defined by

(t0 , t0 ) = I

(t, t0 ) = A(t)(t, t0 )
t

LTI systems
x = Ax

(t, t0 ) = eA(tt0 )

404
Scalar systems. Consider any scalar system described by

x = a(t)x

where x(t) and a(t) are scalars. The solution to this system corresponding to initial condition
x(t0 ) = x0 is given by R t
a( ) d
x(t) = e t0
x0 .
Hence, Rt
a( ) d
(t, t0 ) = e t0

Some properties of the state transition matrix

(a)
(t2 , t1 )(t1 , t0 ) = (t2 , t0 )

(b) (t, t0 ) is invertible and

(t0 , t) = (t, t0 )1

(c)

(t, t0 ) = (t, t0 )A(t0 )
t0

(d)
Z t
det[(t, t0 )] = exp trace [A( )] d
t0

Computation of the state transition matrix. Let X be the solution of

X(t) = A(t)X(t) (24.6a)

X(0) = I (24.6b)

where I is the n n identity matrix. Then X(t) = (t, 0). Hence, using property (b) above,
X(t0 )1 = (0, t0 ) Now use property (a) to obtain

(t, t0 ) = X(t)X(t0 )1 (24.7)

24.3 Stability
Recall that when A is a constant matrix, system (24.4) is asymptotically stable if and only
if all the eigenvalues of A have negative real parts. This leads to the following natural
conjecture: system (24.4) is asymptotically stable if the eigenvalues of A(t) have negative
real parts for all time t. This conjecture is, in general, false. This is demonstrated by the
following example. Actually, in this example, the eigenvalues of A(t) are the same for all t
and have negative real parts. However, the system has unbounded solutions and is unstable.

405
Example 216 (Markus and Yamabe, 1960.) This is an example of an unstable second
order system whose time-varying A matrix has constant eigenvalues with negative real parts.
Consider
1 + a cos2 t 1 a sin t cos t
A(t) =
1 a sin t cos t 1 + a sin2 t
with 1 < a < 2. Here,
e(a1)t cos t et sin t
(t, 0) =
e(a1)t sin t et cos t
Since a > 1, the system corresponding to this A(t) matrix has unbounded solutions. However,
for all t, the characteristic polynomial of A(t) is given by

p(s) = s2 + (2 a)s + 2

Since 2 a > 0, the eigenvalues of A(t) have negative real parts.

24.4 Systems with inputs and outputs

x = A(t)x + B(t)u x(t0 ) = x0 (24.8)

y = C(t)x + D(t)u (24.9)

The solution to (24.8) is uniquely given by

Z t
x(t) = (t, t0 )x0 + (t, )B( )u( ) d
t0

Impulse response matrix

Z t
y(t) = C(t)(t, t0 )x0 + C(t)(t, )B( )u( ) dt + D(t)u(t)
t0

Hence, for x(t0 ) = 0, Z t

y(t) = G(t, )u( ) d
t0

where
G(t, ) := C(t)(t, )B( ) + D( )( )

24.5 DT

x(k) = (k, k0 ) x0
| {z }
state transition matrix

406
Fact 18 (a)
(k0 , k0 ) = I

(b)
(k2 , k1 )(k1 , k0 ) = (k2 , k0 )

(c)
(k + 1, k0 ) = A(k)(k, k0 )

LTI
(k) = Ak

407
408
Chapter 25

Appendix A: Complex stuff

25.1 Complex numbers

Think about ordered pairs (, ) of real numbers , and suppose we define addition oper-
ation and a multiplication operation by

(, ) + (, ) = ( + , + )

(, )(, ) = ( , + )
respectively. It can readily be shown that these two operations satisfy the usual rules of
addition and multiplication for real numbers.
Consider now any two ordered pairs of the form (, 0) and (, 0). Using the above
definitions of addition and multiplication we obtain

(, 0) + (, 0) = ( + , 0)

and
(, 0)(, 0) = (, 0)
(check it out) So we can identify each ordered pair of the form (, 0) with the real number
and the above definitions of addition and multiplication reduce to the usual ones for real
numbers.
Now consider := (0, 1). Using the above definition of multiplication, we obtain

j 2 = (0, 1)(0, 1) = (1, 0)

In other words (identifying (1, 0) with the real number 1),

j 2 = 1

We also note that for any real number ,

j = (0, 1)(, 0) = (0, )

409
hence,
(, ) = (, 0) + (0, ) = + j
If we regard the set C of complex numbers as the set of ordered pairs of real numbers
for which an addition and multiplication are as defined above, we arrive at the following
representation of complex nubers.
A complex number can be written as

= +

where , are real numbers and j 2 = 1. The real numbers and are called the real part
and imaginary part of and are denoted by

:= <() := =()

respectively.
Note
( + j) + ( + j) = + + j( + )
and

( + j)( + j) = + j + j + j 2
= ( ) + j( + )

In other words, we recovered our original definitions of complex addition and multiplication.

If = + , its complex conjugate is defined by

:=

Some properties:

1
<() = ( + )
2
1
=() = ( )
2j

+ = +
=

The absolute value of :

p
|| =
p
= 2 + 2

410
 Suppose p is an n-th order polynomial, i.e.,

p(s) = a0 + a1 s + . . . + an sn

where each coefficient ai is a complex number and an 6= 0. A complex number is a root or

zero of p if
p() = 0
Each p has at least one root and at most n distinct roots. Suppose p has l distinct roots,
1 , 2 , . . . , l ; then
Yl
p(s) = an (s i )mi
i=i

The integer mi is called the algebraic multiplicity of i .

Suppose
p(s) = a0 + a1 s + . . . + an sn
and define
p(s) = a0 + a1 s + . . . + an sn
Then is a root of p iff is a root of p. To see this note that p() = p(), hence

p() = 0 iff p() = 0

Real coefficients. If the coefficients ai of p are real, then the roots occur in complex
conjugate pairs, i.e., if is a root then so is .

25.1.1 Complex functions

The exponential function
2 3 n
e = 1 + + 2!
+ 3!
+ ... + n!
+ ...

It satisfies:
dez
= ez
dz
and

e(+) = e e

Can define cos() and sin(). They satisfy

cos()2 + sin()2 = 1

Also, for any complex

e = cos() + sin()

411
Hence, for any real number ,

e = cos() + sin()

and p
|e | = cos()2 + sin()2 = 1
Also,
e(+) = e e = e cos + e sin
Hence,
|e(+) | = |e | |e) | = e

25.2 Complex vectors and Cn

Read all the material on real vectors and replace the word real with the word complex and
IR with C.
If x Cn , the complex conjugate of x:

x1
x2

x := ..
.
xn

Some properties:

x + y = x + y
x = x

1
<(x) = (x + x)
2
1
=(x) = (x x)
2j

25.3 Complex matrices and Cmn

Read all the material on real matrices and replace the word real with the word complex and
IR with C
If A Cmn , the complex conjugate of A:

a11 a12 . . . a1n
a21 a22 . . . a2n

A := .. .. ..
. . .
am1 am2 . . . amn

412
 Some properties:

A+B = A+B
A = A
AB = AB

1
<(A) = (A + A)
2
1
=(A) = (A A)
2j

The complex conjugate transpose of A:

A := A

T

a11 a21 . . . am1
a12 a22 . . . am2

= .. .. ..
. . .
a1n a2n . . . amn

Some properties:

(A + B) = A + B
(A) = A
(AB) = B A

DEFN. A is hermitian if
A = A

413
414
Chapter 26

Appendix B: Norms

26.1 The Euclidean norm

What is the size or magnitude of a vector? Meet norm.

Consider any complex n vector x.

The Euclidean norm or 2-norm of x is the nonegative real number given by
1
||x|| := (|x1 |2 + . . . + |xn |2 ) 2

Note that
1
||x|| = (x1 x1 + . . . + xn xn ) 2
1
= (x x) 2

If x is real, these expressions become

1
||x|| = (x21 + . . . + x2n ) 2
1
= (xT x) 2

>> norm([3; 4])

ans = 5

>> norm([1; j])

ans = 1.4142

Note that in the last example xT x = 0, but x x = 2.

Properties of || ||. The Euclidean norm has the following properties.

415
 (i) For every vector x,
||x|| 0
and
||x|| = 0 iff x=0

(ii) (Triangle Inequality.) For every pair of fectors x, y,

||x + y|| ||x|| + ||y||

(iii) For every vector x and every scalar .

||x|| = ||||x||

Any real valued function on a vector space which has the above three properties is defined
to be a norm. Two other commonly encountered norms on the space of n-vectors are the
1-norm:
kxk1 := |x1 | + . . . + |xn |
and the -norm:
kxk := max |xi |
i

As a example of a norm on the set of m n matrices A, consider

m X
n
! 12
1
X
kAk = Tr(A0 A) =2 |Aij |2
i=1 j=1

where Tr(M ) denotes the trace of a square matrix M , that is the sum of its diagonal elements.
This norm is called the Frobenius norm and is the matrix analog of the Euclidean norm for
n-vectors. We will meet other matrix norms later.

416
Chapter 27

Appendix C: Some linear algebra

All the definitions and results of this section are stated for real scalars. However, they also
hold for complex scalars; to get the results for complex scalars, simply replace real with
complex and IR with C.

27.1 Linear equations

Many problems in linear algebra can be reduced to that of solving a bunch of linear equa-
tions in a bunch of unknowns. Consider the following m linear scalar equations in n scalar
unknowns x1 , x2 , . . . , xn :

a11 x1 + a12 x2 + + a1n xn = b1

a21 x1 + a22 x2 + + a2n xn = b2
.. (27.1)
.
am1 x1 + am2 x2 + + amn xn = bm

Given the scalars aij and bi , a basic problem is that of determining whether or not the
above system of linear equations has a solution for x1 , x2 , , xn and, if a solution exists,
determining all solutions.
One approach to solving the above problem is to carry out a sequence of elementary
operations on the above equations to reduce them to a much simpler system of equations
which have the same solutions as the original system. There are three elementary operations:

1) Interchanging two equations.

2) Multiplication of an equation by a nonzero scalar.
3) Addition of a multiple of one equation to another different equation.

Each of the above operations are reversible and do not change the set of solutions.
The above approach is usually implemented by expressing the above equations in matrix
form and carrying out the above elementary operations via elementary matrix row operations.

417
Introducing

b1 a11 a12 a1n x1
b2 a21 a22 a2n x2

b= .. , A= .. .. .. and x= .. ,
. . . . .
bm am1 am2 amn xn
the above system of scalar equations can written in matrix form as:
Ax = b (27.2)
where x is an n-vector, b is an m-vector and A is an m n matrix.

Elementary row operations. The above elementary operations on the original scalar
equations are respectively equivalent to the following elementary row operations on the aug-
mented matrix:
A b
associated with the scalar equations.
1) Interchanging two rows.
2) Multiplication of a row by a nonzero scalar.
3) Addition of a multiple of one row to another different row.

The reduced row echelon form of a matrix. Consider any matrix M . For each non-
zero row, the leading element of that row is the first non-zero element of the row. The
matrix M is said to be in reduced row echelon form if its structure satisfies the following four
conditions.
1) All zero rows come after the nonzero rows.
2) All leading elements are equal to one.
3) The leading elements have a staircase pattern.
4) If a column contains a leading element of some row, then all other elements of the column
are zero.
This is illustrated with the following matrix.

0 0 1 0 0
0 0 0 0 1 0

0 0 0 0 0 0 1

M = . .. .. .. .. .. .. .. (27.3)
.. . . . . . . .

0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
So, the matrices

1 0 0 2 0
0 1 0 0 2 1 2 0 0 2 0
0 0 1 3 0

0 1 0 3 0 0 1 0 3 0 0 0 0 1
0 0 0 1 4 0 0 0 1 4
0 0 0 0 0

418
are in reduced row echelon form, whereas the following matrices are not in reduced row
echelon form.

1 2 0 3 1 0 0 3 0 1 2 3 1 2 0 0
0 0 0 0 0 2 0 0 1 0 5 0 0 1 2 0
0 0 1 4 0 0 1 4 0 0 0 0 0 0 0 1

The above matrices respectively violate conditions 1, 2, 3, 4 above.

Fact 19 Every matrix can be transformed into reduced row echelon form by a sequence of el-
ementary row operations. Although the sequence of row operations is not unique, the resulting
row echelon form matrix is unique.

We will denote the row echelon form of a matrix M by rref (M ). The following example
illustrates how to obtain the reduced row echelon form of a matrix.

Example 217
0 2 2 4 2
M = 1 2 3 4 1
2 0 2 0 2
We reduce M to its reduced row echelon form as follows:

0 2 2 4 2
1 2 3 4 1
2 0 2 0 2
Interchange Rows 1 and 2.
1 2 3 4 1
0 2 2 4 2
2 0 2 0 2
Add multiples of row 1 to all suceeding rows.
1 2 3 4 1
0 2 2 4 2
0 4 4 8 4
Multiply row 2 by 1/2.
1 2 3 4 1
0 1 1 2 1
0 4 4 8 4
Add multiples of row 2 to all succeeding rows.
1 2 3 4 1
0 1 1 2 1
0 0 0 0 0
Add multiples of row 2 to all preceeding rows.
1 0 1 0 1
0 1 1 2 1
0 0 0 0 0

Sometimes the leading elements of a reduced row echelon matrix are called pivots.

419
Gauss-Jordan procedure for solving systems of linear equations. Recall the prob-
lem of solving the system of scalar linear equations (27.1) or their vector/matrix counterpart
(27.2). The Gauss-Jordan procedure is a systematic method for solving such equations. One
simply uses elementary row operations to transform the augmented matrix [A b] into its re-
duced row echelon form which we denote by [A b]. Since [A b] is obtained from [A b] by
elementary row operations, the set of vectors x which satisfy Ax = b is exactly the same as
the set which satisfy Ax = b. However, since [A b] is in reduced row echelon form, it can
be simply determined whether or not the equation Ax = b has a solution for x, and, if a
solution exists, to determine all solutions.

Example 218 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an
expression for all solutions and give two solutions.

y 2z = 4
xy = 1
3x 3y + 4z = 9
2y + 10z = 26

Here the augmented matrix is given by

0 1 2 4
1 1 0 1
A b =
3 3

4 9
0 2 10 26

Carrying our elementary row operations, we obtain

0 1 2 4 1 1 0 1 1 1 0 1 1 1 0 1
1 1 0 1 0 1 2 4 4 4
0 1 2 0 1 2
3 3 4 9 3 3 4 9 0 0 4 12 0 0 4 12
0 2 10 26 0 2 10 26 0 2 10 26 0 0 6 18

1 1 0 1 1 1 0 1 1 1 0 1 1 0 0 1
0 1 2 4 0 1 2 4 0 2 0 2
0 1 0 1
0 0 1 3 0 0 1 3 0 0 1 3 0 0 1 3
0 0 6 18 0 0 0 0 0 0 0 0 0 0 0 0

The reduced row echelon form of [A b] (the last matrix above) tells us that the original
equations are equivalent to:

x = 1
y = 2
z = 3

Thus, the original equations have a unique solution given by x = 1, y = 2 and z = 3.

420
 From the above procedure we see that there are three possibilities when it comes to
solving linear equations:
1) A unique solution.
2) Infinite number of solutions.
3) No solution.

421
MATLAB.

help rref

RREF Reduced row echelon form.

R = RREF(A) produces the reduced row echelon form of A.

[R,jb] = RREF(A) also returns a vector, jb, so that:

r = length(jb) is this algorithms idea of the rank of A,
x(jb) are the bound variables in a linear system, Ax = b,
A(:,jb) is a basis for the range of A,
R(1:r,jb) is the r-by-r identity matrix.

[R,jb] = RREF(A,TOL) uses the given tolerance in the rank tests.

Roundoff errors may cause this algorithm to compute a different

value for the rank than RANK, ORTH and NULL.

See also RREFMOVIE, RANK, ORTH, NULL, QR, SVD.

Overloaded methods
help sym/rref.m

Exercises
Exercise 157 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique, and if not unique,
obtain an expression for all solutions.

x1 x2 + 2x3 + x4 = 5
x1 x2 + x3 = 3
2x1 + 2x2 + 2x4 = 2
2x1 2x2 x3 3x4 = 0

Exercise 158 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an
expression for all solutions and given two solutions.

x1 2x2 4x3 = 1
2x1 3x2 5x3 = 1
x1 + 2x3 = 1
x1 + x2 + 5x3 = 2

Exercise 159 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an

422
expression for all solutions and give two solutions.

x1 2x2 x3 2x4 = 1
3x1 + 7x2 3x3 + 7x4 = 4
2x1 x2 + 2x3 x4 = 2

Exercise 160 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an
expression for all solutions and give two solutions.

x2 + x4 = 2
2x1 + 3x2 + 2x3 + 3x4 = 8
x1 + x2 + x3 + x4 = 3

Exercise 161 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an
expression for all solutions and given two solutions.

x1 2x2 4x3 = 1
2x1 3x2 5x3 = 1
x1 + 2x3 = 1
x1 + x2 + 5x3 = 2

Exercise 162 Determine whether or not the following system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an
expression for all solutions and given two solutions.

y 2z = 4
xy = 1
3x 3y + 4z = 9
2y + 10z = 26

Exercise 163 We consider here systems of linear equations of the form

Ax = b

x1
with x = x2 .
x3
For each of the following cases, we present the reduced row echelon form of the augmented
matrix [A b ]. Determine whether or not the corresponding system of linear equations has a
solution. If a solution exists, determine whether or not it is unique; if not unique, obtain an
expression for all solutions and given two solutions.

1 2 0 0 1 0 0 0
1 5 0 0 1 2 0 6 0 0 1 0
(a) 0 0 1 0 (b) 0 0 1 4 (c) (d) 0 1 0 0
0 0 0 1 0 0 1 0
0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 1

423
27.2 Subspaces
Recall that a vector space is a set equipped with two operations, vector addition and mul-
tiplication by a scalar, and these operations must satisfy certain properties. The following
concept generalizes the notion of a line or a plane which passes through the origin.
A non-empty subset S of a vector space is a subspace if for every pair of elements x, y in
S and every pair of scalars , , the vector x + y is contained in S.
In other words, a subset of a vector space is a subspace if it is closed under the operations
of addition and multiplication by a scalar. Note that a subspace can be considered a vector
space (Why?). An immediate consequence of the above definition is that a subspace must
contain the zero vector; to see this consider and to be zero.
Example 219 Consider the set of real 2-vectors x which satisfy
x1 x2 = 0
This set is a subspace of IR2 . Note that the set of vectors satisfying

Figure 27.1: A simple subspace

x1 x2 1 = 0
is not a subspace.
Example 220 Consider any real m n matrix A and consider the set of real n-vectors x
which satisfy
Ax = 0
This set is a subspace of IRn .
Example 221 The set of real matrices of the form



is a subspace of the space of real 2 2 matrices.
Example 222 As a more abstract example of a subspace space, let V be the set of con-
tinuous real-valued functions which are defined on the interval [0, )and let addition and
scalar multiplication be as defined earlier Suppose W be the subset of V which consists of
the differentiable functions V. Then W is a subspace of V.

424
27.3 Basis
27.3.1 Span (not Spam)
Consider a bunch X of vectors. A linear combination of these vectors is a vector which can
be expressed as
1 x1 + 2 x2 + . . . + m xm
where each xi is in the bunch and each i is a scalar.
The span of a bunch X of vectors is the set of all vectors of the form

1 x1 + 2 x2 + . . . + m xm

where each xi is in the bunch and each i is a scalar; it is denoted by span X .

In other words, the span of a bunch of vectors is the set of all linear combinations of
vectors from the bunch.

Exercise 164 Show that span X is a subspace.

Example 223
1 0
span , = IR2
0 1
To see this, note that any element x of IR2 can be expressed as

x1 1 0
= x1 + x2
x2 0 1

Example 224

1 0 0 1 0 0 0 0
span , , , = IR22
0 0 0 0 1 0 0 1

To see this, note that any element A of IR22 can be expressed as:

a11 a12 1 0 0 1 0 0 0 0
= a11 + a12 + a21 + a22
a21 a22 0 0 0 0 1 0 0 1

Fact 20 Consider a bunch X of vectors. The following experiences do not affect span.
(a) Multiplication of one of the vectors by a nonzero scalar.
(b) Replacing a vector by the sum of itself and a scalar multiple of another (different)
vector in the bunch.
(c) Removal of a vector which is a linear combination of other vectors in the bunch.
(d) Inclusion of a vector which is in the span of the original bunch.

425
Span and rref. One can use rref to solve the following problem. Given a bunch of n-
vectors x1 , x2 , , xm and another n-vector b, determine whether or not b is in the span of
the bunch. To solve this problem, consider any linear combination 1 x1 +2 x2 +. . .+m xm of
the vectors x1 , x2 , , xm . Let A be the matrix whose columns are the vectors x1 , x2 , , xm ,
that is,
A = x1 x2 xm
and let be the m-vector whose components are 1 , , m , that is,

= [ 1 2 m ]T .

Then the above linear combination can be expressed as

1
1 2 m
1 2 m

2

1 x + 2 x + . . . + m x = x x x .. = A .
.
m

Thus, b is in the span of the bunch if and only if the equation

A = b

has a solution for . One can determine whether or not a solution exists by considering
M = rref(M ) where
M = [A b] = x1 x2 xm b .

Example 225 Suppose we wish to determine whether or not the vector (1, 1, 1) is a linear
combination of the vectors

2 0 1
1 , 3 , 1 .
1 1 0
Since
2 0 1 1 1 0 0.5 0
rref 1 3 1 1 = 0 1 0.5 0 ,
1 1 0 2 0 0 0 1
it follows that the first vector is not a linear combination of the remaining three.

27.3.2 Linear independence

A bunch of vectors, x1 , x2 , . . . , xm is linearly dependent if there exist scalars 1 , 2 , . . . , m ,
not all zero, such that
1 x1 + 2 x2 + . . . + m xm = 0 (27.4)
The bunch is linearly independent if it is not linearly dependent, that is, the bunch is linearly
independent if and only if (27.4) implies that

1 , 2 , . . . , m = 0

426
 If the bunch consists of a single vector, then it is linearly independent if and only if that
vector is not the zero vector. If a bunch consists of more than one vector, then it is linearly
dependent if and only if one of the vectors can be expressed as a linear combination of the
preceding vectors in the bunch, that is for some j,

xj = 1 x1 + 2 x2 + . . . + j1 xj1

for some scalars i .

Example 226 The following bunches of vectors are linearly dependent.

(i)
0
0

(ii)
1 2
,
1 2

(iii)
1 7 3
2 , 6 , 2
3 5 1
Why?

(iv)
1 0 2 0
,
0 1 0 2

Example 227 The following bunches of vectors are linearly independent.

(a)
1
1

(b)
1 0
,
0 1

(c)
1 7
2 , 6
3 5

427
 (d)
0 0 0
1 0 0

2 5 0
, ,
3 6 0

4 4 203
494 74 11
To see this:
0 0 0 0
1 0 0 0

2 5 0 0
1

+ 2

+ 3

=



3 6 0 0
4 4 203 0
494 74 11 0
implies
1 = 0
21 + 52 = 0
41 + 42 + 2033 = 0
The first equation implies 1 = 0; combining this with the second equation yields
2 = 0; combining these results with the last equation produces 3 = 0. So, no
nontrivial linear combination of this bunch equals zero; hence the bunch is linearly
independent.

(e)
1 0 0 1 0 0 0 0
, , ,
0 0 0 0 1 0 0 1

Exercise 165 Useful generalization of example 227 (d)

Consider a bunch of m vectors in IRn (m n) which have the following structure.

0 0 0
.. .. ..
. . .
. .
0 . .
1 . .
xn 1 .
. , 0 , . . . , ..
. 2
. xn2 0
. .
.. .. xm
nm
.. .. ..
. . .

x1n1 , x2n2 , . . . , xm
nm 6= 0

In words, the first ni 1 components of the i-th vector are zero; its ni component is nonzero;
and n1 < n2 < . . . < nnm .
Prove that this bunch is linearly independent.

428
 Consider a bunch
x1 , x 2 , . . . , x m
of vectors. It can readily be shown that:
Fact 21 The following experiences do not affect linear independence status.
(a) Multiplication of one of the vectors by a nonzero scalar.

(b) Replacing a vector by the sum of itself and a scalar multiple of another (different)
vector in the bunch.

(c) Removal of a vector from the bunch.

(d) Inclusion of a vector which is not in the span of the original bunch.

Using rref to determine linear independence. Suppose we want to determine whether

or not a bunch of n-vectors, x1 , x2 , , xm , are linearly independent. We can achieve this
using rref. To see this, consider any linear combination 1 x1 +2 x2 +. . .+m xm of the vectors
x1 , x2 , , xm . Let A be the n m matrix whose columns are the vectors x1 , x2 , , xm ,
that is,
A = x1 x2 xm ,
and let be the m-vector whose components are 1 , , m , that is,

= [ 1 2 m ]T .

Then the above linear combination of vectors can be expressed as

1

2

1 x1 + 2 x2 + . . . + m xm = x1 x2 xm .. = A .
.
m

Thus the linear combination is zero if and only if A = 0. Letting A be the reduced row
echelon form of A, it follows that satisfies A = 0 if and only if A = 0. So, the columns
of A are linearly independent if and only if the columns of A are linearly independent. More-
over, a column of A is a linear combination of the preceding columns of A if and only if the
corresponding column of A is the same linear combination of the preceding columns of A.
From the above observations, we can obtain the following results where A is the reduced row
echelon form of A.

(a) The vectors are linear independent if and only if every column of A contains a leading
element, that is, the number of leading elements of A is the same as the number of vectors.

(b) If A has a zero column, then the corresponding x-vector is zero; hence the vectors are
linearly dependent.

429
(c) If a non-zero column of A does not contain a leading element, then the corresponding
vector can be written as a linear combination of the preceding vectors, hence the vectors are
linearly dependent.

Example 228 Show that the vectors,

1 1 1
1 1 1
, ,
1 1 , 3
1 0 2

are linear dependent.

Solution:

1 1 1 1 0 2
1 1
1 0 1 1
rref
1 1 = .
3 0 0 0
1 0 2 0 0 0
The vectors are linearly dependent. The third vector is the twice the first vector minus the
second vector.

Example 229 Are the vectors

1 1 1
1 , 1 , 1 ,
2 0 1

linearly independent?
Solution:

1 1 1 1 0 0
rref 1 1 1 = 0 1 0 .
2 0 1 0 0 1
The vectors are linearly independent.

430
27.3.3 Basis
A basis for a vector space is a bunch of linearly independent vectors whose span is the space.

A vector space is called finite dimensional if it has a finite basis; otherwise it is infinite
dimensional.

For a finite dimensional vector space V, it can be shown that the number of vectors in
every basis is the same; this number is called the dimension of the space and is denoted
dim V. We refer to a vector space with dimension n as an n-dimensional space. In an
n-dimensional space, every bunch of more than n vectors space must be linearly dependent,
and every bunch of n linearly independent vectors must be a basis.

Example 230 The following vectors constitute a basis for IR2 (Why?) :

1 0
,
0 1

Hence IR2 is a 2-dimensional space (Big surprise!).

Example 231 The following vectors constitute a basis for IR22 (Why?) :

1 0 0 1 0 0 0 0
, , ,
0 0 0 0 1 0 0 1

Hence IR22 is a 4-dimensional space.

Example 232 The following vectors constitute a basis for the subspace of example 221.

1 0 0 1 0 0
, ,
0 0 1 0 0 1

Exercise 166 Give an example of a basis for IR3 and for IR23 . What are the dimensions
of these spaces?

Coordinates. If e1 , e2 , . . . , en is a basis for an n-dimensional vector space, then every

vector x in the space has a unique representation of the form:

x = 1 e1 + 2 e2 + . . . + n en

The scalars 1 , 2 , , n are called the coordinates of x wrt the basis e1 , e2 , . . . , en . Thus
there is a one-to-one correspondence between any real n-dimensional vector space and IRn .

1
2

x .
..
n

431
 Any linear map from a real n-dimensional V space to a real m-dimensional space W can
be uniquely represented by a real mn matrix; that is, there is a one-to-one correspondence
between the set of linear maps from V to W and IRmn . This is achieved by choosing bases
for V and W.

27.4 Range and null space of a matrix

27.4.1 Null space
Consider any real m n matrix A.
The null space of A, denoted by N (A), is the set of real n-vectors x which satisfy

Ax = 0

that is,
N (A) := {x IRn : Ax = 0}

N (A) is a subspace of IRn . (Exercise)

The nullity of A is the dimension of its null space, that is,

nullity A := dim N (A)

Determining the null space of a matrix A simply amounts to solving the homogeneous
equation Ax = 0. This we can do by obtaining rref (A).

Example 233 A null space calculation

Lets compute a basis for the null space of the following matrix.

0 2 2 4 2
A= 1 2 3 4 1
2 0 2 0 2

We saw in Example 217 that the reduced row echelon form of A is given by

1 0 1 0 1
A = 0 1 1 2 1
0 0 0 0 0

432
Hence, the vectors in the null space of A are those vectors x which satisfy Ax = 0, that is,
x1 + x3 + x5 = 0
x2 + x3 + 2x4 x5 = 0
these equations are equivalent to
x1 = x3 x5
x2 = x3 2x4 + x5
Hence, we can consider x3 , x4 , x5 arbitrary and for any given x3 , x4 , x5 , the remaining com-
ponents x1 and x2 are uniquely determined; hence, all vectors in the null space must be of
the form
x3 x5 1 0 1
x3 2x4 + x5 1 2 1

x3
= x3 1 + x4 0 + x5 0
x4 0 1 0
x5 0 0 1
where x3 , x4 , x5 are arbitrary. Thus the null space of A is the subspace spanned by the three
vectors:
1 0 1
1 2 1

1 0 0

0 1 0
0 0 1
Since the three vectors are linearly independent, they form a basis for the null space of A;
hence the nullity of A is 3.
Example 234 Consider any non-zero (angular velocity) matrix of the form:

0 3 2
A = 3 0 1
2 1 0
The null space of A is the set of vectors x which satisfy:
3 x2 + 2 x3 = 0
3 x1 1 x3 = 0
2 x1 + 1 x2 = 0
Every solution to these equations is a scalar multiple of

1
x = 2
3
Hence, the null space of A is the 1-dimensional subspace of IR3 spanned by this vector.

The set of equilibrium states of a continuous LTI system x = Ax is the null space of A.

The set of equilibrium states of a discrete LTI system x(k+1) = Ax(k) is the null space of
A I.

433
27.4.2 Range
Consider a real m n matrix A.
The range of A, denoted by R(A), is the set of m-vectors of the form Ax where x is an
n-vector, that is,
R(A) := {Ax : x IRn }

Thus, the range of A is the set of vectors b for which the equation Ax = b has a least one
solution for x.

R(A) is a subspace of IRm . (Exercise)

The rank of A is the dimension of its range space, that is,

rank A := dim R(A)

Since

A= a1 a2 . . . an

where a1 , a2 , . . . , an are the columns of A, it should be clear that

x1

x2

Ax = a1 a2 . . . an .. = x1 a1 + x2 a2 + . . . + xn an
.
xn

Hence, the range of a matrix is the set of all linear combinations of its columns. Thus the
rank of A equals the maximum number of linearly independent columns of A.
From Fact 20 it should be clear that if a matrix A is obtained from a matrix A by a
sequence of the following operations then A and A have the same range.

(a) Interchanging two columns.

(b) Mutiplication of one of the columns by a nonzero scalar.

(c) Replacing a column by the sum of itself and a scalar multiple of another (different)
column in the bunch.

(d) Removal of a column which is a linear combination of other columns in the bunch.

(e) Inclusion of a column which is a linear combination of other columns in the bunch.

Calculation of range basis and rank with rref

Here, we present two ways to calculate a basis for the range of a matrix using rref.

434
Method one: rref(A) A basis for the range of A is given by the columns of A corre-
sponding to the columns of rref(A) which contain leading elements. Thus the rank of A is
the number of leading elements in rref(A). This method yields a basis whose members are
columns from A.

Method two: rref(AT ) Let B be the reduced row echelon form of AT , that is

B = rref(AT ) .

Then the nonzero columns of B T form a basis for the range of A. Thus the rank of A is the
number of leading elements in rref(AT ).
From the above considerations, it follows that, for any matrix A,

rank AT = rank A

hence,

rank A = maximum number of linearly independent columns of A

= maximum number of linearly independent rows of A

Example 235 (A range calculation) Lets compute a basis for the range of the matrix,

0 1 1 2
A= 1 2 3 4
2 0 2 0

Using Method one, we compute

1 0 1 0
rref(A) = 0 1 1 2
0 0 0 0

Since the leading elements of rref(A) occur in columns 1 and 2, a basis for the range of A is
given by columns 1 and 2 of A, that is

0 1
1 , 2 .
2 0

Also, the rank of A is two.

Using Method two, we compute

1 0 4
0 1 2
rref(AT ) =
0

0 0
0 0 0

435
Hence,
1 0 0 0
rref(AT )T = 0 1 0 0
4 2 0 0
Since columns 1 and 2 are the non-zero columns of rref(AT )T , a basis for the range of A is
given by columns 1 and 2 of rref(AT )T , that is

1 0
0 , 1 .
4 2

Again we obtain that the rank of A is two.

Fact 22 If A is an m n matrix,

dim R(A) + dim N (A) = n

Fact 23 If A and B are two matrices which have the same numbers of columns, then

N (A) = N (B) if and only if R(AT ) = R(B T )

Exercise 167 Picturize the range and the null space of the following matrices.

2 1 1 1
2 1 1 1

Exercise 168

(a) Find a basis for the null space of the matrix,

1 2 3 4
A= 2 3 4 5 .
3 4 5 6

(b) What is the nullity of A?

Check your answer using MATLAB.

Exercise 169

(a) Find a basis for the range of the matrix,

1 2 3 4
A= 2 3 4 5 .
3 4 5 6

(b) What is the rank of A?

436
Check your answer using MATLAB.
Exercise 170 Obtain a basis for the range of each of the following matrices. Also, what is
the rank and nullity of each matrix?

4 1 1 0 1 0 1 2 3 4 5
3 2 3 0 0 0 2 3 4 1 2
1 3 0 0 0 1 3 4 5 0 0
Check your answers using MATLAB.
Exercise 171 Obtain a basis for the the null space of each of the matrices in exercise 170.
Check your answers using MATLAB

27.4.3 Solving linear equations revisited

Recall the problem of solving the following m scalar equations for n scalar unknowns x1 , x2 , . . . , xn :
a11 x1 + a12 x2 + . . . + a1n xn = b1
a21 x1 + a22 x2 + . . . + a2n xn = b2
..
.
am1 x1 + am2 x2 + . . . + amn xn = bm
These equations can be written in matrix form as:
Ax = b
where x is n 1, b is m 1 and A is m n.

This matrix equation has at least one solution for x if and only if b is in the range of A.
This is equivalent to
rank (A b) = rank A
The matrix equation has at least one solution for every b if and only if the range of A is
m
R , that is, the rank of A is m, or equivalently, the nullity of A is n m; hence n m.

If x is any particular solution of Ax = b, then all other solutions have the form
x + e
where e is in the null space of A.
To see this:

Hence, a solution is unique if and only if the nullity of A is zero or equivalently, the rank
of A is n; hence n m.

It follows from the above that A is invertible if and only if its rank is m and its nullity is
0; since rank A + nullity A = n, we must have m = n, that is A must be square.

437
MATLAB A solution (if one exists) to the equation Ax = b can be solved by
x= A\b

>> help orth

ORTH Orthogonalization.
Q = orth(A) is an orthonormal basis for the range of A.
Q*Q = I, the columns of Q span the same space as the columns
of A and the number of columns of Q is the rank of A.

See also SVD, ORTH, RANK.

>> help rank

RANK Number of linearly independent rows or columns.

K = RANK(X) is the number of singular values of X
that are larger than MAX(SIZE(X)) * NORM(X) * EPS.
K = RANK(X,tol) is the number of singular values of X that
are larger than tol.

>> help null

NULL Null space.

Z = null(A) is an orthonormal basis for the null space of A.
Z*Z = I, A*Z has negligible elements, and the number of columns
of Z is the nullity of A.

See also SVD, ORTH, RANK.

Exercise 172 Using MATLAB, compute the rank of the matrix

1 0
0
for
= 1, 103 , 1010 , 1015 , 1016
Any comments?

Range and null space of linear maps

Suppose A is a linear map from an n-dimensional space V to an m-dimensional space W.
Then the above definitions and results for matrices can be generalized as follows.

Range of A
R(A) := {Ax : x V}

438
 R(A) is a subspace of W.

rank A := dim R(A)

Null space of A

N (A) := {x V : Ax = 0}
N (A) is a subspace of V.

nullity of A := dim N (A)

Fact 24
dim R(A) + dim N (A) = n

Suppose W = V. Then A is invertible if and only if rank A = n.

Suppose b is given vector in W and consider the equation.

Ax = b

This equation has a solution for x if and only if b is in the range of A. If x is any
particular solution of Ax = b, then all other solutions have the form

x + e

where e is in the null space of A.........

439
27.5 Coordinate transformations
Suppose we have n scalar variables x1 , x2 , . . . , xn and we implicitly define new scalar variables
1 , 2 , . . . , n by
x = U
where U is an n n invertible matrix. Then, is explicitly given by

= U 1 x

We can obtain a geometric interpretation of this change of variables as follows. First,

observe that

x1 1 0 0
x2 0 1 0

.. = x1 .. + x2 .. + . . . + xn ..
. . . .
xn 0 0 1
| {z } | {z } | {z } | {z }
x e1 e2 en
that is the scalars x1 , x2 , . . . , xn are the coordinates of the vector x wrt the standard basis
e1 , e2 , . . . , en , or, x is the coordinate vector of itself wrt the standard basis. Suppose

U = u1 . . . u j . . . u n

that is, uj is the j-th column of U . Since U is invertible, its columns u1 , u2 , . . . , un are
linearly independent; hence they form a basis for IRn . Since x = U can be written as

x = 1 u1 + 2 u2 + . . . + n un

we see that 1 , 2 , . . . , n are the coordinates of x wrt to the new basis u1 , u2 , , un and the
vector is the coordinate vector of the vector x wrt this new basis. So x = U defines a
coordinate transformation.

27.5.1 Coordinate transformations and linear maps

Suppose we have a linear map from IRn into IRm defined by

y = Ax

where A is an m n matrix. Suppose we introduce coordinate transformations

x = U , y =V

where U and V are invertible matrices. Then the map is described by

where
= V 1 AU

440
Recalling that the columns u1 , u2 , . . . , un of U form a basis for IRn , and the columns v 1 , v 2 , . . . , v m
of V form a basis for IRm , we have now the following very useful result.

Useful result: Suppose

Auj = 1j v 1 + 2j v 2 + . . . + mj v m

Then the matrix = V 1 AU is uniquely given by

ij = ij

that is ij is the i-th coordinate of the vector Auj wrt the basis v 1 , v 2 , . . . , v m , or, the j-th
column of is the coordinate vector of Auj wrt the basis v 1 , v 2 , . . . , v m .

Example 236 Suppose A is a 2 2 matrix and there are two linearly independent vectors
u1 and u2 which satisfy:

Au1 = 2u1 and Au2 = u1 + 2u2

Letting U = u1 u2 , we can use the above useful result to obtain that

1 2 1
U AU =
0 2

Example 237 Suppose A is a 3 2 matrix while U = u1 u2 and V = v 1 v 2 v 3
are invertible 3 3 and 2 2 matrices, respectively, which satisfy

Au1 = v1 + 2v3 Au2 = 2v2 + v3

Then, applying the above useful result, we obtain

1 0
V 1 AU = 0 2
2 1

Proof of useful result. Premultiply = V 1 AU by V to get:

V = AU

On the right we have

AU = A u 1 . . . uj . . . u n = Au1 . . . Auj . . . Aun

441
And on the left,

11 . . . 1j . . . 1n
21 . . . 2j . . . 2n

.. .. ..

V = v1 v2 . . . . . . vm . . .
.. .. ..
. . .
m1 . . . mj . . . mn

. .. . .. !
11 v + + m1 v ..
1 m
...... . 1j v 1 + . . . + mj v m .. ...... 1
. 1n v + . . . + mn v m
= | {z }
j-th column
Comparing the expressions for the j-th columns of AU and V yields

1j v 1 + . . . + mj v m = Auj = 1j v 1 + . . . + mj v m

Since the {v 1 , . . . , v m } is a basis we must have ij = ij .

27.5.2 The structure of a linear map

Consider again a linear map from IRn into IRm defined by

y = Ax

where A is an m n matrix. Suppose the rank of A is r. Then the dimension of the null
space of A is n r. Choose any basis

{v 1 , . . . , v r }

for the range of A and extend it to a basis

{v 1 , v 2 , . . . , v m }

for IRm . Choose any basis

{ur+1 , . . . , un }
for the null space of A and extend it to a basis

{u1 , u2 , . . . , un }

for IRn .
If we introduce the coordinate transformations

y =V , x = U

where
V = [v 1 . . . v m ] , U = [u1 . . . un ]

442
the original linear map is described by

where
= V 1 AU .
Recall the useful result on coordinate transformations. Since the vectors uj , j = r + 1, . . . , n
are in the null space of A we have

Auj = 0 for j = r + 1, . . . , n

Hence, the last r columns of are zero. For j = 1, . . . , r the vector Auj is in the range of A.
Hence it can be expressed as a linear combination of the vectors v 1 , . . . , v r . Consequently,
the last m r rows of are zero and has the following structure:

11 0
=
0 0

where 11 IRrr . Since the rank of A equals rank which equals rank 11 , the matrix 11
must be invertible.
If we decompose and as

1 1
= , =
2 2

where 1 IRr , 2 IRnr , 1 IRr , 2 IRmr , then the linear map is described by

1 = 11 1
2 = 0

where 11 is invertible.
Also, we have shown that any matrix any m n matrix A can be decomposed as

11 0
A=V U 1
0 0

where U , V , and are square invertible matrices of dimensions n n, m m, and r r,

respectively, and r is the rank of A.

443
444
Chapter 28

Appendix D: Inner products

28.1 The usual scalar product on IRn and Cn

Consider any pair x, y of real n-vectors. The scalar product (inner product) of x and y is
the real number given by

hy, xi = y1 x1 + . . . + yn xn .

Regarding x and y as real column matrices, we have

hy, xi = y T x .

Suppose x and y are complex n-vectors. If we use the above definition as the definition
of inner product, we wont have the property that hx, xi 0. To see this, note that x =
results in x2 = 1. So we define the scalar product (inner product) of x and y to be the
complex number given by

hy, xi := y1 x1 + . . . + yn xn

When x and y are real this definition yields the usual scalar product on IRn . Regarding x
and y as column matrices, we have
hy, xi = y x
where y is the complex conjugate transpose of y.
We can obtain the usual Euclidean norm from the usual scalar product by noting that

hx, xi = x1 x1 + . . . + xn xn
= |x1 |2 + |x2 |2 + . . . + |xn |2
= kxk2 .

Hence
1
kxk = hx, xi 2 .

445
Properties of h, i. The usual scalar product on IRn or Cn has the following properties.

(i) For every pair of vectors x and y,

hx, yi = hy, xi

(ii) For every vector y, every pair of vectors x1 , x2 , and every pair of scalars 1 , 2 ,

hy, 1 x1 + 2 x2 i = 1 hy, x1 i + 2 hy, x2 i

(iii) For every vector x,

hx, xi 0
and
hx, xi = 0 if and only if x=0

For real vectors, property (i) reduces to hx, yi = hy, xi. By utilizing properties (i) and
(ii), we obtain that for every vector x, every pair of vectors y 1 , y 2 , and every pair of scalars
1 , 2 ,
h1 y 1 + 2 y 2 , xi = 1 hy 1 , xi + 2 hy 2 , xi

Any scalar valued function on a real (or complex) vector space which has the above three
properties is defined to be an inner product. A vector space equipped with an inner product
is called an inner product space. For example, suppose V is the vector space of all real valued
continuous functions defined on the interval [0, 1]. For any two functions f and g in V, let
Z 1
hf, gi = f (t)g(t) dt .
0

One may readily show that properties (i)-(iii) above hold.

28.2 Orthogonality
Suppose V is a real (or complex) vector space which is equipped with an inner product h, i.

A vector y is said to be orthogonal to a vector x if

hy, xi = 0

For example, in IR2 the vector (1, 1) is orthogonal to vector (1, 1).
If y is orthogonal to x, then (using property (ii) above) x is orthogonal to y; so, we
usually just say x and y are orthogonal.

446
 Figure 28.1: Orthogonality

28.2.1 Orthonormal basis

Suppose V is finite dimensional and

b1 , b2 , . . . bn

is a basis for V. Then this basis is an orthonormal basis if for every i, j,

hbi , bi i = 1
hbi , b j i = 0 when i 6= j

that is each basis vector has magnitude 1 and is orthogonal to all the other basis vectors.

Unitary coordinate transformations

Suppose b1 , b2 , . . . , bn is an orthonormal basis for Cn and consider the transformation matrix

T = b1 b2 . . . bn

Then

b1
b2
1 2
T T = .. b b . . . bn
.
bn

b1 b1 b1 b2 . . . b1 bn 1 0 ... 0
b2 b1 b2 b2 . . . b2 bn 0 1 ... 0

= .. .. .. = .. .. .. ,
. . . . . .
n 1 n 2 n n
b b b b ... b b 0 0 ... 1

that is, T T = I.
A square complex matrix T is said to be unitary if

T T = I .

If T is unitary, then
T 1 = T .

447
Example 238 The following matrix is unitary for any .

cos sin
T =
sin cos

Consider a unitary coordinate transformation

x = T

with T unitary. Then for any two n vectors 1 and 2 ,

hT 1 , T 2 i = (T 1 ) T 2

= 1 T T 2

= 1 2
= h 1 , 2 i

hence, hT 1 , T 2 i = h 1 , 2 i; that is, unitary transformations preserve the scalar product

and, hence, the Euclidean norm.

448
28.3 Hermitian matrices
In this section, the square (n n) complex matrix P is hermitian, that is

P = P

If P is real, then hermitian is equivalent to symmetric, that is, P T = P .

For any complex n-vector x, the scalar x P x is real:

x P x = (x P x)
= x P x
= x P x

Since x P x = x P x, the scalar x P x is real.

Every eigenvalue of P real. Since is an eigenvalue of P , there is a nonzero vector v

such that
P v = v
Hence
v P v = v v = ||v||2
Since v P v is real and ||v||2 is nonzero real, must be real.

Suppose v1 and v2 are eigenvectors corresponding to different eigenvalues of P . Then v1

and v2 are orthogonal:

P v 1 = 1 v1
implies
v2 P v 1 = 1 v2 v1
Similarly,
v1 P v 2 = 2 v1 v2
Taking the complex conjugate transpose of both sides of this last expression and using
P = P and 2 = 2 yields

v2 P v 1 = 2 v2 v1

Equating the two expressions for v2 P v1 yields

1 v2 v1 = 2 v2 v1

Since 1 6= 2 , we must have v2 v1 = 0, that is, v2 is orthogonal to v1 .

Actually, the following more general result is true.

Fact 25 Every hermitian matrix has an orthonormal basis of eigenvectors.

449
 It now follows that every hermitian matrix P can be expressed as
P = T T
where T is a unitary matrix and is a real diagonal matrix whose diagonal elements are
the eigenvalues of P and the number of times an eigenvalue appears equals its algebraic
multiplicity.

28.4 Positive and negative (semi)definite matrices

28.4.1 Quadratic forms
For any square n n matrix P we can define an associated quadratic form:
n X
X n

x Px = pij xi xj .
i=1 j=1

Quadratic forms arise naturally is mechanics. For example, suppose one considers a rigid
body and chooses an orthogonal coordinate system with origin at the body mass center. If
is a real 3-vector consisting of the components of the angular velocity vector of the body
and I is the 3 3 symmetric inertia matrix of the body, then the rotational kinetic energy
of the body is given by 12 T I.
If P is hermitian (P = P ) then, as we have already shown, the scalar x P x is real for
all x Cn . The following result provides useful upper and lower bounds on x P x .

If P is a hermitian n n matrix, then for all x Cn

min (P )||x||2 x P x max (P )||x||2
where min (P ) and max (P ) denote the minimum and maximum eigenvalues of P respectively.

Proof. Letting = T x, we have

x P x = x T T x
=
= 1 1 1 + . . . + n n n
= 1 |1 |2 + . . . + n |n |2
Since, i max (P ) for i = 1, . . . , n we have i |i |2 max (P )|i |2 ; hence

x P x max (P ) |1 |2 + . . . + |n |2 = max (P )||||2 ;
hence x P x max (P )||||2 . Also |||| = ||x||; hence,
x P x max (P )||x||2
In an analogous fashion one can show that
x P x min (P )||x||2

450
28.4.2 Definite matrices

DEFN. A hermitian matrix P is positive definite (pd) if

x P x > 0 for all nonzero x in Cn
We denote this by P > 0.

The matrix P is negative definite (nd) if P is positive definite; we denote this by P < 0.

Example 239 For

1 1
P =
1 2
we have (note the completion of squares trick)
x P x = x1 x1 x1 x2 x2 x1 + 2x2 x2
= (x1 x2 ) (x1 x2 ) + x2 x2
= |x1 x2 |2 + |x2 |2
Clearly, x P x 0 for all x. If x P x = 0, then x1 x2 = 0 and x2 = 0; hence x = 0. So,
P > 0.
Fact 26 The following statements are equivalent for any hermitian matrix P .
(a) P is positive definite.
(b) All the eigenvalues of P are positive.
(c) All the leading principal minors of P are positive; that is,
p11 > 0

p11 p12
det > 0
p21 p22
..
.
det(P ) > 0

Example 240 Consider

1 1
P =
1 2
Since p11 = 1 > 0 and det(P ) = 1 > 0, we must have P > 0.
>> eig(P)
ans =
0.3820
2.6180
Note the positive eigenvalues.

451
28.4.3 Semi-definite matrices

DEFN. A hermitian matrix P is positive semi-definite (psd) if

x P x 0 for all x in Cn
We denote this by P 0

The matrix P is negative semi-definite (nsd) if P is positive semi-definite; we denote this

by P 0

Fact 27 The following statements are equivalent for any hermitian matrix P .
(a) P is positive semi-definite.
(b) All the eigenvalues of P are non-negative.
(c) All the principal minors of P are non-negative.
Example 241 This example illustrates that non-negativity of only the leading principal
minors of P is not sufficient for P 0. One needs non-negativity of all the principal minors.

0 0
P =
0 1
We have p11 = 0 and det(P ) = 0. However,
x P x = |x2 |2
hence, P is not psd. Actually, P is nsd.
Fact 28 Consider any m n complex matrix M and let P = M M . Then
(a) P is hermitian and P 0
(b) P > 0 if and only if rank M = n.
Example 242
1 1
P =
1 1
Since
1
P = 1 1
1
and
rank 1 1 =1
P 0 but P is not pd.
Exercise 173 (Optional)
Suppose P is hermitian and T is invertible. Show that P > 0 if and only if T P T > 0.
Exercise 174 (Optional)
Suppose P and Q are two hermitian matrices with P > 0. Show that P + Q > 0 for
all real sufficiently small; that is, there exists > 0 such that whenever || < , one has
P + Q > 0.

452
28.5 Singular value decomposition
In this section we introduce a matrix decomposition which is very useful for reliable numerical
computations in linear algebra. First, some notation.
The diagonal elements of an m n matrix A are
A11 , A22 , . . . , App where p = min{m, n}
We say that A is diagonal if all its nondiagonal elements are zero, that is, Aij = 0 for i 6= j.
Examples are:

1 0 1 0 0 0
1 0 1
, 0 2 , 0 2 0 0 , .
0 2 0
0 0 0 0 0 0
Theorem 35 (Singular value decomposition) If A is a complex m n matrix, then
there exist unitary matrices U Cmm and V Cnn such that
A = U V
where is a diagonal matrix of the same dimension as A and with real nonnegative diagonal
elements:
1 2 . . . p 0 p = min{m, n}
If A is real, then U and V are real.

So, if A is square (m = n), then

1 0 0
0 2 0

= .. .. . .
. . ..
0 0 n

When A is fat (m < n), we have

1 0 0 0 0
0 2 0 0 0

= .. .. . .. .. .
. . .. . .
0 0 m 0 0

Finally, a tall matrix (m > n) yields

1 0 0
0 2 0
.. .
..
. . ..

= 0 0 n .

0 0 0
.. .. ..
. . .
0 0 0

453
 The scalar i is called the i-th singular value of A. Let r be the smallest nonzero singular
value of A. Then r is smaller than both m and n. Moreover, can be expressed as

11 0
=
0 0

where 11 is the square diagonal invertible matrix given by

1 0 . . . 0
0 2 0

11 = . . ..
0 0 . .
0 0 . . . r

If
11 0
U V
0 0
is a singular value decomposition of A, then
2
11 0
A A=V V
0 0

Hence, the nonzero singular values of A are uniquely given by

p
i = i

where 1 , . . . , r are the nonzero eigenvalues of the hermitian positive semidefinite matrix
A A.

Example 243 The matrix

1.2 0.64 0.48
A=
1.6 0.48 0.36

has the following singular value decomposition

1 0 0
0.6 0.8 2 0 0
A= 0 0.8 0.6
0.8 0.6 0 1 0
| {z }| {z } 0 0.6 0.8
U | {z }
V

Useful facts

The maximum (or largest) singular value of a matrix satisfies the properties of a norm.
Thus, if we consider the set of m n matrices A and define

||A|| = max (A)

454
where max (A) denotes the largest singular of A, then the following properties hold:

(a) ||A|| 0 and kAk = 0 if and only if A = 0

(b) ||Ak = ||kAk
(c) kA + Bk kAk + kBk .

In addition to the above usual norm properties, this norm also satisfies

kAxk
kAk = sup
x6=0 kxk

where the above vector norms are the usual euclidean vector norms. Thus,

kAxk kAkkxk

for all x.

Using the above norm, we can define the distance between two matrices A and B of the
same dimensions as kA Bk.

The rank of A equals the number of its nonzero singular values.

Suppose k is a nonegative integer strictly less than the rank r of A. Then in a certain
sense (which can be made precise), a matrix of rank k which is closest to A is given by

Ak = U k V

where k is the diagonal matrix with diagonal elements

1 , 2 , . . . , k , 0, . . . , 0
| {z }
pk zeros

that is, the first k diagonal elements of k are the same as those of and the remaining
diagonal elements are zero. Also, the distance between Ak and A is k+1 . So, the smallest
nonzero singular value r of A is a measure of the distance between A and the closest matrix
of rank r1.

Suppose A has rank r and

U = u1 u2 . . . u m , V = v1 v2 . . . vn

Then the vectors

u1 , u 2 , . . . , u r
form an orthonormal basis for the range of A and the vectors

v r+1 , v r+2 , . . . , v n

form an orthonormal basis for the null space of A

455
MATLAB

>> help svd

SVD Singular value decomposition.

[U,S,V] = SVD(X) produces a diagonal matrix S, of the same
dimension as X and with nonnegative diagonal elements in
decreasing order, and unitary matrices U and V so that
X = U*S*V.

By itself, SVD(X) returns a vector containing diag(S).

[U,S,V] = SVD(X,0) produces the "economy size"

decomposition. If X is m-by-n with m > n, then only the
first n columns of U are computed and S is n-by-n.

456
Example 244
A =

1 1 1
1 -1 0
1 1 1
1 1 1

[u s v] = svd(A)

u =
0.5774 -0.0000 -0.8165 -0.0000
-0.0000 -1.0000 -0.0000 -0.0000
0.5774 -0.0000 0.4082 -0.7071
0.5774 -0.0000 0.4082 0.7071

s =
3.0000 0 0
0 1.4142 0
0 0 0
0 0 0

v =
0.5774 -0.7071 0.4082
0.5774 0.7071 0.4082
0.5774 0.0000 -0.8165
So A has rank 2 and a basis for its range is given by

0.5774 0
0
, 1 .
0.5774 0
0.5774 0

A basis for the null space of A is given by

0.4082
0.4082
0.8165

457
458
Bibliography

[1] Chen, C.T., Linear System Theory and Design, 3rd Ed., Oxford University Press, 1998.

[2] T. Kailath, T., Linear Systems, Prentice-Hall Information and Systems Science Series,
1979.

[3] Khalil, H., Nonlinear Systems, 3rd Ed., Prentice-Hall, 2002.

[4] Vidasagar, M., Nonlinear Systems Analysis, 2nd Ed., Prentice-Hall, 1993.

459