Control Systems

Linear Algebra topics

L. Lanari

Wednesday, October 8, 2014

Matrices
Terminology
and notation

M =

m11
m21
..
.

m12
m22
..
.

mp1

mp,q1

M = {mij : i = 1, . . . , p
transpose

vector (column)
row vector
scalar

mij

&

mpq

j = 1, . . . , q}

11
M :pp

rectangular matrix

M :pq

Wednesday, October 8, 2014

= mji

v :n1
vT : 1 n

square matrix

Lanari: CS - Linear Algebra

mij

m1q
m2q
..
.

(p = q)
2

Matrices
diagonal matrix

block diagonal matrix

m1

0
0

m2
..
.

M1
0
0

0
M2
0

m11

m12

m22
..
.

upper triangular matrix

upper block triangular

matrix
Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

..
.
..
.
0

0
0

M11
0

0
mn

0
0
M3

..
.
..
.
0

M12
M22

m1n
m2n
..
.
mnn

M = diag{mi },

i = 1, . . . , p

M = diag{Mi },

i = 1, . . . , k

lower triangular matrix

strictly lower triangular matrix

lower block triangular matrix

strictly lower block triangular matrix
3

Matrices

a11
..
.

an1

..
.

gives

1
a1n
.. ..
. .

ann

a11 1
..
.

an1 1

det

det

M11
0
M1
0

M12
M22
0
M2

..
.

a11
1 0
0
.. = ..
.. ..
..
.
.

.
.
.
.
.
.
an1
n
0 n

a11 1 a1n 1
a1n n
..
.. = ..
.
.
.
.
. .
an1 n ann n
ann n

..
.

a1n
..
.

ann

= det(M11 ) det(M22 )

= det(M1 ) det(M2 )

determinant of a diagonal matrix = product of the diagonal terms

special case

special case

useful for the eigenvalue computation

Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

Matrices
square matrix

1 1

(AB)

non-singular
(invertible)

det(M ) = 0

=A
= B 1 A1

M = diag{mi }

1
= diag{ }
mi

A1 A = AA1 = I
A0 = I

T
det A = det A
Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

Eigenvalues & Eigenvectors

A
u

Au

generic transformation

Au
u

particular directions
such that Au is parallel to u

Au = u

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

scalar
(scaling factor)

or A u

Eigenvalues & Eigenvectors

eigenvalue i

Aui = i ui
eigenvector ui
for non-trivial solution

ui = 0

(i I A)ui = 0 (scaling)

det(i I A) = 0

characteristic polynomial (order n = dim(A))

pA () = det(I A) = n + an1 n1 + an2 n2 + + a1 + a0

eigenvalue i solution of

pA () = det(I A) = 0

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

Eigenvalues & Eigenvectors

pA () = det(I A) = n + an1 n1 + an2 n2 + + a1 + a0
polynomial with real coefficients.
The set of the n solutions of pA () = 0

generic eigenvalue

is defined as the spectrum of A: (A)

i R
i C

also i is a solution

(i , i ) pairs

therefore
if

i R

i C

then

ui real components
ui complex components

eigenvalue i solution of pA () = 0 with multiplicity ma(i )

Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

ui
algebraic
multiplicity
8

Eigenvalues & Eigenvectors

special cases

0
0

diagonal
matrix

triangular
matrix

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

m1

0
m2
..
.

m11

m12

m22
..
.

0
0

..
.
..
.
0

..
.
..
.
0

0
mn
m1n
m2n
..
.
mnn

eigenvalues = {mi}

elements on the
main diagonal

eigenvalues = {mii}

Eigenvalues & Eigenvectors

A

similarity transformations

T AT 1 same eigenvalues as A

(proof)

det(T ) = 0
eigenvalues are invariant under similarity transformations
eigenvector associated to i is not unique

A(ui ) = Aui = i ui = i (ui )

all belong to the same linear subspace

Aui = i ui
viT A = viT i = i viT
with

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

ui

right eigenvector

viT

left eigenvector

viT uj = ij

10

Eigenvalues & Eigenvectors

i eigenvalue
with ma(i ) > 1

1
A1 =
0

1
A2 =
0

one or more linearly independent eigenvectors

0
1

pA () = ( 1 )2

,
,

u11 =

1
0

with = 0,

u12 =
only

0
1

u1 =

1
0

A1 and A2 same eigenvalues with same m.a.

Linear subspace (eigenspace) associated to an eigenvalue i

Vi = {u Rn |Au = i u}
dim(Vi ) = mg(i )

geometric multiplicity

mg(i ) = dim [Ker(A i I)] = n rank(A i I)

Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

11

Eigenvalues & Eigenvectors

useful property
A1
A2

=
=

1 mg(i ) ma(i ) n

1
0

0
1

1
0

0.5 0 0.5
P = 0 1 0
0.5 0 0.5

, (A1 1 I) =
, (A2 1 I) =

0
0

0
0

0
0

projection matrix


0
1
u1 = 1 u2 = 0
1 = 2 = 1
0
1
ma(1 ) = 2 = mg(1 )

1
3 = 0
u3 = 0
1

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

mg(1 ) = 2 = ma(1 )
mg(1 ) = 1 < ma(1 )

u3
u2
u1
45

12

Diagonalization
Def.
An (n x n) matrix A is said to be diagonalizable if there exists
an invertible (n x n) matrix T such that TAT -1 is a diagonal matrix
Th.
An (n x n) matrix A is diagonalizable if and only if it has n
independent eigenvectors
Since the eigenvalues are invariant under similarity transformations
if A diagonalizable
T AT 1 = = diag{i }, i = 1, . . . , n

eigenvalues of A
Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

13

Diagonalization
We need to find T
i

ui

n linearly independent
(by hyp.)

Aui = i ui ,

in matrix form

A u1 u2

u1

u2

non-singular

un

i = 1, . . . , n

un

u1

u2

AU = U

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

U=

un

0
2

..
.

0
0

14

Diagonalization
AU = U being U non-singular, we can define T such that T 1 = U
AT 1 = T 1 A = T 1 T = T AT 1

therefore the diagonalizing similarity transformation is T s.t.

= U = u1

u2

un

A: (n x n) is diagonalizable if and only if

mg(i) = ma(i) for every i

Distinct eigenvalues (real and/or complex)

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

=
=

A diagonalizable

15

Diagonalization
(i , i )

Complex eigenvalues?

i = i + ji

eigenvalue

ui = uai + jubi

eigenvector

2 choices
diagonalization
T

= ui

ui

Di = T AT

i
=
0

0
i

complex
elements

or real block 2 x 2
T

= uai

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

ubi

Mi = T AT

i
=
i

i
i

real
elements

real system representation for complex eigenvalues

16

Diagonalization
Simultaneous presence of real and complex eigenvalues
If A diagonalizable, there exists a non-singular matrix R such that

RAR1 = diag {r , Mr+1 , Mr+3 , . . . , Mq1 }

real eigenvalues

r = diag{1 , . . . , r }
Mi = T AT

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

i
=
i

i
i

complex eigenvalues

17

Spectral decomposition
Hyp: A diagonalizable

or Eigendecomposition

A = U U 1

columns (linearly independent)

v1T
v2T

rows

U = u1 u2 un
U 1 = .
..
T
v
n
U 1 U = I viT uj = ij , i, j = 1, . . . , n
T
v1
v2T

1
A = U U = 1 u 1 2 u 2 n u n .
..

vnT

spectral form of A

A=

i ui viT

i=1

Lanari: CS - Linear Algebra

Wednesday, October 8, 2014

18

Spectral decomposition
A=

i ui viT

i=1

column

A=

2 1
0 1

1 = 2

row

(n x n)

is the projection matrix on the

invariant subspace generated by ui

1
u1 =
0

v1T = 1 1

2 = 1
P1 =

Pi = ui viT

u2 =

1 1
0 0

1
1

v2T = 0

P2 =

0
0

1
1

P1v
v
u1
u2
P2v

v=
Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

2
1
19

Not diagonalizable case

Since in general 1 mg(i ) ma(i ) n
if A not diagonalizable then

mg(i ) < ma(i )

mg(i )

Jordan
block
of dimension nk

i
0
..
.

Jk =

0
0

blocks of dimension nk

1
i
..
.

0
1
..
.

i
0

..

0
0
..
.

the knowledge of
this dimension is
out of scope

nk nk

1
i

Jk i I Null space has dimension 1

Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

20

Not diagonalizable case

Generalized eigenvector

chain of nk generalized eigenvectors

Jordan canonical form (block diagonal)

example: unique eigenvalue i of matrix A (n x n)
mg(i ) = p
ma(i ) = n =

nk

i=1

T :

T AT

=J =

J1
..

.
Jp

with Jk Rnk nk Jordan block of dim nk

Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

21

Special cases

if ma(i) = 1 then mg(i) = 1

if mg(i) = 1 then only one Jordan block of dimension ma(i)

1
0

1
1

ma(1 ) = 2

1 = 1

mg(1 ) = 1

if i unique eigenvalue of matrix A and if rank(iI - A) = ma(i) - 1

then only one Jordan block of dimension ma(i)
From the rank-nullity theorem
A i I : Rn Rn
dim (Rn ) = dim (Ker(A i I)) + dim (Im(A i I))

n
Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

nullity(A - iI)

rank(A - iI)

22

Summary
A
diagonalizable

T s.t. T AT 1 =

mg(i) = ma(i)
for all i

= diag{i }

for real & complex i

r = diag{1 , . . . , r }

alternative choice
for complex i

i i
Mi =
i i
A=

i ui viT

spectral form

i=1

Jordan blocks
A
not diagonalizable
mg(i) < ma(i)
Lanari: CS - Linear Algebra
Wednesday, October 8, 2014

T s.t.

T AT 1 = diag{Jk }

block diagonal

i
..
.
Jk =
0
0

1
..
.

..
.
i
0

0
..
.
Rnk nk
1
i
23