Eigenvalue Problems

1. Eigenvalues and eigenvectors

2. Vector spaces
3. Linear transformations
4. Matrix diagonalization
 The Eigenvalue Problem

Consider a nxn matrix A

Vector equation: Ax = x
» Seek solutions for x and 
»  satisfying the equation are the eigenvalues
» Eigenvalues can be real and/or imaginary; distinct and/or
repeated
» x satisfying the equation are the eigenvectors

Nomenclature
» The set of all eigenvalues is called the spectrum
» Absolute value of an eigenvalue:
 j  a  ib   j  a2  b2

» The largest of the absolute values of the eigenvalues is

called the spectral radius
 Determining Eigenvalues

Vector equation
» Ax = x  (A-x = 0
» A- is called the characteristic matrix

Non-trivial solutions exist if and only if:
a11   a12  a1n
a21 a22    a2 n
det(A  I )  0
   
an1 an 2  ann  

» This is called the characteristic equation


Characteristic polynomial
» nth-order polynomial in 
» Roots are the eigenvalues {1, 2, …, n}
 Eigenvalue Example

Characteristic matrix
1 2  1 0 1   2 
A  I        
3  4 0 1   3  4  

Characteristic equation
A  I  (1   )(4   )  (2)(3)  2  3  10  0


Eigenvalues: 1 = -5, 2 = 2
 Eigenvalue Properties

Eigenvalues of A and AT are equal

Singular matrix has at least one zero eigenvalue

Eigenvalues of A-1: 1/1, 1/2, …, 1/n

Eigenvalues of diagonal and triangular matrices are
equal to the diagonal elements

Trace
n
Tr ( A)    j
j 1


Determinant
n
A  j
j 1
 Determining Eigenvectors

First determine eigenvalues: {1, 2, …, n}

Then determine eigenvector corresponding to
each eigenvalue:
( A  I )x  0  ( A  k I )x k  0


Eigenvectors determined up to scalar multiple

Distinct eigenvalues
» Produce linearly independent eigenvectors

Repeated eigenvalues
» Produce linearly dependent eigenvectors
» Procedure to determine eigenvectors more complex (see
text)
» Will demonstrate in Matlab
 Eigenvector Example

Eigenvalues
1 2  1  5
A   2
3  4  2


Determine eigenvectors: Ax = x
x1  2 x2  x1 (1   ) x1  2 x2  0

3 x1  4 x2  x2 3 x1  (4   ) x2  0

Eigenvector for 1 = -5
6 x1  2 x2  0  0.3162 1
 x1    or x1   
3 x1  x2  0  0.9487   3

Eigenvector for 1 = 2
 x1  2 x2  0 0.8944  2
 x2    or x 2   
3 x1  6 x2  0 0.4472 1 
 Matlab Examples
>> A=[ 1 2; 3 -4]; >> A=[2 5; 0 2];
>> e=eig(A) >> e=eig(A)
e= e=
2 2
-5 2
>> [X,e] = eig(A) >> [X,e]=eig(A)
X= X=
0.8944 -0.3162 1.0000 -1.0000
0.4472 0.9487 0 0.0000
e= e=
2 0 2 0
0 -5 0 2
 Vector Spaces

Real vector space V
» Set of all n-dimensional vectors with real elements
» Often denoted Rn
» Element of real vector space denoted x  V

Properties of a real vector space
» Vector addition
ab  ba a0a
(u  v)  w  u  ( v  w ) a  (a)  0
» Scalar multiplication

c(a  b)  ca  cb c(ka)  (ck )a

(c  k )a  ca  ka 1a  a
 Vector Spaces cont.

Linearly independent vectors
» Elements: a (1) , a (2) ,  , a (m)  V
» Linear combination: c1a (1)  c2a (2)    cma (m)  0
» Equation satisfied only for cj = 0

Basis
» n-dimensional vector space V contains exactly n linearly
independent vectors
» Any n linearly independent vectors form a basis for V
» Any element of V can be expressed as a linear
combination of the basis vectors

Example: unit basis vectors in R3
1 0  0  c1 
x  c1a (1)  c2a (2)  c3a ( 3)  c1 0  c2 1  c3 0  c2 
0 0 1 c3 
 Inner Product Spaces

Inner product
n
a b  (a, b)  a  b   ak bk  a1b1  a2b2    anbn
T

k 1

Properties of an inner product space
(q1a  q2b, c)  q1 (a, c)  q2 (b, c)
(a, c)  (c, a)
(a, a)  0 (a, a)  0 if and only if a  0

Two vectors with zero inner product are called orthogonal

Relationship to vector norm
» Euclidean norm

a  (a, a)  aT a  a12  a22    an2

» General norm
(a, b)  a b ab  a  b
» Unit vector: ||a|| = 1
 Linear Transformation

Properties of a linear operator F
F ( v  x)  F ( v )  F (x) F (cx)  cF (x)

» Linear operator example: multiplication by a matrix

» Nonlinear operator example: Euclidean norm

Linear transformation
Elements x  Rn , y  Rm
Operator A  R mxn
Transformation y  Ax


Invertible transformation
Dimensions x  R n , y  R n , A  R nxn
Transformation y  Ax
Inverse Transformation x  A 1y

» Often called a coordinate transformation

 Orthogonal Transformations

Orthogonal matrix
» A square matrix satisfying: AT = A-1
» Determinant has value +1 or -1
» Eigenvalues are real or complex conjugate pairs with
absolute value of unity
» A square matrix is orthonormal if:
0 if j  k
a j ak  
T

1 if j  k

Orthogonal transformation
» y = Ax where A is an orthogonal matrix
» Preserves the inner product between any two vectors
u  Aa, v  Ab  u  v  a  b
» The norm is also invariant to orthogonal transformation
a  u b  v
 Similarity Transformations

Eigenbasis
» If a nxn matrix has n distinct eigenvalues, the
eigenvectors form a basis for Rn
» The eigenvectors of a symmetric matrix form an
orthonormal basis for Rn
» If a nxn matrix has repeated eigenvalues, the
eigenvectors may not form a basis for Rn (see text)

Similar matrices
» Two nxn matrices are similar if there exists a
nonsingular nxn matrix P such that: A ˆ  P 1AP
» Similar matrices have the same eigenvalues
» If x is an eigenvector of A, then y = P-1x is an
eigenvector of the similar matrix
 Matrix Diagonalization

Assume the nxn matrix A has an eigenbasis

Form the nxn modal matrix X with the eigenvectors
of A as column vectors: X = [x1, x2, …, xn]

Then the similar matrix D = X-1AX is diagonal with
the eigenvalues of A as the diagonal elements

 a11 a12  a1n  1 0  0

a a22  a2 n  0   0 
A  21
 D  X 1AX   2

        
   
an1 an 2  ann  0 0  n 


Companion relation: XDX-1 = A
 Matrix Diagonalization Example

1 2  1  2
A  1  5, x1    2  2, x 2   
3  4  3 1 
 1 2 1 1  2
X   x1 x 2    
1
X   
 3 1 7 3 1 
1 1 1  2 1 2   1 2
D  X AX       
7 3 1  3  4   3 1 
1 1 1  2  5 4  5 0
D  X AX   
7 3 1   15 2  0 2
  
 Matlab Example
>> A=[-1 2 3; 4 -5 6; 7 8 -9];
>> [X,e]=eig(A)
X=
-0.5250 -0.6019 -0.1182
-0.5918 0.7045 -0.4929
-0.6116 0.3760 0.8620
e=
4.7494 0 0
0 -5.2152 0
0 0 -14.5343
>> D=inv(X)*A*X
D=
4.7494 -0.0000 -0.0000
-0.0000 -5.2152 -0.0000
0.0000 -0.0000 -14.5343