ENGMAE 200A: Engineering Analysis I

Matrix Eigenvalue Problems

Instructor: Dr. Ramin Bostanabad

ROADMAP

• Definitions and how to calculate eigenvalues and eigenvectors

• A few examples with useful results

• Eigenbases and their applications in:

• Similar matrices

• Diagonalization of matrices

• Matrix Classification

2
DEFINITIONS

• What is the difference between the results of these multiplications:

 
vs.

• A matrix eigenvalue problem considers the vector equation:

Ax = λx (1)
• Here A is a given square matrix, λ an unknown scalar, and x an
unknown vector. In a matrix eigenvalue problem, the task is to
determine λ’s and x’s

3
TERMINOLOGY
• λ’s for which (1) has a solution x ≠ 0 is called: an eigenvalue,
characteristic value, or latent root.
• Solutions x ≠ 0 of (1) are called the eigenvectors, characteristic
vectors of A corresponding to that eigenvalue λ.
• The set of all the eigenvalues of A is called the spectrum of A. We
shall see that the spectrum consists of at least one eigenvalue and at
most of n numerically different eigenvalues.
• The largest of the absolute values of the eigenvalues of A is called the
spectral radius of A.
4
HOW TO FIND EIGENVALUES AND EIGENVECTORS
We use an example:
  −5 2
[
𝑨=
2 −2 ]
1
𝑥1 𝑥1  −5 𝑥 1+2 𝑥 2= 𝜆 𝑥 1
  −5 2
𝐀𝐱=
2 [ −2 ][ ] [ ]
𝑥2
=𝜆
𝑥2
2
2 𝑥 1 − 2 𝑥 2= 𝜆 𝑥 2
3
  −5 − 𝜆 ¿ 𝑥 4
( 1 +             2 𝑥 2=0
2 𝑥 1 +(− 2− 𝜆) 𝑥 2=0
( 𝐀 − 𝜆 𝐈 ¿ 𝐱=0
 

 
−5−𝜆 2 2
5
𝐷(𝜆)=det(𝐀−𝜆𝐈)=¿ ∨¿=(−5−𝜆)(−2−𝜆)−4=𝜆 +7𝜆+6=0
2 −2−𝜆 5
HOW TO FIND EIGENVALUES AND EIGENVECTORS
• D(λ) is the characteristic determinant or, if expanded, the
characteristic polynomial.
• D(λ) = 0 the characteristic equation of A.

Example Continued:
• Solutions of D(λ) = 0 are λ1 = −1 and λ2 = −6. These are the
eigenvalues of A.
• Eigenvectors for λ1 = −1:
 − 5 𝑥 1+2 𝑥 2= 𝜆 𝑥 1 − 4 𝑥1 +2 𝑥2 =0 1
2 𝑥 1 − 2 𝑥 2= 𝜆 𝑥 2
→
2 𝑥 1 − 𝑥2 =0 []
→ 𝑥 2=2 𝑥 1 → 𝒙 1=
2
 
• Eigenvectors for λ2 = −6:
show that
6
OPENING EXAMPLE REVISITED

• What is the difference between the results of these multiplications:

 
vs.

• Obtain the eigenvalues and eigenvectors.

• The eigenvalues are {10, 3}. Corresponding eigenvectors are [3 4] T

and [−1 1]T, respectively.

7
GENERAL CASE

• How to find the eigenvalues and eigenvectors of:

Ax = λx (1)
a11 x1    a1n xn   x1 ( a11   )x1  a12 x2    a1n xn  0
a21 x1    a2 n xn   x2 a21 x1  ( a22   )x2    a2 n xn  0
 
an1 x1    ann xn   xn . an1 x1  an 2 x2    ( ann   )xn  0.

a11   a12  a1n

a21 a22    a2 n
( A   I ) x  0. D( )  det( A   I)   0.
   
an1 an 2  ann  
8
EXAMPLE
Find the eigenvalues and eigenvectors:
  −2 2 −3

[
𝐀= 2
−1
1
−2
−6
0 ]
 •
Characteristic equation: −λ3 − λ2 + 21λ + 45 = 0
• Roots: λ1 = 5, λ2 = λ3 = −3.
• Form and then use Gauss elimination:
  −7 2 −3   − 7 2 −3

 
[
𝐀 − 𝜆𝟏 𝐈 = 𝐀 − 5 𝐈 = 2
−1
−4
−2 −5][
−6 → 0
0
− 24/ 7
0
− 48/ 7
0 ]
• Hence, the rank of is 2. 9
EXAMPLE CONTINUED
 
• Choosing x3 = −1 we have x2 = 2 from and then x1 = 1 from −7x1 + 2x2
− 3x3 = 0. So:
x1 = [1 2 −1]T

• For λ = −3 the characteristic matrix:

  1 2 −3 1 2 −3

[
𝐀 − 𝜆 𝐈 = 𝐀 +3 𝐈 = 2
−1
4
−2 3 ][
−6 → 0
0
0
0
0
0 ]
10
EXAMPLE CONTINUED
  1 2 −3 1 2 −3

[
𝐀 − 𝜆 𝐈 = 𝐀 +3 𝐈 = 2
−1
4
−2 3][
−6 → 0
0
0
0
0
0 ]
• From x1 + 2x2 − 3x3 = 0 we have x1 = −2x2 + 3x3. Choosing x2 = 1, x3 = 0

and x2 = 0, x3 = 1, we obtain two linearly independent eigenvectors of A

corresponding to λ = −3:

 
and

11
EXAMPLE
Find the eigenvalues and eigenvectors:

  0 1
[
𝐀=
−1 0 ]
 
• , and
• and .

 •
What are the eigenvalues and eigenvectors of ?

12
EXAMPLE ON LAND USE REVISITED
 
The 2004 state of land use in a city of of built-up area is:
C: Commercially Used 25%
I: Industrially Used 20%
R: Residentially Used 55%

• The transition probabilities for 5-year intervals are given by A and

remain practically the same over the time considered

13
EXAMPLE ON LAND USE
What is the limit state?
 

• Definition of limit state

• How to find it systematically

 
• Solution: Ax=x so (verify!). Then .

14
EIGENVALUES OF SPECIAL MATRICES
Symmetric, Skew-Symmetric, and Orthogonal:

A real square matrix A = [ajk] is called symmetric if transposition leaves it

unchanged:
(1) AT = A, thus akj = ajk,

skew-symmetric if transposition gives the negative of A:

(2) AT = −A, thus akj = −ajk,

orthogonal if transposition gives the inverse of A:

(3) AT = A−1. 15
EIGENVALUES OF SPECIAL MATRICES
Theorem: Eigenvalues of Symmetric and Skew-Symmetric Matrices
(a) The eigenvalues of a symmetric matrix are real.

(b) The eigenvalues of a skew-symmetric matrix are pure imaginary or

zero.
Examples:

  −3 1 5   0 9 − 12
𝐀= 1
5[ 0
−2
−2
4 ] [
𝐁= − 9
12
0
−2 0
20
0 ]
16
USEFUL THEOREMS
 
We accept and use the following without proof:
1. The eigenvalues of a real matrix are real or complex conjugate in
pairs.
2. has an inverse if and only if all its eigenvalues are nonzero.
For with eigenvalues
3. The eigenvalues of are the inverse of those of .
4. The sum of the main diagonal entries, called the trace of A, equals the
sum of the eigenvalues of A
5. Eigenvalues of are
17
ORTHOGONAL TRANSFORMATIONS
Orthogonal transformations: Transformations like y = Ax where A is an
orthogonal matrix.
• With each vector x in Rn such a transformation assigns a vector y in
Rn.
• Example: The plane rotation through an angle θ

  𝑦1 cos 𝜃 − sin 𝜃 𝑥1
𝐲=
[ ] [
𝑦2
=
sin 𝜃 cos 𝜃 ][ ]
𝑥2

is an orthogonal transformation.

18
THEOREM: INVARIANCE OF INNER PRODUCTS
An orthogonal transformation preserves the value of the inner product:
  𝑏1
T
𝐚 ∙ 𝐛=𝐚 𝐛= [ 𝑎1 ⋯ 𝑎𝑛 ] ⋮
[]
𝑏𝑛

That is, for any a and b in Rn, orthogonal n × n matrix A, and u = Aa, v =
Ab we have u · v = a · b.
Hence the transformation also preserves the length or norm of any
vector a in Rn
  T
‖𝐚 ‖=√ 𝐚 𝐚
19
MORE THEOREMS
• Orthonormality of Column and Row Vectors: A real square matrix is
orthogonal if and only if its column vectors a1, … , an (and also its row
vectors) form an orthonormal system:
 
0     if    𝑗≠ 𝑘  
𝑇
𝐚 𝐚𝑘 =
𝑗 {
1     if    𝑗=𝑘
• Determinant of an Orthogonal Matrix: Has the value +1 or −1.

• Eigenvalues of an orthogonal matrix A: Real or complex conjugates

in pairs and have absolute value 1.

20
EXAMPLE
Orthogonal matrix:
  2 1 2

[
𝐀= −2
1
2
2 −2]
1 ×
1
3

 
• Characteristic equation:
• One of the eigenvalues must be real (why?)
• What are the other eigenvalues? What is their norm?

21
ROADMAP

• Definitions and how to calculate eigenvalues and eigenvectors

• A few examples with useful results

• Eigenbases and their applications in:

• Similar matrices

• Diagonalization of matrices

• Matrix Classification

22
BASIS OF EIGENVECTORS
Theorem: If an n × n matrix A has n distinct eigenvalues, then A has a
basis of eigenvectors x1, … , xn for Rn.

Theorem: A symmetric matrix has an orthonormal basis of eigenvectors

for Rn.

Example:
  −2 2 −3 λ1 = 5   1 −2 3
𝐀= 2
−1[ 1
−2
−6
0 ] Slide 8
λ2 = −3
λ3 = −3
[ ] [ ] []
𝐱1 = 2 , 𝐱 2= 1 , 𝐱 3= 0
−1 0 1

23
SIMILARITY OF MATRICES
Definition: An n × n matrix  is called similar to an n × n matrix A if

 = P−1AP

for some (nonsingular!) n × n matrix P. This transformation, which gives Â

from A, is called a similarity transformation.

Theorem: If  is similar to A, then  has the same eigenvalues as A.

Furthermore, if x is an eigenvector of A, then y = P−1x is an eigenvector of
 corresponding to the same eigenvalue.

24
EXAMPLE
Let’s revisit the matrix in slide 2:
  6 3
[
𝑨=
4 7 ]
 
Let’s choose which gives

 
Then:

25
DIAGONALIZATION OF A MATRIX
Theorem: If an n × n matrix A has a basis of eigenvectors, then

D = X−1AX

is diagonal, with the eigenvalues of A as the entries on the main diagonal.

Here X is the matrix with these eigenvectors as column vectors.

Also:

Dm = X−1AmX (m = 2, 3, … ).

26
EXAMPLE
Diagonalize:
      7.3 0.2 −3.7

[
𝐀 = −11.5
  17.7
1.0
1.8
   5.5
− 9.3 ]
Solution:

  − 0.7   0.2   0.3 −3 −4 0 3 0 0

−1

[
𝐃= 𝐗 𝐀𝐗 = − 1.3
  0.8
− 0.2
  0.2
  0.7
− 0.2 ][ 9
−3
4
− 12 0][
0 = 0
0
−4
0
0
0 ]
27
QUADRADIC FORMS
Definition: A quadratic form Q in the components x1, … , xn of a vector x
is a sum n2 of terms: n n
Q  x Ax   a jk x j xk
T

j 1 k  1

 a11 x12  a12 x1 x2    a1n x1 xn

 a21 x2 x1  a22 x2 2    a 2 n x2 xn
 
 an1 xn x1  an 2 xn x2    ann xn 2

A = [ajk] is called the coefficient matrix of the form. We may assume that
A is symmetric (?).
28
EXAMPLE
Let
 3 4   x1 
x Ax   x1 x2  
T
 x 
 6 2  2
 3 x12  4 x1 x2  6 x2 x1  2 x2 2
 3 x12  10 x1 x2  2 x2 2 .

Here 4 + 6 = 10 = 5 + 5. So:
 3 5   x1 
x Cx   x1 x2  
T
 x 
 5 2  2
 3 x12  5 x1 x2  5 x2 x1  2 x2 2
 3 x12  10 x1 x2  2 x2 2 .
29
MANIPULATING QUADRATIC FORMS
 
• Say you are given something like this:

• How can you convert it to a canonical form:

• Applications are many! We will look into an important one.

30
MANIPULATING QUADRATIC FORMS
• Symmetric coefficient matrix A has an orthonormal basis of
eigenvectors (Theorem 2 on slide 23). So, if we take these as column
vectors, we obtain a matrix X that is orthogonal, so that X−1 = XT:
A = XDX−1 = XDXT.
• Substitution: Q = xTXDXTx.
• Set XTx = y. Since X−1 = XT, we have X−1x = y and so x = Xy.

• We also have xTX = (XTx)T = yT and XTx = y.

• Now Q becomes

Q = yTDy = λ1y12 + λ2y22 + … + λnyn2 31

THEOREM
Principal Axes Theorem: The substitution x = Xy transforms a quadratic
form
n n
Q  x T Ax   a jk x j xk ( akj  a jk )
j 1 k 1

to the principal axes form or canonical form

Q = yTDy = λ1y12 + λ2y22 + … + λnyn2

λ1, … , λn are the (not necessarily distinct) eigenvalues of the (symmetric!)

matrix A, and X is an orthogonal matrix with corresponding eigenvectors
x1, … , xn, respectively, as column vectors.

32
EXAMPLE
What type of conic section the following quadratic form represents
Q  17 x12  30 x1 x2  17 x2 2  128.

 17 15   x1 
Solution. We have Q = xTAx, where A  , x   .
 15 17   x2 

Characteristic equation (17 − λ)2 − 152 = 0 with roots λ1 = 2, λ2 = 32. So:

y12 y2 2
Q  2 y1  32 y2 .
2 2
2
 2  1.
8 2

What is the direction of the principal axes in the x 1x2-coordinates?

33
COMPLEX MATRICES
A square matrix A = [akj] is called

Hermitian if ĀT = A, that is, ākj = ajk

skew-Hermitian if ĀT = −A, that is, ākj = −ajk
unitary if ĀT = A−1

• These are generalization of symmetric, skew-symmetric, and

orthogonal matrices in complex spaces.

• For example (Theorem on invariance of Inner Product): The unitary

transformation y = Ax with a unitary matrix A, preserves the value of
the inner product and norm.
34
GENERALIZING THE THEOREMS
• The eigenvalues of a Hermitian matrix (and thus of a symmetric matrix)
are real.

• The eigenvalues of a skew-Hermitian matrix (and thus of a skew-

symmetric matrix) are pure imaginary or zero.

• The eigenvalues of a unitary matrix (and thus of an orthogonal matrix)

have absolute value 1.

35
ROADMAP

• Definitions and how to calculate eigenvalues and eigenvectors

• A few examples with useful results

• Eigenbases and their applications in:

• Similar matrices

• Diagonalization of matrices

• Matrix Classification

36
MATRIX CLASSIFICATION
 
• Positive (semi-) definite, negative (semi-) definite, and indefinite matrices:

• The Hermitian matrix is said to be positive (negative) definite if the scalar is

strictly positive (negative) for every non-zero , i.e., ().

• Replace with to get the definition for positive (negative) semidefinite.

• For instance: if for any , then is negative semidefinite.

• * denotes conjugate transpose. Replace it via “transpose” if is real.

• How does this connect to “quadratic forms” that we just studied?

37
REVISITING THE EXAMPLE
 
• We reformulated the problem into a sum of quadratic terms:

 
• Can we say anything about the sign of just by looking at for ?

• What does this say about ?

 
• What are the coefficients in ?

• What does this suggest?

38
MATRIX CLASSIFICATION WITH EIGENVALUES
 
The Hermitian matrix is:

• Positive definite if and only if all of its eigenvalues are positive.

• Positive semi-definite if and only if all of its eigenvalues are non-negative.

• Negative definite if and only if all of its eigenvalues are negative

• Negative semi-definite if and only if all of its eigenvalues are non-positive.

• Indefinite if and only if it has both positive and negative eigenvalues.

What if the eigenvalues are complex?

39
EXAMPLES
• Classify the following matrices:

  6 1   3 8   2 4   2 4
[ 1 7 ] [ 8 4 ] [ 4 8 ] [ 4 −8 ]

40
EXTRA

41
SOME THEOREMS
• Theorem 1: The eigenvalues of a square matrix A are the roots of D(λ)
= 0.
• An n × n matrix has at least one eigenvalue and at most n
numerically different eigenvalues.

• Theorem 2: If w and x are eigenvectors of a matrix A corresponding to

the same eigenvalue λ, so are w + x (provided x ≠ −w) and kx for any
k ≠ 0.
• Eigenvectors corresponding to one and the same eigenvalue λ of A,
together with 0, form a vector space, called the eigenspace of A
corresponding to that λ.
• An eigenvector x is determined only up to a constant factor. So, we
can normalize x.
42