Eigenvalues, eigenvectors, diagonalization

Phan Thi Khanh Van

E-mail: [email protected]

May 13, 2021

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 1 / 26
Table of contents

1 Eigenvalues, eigenvectors

2 Diagonalization

3 Leslei model: Population growth

4 Markov chain

5 Application of singular value decomposition (SVD)

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 2 / 26
Eigenvalues, eigenvectors
Let A ∈ Mn be a square matrix . λ is called an eigenvalue of A if there
exist x 6= 0, x ∈ Rn satisfying: Ax = λx. x is then called an eigenvector
of A associated with λ.

Example 1
       
1 2 1 2 1
A= . Are e1 = , e2 = , e3 = eigenvectors of A?
2 1 1 3 −1
    
1 2 1 3
Ae1 = = = 3e1 - eigenvector with λ = 3.
2 1 1 3
    
1 2 2 8
Ae2 = = 6= λe2 - e2 is not an eigenvector.
2 1 3 7
    
1 2 1 −1
Ae3 = = = −e3 - eigenvector with λ = −1
2 1 −1 1

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 3 / 26
Properties
If x is an eigenvector of A associated with the eigenvalue λ then:
1 αx, α 6= 0 is also an eigenvector of A assosiated with λ
2 λm is an eigenvalue of Am , x is the corresponding eigenvector.
3 If A is invertible, then λ 6= 0 and 1/λ is an eigenvalue of A−1 , x is the
corresponding eigenvector.

Example 2
 
1 2
Given A = , is λ = 1 an eigenvalue of A?
0 3

Consider the equation

 Ax = λx = 1.x  =x 
0 2 0 0 2 0
⇔ (A − I )x = 0 ⇔ ∼
0 2 0 0 0 0
(
x1 = m
⇔ .
x2 = 0
We can find x = (m, 0) 6= 0 (if m 6= 0) such that Ax = x.
Hence, λ = 1 is an eigenvalue of A.
(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 4 / 26
Remark
λ is an eigenvalue of A ∈ Mn
⇔ ∃x 6= 0, x ∈ Rn satisfying: Ax = λx
⇔ (A − λI )x = 0 has nontrivial solution
⇔ det(A − λI ) = 0 - characteristic equation of A.
The polynomial PA (λ) = det(A − λI ) is called the characteristic
polynomial of A.

Example 2
 
1 1 m
Find m such that 2 is an eigenvalue of A = −2 1 1
4 −1 −2

2 is an eigenvalue of A ⇔ |A − 2I | = 0
−1 1 m
3
⇔ −2 −1 1 = 0 ⇔ 6m − 9 = 0 ⇔ m = 2
4 −1 −4

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 5 / 26
How to find the eigenvectors, eigenvalues
1 Find PA (λ) = |A − λI |, and then solve |A − λI | = 0 to find λ.
2 For every λi solve the system (A − λi I )x = 0 to find the corresponding
eigenvectors x.
3 The subspace of Rn of all eigenvectors assosiated with λi is called the
eigenspace of A assosiated with λi , and is denoted by Eλi .

Theorem
All vector in the bases of the eigenspaces form a linearly independent set.

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 6 / 26
Example 3
 
2 3
Find all eigenvalues and eigenvectors of: A =
4 3
2−λ 3
|A − λI | = = (2 − λ)(3 − λ) − 12
4 3−λ
= λ2 − 5λ − 6 = 0 ⇔ λ = −1 ∨ λ = 6
λ = −1: Consider
  the system
 (A + I )x = 0  
3 3 0 −1 −1
⇔x =m , Eigenspace: E−1 = span
4 4 0 1 1
λ
 = 6: Consider
 the system  − 6I )x = 0
 (A  
−4 3 0 3 3
⇔x =m , Eigenspace: E6 = span
4 −3 0 4 4

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 7 / 26
Example 4
 
1 2 3
Find all eigenvalues and eigenvectors of: A = −2 6 6 
4 −12 −11

|A − λI | = −λ3 + trace(A)λ2 − (A11 + A22 + A33 )λ + |A|

= −λ3 − 4λ2 − (6 − 23 + 10)λ + 10 = 0
⇔ λ = −5 ∨ λ = −1 ∨ λ = 2
 = −5: Consider the system
λ  (A + 5I )x = 0 
6 2 3 0 0 35 21 0
 −2 11 6 0  ∼  −2 11 6 0 
4 −12 −6 0 0 10 6 0
⇔ x = m(−3, −6, 10), Eigenspace: E−5 = span {(−3, −6, 10)} ,.
 = −1: Consider (A +
λ  I )x = 0 
2 2 3 0 2 2 3 0
 −2 7 6 0 ⇔ 0 9 9 0 
4 −12 −10 0 0 −16 −16 0
⇔ x = m(−1, −2, 2). Eigenspace: E−1 = span {(−1, −2, 2)}
(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 8 / 26
λ = 2: Consider (A − 
 2I )x =
0 
−1 2 3 0 −1 2 3 0
 −2 4 6 0 ⇔ 0 0 0 0 
4 −12 −13 0 0 −4 −1 0
⇔ x = m(10, −1, 4)
Eigenspace: E2 = span {(10, −1, 4)} , dim(E2 ) = 1.

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 9 / 26
Example 5
 
1 3 3
Find all eigenvalues, eigenvectors of A = −3 −5 −3
3 3 1

|A − λI | = −λ3 − 3λ2 − (4 + 4 − 8)λ + 4 = 0

⇔ λ = −2, (multiplicity =2), ∨λ = 1, (multiplicity=1)
λ = 1: Consider (A 
 − I )x = 0
0 3 3 0
 −3 −6 −3 0  ⇔ x = m(1, −1, 1)
3 3 0 0
E1 = span {(1, −1, 1)}, dim(E1 ) = 1
λ = −2: Consider (A
  + 2I )x = 0
3 3 3 0  
 −3 −3 −3 0  ∼ 1 1 1 0
3 3 3 0
⇔ x = m(−1, 1, 0) + n(−1, 0, 1)
E−2 = span {(−1, 1, 0), (−1, 0, 1)}, dim(E−2 ) = 2.
(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 10 / 26
Example 6
 
5 6 −3
Find all eigenvalues, eigenvectors of A = −1 0 1 
2 2 −1

|A − λI | = −λ3 + 4λ2 − (6 − 2 + 1)λ + 2 = 0 ⇔ λ = 1 ∨ λ = 2

 − I )x = 0
λ = 1: Consider (A  
4 6 −3 0 0 2 1 0
 −1 −1 1 0  ∼  −1 −1 1 0 
2 2 −2 0 0 0 0 0
⇔ x = m(3, −1, 2), E1 = span {(3, −1, 2)}, dim(E1 ) = 1
 − 2I )x = 0
λ = 2: Consider (A  
3 6 −3 0 0 0 0 0
 −1 −2 1 0  ∼  −1 −2 1 0 
2 2 −3 0 0 −2 −1 0
⇔ x = m(4, −1, 2) E2 = span {(4, −1, 2)}, dim(E2 ) = 1

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 11 / 26
Exercises
Find the eigenvalues, eigenvectors of the following matrices
 
15 −18 −16
1 A =  9 −12 −8 
4 −4 −6
 
1 −1 −1
2 A = 2 −2 −4
0 −4 −3

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 12 / 26
Similar matrices
A matrix A is called similar to B if there exists an invertible matrix
P : A = PBP −1 .

Diagonalizable matrix
Let A ∈ Mn be a square matrix. A is called diagonalizable if it is similar to
a diagonal matrix: ∃P : P −1 AP = D, where D is a diagonal matrix.

Theorem
1 A is diagonalizable ⇔ A has n independent eigenvectors
2 If A has exactly n distinct eigenvalues then A is diagonalizable.

Theorem
If A is diagonalizable then we can choose P = [e1 ..en ] where the column
vectors ei are independent eigenvectors of A, D = diag (λ1 ...λn ), λi −
corresponding eigenvalues.

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 13 / 26
How to diagonalize a matrix
1 Solve |A − λI | = 0 to find the eigenvalues λ1 , ...λn
2 Solve the system (A − λ)x = 0 to find the corresponding eigenvectors
of A: e1 , e2 ...en
 
λ1 0 ... 0
 0 λ2 ... 0 
D =  ..  , P = [e1 e2 ...en ] (col. vectors)
3
 
. 
0 0 ... λn

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 14 / 26
Consider the examples 3,4,5,6
1. Example 3:
A has 2 independent
 eigenvectors
 ⇒ A is diagonalizableA = PDP −1 .
−1 0 −1 1
D= ,P=
0 6 3 4
2. Example 4:
 ⇒ A is diagonalizable
A has3 distinct eigenvalues
  A = PDP −1 .
−5 0 0 −3 −1 10
D = 0 −1 0 , P = −6 −2 −1
  
0 0 2 10 2 4
3. Example 5:
A has3 independent eigenvectors
 ⇒ A is diagonalizableA = PDP −1 .
−2 0 0 −1 −1 1
D = 0 −2 0 , P = 1
   0 −1
0 0 1 0 1 1
4. Example 6:
A is not diagonalizable.

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 15 / 26
Remark
If A is diagonalizable A = PDP −1 then
An =(PDP −1 )(PDP −1  )...(PDP −1 ) = PD n P −1
λn1 0 ... 0
 0 λn ... 0 
2  −1
= P  .. P

. 
0 0 ... λnn

Consider Example 4
 2018 
5 0 0
A2018 = PD 2018 P −1 =P 0 1 0  P −1
0 0 22018

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 16 / 26
Exercises
Diagonalize the following matrices (if it is possible)
   
2 5 −1 3 −1
1 A=
−1 3 5 A = −3 5 −1.
  −3 3 1
1 2 −2
2 A = 1 0 3 . Find B : B 3 = A
1 3 0 1 0 0
Find A2017
6 A = 1 −1 1
  2 0 1
2 0 −6  
3 A = 1 3 −2 2 2 3
1 0 1 7 A = 2 5 6
  3 6 2
0 1 0  
4 A = −4 4 0 2 −1 −1
−2 1 2
8 A = −1 2 −1
−1 −1 2

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 17 / 26
Population growth

Matrices can be used to form models for population growth. The first step
in this process is to group the population into age classes of equal duration.
For instance, if the maximum life span of a member is L years, then the age
classes are represented by the n intervals: First age class: [0, Ln ), Second
(n−1)L
age class [ Ln , 2L
n ), n−th age class: [ n , L). The number of population
members  in each age class is then represented
 by the age distribution
x1 Number in first age class
 x2 Number in second age class
vector  . . .


xn Number in n-th age class

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 18 / 26
Example 1
A population of rabbits raised in a research laboratory has the
characteristics listed below.
(a) Half of the rabbits survive their first year. Of those, half survive their
second year. The maximum life span is 3 years.
(b) During the first year, the rabbits produce no offspring. The average
number of offspring is 6 during the second year and 8 during the third year.
The laboratory population now consists of 24 rabbits in the first age class,
24 in the second, and 20 in the third. Find a stable age distribution vector

To find a stable age distribution vector, we have to find x : Ax = λx.

|A − λI | = 0 ⇔ λ = −1 ∨ λ = 2.
Choosing the positive value λ = 2.
To find a corresponding
  eigenvector, row reduce the matrix (A − 2I ) to
16
obtain x =  4 .
1

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 19 / 26
Example 2

Two competing companies offer 15000 subscribers.

satellite television service to a city
with 100000 households. The
transition
 matrix is 
0.7 0.15 0.15
A = 0.2 0.8 0.15 (see the
0.1 0.05 0.7
figure below for the changes in
satellite subscriptions each year).
Company A now has 10000
subscribers and Company B has

Using the method of diagonalizing the transition matrix to find the numbers
of subscribers after 50 years. Round each answer to the nearest integer.

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 20 / 26
Let x, y be the numbers of subscribers of companies A and B, and
z = 100000 − (x + y ) be the numbers of households that don’t use any
services.
  At this  time: x0 = 10000, y0 = 15000, z0 = 75000.
x0 10000,
y0  =  15000 
z0 75000
   
x100 x0
After 50 years y100  = A50 . y0 
z100 z0
Use the method of diagonalization: A = PDP −1 , we can find
A50 = PD 50 P −1 .

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 21 / 26
SVD

Let A be an m × n matrix. Then AAT and AT A are 2 symmetric matrices.

Hence, they can be orthogonally diagonalized:
AAT = QD1 Q T , AT A = PD2 P T
We can easily prove that AAT and AT A have the same set of nonzero
eigenvalues and these values arepositive: 2 2 2
 λ1 > λ2 > .... > λr > 0. Then,
D 0
A = QΣP T (SVD), where Σ = - an m × n matrix;
0 0
D = diag ([λ1 , λ2 ..., λr ])- a diagonal matrix of order r . λi are called
singular values of A, Q = [q1 ...qm ], P = [p1 ...pn ].
⇒ A = λ1 q1 p1T + λ2 q2 p2T + ... + λr qr prT .
A = Qr DPrT -compact SVD of A.

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 22 / 26
Example
 
3 2 2
Find the SVD of A =
2 3 −2
 
17 8
AAT = , Orthogonally diagonalize: AAT = QD1 Q T where
8 17
   
25 0 1 1 1
D1 = , Q= 2√ .
0 9 1 −1
 
13 12 2
AT A = 12 13 −2, Orthogonally diagonalize:
2 −2 8
−2
 1
√1

  √
25 0 0 2 18 3
A = PD2 P T , where D2 =  0 9 0 , P =  √12 √−1 1 23  .

18
0 0 0 4 1
0 √
18  3

5 0 0
Hence, the SVD of A: A = QΣP T , where Σ = .
0 3 0

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 23 / 26
Application of SVD in image compression
The singular values σi are ordered σ1 ≥ σ2 ≥ ... ≥ σn ≥ 0. Significant
compression of the image is possible if the spectrum of singular values has
only a few very strong entries.
A ≈ Ak = σ1 q1 p1T + σ2 q2 p2T + ... + σk qk pkT .

Example
We have an image flower .jpg .
Transform the image into a matrix using Matlab:
A=rgb2gray(imread(’flower.jpg’));, A is 432 × 432.
Change to the double type: A = im2double(A);
Find SVD of A: [U S V]=svd(A); sigma = diag(S);

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 24 / 26
The original image:
figure; subplot(4,2,1); imshow(A); title(’Original’);
Choose the number k to approximate A: ranks=[200,100,50,30,20,10,5];
for i = 1 : length(ranks)
approxsigmas = sigma; approxsigmas(ranks(i):end)=0;
ns=length(sigma); approxS = S; approxS(1:ns,1:ns)=diag(approxsigmas);
approxflower = U*approxS*V’; subplot(4,2,i+1),imshow(approxflower);
title(sprintf(’When k = %s \n’, ranks(i))); end

(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 25 / 26
(Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 26 / 26