1

0402340 – Engineering Computation and

Linear Algebra
Chapter 5

Eigenvalues and Eigenvectors

Dr. Raouf Fareh

 2

Eigenvalues and Eigenvectors

• Dynamic physical systems can be modeled using systems
of linear differential equations.
• The dynamic nature comes from energy storage elements
within the system:
– (e.g. mechanical systems: rotating masses, compressed springs,
and pressurized containers,
– electrical systems: charged capacitors, inductors,
• In the absence of an external input or excitation, the
distribution of energy within the system:
– will decay to a minimum energy state if the system is stable, or
– will oscillate between states if the system is unstable.
 3

Eigenvalues and Eigenvectors

• Eigenvalues of the coefficient matrix A of a given system
determines the:
– the rate of decay of the natural modes of a stable system, and
– the frequency of the oscillation or rate of growth of the natural
modes of an unstable system.
• Eigenvalues are defined as follows: For a given nxn
matrix A: λ is the eigenvalue of A if there exist a non-zero
vector V such that:
AV = λV,
where A is nxn matrix, vector V is the eigenvector of A,
• Eigenvalues can be complex or real numbers.
 4

Eigenvalues and Eigenvectors

• The condition AV = λV, can be re-written as:
[A–λI]V=0
where I is nxn identity matrix,
• Now, in order for a non-zero vector V to satisfy this
equation, [ A – λ I ] must not be invertible,
i.e. det(A – λ I ) = | A – λ I | = 0.
C(λ) = | A – λ I | = 0 is called the characteristic equation
• Hint: if [ A – λ I ] is invertible then:
[ A – λ I ] V = 0 => V = [ A – λ I ]-1 . 0 => V = 0
• Which contradict the original assumption.
 5

Calculating Eigenvalues
• Given the following system: AV = λV =>
AV – λV = 0 =>
[ A– λI ] V = 0
• Where λ is a scalar, A is nxn matrix, I is nxn Identity
matrix.
(! a11 a12 ... a1n $ ! $, ! v1 $ ! 0 $
*# & # 1 0 ... 0 *
& # & # &
*# a21 a22 ... a2n & # 0 1 ... 0 &*- × # v2 & # 0 &
)# &− λ#     &* # &=#  &
*#     &  &
*# # 0 0 ... 1 &* # # 0 &
+" an1 an2 ... ann &% " %. #" vn &% " %
 6

Calculating Eigenvalues
(! a11 a12 ... a1n $ ! λ $, ! v1 $ ! 0 $
*# 0 ... 0
& # &* # & # &
*# a21 a22 ... a2n & # 0 λ ... 0 &*- × # v2 & # 0 &
)# &−#     &* # &=#  &
*#     &  &
*# # 0 0 ... λ &* # # 0 &
+" an1 an2 ... ann &% " %. #" vn &% " %

" a −λ a12 ... a1n % " v1 % " 0 %

$ 11 ' $ ' $ '
$ a21 a22 − λ ... a2n ' $ v2 ' $ 0 '
$ '×$ '=$  '
$     ' $  '
$ 0 '
$# an1 an2 ... ann − λ '& $# vn '& # &
 7

Calculating Eigenvalues
• The characteristic equation of the system can be found as:
C(λ) = det [A – λI] => C(λ) = |A – λI|
• C(λ) is the characteristic equation:
a11 − λ a12 ... a1n
a21 a22 − λ ... a2n
C(λ ) =
   
an1 an2 ... ann − λ
• C(λ) can be re-written as a polynomial:
C(λ) = an λn + an-1 λn-1 + an-2 λn-2 + … + a1 λ +a0
• Eigenvalues are the roots of the equation C(λ) = 0.
• Substituting the roots λ in the system [A – λ I ] V = 0 to
get V the eignvector for each λ
 8

Example 1: Calculating Eigenvalues

" 2 −4 %
• Given matrix A = $ ' , then:
# −1 −1 &
" 2 −4 % " 1 0 % " 2−λ −4 %
A − λI = $ '− λ$ '=$ '
# −1 −1 & # 0 1 & # −1 −1− λ &
2−λ −4
⇒ C(λ ) = =0
−1 −1− λ
• Then: C(λ) = (2 – λ)(–1 – λ) – (-1)(-4) = λ2 – λ – 6 = 0 =>
(λ – 3)(λ + 2) = 0 the eigenvalues are λ1,2 = 3, – 2
 9

Example 1: Calculating Eigenvectors

" 2 −4 %
• Given A = $ ' , and its eigenvalues λ1,2 = 3, –2
# −1 −1 &
• The eigenvectors V1 and V2 that corresponds to λ1 and λ2
are calculated using the equation:
[ A– λI ] V = 0 => [ A– λ1I ] V1 = 0 and [ A– λ2I ] V1 = 0
" 2 − 3 −4 %" v1,1 % " %" v %
$ '$ ' = $ −1 −4 '$ 1,1 '= 0
# −1 −1− 3 &$# v2,1 ' # −1 −4 &$ v2,1
& #
'
&
• Duplicate of same equation:
–v1,1 – 4 v2,1 = 0 and –v1,1 – 4v2,1 = 0
• We forced [ A– λI ] to be singular (|A– λI| = 0 ) to
calculate the eigenvalues, so this is expected.
 10

Example 1: Calculating Eigenvectors

• Assume that v2,1= t then v1,1= – 4t
• All eigenvectors corresponding to λ1= 3 are multiples of
the vector: " −4 %
V1 = $ '
# 1 &
• Considering the eigenvalue λ2 = –2, then:
" 2+2 %" v1,2 % " %" v %
$
−4
'$ ' = $ 4 −4 '$ 1,2 '= 0
# −1 −1+ 2 &$# v2,2 ' # −1 1 &$ v2,2
& #
'
&
• Duplicate of same equation: –v1,2 + v2,2 = 0
• Assume that v1,2= t then v2,2= t
• All eigenvectors corresponding to ! 1 $
V2 = # &
λ2= – 2 are multiples of the vector: " 1 %
 11

Example 2: Calculating Eigenvalues

" 5 8 16 %
• Given the matrix: A = $ 4 1 8 '
'
$
$ −4 −4 −11 '
# &
• Calculate the eigenvalues:

5− λ 8 16
C(λ ) = A − λ I = 4 1− λ 8 =0
−4 −4 −11− λ

C(λ) = (5 – λ)*{(1 – λ)*(-11 – λ) – (8)(-4)} – 8*{4(-11-λ)-

8*(-4)}+16*{4*(-4)-(-4)*(1-λ)} = 0
Þ (λ-1)(λ+3)2 = 0
Þthe eigenvalues are λ1,2,3 = 1, – 3, – 3,
 12

Example 2: Calculating Eigenvectors

• Eigenvector corresponds to λ2,3 = -3:
8v1 + 8v2 +16v3 = 0 " ' v1 + v2 + 2v3 = 0
$$ $$
4v1 + 4v2 + 8v3 = 0 # ⇒ ( v1 + v2 + 2v3 = 0
$ $
−4v1 − 4v2 − 8v3 = 0 $% $) v1 + v2 + 2v3 = 0

• Consider v3 = t and v2 = s, then v1 = -2t – s, the

corresponding eigenvector is:

v1 = −2t − s " −2 % " −1 %

$ ' $ ' Two
v2 = 0t + s V2,3 = $ 0 ' t + $ 1 ' s dimensional
$ 1 ' $ 0 ' vector
# & # &
v3 = t + 0s
 13

Example 2: Calculating Eigenvectors

• Eigenvector corresponds to λ1 = 1:
4v1 + 8v2 +16v3 = 0 " ' v1 + 2v2 + 4v3 = 0
$$ $$
4v1 + 0v2 + 8v3 = 0 # ⇒ ( v1 + 2v3 = 0
$ $
−4v1 − 4v2 −12v3 = 0 $% $) v1 + v2 + 3v3 = 0

• Add equations (1) and (2): 2v1+2v2+6v3 = 0 similar to

equation (3) => singular system.
• Consider v3 = t then v1 = -2t and v2 = -t, the corresponding
eigenvector is: " −2 %
$ '
V1 = $ −1 '
$ 1 '
# &
 14

Example 3: Calculating Eigenvalues

• Consider the following system of equations:
x1 + x2 + x3 =1
x1 + x2 + x3 =2
x1 + x2 + x3 =3
• Find the eigenvalues of the system such that: λ3 ≥ λ2 ≥ λ1
• Solution: ! $ ! x $ ! $
1 1 1 1 1
# & # & # &
# 1 1 1 & × # x2 & = # 2 &
# 1 1 1 & # x & # 3 &
" % #" 3 &% " %
AX = λX
Þ AX – λX = 0
Þ [A – λI] X = 0
 15

Example 3: Calculating Eigenvalues

• Solution: " 1− λ 1 1 % " x1 % " 0 %
$ ' $ ' $ '
$ 1 1− λ 1 ' × $ x2 ' = $ 0 '
$ 1 1 1− λ ' $ x ' $ 0 '
# & $# 3 '& # &
1− λ 1 1
C(λ ) = 1 1− λ 1
1 1 1− λ
C(λ) = (1 – λ){(1 – λ)2 – 1} – 1{(1 – λ – 1}+1{ 1 – (1 – λ )}
C(λ) = λ2 (λ – 3)
• To find the eigenvalues of the system, let C(λ) = 0
λ2 (λ – 3) = 0 => λ = 0, 0, 3 , given that λ3 ≥ λ2 ≥ λ1 then
the eigenvalues of the system are λ3 = 3, λ2 = 0, λ1= 0
 16

Example 3: Calculating Eigenvectors

• Eigenvector corresponds to λ1,2 = 0:
x1 + x2 + x3 =0
x1 + x2 + x3 =0
x1 + x2 + x3 =0

• Consider x1 = t and x2 = s, then x3 = -t – s, the

corresponding eigenvector is:

x1 = t + 0s " 1 % " 0 %
$ ' $ ' Two
x2 = 0t + s X1,2 = $ 0 ' t + $ 1 ' s dimensional
$ −1 ' $ −1 ' vector
x3 = −t − s # & # &
 17

Example 3: Calculating Eigenvectors

• Eigenvector corresponds to λ3 = 3:
-2x1 + x2 + x3 =0
x1 + -2x2 + x3 =0
x1 + x2 + -2x3 =0

• Consider x1 = t and x2 = t, then x3 = t, the corresponding

eigenvector is:
! 1 $
# &
X3 = # 1 &
# 1 &
" %
 18

Example 4: Calculating Eigenvalues

" −3 −1 %
• Given matrix A = $ ' , then:
# 5 1 &
" −3 −1 % " 1 0 % " −3− λ −1 %
A − λI = $ '− λ$ '=$ '
# 5 1 & # 0 1 & # 5 1− λ &
−3− λ −1
⇒ C(λ ) = =0
5 1− λ
• Then: C(λ) = (1 – λ)(–3 – λ) – (-1)(5) = λ2 + 2λ + 2 = 0
=> the eigenvalues are λ1,2 = – 1 i
 19

Example 4: Calculating Eigenvectors

" −3 −1 %
• Given A = $ ' , and its eigenvalues λ1 = –1 – i
# 5 1 &
• Find the eigenvector:
" −3− (−1− i) −1 %" x % " %" x1 %
$ '$ 1 ' = $ −2 + i −1 '$ '= 0
$# 5 1− (−1− i) '&$# x2 '& # 5 2+i &$# x2 '&
• Multiply the first row by the complex conjugate (–2 – i):
(−2 − i) × # −2 + i −1 &# x1 & # 5 2 + i &# x1 &
% (% (=% (% (= 0
$ 5 2+i '%$ x2 (' $ 5 2 + i '%$ x2 ('
• Multiple equations, assume x1 = t, x2 = ( – 2 +i )t,
corresponding eigenvector is:
" 1 %
X =$ '
# −2 + i &
 20

Example 4: Calculating Eigenvectors

" −3 −1 %
• Given A = $ ' , and its eigenvalues λ1 = –1 + i
# 5 1 &
• Find the eigenvector:
" −3− (−1+ i) −1 %" x % " %" x1 %
$ '$ 1 ' = $ −2 − i −1 '$ '= 0
$# 5 1− (−1+ i) '&$# x2 '& # 5 2−i &$# x2 '&
• Multiply the first row by the complex conjugate (–2 + i):
(−2 + i) × # −2 − i −1 &# x1 & # 5 2 − i &# x1 &
% (% (=% (% (= 0
$ 5 2−i '%$ x2 (' $ 5 2 − i '%$ x2 ('
• Multiple equations, assume x1 = t, x2 = ( – 2 – i )t,
corresponding eigenvector is:
" 1 %
X =$ '
# −2 − i &
 21

Dominant Eigenvalue and Stability

• Spectral radius of A is given as the maximum modulus of
the given eigenvalues: Δ n
ρ ( A) = max { λk }
k=1

• A dominant eigenvalue λ1 is an eigenvalue of a matrix

Anxn if it has the largest magnitude compared to any other
eigenvalue: |λ1| > |λ2| ≥ |λ3|…. ≥ |λn-1| ≥ |λn|, i.e. the
eigenvalue that corresponds to the spectral radius of A
• The Eigenvector V1 corresponding to λ1 is called the
dominant eigenvector.
• A discrete time linear system AX = b is stable if and only
if:
– | λi | < 1, n ≥ i ≥ 1 OR
– its spectral radius is less than unity: ρ(A) < 1
 22

Dominant Eigenvalue
• Example: given the following system:
" 10 −7 −2 % has the following eignvalues: 10,
$ ' , 7 + 3i and 7 − 3i
A = $ 0 9 −31 '
2 2
$ 0 1 −2 '
# &
• The modulus of the shown eignvalues are: 10, and 13
• The dominant eigenvalue is 10 and the spectral radius of
A is 10.
• Consider the matrix:
has the following eignvalues: 1, −11+ 3i
" 1 −7 −2 % 2
$ ' , and −11− 3i
A = $ 0 0 −31 ' 2
$ 0 1 −11 ' The modulus of the shown eignvalues are:
# & 1, and 31 => No single dominant
eigenvalue
 23

Eigenvalues and Trace Function

• For a Given Anxn, the following function is called the
trace of an nxn matrix A: Trace (A) = a11+a22+ …+ann
i.e. the sum of the diagonal elements.
• The Trace function is important because it appears in the
characteristic equation.
• For a given matrix Anxn the characteristic equation is:
C(λ) = - λ3 + [a11+ a22+ a33]*λ2 + a11 − λ a12 a13
a11*[a22*a33 - a23*a32] + C(λ ) = a21 a22 − λ a23
a12*[a23*a31- a21*a33] + a31 a32 a33 − λ
a13*[a21*a32 - a22*a31]
+λ*[- a11*a22 - a11*a33 - a22*a33+a12*a21 +a13*a31+ a23*a32]

C(λ) = - λ3 + Trace(A)*λ2 + det(A)+…

 24

Eigenvalues of Triangular Matrix

• If a matrix Anxn is triangular (upper or lower):
– the eigenvalues are the diagonal elements,
– the trace is the sum of the eigenvalues or diagonal elements and
– the determinant is the product of the eigenvalues (the
determinant of a triangular matrix is the multiplication of the
diagonal elements)
C(λ) = - λ3 + [a11+ a22+ a33]*λ2 +a11*a22*a33 a11 − λ a12 a13
- λ*[ a11*a22+ a11*a33 +a22*a33] C(λ ) = 0 a22 − λ a23
0 0 a33 − λ
The roots of this equation are: a11, a22, and a33
The eignvalues are the diagonal elements of the OR
given matrix. a11 − λ 0 0
C(λ ) = a21 a22 − λ 0
• For a Triangular matrix: a31 a32 a33 − λ
– Trace(A) = λ1 + λ2 + … + λn
– det(A) = |A| = λ1 λ2 … λn
 25

Eigenvalues of General NxN Matrix

• For a general matrix A, the characteristic equation is:
C(λ) = - λ3 + [a11+ a22+ a33]*λ2 + a11 − λ a12 a13
a11*[a22*a33 - a23*a32] – C(λ ) = a21 a22 − λ a23
a12*[a21*a33 - a23*a31] + a31 a32 a33 − λ
a13*[a21*a32 - a22*a31]
+λ*[- a11*a22 - a11*a33 - a22*a33+a12*a21 +a13*a31+ a23*a32]
• It can also be written as: C(λ) = (λ1-λ). (λ2-λ) …(λn-λ)
• If λ is substituted by zero, it is equal to the determinant of
A or |A-λI| = |A-0I| = |A|.
• Thus for any general matrix NxN A:
– Trace(A) = λ1 + λ2 + … + λn
– det(A) = |A| = λ1 λ2 … λn
 More on Eigenvalues
26

• If a matrix A has eigenvalue λ with corresponding

eigenvector X, then for any k = 1, 2, ... ; Ak has
eigenvalue λk corresponding to the same eigenvector X.
Proof: AX = λX
Þ A with eigenvalues λ
Þ A A X = A λX
Þ A2 X = λ A X = λ λ X
• So based on the same concept:
Þ A2 X = λ2 X
Þ A2 with eigenvalues λ2
Þ A3 X = λ3 X given that A3 with eigenvalues λ3
:
Þ An X = λn X given that An with eigenvalues λn
 27

Power Method
• Power method is an iterative method (convergence
conditions applies)
• We use the power method to compute the dominant eigen
pair (dominant Eigenvalue & dominant Eigenvector).
• Knowing the the dominant eigenvalue helps determine the
stability of the system.
• The convergence condition is that the matrix A has a
single dominant Eigenvalue.
• Power Method might not converge for multiple dominant
eigenvalues of the same magnitude, or dominant complex
eigenvalues.
 28

Power Method
• An Eigen vector V is said to be normalized if the
coordinates of the largest magnitude is equal to unity (the
largest coordinate in the vector V is equal to 1).
• Normalizing an eigenvector: ! V $
! V $ Normalized V: # 1
&
# 1 & 1 # V2 &
# V2 & Where C = Vj and V= # &
V =# & C
#  &
#  & Vj = max { Vi } #" Vn &%
#" Vn &% n≥i≥1

• Suppose that Anxn has a dominant eigenvalue λ and that

there is a unique normalized eigenvector V that
corresponds to λ.
• This eigen-pair (λ & V) can be found by the following
iterative method which is called the power method.
 29

Power Method: Find Dominant Eignvalue

! $
1. Start with Vector: #
1
&
1
X0 = # &
#  &
# 1 &
" %
2. Generate the sequence Xk recursively using the
following equations:
Yk = A Xk
1
and X k+1 = Yk
Ck+1
Where Ck+1 is the element of the largest magnitude in YK
(in the case of a tie, choose the coordinate that comes first).

3. The sequences Xk and Ck will converge to V and λ,

respectively:
lim X k = V and lim Ck = λ k→∞
k→∞
 30

Example: Power Method

• Use the power method to find the dominant eigenvalue
and its corresponding eigenvector for the matrix:
" 0 11 −5 %
$ '
• Solution: A = $ −2 17 −7 '
! 1 $ $ −4 26 −10 '
# & First iteration (k=0) # &
X0 = # 1 &
# 1 & "
" % 0 11 −5 % " 1 % " 6 % " 1/ 2 %
$ ' $ ' $ ' $ '
Yo = AX 0 = $ −2 17 −7 ' × $ 1 ' = $ 8 ' = 12 × $ 2 / 3 '
$ −4 26 −10 ' $ 1 ' $ 12 ' $ 1 '
# & # & # & # &
" 1/ 2 %
1 $
12 × $ 2 / 3
'
X1 = Y0 → Y0 = C1 X1 ' = C1 X1
C1 $ 1
#
'
&
 31

Example: Power Method

Second iteration (k=1)
" 0 11 −5 % " 1 / 2 % " 7/3 % " 7 /16 %
$ ' $ ' $ ' 16 $ '
Y1 = AX1 = $ −2 17 −7 ' × $ 2 / 3 ' = $ 10 / 3 ' = × $ 5 / 8 '
$ −4 26 −10 ' $ 1 ' $ 16 / 3 ' 3 $ 1 '
# & # & # & # &
" 7 /16 %
16 $ '
Y1 = C2 X 2 → $ 5 / 8 ' = C2 X 2
3$ '
# 1 &
 32

Example: Power Method

 33

Example: Power Method

• Doing more iterations, our results become:
! 1/ 2 $ ! 7 /16 $ ! 5 /12 $
# & 16 # & 9# &
C1 X1 = 12 # 2 / 3 & C2 X 2 = # 5 / 8 & C3 X3 = # 11 /18 &
# 1 & 3# & 2# &
" % " 1 % " 1 %
! 31 / 76 $ ! 2/5 $
38 # & # &
C4 X 4 = # 23 / 38 & C16 X16 = 4 # 3 / 5 &
9# & # 1 &
" 1 % " %
• The sequence of vectors converges to: !
2/5 $
and the sequence of # &
V =# 3/5 &
constants converges to λ = 4 # 1 &
" %