Lecture Notes: Eigenvalues and Eigenvectors

Yufei Tao
Department of Computer Science and Engineering
Chinese University of Hong Kong
[email protected]

1 Definitions
Let A be an n × n matrix. If there exist a real value λ and a non-zero n × 1 vector x satisfying

Ax = λx (1)

then we refer to λ as an eigenvalue of A, and x as an eigenvector of A corresponding to λ.

Example 1. Consider
 
1 −1
A =
2 4

You can easily verify that

    
1 −1 1 1
= 3
2 4 −2 −2
 
1
Hence, 3 is an eigenvalue of A. Vector is an eigenvector of A corresponding to 3.
−2

2 Finding All Eigenvalues

Moving λx in (1) to the left hand side gives:

(A − λI)x = 0

where I is the n × n identity matrix. Introducing B = A − λI, we can re-write the above as

Bx = 0 (2)

Let us consider the above as an equation about x. We know that if det(B) 6= 0, the above equation
has a unique solution x = 0. However, this is not what we want. Remember that our goal is to
find an eigenvector x of A, which needs to be a non-zero vector. Therefore, we must choose λ
appropriately to make det(B) = 0. This provides us a way to find the eigenvalues of A.

Example 2. Consider the matrix A in Example 1.

B = A − λI
 
1−λ −1
=
2 4−λ

1
Hence:

det(B) = (1 − λ)(4 − λ) + 2
= λ2 − 5λ + 6.

To make det(B) = 0, we can set λ to λ1 = 3 and λ2 = 2. These are all the eigenvalues of A.

In general, det(B) = det(A − λI) is a polynomial function of λ. We refer to the function as

the characteristic polynomial of A. For instance, in Example 2, the characteristic polynomial of A
is λ2 − 5λ + 6. The eigenvalues of A are precisely the solutions of λ in

det(A − λI) = 0. (3)

The above equation is called the characteristic equation of A.

Lemma 1. An n × n matrix A can have at most n distinct eigenvalues.

Proof. The characteristic polynomial of A is a polynomial of degree n. Hence, Equation (3) can
have at most n distinct roots of λ.

3 Finding All Eigenvectors

Let λ be a value satisfying (3), namely, λ is an eigenvalue of A. In this case, Equation (2) has
infinitely many solutions x (because det(B) = 0); we denote by EigenSpace(λ) the set of all
those solutions x. The eigenvectors of A corresponding to λ are exactly the non-zero vectors in
EigenSpace(λ).

Example 3. Consider again the matrix A in Example 1. We know from Example 2 that it has
two eigenvalues: λ1 = 3 and λ2 = 2.
 
x1
Let us first look for the eigenvectors of A for λ1 = 3. Namely, we want to find x = to
x2
satisfy:

(A − λ1 I)x = 0 ⇒
    
1−3 −1 x1 0
= ⇒
2 4−3 x2 0
    
−2 −1 x1 0
=
0 0 x2 0
 
x1
Hence, any satisfying −2x1 − x2 = 0 is a solution to the above system. The set of such
x2
vectors can be represented in a parametric form: x1 = t and x2 = −2t for any t ∈ R. This set—
which we denote as EigenSpace(λ1 )—has dimension 1. Every non-zero vector in EigenSpace(λ1 ) is
an eigenvector corresponding to λ1 .

2
  
x1
Similarly, to obtain the eigenvectors of A for λ2 = 2, we want x = to satisfy:
x2
(A − λ2 I)x = 0 ⇒
    
1−2 −1 x1 0
= ⇒
2 4−2 x2 0
    
−1 −1 x1 0
=
0 0 x2 0
 
x1
Hence, any satisfying −x1 − x2 = 0 is a solution to the above system. The set of such
x2
vectors can be represented in a parametric form: x1 = t and x2 = −t for any t ∈ R. This set,
denoted as EigenSpace(λ2 ), also has dimension 1. Every non-zero vector in EigenSpace(λ2 ) is an
eigenvector corresponding to λ2 .

Example 4. Consider
 
4 6 0
A =  −3 −5 0 
−3 −6 1
Its characteristic equation is:
det(A − λI) = 0 ⇒
4−λ 6 0
−3 −5 − λ 0 = 0 ⇒
−3 −6 1−λ
4−λ 6
(1 − λ) = 0 ⇒
−3 −5 − λ
(1 − λ)((4 − λ)(−5 − λ) + 18) = 0 ⇒
(λ − 1)2 (λ + 2) = 0.
Hence, A has two eigenvalues: λ1 = 1 and λ2 = −2.
 
x1
To look for the eigenvectors of A for λ1 = 1, we seek x =  x2  satisfying
x3
(A − λ1 I)x = 0 ⇒
  
3 6 0 x1
 −3 −6 0   x2  = 0 ⇒
−3 −6 0 x3
  
3 6 0 x1
 0 0 0   x2  = 0 ⇒
0 0 0 x3
 
x1
Hence, any  x2  satisfying 3x1 + 6x2 = 0 is a solution to the above system. The set of
x3
such vectors can be represented in a parametric form: x1 = 2u, x2 = −u, and x3 = v for any

3
(u, v) ∈ R2 . The set is denoted as EigenSpace(λ1 ), and has dimension 2. Every non-zero vector in
EigenSpace(λ1 ) is an eigenvector corresponding to λ1 .
 
x1
To look for the eigenvectors of A for λ2 = −2, we seek x =  x2  satisfying
x3

(A − λ2 I)x = 0 ⇒
  
6 6 0 x1
 −3 −3 0   x2  = 0 ⇒
−3 −6 3 x3
  
−1 −1 0 x1
 0 1 −1   x2  = 0 ⇒
0 0 0 x3
 
x1
Hence, any  x2  satisfying
x3

−x1 − x2 = 0
x2 − x3 = 0

is a solution to the above system. The set of such vectors can be represented in a parametric
form: x1 = −t, x2 = t, and x3 = t for any t ∈ R. This set is denoted as EigenSpace(λ2 ), and has
dimension 1. Every non-zero vector in EigenSpace(λ2 ) is an eigenvector corresponding to λ2 .

The vector space EigenSpace(λ) is referred to as the eigenspace of the eigenvalue λ. The
dimension of EigenSpace(λ) is referred to as the geometric multiplicity of λ.

Appendix: Algebraic Multiplicity of Eigenvalues (Not Required by

the Syllabus)
Recall that the eigenvalues of an n × n matrix A are solutions to the characteristic equation (3) of
A. Sometimes, the equation may have less than n distinct roots, because some roots may happen
to be the same. In general, if λ1 , λ2 , ..., λk are the distinct roots, then we must be able to re-write
equation (3) as:

(λ − λ1 )t1 (λ − λ2 )t2 ...(λ − λk )tk = 0

Pk
where t1 , t2 , ..., tk are positive integers satisfying i=1 ti = n. We refer to ti as the algebraic
multiplicity of λi , for each i ∈ [1, k]. It is worth mentioning that some of these roots can be complex
numbers, although in this course we will focus on matrices with only real-valued eigenvalues.

Example 5.

• In Example 2, the algebraic multiplicities of the eigenvalues λ1 = 3 and λ2 = 2 are both 1.

• In Example 4, the algebraic multiplicity of the eigenvalue λ1 = 1 is 2, and that of the

eigenvalue λ2 = −2 is 1.

4
 We state the next result without proof:

Lemma 2. The geometric multiplicity of an eigenvalue is at most its algebraic multiplicity.

In all the examples we have seen, the geometric multiplicity has always been the same as its
algebraic multiplicity. However, this is not always true, as you can see from the example below.

Example 6. Consider
 
−1 1 0
A =  −4 3 0 
1 0 2

Its characteristic equation is

−1 − λ 1 0
−4 3−λ 0 = 0 ⇒
1 0 2−λ
(λ − 1)2 (λ − 2) = 0.

Hence, A has only 2 eigenvalues: λ1 = 1 and λ2 = 2. They have algebraic multiplicities 2 and 1,
respectively.

Let us now focus on the eigenvalue λ1 = 1.We will see that its geometric multiplicity is 1. To
x1
find its corresponding eigenvectors x =  x2 , we solve the equation:
x3

(A − λ1 I)x = 0 ⇒
  
−2 1 0 x1
 −4 2 0   x2  = 0 ⇒
1 0 1 x3
  
−2 1 0 x1
 0 1/2 1   x2  = 0 ⇒
0 0 0 x3

Hence, we know that

−2x1 + x2 = 0
(1/2)x2 + x3 = 0
 
−t
Therefore, the set of solutions to the above problem is the set EigenSpace(λ1 ) of vectors  −2t 
t
for all t ∈ R. This set has dimension 1.