MATH 351: Linear Algebra

Vector Spaces

Course Instructors:
Abdullah Abubakar
Kenneth Dadedzi
Ralph A. Twum

First Semester, 2021

Department of Mathematics
University of Ghana
Outline

1 Motivating Examples: The Cartesian Plane and R3

2 Definition of a Vector Space

3 Examples

1
Motivating Examples: The
Cartesian Plane and R3
The Space R2

• In Physics and other fields, a vector is defined as a quantity with magnitude and
direction. What this means is that we need more than one number to describe
vectors.
• We have discussed vectors in the plane R2 , representing a vector ~v "by coordinates:
#
v1
~v = (v1 , v2 ), i, j-notation: ~v = v1 i + v2 j, or column notation: ~v = .
v2
• We saw that we can add (or subtract ) vectors component-wise: If ~v = (v1 , v2 )
and w
~ = (w1 , w2 ), then

~v + w
~ = (v1 + w1 , v2 + w2 ).

• We can also scale a vector: for ~v = (v1 , v2 ) and r a real number, r · ~v = (rv1 , rv2 ).

2
The Space R2

• In addition, we discussed the scalar (or dot) product: given ~v = (v1 , v2 ) and
w
~ = (w1 , w2 ),
~v · w
~ = v 1 w1 + v 2 w2 .

This allows us to perform certain computations, like finding the length of a vector
and finding the angle between vectors.
• We can also consider transformations of vectors: rotations, reflections, scaling
and translations.

3
3-space, R3

• Similar computations can be performed for vectors in space, where we represent  

v1
vectors by three coordinates: ~v = (v1 , v2 , v3 ), ~v = v1 i + v2 j + v3 k, or ~v =  v2 .
 

v3
• The inner product is defined for ~v = (v1 , v2 , v3 ) and w ~ = (w1 , w2 , w3 ) as

~v · w
~ = v1 w1 + v2 w2 + v3 w3 .

• In both R2 and R3 , the vector representing the origin is denoted 0 : (0, 0) in R2

and (0, 0, 0) in R3 .

4
Definition of a Vector Space
What is a Vector Space?

• What we are going to do in this course is to abstract the properties described for
R2 and R3 , in order to apply them to many kinds of mathematical concepts. This
is what makes Linear Algebra so powerful: the techniques can be applied to many
mathematical systems which can be linearized1 .

1
Think back to linear approximations you learnt in Calculus.
5
What is a Vector Space?

• What we are going to do in this course is to abstract the properties described for
R2 and R3 , in order to apply them to many kinds of mathematical concepts. This
is what makes Linear Algebra so powerful: the techniques can be applied to many
mathematical systems which can be linearized1 .
• In addition, because these techniques can be programmed, computing resources
can be deployed to solve some seemingly intractable problems, if a linear model of
such problems can be obtained.
• We will start by defining a vector space, and then consider examples.

1
Think back to linear approximations you learnt in Calculus.
5
Definition of Vector Space

Definition
A vector space V over a field F is a set together with two operations:
vector addition + : V × V → V , and scalar multiplication · : F × V → V , satisfying
the following properties: given u, v, w ∈ V and r, s ∈ F, the following hold:

1 v + w ∈ V (i.e. V is closed under vector addition)

2 r · v ∈ V (i.e. V is closed under scalar multipication)
3 v+w =w+v
4 u + (v + w) = (u + v) + w
5 There exists a zero vector, denoted 0, such that v + 0 = v
6 Given v ∈ V , there exists a vector (denoted (−v)) such that v + (−v) = 0

6
Definition of Vector Space

Definition
7 (r + s) · v = r · v + s · v
8 r · (v + w) = r · v + r · w
9 (rs) · v = r · (s · v)
10 1 · v = v, where 1 is the identity element in the group F \ {0}.

2
When vector multiplication is defined, we have an algebra.
7
Definition of Vector Space

Definition
7 (r + s) · v = r · v + s · v
8 r · (v + w) = r · v + r · w
9 (rs) · v = r · (s · v)
10 1 · v = v, where 1 is the identity element in the group F \ {0}.

Remark
Notice that the properties described above indicates that (V, +) is an abelian group,
with the zero vector as the identity element. Notice also that only vector addition and
scalar multiplication are defined: we do not have a notion of vector multiplication2 .
2
When vector multiplication is defined, we have an algebra.
7
Examples
Examples

Example
The space Rn : As we saw in the introduction, Rn is a vector space over the field R of
real numbers: for v = (v1 , v2 , . . . vn ), w = (w1 , w2 , . . . , wn ), and for r ∈ R we have

v + w = (v1 + w1 , v2 + w2 , . . . , vn + wn )
r · v = (rv1 , rv2 , . . . rvn )

The zero vector is the n-tuple (0, 0, . . . , 0). We can check to see that all the properties
are satisfied.
Example
Sequences: We can extend the above example to consider (real) sequences: maps from
N to R: the sum of two sequences s and t gives a sequence (s + t): for n ∈ N,
(sn + tn ) = (s + t)n .
8
Examples

Example
Real-Valued Functions: Suppose f : R → R and g : R → R are real-valued functions,
and let s be a real number.
Then f + g gives another real-valued function; so does s · f , where (s · f )(t) = s(f (t)).
We can check to see that all the properties are satisfied.

9
Examples

Example
Real-Valued Functions: Suppose f : R → R and g : R → R are real-valued functions,
and let s be a real number.
Then f + g gives another real-valued function; so does s · f , where (s · f )(t) = s(f (t)).
We can check to see that all the properties are satisfied.
We can consider functions satisfying some additional properties: polynomial functions,
continuous functions, differentiable functions, solutions to differential equations, and
others.

9
When is a set a vector space?

• The main properties to check are closure under vector addition and scalar
multiplication. If one of these conditions is not satisfied then V is not a vector
space3 .

3
In some cases, some of the other properties may fail.
4
It also does not contain a zero vector.
10
When is a set a vector space?

• The main properties to check are closure under vector addition and scalar
multiplication. If one of these conditions is not satisfied then V is not a vector
space3 .
• As an example, consider V = {(x, y) ∈ R2 |x + y = 1}. Notice that this describes
the points (vectors) that make up the line y = 1 − x. Consider the intercepts
(0, 1) and (1, 0). These belong to the set V , but (0, 1) + (1, 0) = (1, 1) does not.
Therefore V is not closed under vector addition, and is not a vector space. You
can check that it fails the test for scalar multiplication as well.4

3
In some cases, some of the other properties may fail.
4
It also does not contain a zero vector.
10
When is a set a vector space?

Let us consider another example: let V = {(x, y) ∈ R2 |x2 = y 2 }, describing two

perpendicular lines through the origin, with gradients 1 and −1:

11
When is a set a vector space?

Is V as described a vector space? Notice that V is closed under scalar multiplication:

V is composed of points (a, a) or (a, −a): multiplying through by a real number
preserves the nature of the points. It also contains the zero vector (0, 0).

12
When is a set a vector space?

Is V as described a vector space? Notice that V is closed under scalar multiplication:

V is composed of points (a, a) or (a, −a): multiplying through by a real number
preserves the nature of the points. It also contains the zero vector (0, 0).
Consider (2, 2) and (3, −3): both belong to V , but (2, 2) + (3, −3) = (5, −1) does
not. Therefore V is not a vector space.

12
When is a set a vector space?

Is V as described a vector space? Notice that V is closed under scalar multiplication:

V is composed of points (a, a) or (a, −a): multiplying through by a real number
preserves the nature of the points. It also contains the zero vector (0, 0).
Consider (2, 2) and (3, −3): both belong to V , but (2, 2) + (3, −3) = (5, −1) does
not. Therefore V is not a vector space.
We see that it is not enough to consider the geometric picture of the set V ; we need
to test the points to determine if it satisfies all the properties of a vector space.

12
When is a set a vector space?

Is V as described a vector space? Notice that V is closed under scalar multiplication:

V is composed of points (a, a) or (a, −a): multiplying through by a real number
preserves the nature of the points. It also contains the zero vector (0, 0).
Consider (2, 2) and (3, −3): both belong to V , but (2, 2) + (3, −3) = (5, −1) does
not. Therefore V is not a vector space.
We see that it is not enough to consider the geometric picture of the set V ; we need
to test the points to determine if it satisfies all the properties of a vector space.
In the following slides, we will consider other tools for determining if a set is a vector
space.

12
MATH 351: Linear Algebra
Subspaces

Course Instructors:
Kenneth Dadedzi
Ralph A. Twum

First Semester, 2022

Department of Mathematics
University of Ghana
Outline

1
Definition and Examples
Definition of a Subspace

When is a set a vector space?

• In the previous set of slides, we considered subsets of R2 , determining whether
they were vector spaces or not.
• This leads us to the definition of a subspace:

2
Definition of a Subspace

When is a set a vector space?

• In the previous set of slides, we considered subsets of R2 , determining whether
they were vector spaces or not.
• This leads us to the definition of a subspace:

Definition
A subset W of a vector space V (over a field F) is called a subspace of V if W is itself
a vector space under the same operations of vector addition and scalar multiplication
as V .

Let us consider some examples of subspaces.

2
Example
Let V = R2 , the Cartesian plane. Consider the line y = x. This consists of points
(x, x) ∈ R2 , where x ∈ R. It is straightforward to verify that L = {(x, x)|x ∈ R} is a
vector space under the familiar operations of vector addition and scalar multiplication
in R2 .1

1
In fact, all lines through the origin are subspaces in R2 .
2
Verify that this is indeed a vector space!
3
Example
Let V = R2 , the Cartesian plane. Consider the line y = x. This consists of points
(x, x) ∈ R2 , where x ∈ R. It is straightforward to verify that L = {(x, x)|x ∈ R} is a
vector space under the familiar operations of vector addition and scalar multiplication
in R2 .1

Example
Consider the vector space
(" M2×2 (R)
# of matrices ) of dimension 2 × 2 with real
a b
coefficients.2 Let S = |a, b, c ∈ R . Then we can check to see that S is
0 c
closed under
" # vector addition and scalar multiplication. S also includes the zero vector
0 0
. You can verify that the rest of the axioms are satisfied. Therefore S is a
0 0
subspace of M2×2 (R).
1
In fact, all lines through the origin are subspaces in R2 .
2
Verify that this is indeed a vector space!
3
Example
Consider the vector space V of all real-valued functions f : R → R. Let S denote the
subset of V composed of functions which satisfy the ordinary differential equation

d2 f
+ f = 0.
dt2

Note that S is well-defined, since it contains the 0 function3 , and the functions cos t
and sin t (Check by differentiating the functions!).
We can verify that for f, g ∈ S and any real number r, f + g ∈ S and r · f ∈ S. The
other properties follow similarly. Therefore S is a subspace of V .4

3
The function which maps all t ∈ R to 0.
4
This example illustrates the usefulness of Linear Algebra in Differential Equations (and Functional
Analysis).
4
The Subspace Criterion
If S is a subset of a vector space V , we find that we do not need to check all the
properties for S before concluding whether or not S is a subspace of V . It suffices to
check closure under the inherited operations:

5
If S is a subset of a vector space V , we find that we do not need to check all the
properties for S before concluding whether or not S is a subspace of V . It suffices to
check closure under the inherited operations:
Lemma
(The Subspace Criterion)
Let S be a non-empty subset of a vector space V . Then the following statements are
equivalent:

1 S is a subspace of V .
2 For any two vectors s1 , s2 ∈ S and scalars r1 , r2 ∈ F, r1 s1 + r2 s2 ∈ S (i.e. S is
closed under linear combinations of pairs of elements of S).
3 For vectors s1 , s2 , . . . , sn ∈ S and scalars r1 , r2 , . . . rn ∈ F,
r1 s1 + r2 s2 + · · · + rn sn ∈ S (i.e. S is closed under arbitrary linear combinations).

5
 Note
When proving three equivalent statements, we usually prove the following logical
expression:
(1) ⇒ (2) ⇒ (3) ⇒ (1).

Proof.
(1) ⇒ (2) If S is a subspace of V , then S is automatically closed under vector addition and
scalar multiplication. Therefore r1 s1 + r2 s2 = s0 + s00 for s0 , s00 ∈ S, which lies in
S.

6
 Note
When proving three equivalent statements, we usually prove the following logical
expression:
(1) ⇒ (2) ⇒ (3) ⇒ (1).

Proof.
(1) ⇒ (2) If S is a subspace of V , then S is automatically closed under vector addition and
scalar multiplication. Therefore r1 s1 + r2 s2 = s0 + s00 for s0 , s00 ∈ S, which lies in
S.
(2) ⇒ (3) We can proceed by induction, with n = 2 being the case above. Suppose that
r1 s1 + r2 s2 + · · · + rk sk lies in S. Now

r1 s1 + r2 s2 + · · · + rk sk + rk+1 sk+1 = s0 + rk+1 sk+1 ,

which is a linear combination of two pairs of elements of S, and so lies in S.

Hence the statement holds by induction.
6
 Proof.
(3) ⇒ (1) If S is closed under arbitrary linear combinations, then in particular, S is closed
under vector addition and scalar multiplication. The rest of the properties of a
vector space are straightforward to establish. Therefore the three statements are
equivalent.

5
This is essentially what we used in the previous set of slides to check if the graph of y 2 = x2 is a
vector space.
7
 Proof.
(3) ⇒ (1) If S is closed under arbitrary linear combinations, then in particular, S is closed
under vector addition and scalar multiplication. The rest of the properties of a
vector space are straightforward to establish. Therefore the three statements are
equivalent.

• What this means is that, to verify if S is a subspace of V , we can use any of the
methods in the three statements described above.
• We will usually check if S is closed under linear combinations of pairs of elements
of S.5
5
This is essentially what we used in the previous set of slides to check if the graph of y 2 = x2 is a
vector space.
7
Spanning Sets
The span of a vector

Let us consider again the graph S of the line y = x in R2 . We saw that this is a vector
space.
Notice that for each x ∈ R, the vector (x, x) lies in S. In particular, the vector
(1, 1) ∈ S. We can write (x, x) as x · (1, 1). This means that multiplying (1, 1) by a
scalar gives us a vector in S, and so we can describe S using just this one vector!.
We say that S is generated (or spanned) by (1, 1).

8
The span of a vector

Let us consider again the graph S of the line y = x in R2 . We saw that this is a vector
space.
Notice that for each x ∈ R, the vector (x, x) lies in S. In particular, the vector
(1, 1) ∈ S. We can write (x, x) as x · (1, 1). This means that multiplying (1, 1) by a
scalar gives us a vector in S, and so we can describe S using just this one vector!.
We say that S is generated (or spanned) by (1, 1).
Definition
Let s be a vector in S. The span of s, denoted span({s}), is the set of linear
combinations of s: span({s}) = {c · s|c ∈ F}. Extending the concept, let
s1 , s2 , . . . , sn ∈ S. The span of {s1 , s2 , . . . , sn } is the set
{c1 · s1 + c2 · s2 + · · · + cn · sn |c1 , c2 , . . . , cn ∈ F}.

8
The usefulness of the concept of the span is found in the following result:
Lemma
Let S be a subset of V . Then span(S) is a subspace of V .

Before we prove the lemma, let us consider the example of V = R2 : vectors in the
plane are denoted by Cartesian coordinates: ~v = (x, y). We can rewrite ~v as follows:

~v = (x, y)
= (x, 0) + (0, y)
= x · (1, 0) + y · (0, 1)

In MATH 123 (Vectors and Geometry), we denoted (1, 0) by i and (0, 1) by j, so that
~v = xi + yj. We see that R2 is spanned by the set {i, j}: we can describe each vector
in the plane by using the horizontal and vertical unit vectors.

9
Proof of Lemma
m
X n
X
Let u and v be in span(S). Then u = ai · si , and v = bj · tj , where si , tj all
i=1 j=1
belong to the set S and ai , bj are scalars. We show that any linear combination of u
and v is also in the span of S.
For scalars c, d, we have

m
X n
X
cu + dv = c ai · si + d bj · tj
i=1 j=1
m
X n
X
= c · si + d · tj
i=1 j=1

= c · (a1 · s1 ) + · · · + c · (am · sm ) + d · (b1 · t1 ) + · · · + d · (bn · tn )

Therefore cu + dv is a linear combination of elements of S, and so lies in the span of
10
S. Therefore span(S) is a subspace of V .
Examples

Example
Consider S = {(1, 0), (0, 1)}. As we have seen, span(S) = R2 (where the scalars are
real numbers). We can also write the vectors as column vectors, as we do in the next
example.

11
Examples

Example
Consider S = {(1, 0), (0, 1)}. As we have seen, span(S) = R2 (where the scalars are
real numbers). We can also write the vectors as column vectors, as we do in the next
example.

Example    
 1
 0  
3
Consider the vector space V = R , and let S =  0  ,  1  . What is the span
   
 
0 0
 
of S? Notice that both vectors in S have the third coordinate being 0, so any linear
combination of the vectors will also have the third coordinate as zero.
From the above example, we notice that the span of S gives us the xy-plane in R3 , as
shown in the following diagram.
11
12
Examples

Example
Suppose S = {0} is the set consisting of the zero vector in V . What is the span of S?
Notice that any linear combination of the zero vector gives the zero vector. Therefore
span({0}) = {0}, which is a subspace of S.
In fact, for the empty set ∅, we have span(∅) = {0}! This is because the span of a set
(even the empty set) is a subspace, and so for a set with no elements, we get the
trivial or empty sum for each (non existing) element, which leaves us with the subspace
consisting of just the zero vector.6

6
We can define the span of S as the smallest subspace containing all elements of S, or as the
intersection of all subspaces containing S. In the case of the empty set, we obtain span(∅) = {0}.
13
When is W a subspace of V ?

• We have seen how to use the subspace criterion to determine if a set W is a

subspace of V .
• We can use the concept of the span as well. If we can show that W is the span of
some set S, then W is a subspace of V . Let us consider some examples:

14
When is W a subspace of V ?

• We have seen how to use the subspace criterion to determine if a set W is a

subspace of V .
• We can use the concept of the span as well. If we can show that W is the span of
some set S, then W is a subspace of V . Let us consider some examples:

Example
Consider the vector space
(" M2×2 (R)
# of matrices ) of dimension 2 × 2 with real
a b
coefficients. Let W = |a, b, c ∈ R . An element A ∈ W is of the form
0 c
" #
a b
. We will rewrite A as follows:
0 c

14
Example " #
a b
A=
0 c
" # " # " #
a 0 0 b 0 0
= + +
0 0 0 0 0 c
" # " # " #
1 0 0 1 0 0
=a· +b· +c· .
0 0 0 0 0 1
Therefore, A is a linear combination of elements of the set
(" # " # " #)
1 0 0 1 0 0
S= , , .
0 0 0 0 0 1
We conclude that W = span(S), and hence is a subspace of M2×2 (R).

15
  
1
Let us consider the vector u =  0 . Is u in the span of the set
 

3
   

 2 1  
S =  1  ,  −1  ?
   
 
−1 1
 

We check to see if there are scalars x, y such that

     
1 2 1
 0  = x 1  + y  −1  .
     

3 −1 1

16
This gives us the system of equations below:

2x + y = 1
x−y =0
−x + y = 3
We must find x and y which works for all three equations. If these do not exist, then
u is not in the span of the set S.
Check to see that there do not exist any x and y which satisfy all three equations. The
diagram below illustrates this.
We conclude that u is not in the span of the set S.7

7
This means that span(S) is a proper subspace of the vector space R3 .
17
18
Linear Dependence and Linear
Independence
• We have seen that given a set S of a vector space V , the span of S is a subspace
of V . This gives us a way of describing (sub)spaces without necessarily listing all
the elements of the space.
• What we now want to do is to obtain a minimal set that spans a space. We do so
by exploring the concepts of linear dependence and linear independence. Let us
begin with an example.

19
 • We have seen that given a set S of a vector space V , the span of S is a subspace
of V . This gives us a way of describing (sub)spaces without necessarily listing all
the elements of the space.
• What we now want to do is to obtain a minimal set that spans a space. We do so
by exploring the concepts of linear dependence and linear independence. Let us
begin with an example.
(" # " # " #)
2 1 0 1
Consider the vector space V = R , and the set S = , , .
0 1 1
" #
1
Notice that span(S) = V . However, the vector can be expressed in terms of the
1
" # " # " #
1 1 0
other two vectors: = + .
1 0 1

19
 " #
1
• What this means is that taking from the set S will not affect span(S):
1
(" #)!
1
span S \ = R2 .
1
• We say that the set S is linearly dependent: some of the elements in S can be
expressed in terms of the others.

20
 " #
1
• What this means is that taking from the set S will not affect span(S):
1
(" #)!
1
span S \ = R2 .
1
• We say that the set S is linearly dependent: some of the elements in S can be
expressed in terms of the others.

There are many ways to define linear dependence(and linear independence). We will
use the one below:
Definition
Let S be a subset of a vector space V . S is linearly independent if for elements in S,
the linear equation c1 s1 + c2 s2 + · · · + cn sn = 0 for s1 , s2 , . . . , sn ∈ S has only the
trivial solution c1 = c2 = · · · = cn = 0. If at least one of the ci 6= 0, then S is linearly
dependent.

20
• From the definition, we see that S is linearly independent if none of the vectors in
S can be expressed as a linear combination of the others.
• If no vector can be expressed in terms of the others, then S is linearly independent.

21
 • From the definition, we see that S is linearly independent if none of the vectors in
S can be expressed as a linear combination of the others.
• If no vector can be expressed in terms of the others, then S is linearly independent.

Lemma
Let S be a subset of a vector space V , and let v be a vector in V . Then
span(S ∪ {v}) = span(S) if and only if v ∈ span(S).

Proof.
(⇒) If span(S ∪ {v}) = span(S), then v 6∈ span(S) is impossible, because it would
make the sets span(S ∪ {v}) and span(S) not equal.

21
Proof.
(⇐) Suppose v ∈ span(S). Then v can be expressed as a linear combination:

v = b1 s1 + b2 s2 + · · · + bn sn ,

where bi are scalars and si ∈ S. We know that span(S) ⊆ span(S ∪ {v}). We

need to prove the reverse inclusion.
Let w ∈ span(S ∪ {v}). Then there are scalars ci and vectors ti ∈ S such that

w = c1 t1 + c2 t2 + · · · + cm tm + cm+1 v
= c1 t1 + c2 t2 + · · · + cm tm + cm+1 (b1 s1 + b2 s2 + · · · + bn sn )
= c1 t1 + c2 t2 + · · · + cm tm + b1 cm+1 s1 + b2 cm+1 s2 + · · · + bn cm+1 sn .

This is a linear combination of elements si , ti ∈ S, and so w ∈ span(S), Therefore

span(S) = span(S ∪ {v}).

22
Corollary
Let S be a subset of a vector space V , and s ∈ S. Then span(S \ {s}) = span(S) if
and only if s can be written as a linear combination of the rest of the elements of S.

23
Corollary
Let S be a subset of a vector space V , and s ∈ S. Then span(S \ {s}) = span(S) if
and only if s can be written as a linear combination of the rest of the elements of S.
Example
Let V = P3 (R), the vector space of polynomials of degree 3, with real coefficients.
Consider the set S = {3 − x + 9x2 , 5 − 6x + 3x2 , 1 + x − 5x2 }. Is S linearly
independent? For scalars a, b and c, we consider the equation

a(3 − x + 9x2 ) + b(5 − 6x + 3x2 ) + c(1 + x − 5x2 ) = 0.

Comparing coefficients, we have

3a + 5b + c = 0
−a − 6b + c = 0
9a + 3b − 5c = 0
23
From MATH 126 and MATH 224, we know that we can convert the system into a
matrix equation:

    
3 5 1 a 0
−1 −6 1    0 .
b =
    

9 3 −5 c 0

We know that a = b = c = 0 is one solution. Is it the only solution? Let us find the
determinant of the matrix:

3 5 1
−1 −6 1 = 3(30 − 3) − 5(5 − 9) + 1(−3 + 54) = 81 + 20 + 51 = 152.
9 3 −5
Therefore a = b = c = 0 is the only solution, and the set is linearly independent.
24
MATH 351: Linear Algebra
Bases for Vector Spaces

Course Instructors:
Abdullah Abubakar
Kenneth Dadedzi
Ralph A. Twum

First Semester, 2021

Department of Mathematics
University of Ghana
Outline

1
Definition and Examples
Introduction

In the previous lecture, we discussed linear dependence and independence of sets.

We want to use these results to describe vector spaces in terms of bases:

2
The space R2

Example
For vectors in the space R2 , we have
" # " # " #
x x 0
= +
y 0 y
" # " #
1 0
=x· +y·
0 1

3
The space R2

" #
x
So each vector can be uniquely represented as a linear combination of elements
y
in the set (" # " #)
1 0
S= , .
0 1

We call S a basis for the vector space R2 .

4
Main Definition

Definition (Basis)
Let V be a vector space over a field F. An indexed set B in V is a basis for V if

1 B spans V
2 B is linearly independent

Remark
Note that the definition refers to ”a basis” for a vector space. We can have several
bases for the same vector space.

5
Examples

Example
(" # " #)
1 0
S= , is a basis for R2 . . .
0 1
(" # " #)
1 1
but so is S1 = , ...
1 −1
(" # " #)
2 3
and S2 = , , and many more
1 2

6
Examples

In general, for Rn the set

       

 1 0 0 0 

 



 0  
  1  
  0 


 0 


       
B=  0 , 0 , 1 ,··· , 0 
.. .. .. ..
       
       


 .   .   .   . 


 
0 0 0 1
 

is a basis for Rn , called the standard or natural or canonical basis.

7
Examples

Similarly, for Pn (R), B = {1, x, x2 , x3 , . . . , xn } is the standard basis.

As another example, consider the vector space of 2 × 2 matrices with real entries:
(" # )
a b
V = |a, b, c, d ∈ R
c d

Can we think of a basis for V ?

8
Examples

Let us remember the vector space V of real sequences (xn )n∈N :

The standard basis for V is the set
 


 (1, 0, 0, . . . , 0, 0, . . . ) 


 
(0, 1, 0, . . . , 0, 0, . . . )

 


 


 

(0, 0, 1, . . . , 0, 0, . . . )

 


 

 
 .. 
B= .

 

(0, 0, 0, . . . , 1, 0, . . . )

 


 

 


 (0, 0, 0, . . . , 0, 1, . . . ) 

 
..

 

 
 . 

9
Characterization of Bases

Theorem
Let V be a vector space. A subset B of V is a basis if and only if each vector v ∈ V
has a unique representation as a linear combination of elements of B.

Proof.
Since B is a basis of V , B spans V , and elements of B are linearly independent.
Suppose an element v of V has two representations as linear combinations of elements
of B:

10
Characterization of Bases

Proof. (continued)
v = c1 b1 + c2 b2 + · · · + cn bn , and
v = d1 b1 + d2 b2 + · · · + dn bn

Subtracting the two equations, we get

0 = (c1 − d1 )b1 + (c2 − d2 )b2 + · · · + (cn − dn )bn .

Since B is linearly independent, ci = di for each i = 1, 2, . . . n.

11
Characterization of Bases

Proof. (continued)
Conversely, suppose each v ∈ V can be expressed uniquely in terms of elements of B.
Then B spans V .
Also the zero vector 0 has a unique representation in terms of elements of B. In fact,
we have
0 = c1 b1 + c2 b2 + · · · + cn bn .

Since c1 = c2 = · · · = cn = 0 is a solution, it is (by uniqueness) the only solution.

Therefore B is also linearly independent, so it is a basis for V .

12
Coordinate Representations
Definition
Let V be a vector space, and suppose B is a basis for V . If an element v in V can be
written uniquely in terms of elements of B as v = c1 b1 + c2 b2 + · · · cn bn , then the
coordinate representation of v with respect to B is given by
 
c1
 c2 
 
RepB (v) =  . 

.
 .. 
cn

13
Examples

Example " # " #

2 3 3
In R , the vector has the coordinate representation with respect to the
4 4
standard basis (perhaps this is why it is ”the standard basis”).
(" # " #)
1 1
With respect to the basis S1 = , , what will be the coordinate
1 −1
representation?

14
Examples

Find the coordinate representation of the following vectors:

1 1 + x + x2 ; B = {1, 1 + x, x + x2 }
       
1  1
 1 1  
 0 ; B =  1  ,  1  ,  0 
       
2
 
3 1 0 0
 

ex ; B = 1, x, x2 , x3 , . . . , (think of Maclaurin series)


3

15
Dimension
The Dimension of a Vector Space

We have seen that vector spaces may have (infinitely) many bases.
These bases share a similar property, which may be used as an invariant property of the
vector space.
We will mostly focus on vector spaces with bases consisting of finitely many vectors.

16
The Dimension of a Vector Space

Definition
A vector space V is finite-dimensional if it has a basis with finitely many vectors.

Example
1 Rn is a finite-dimensional vector space. The canonical basis has n vectors.
2 P (R) is not a finite-dimensional vector space - no finite basis exists.

17
Lemma
Let B = {b1 , b2 , . . . bn } be a basis for a vector space V . Suppose there exists an
element v ∈ V such that v = c1 b1 + c2 b2 + · · · + cn bn , with ci 6= 0. Then exchanging
bi with v gives another basis for V .

Proof(Sketch)
• If the new basis is B1 , the linear equation d1 b1 + · · · + di v + · · · + dn bn = 0
reduces to a linear equation involving the members of B, which gives us di = 0.
So B1 is linearly independent.

18
Proof(Sketch)
• Each linear combination of elements of B1 can be rearranged to form a linear
combination of elements of B, so span(B1 ) ⊆ span(B).
• Each linear combination of elements of B including bi can be rearranged to form a
linear combination of elements of B1 , so span(B) ⊆ span(B1 ).

19
Theorem
In any finite-dimensional vector space, all bases have the same number of elements.

Proof(Sketch)
Suppose B and B1 are two finite bases of the same vector space:

B = {b1 , b2 , . . . , bn }
B1 = {β1 , β2 , . . . , βm }

20
Proof(Sketch)
Suppose B and B1 are two finite bases of the same vector space:

B = {b1 , b2 , . . . , bn }
B1 = {β1 , β2 , . . . , βm }

Using the previous lemma, we can swap β1 with an element from B, giving a new
basis. By induction, we can replace all elements of B with those from B1 . We cannot
have any elements of B1 left over because of linear independence, so n = m.

21
Dimension

Definition
Let V be a vector space over a field F. The dimension of V is the cardinality of any of
its bases, i.e. the number of elements in any of its bases.

Example
1 dim Rn = n.
2 dim Mn×n (R) = n2 .
3 dim{0} = 0.
4 dim Pn (R) = n + 1.
5 dim P (R) = ∞.

22
Corollary
Any linearly independent set can be expanded to a basis.

Corollary
Any spanning set can be reduced to a basis.

Corollary
Let V be an n-dimensional vector space. A set S with n vectors is linearly independent
if and only if it spans V .

23
Operations on Subspaces
Let V be a vector space and U, W subspaces of V . We will try to form new subspaces
using U and W .

1 U ∩ W = {v ∈ V |v ∈ U and v ∈ W }. This is a subspace of V (Prove it.)

2 U ∪ W = {v ∈ V |v ∈ U or v ∈ W }. Is U ∪ W a subspace of V ? Why or why
not?
3 U + W = {u + w|u ∈ U, w ∈ W }. This is called the sum of U and W . Is U + W
a subspace of V ? Why or why not?

24
Example      

 a 
 
 0 

3
Consider V = R , U =  b  |a, b, ∈ R , and W =  x  |x, y, ∈ R .
   
   
0 y
   

1 What is U ∩ W ?
2 What is U + W ?

25
Theorem
Suppose U and W are subspaces of a finitely generated vector space V . Then

dim(U + W ) + dim(U ∩ W ) = dim(U ) + dim(W ).

Proof
We suppose that the ordered set {v1 , v2 , . . . , vk } is a basis for the intersection U ∩ W .
Since U ∩ W is a subspace of U , we can complete the set {v1 , v2 , . . . , vk } to a basis of
U : {u1 , u2 , . . . , uj , v1 , v2 , . . . , vk }

26
Proof - Continued
Similarly, we can complete {v1 , v2 , . . . , vk } to a basis of W :
{v1 , v2 , . . . , vk , w1 , w2 , . . . , wn }.
We claim that the set B = {u1 , u2 , . . . , uj , v1 , v2 , . . . , vk , w1 , w2 , . . . , wn } is a basis for
U + W:

1 Given v = u + w ∈ U + W , we can write u and w in terms of the set B, therefore

span(B) = U + W .
2 Suppose we have the equation
j
X k
X n
X
ci ui + di vi + fi wi = 0.
i=1 i=1 i=1

27
Proof - Continued
We can rewrite the equation as
j
X k
X n
X
ci ui = (−di )vi + (−fi )wi .
i=1 i=1 i=1

The left and right hand sides of the equation are equal, and belong to both U and W ,
so to U ∩ W .
So the u0 s can be expressed in terms of the v 0 s, which is not possible since the set
{u1 , u2 , . . . , uj , v1 , v2 , . . . , vk } is a basis for U .

28
Proof - Continued
So ci = 0 for all i = 1, 2, . . . , j. This gives us the equation
k
X n
X
(di )vi + (fi )wi = 0
i=1 i=1

Since the set {v1 , v2 , . . . , vk , w1 , w2 , . . . , wn } is a basis for W , all d0 s and f 0 s are 0,

demonstrating the linear independence of B.

29
Proof - Continued
From the above, we have dim(U + V ) = j + k + n, dim(U ) = j + k, dim(V ) = k + n,
and dim(U ∩ V ) = j. We rearrange to get the formula:

(j + k + n) + k = (j + k) + (k + n)
dim(U + W ) + dim(U ∩ W ) = dim(U ) + dim(W )

30
Example
1 Suppose U and W are eight-dimensional subspaces of a 10 dimensional subspace.
Find all possible values for dim(U ∩ W ).
2 Suppose U is a six-dimensional subspace and W is an eight-dimensional subspace
of R10 . Find the smallest possible dimension of U ∩ W .

31
Definition
Let V be a vector space and U , W subspaces of V . We say V is the direct sum of U
and W if V = U + W and U ∩ W = 0.
We write V = U ⊕ W .

If V = U ⊕ W , each vector v ∈ V has a unique decomposition v = u + w, u ∈ U ,

w ∈ W.

32
Example
(" # ) (" # )
2 a 0
1 Let V = R and consider U = |a ∈ R and W = |b ∈ R .
0 b
Then V = U ⊕ W .
2 Let V = R3 . Write down possible candidates for subspaces U and W such that
V = U ⊕ W.

Theorem
Suppose V is a finite dimensional vector space, and V = U ⊕ W , where U and W are
subspaces of V . Then
dim(V ) = dim(U ) + dim(W ).

33
MATH 351: Linear Algebra
Introduction to Linear Transformations

Course Instructors:
Kenneth Dadedzi
Ralph A. Twum

First Semester, 2022

Department of Mathematics
University of Ghana
Outline

1 Definition of a Linear Transformation

2 Examples of Linear Transformations

3 The Induced Subspaces

1
Definition of a Linear
Transformation
Introduction

We have discussed vector spaces, subspaces, bases and dimension, with examples.
Now we want to consider maps between vector spaces - linear transformations.
We will see that with a choice of suitable bases, we can represent a linear
transformation by a matrix.

2
Definition
Suppose V and W are vector spaces (over the same field). A map T : V → W is
called a linear transformation1 if T satisfies the following properties:
for all v, v1 , v2 ∈ V , λ ∈ F,

• T (v1 + v2 ) = T (v1 ) + T (v2 )

• T (λv) = λT (v).2

The notion of linear transformation is so useful; there are many examples.

1
In the Hefferon book, it is called a homomorphism
2
Note that T maps the zero vector of V to the zero vector of W .
3
Examples of Linear Transformations
Example  
a11 a12 · · · a1n
 a
 21 a22 · · · a2n 
An m × n matrix A =   is a linear transformation from

 ··· ··· ··· ··· 
am1 am2 · · · amn
n m
V = R to W = R over the field of real numbers.
For vectors v1 , v2 ∈ Rn , we have. A(v1 + v2 ) = Av1 + Av2 . Also, A(λv1 ) = λ · (Av1 ).
Therefore multiplication by a matrix A is a linear transformation. We will see that,
with a choice of bases for our vector spaces, we can go in the other direction -
representing a linear transformation by a matrix.

4
Example
Define D : P (R) → P (R) by D(f ) = f 0 and : P (R) → P (R) by (f ) = f (t) dt.
R R R

These are linear transformations of the (infinite-dimensional) vector space of

polynomial functions, P (R).
For polynomials p(t) and q(t), we know from calculus that

[p(t) + q(t)]0 = p0 (t) + q 0 (t)

(kp(t))0 = kp0 (t)
Z Z Z
[p(t) + q(t)] dt = p(t) dt + q(t) dt
Z Z
(kp(t)) dt = k p(t) dt

This tells us that the derivative and the integral are linear transformations of P (R).3
3
In Functional Analysis, these maps are called linear operators.
5
Example
Let V R2 and W = R1 . The projection map P : V → W which is defined by
" = #!
a
P = a is a linear transformation:
b
" #!
a+c
P =a+c
b+d
" #! " #!
a c
P +P =a+c
b d
" #! " #!
a ra
P r =P = ra
b rb
" #!
a
r·P = ra
b

Therefore the projection map defined above is a linear transformation.

6
Example
of 2 × 2 #!
Let V be the vector space " matrices with real entries. The trace operator
a b
T : V → R defined by T = a + d is a linear transformation: For matrices
c d
" # " #
a b e f
and , we have
c d g h
" #!
a+e b+f
T = (a + e) + (d + h) = (a + d) + (e + h)
c+g d+h
" #! " #!
a b e f
=T +T .
c d g h
" #! " #!
a b ra rb
T r =T = ra + rd = r(a + d)
c d rc rd
" #!
a b
=r·T .
c d 7
The Induced Subspaces
Definition
Let V and W be vector spaces. We denote the set of linear transformations from V to
W by L(V, W ).

Theorem
L(V, W ) is a vector space.

4
The 0 map sends everything from V to the zero vector 0W in W .
8
Definition
Let V and W be vector spaces. We denote the set of linear transformations from V to
W by L(V, W ).

Theorem
L(V, W ) is a vector space.

Proof
• L(V, W ) is nonempty - it contains the 0 map.4
• For T, S in L(V, W ), T + S ∈ L(V, W ) and λT ∈ L(V, W ). We can verify all the
vector space properties of L(V, W ).

4
The 0 map sends everything from V to the zero vector 0W in W .
8
If V is a vector space, then L(V, R) is called the dual space to V , and denoted V ∗ . It
has the same dimension as V if V is finite dimensional.
Definition
Let T : V → W be a linear transformation. The kernel of T , denoted Ker(T ) is
defined as the set Ker(T ) = {v ∈ V : T (v) = 0W }.
The image of T , denoted Im(T ) is defined as the set
Im(T ) = {w ∈ W : w = T (v) for some v ∈ V }.

9
If V is a vector space, then L(V, R) is called the dual space to V , and denoted V ∗ . It
has the same dimension as V if V is finite dimensional.
Definition
Let T : V → W be a linear transformation. The kernel of T , denoted Ker(T ) is
defined as the set Ker(T ) = {v ∈ V : T (v) = 0W }.
The image of T , denoted Im(T ) is defined as the set
Im(T ) = {w ∈ W : w = T (v) for some v ∈ V }.

Remark
Ker(T ) is a subspace of V and Im(T ) is a subspace of W .

9
Definition
Let T : V → W is a linear transformation. Then the nullity of T is the dimension of
Ker(T ), and the rank of T is the dimension of Im(T ).

5
We assume that V and W are finite dimensional.
10
Definition
Let T : V → W is a linear transformation. Then the nullity of T is the dimension of
Ker(T ), and the rank of T is the dimension of Im(T ).

Theorem (Rank-Nullity Theorem)

Let T : V → W be a linear transformation.5 Then

Nullity(T ) + Rank(T ) = dim(V ).

5
We assume that V and W are finite dimensional.
10
Proof
Let BN = {b1 , b2 , . . . , bk } be a basis for Ker(T ). We can complete BN to a basis of V ,
giving B = {b1 , b2 , . . . , bk , vk+1 , . . . , vn }.
We claim that {T (vk+1 ), T (vk+2 ), . . . , T (vn )} is a basis for Im(T ).

11
Proof - Continued
Let v ∈ V and consider T (v) ∈ Im(T ). We have

v = d1 b1 + · · · + dk bk + ck+1 vk+1 + · · · + cn vn

12
Proof - Continued
Let v ∈ V and consider T (v) ∈ Im(T ). We have

v = d1 b1 + · · · + dk bk + ck+1 vk+1 + · · · + cn vn
T (v) = d1 T (b1 ) + · · · + dk T (bk )
+ ck+1 T (vk+1 ) + · · · + cn T (vn )
T (v) = d1 (0W ) + · · · + dk (0W )
+ ck+1 T (vk+1 ) + · · · + cn T (vn )
T (v) = ck+1 T (vk+1 ) + · · · + cn T (vn )

Therefore T (v) is a linear combination of elements of the set

{T (vk+1 ), T (vk+2 ), . . . , T (vn )}, so span({T (vk+1 ), T (vk+2 ), . . . , T (vn )}) = Im(T ).

12
Proof - Continued
Suppose
n
X
0W = cj T (vj )
j=k+1
 
n
X
Then 0W = T  cj vj  .
j=k+1

n
X
This means cj vj ∈ Ker(T ), so
j=k+1

13
Proof - Continued
Suppose
n
X k
X
cj vj = di bi
j=k+1 i=1
k
X n
X
(−di )bi + cj vj = 0V
i=1 j=k+1

But since the set B = {b1 , b2 , . . . , bk , vk+1 , . . . , vn } is linearly independent,

ci = di = 0. Therefore {T (vk+1 ), T (vk+2 ), . . . , T (vn )} is linearly independent as well.

14
MATH 351: Linear Algebra
Inner Product Spaces

Course Instructors:
Abdullah Abubakar
Kenneth Dadedzi
Ralph A. Twum

First Semester, 2021

Department of Mathematics
University of Ghana
Outline

1 Inner Products

2 Orthogonal Sets

3 The Gram-Schmidt Process

1
Inner Products
Introduction

In our work on vectors in the plane and in space, we made use of the scalar or dot
product, and considered some applications of this product to determine the length of a
vector, and acute angles between vectors.
A similar idea exists in an arbitrary vector space. We introduce the concept of an inner
product in a vector space, and consider how to construct an orthonormal basis given
any basis of a (finite dimensional) vector space.

2
Definition
Let V be an n-dimensional real vector space. An inner product on V is a map
h·, ·i : V × V → R1 satisfying the following properties:

(I1) hx, yi = hy, xi ∀x, y ∈ V

(I2) The map is bilinear, i.e. hλx, yi = λhx, yi, and hx, µyi = µhx, yi, for all x, y ∈ V
and λ, µ ∈ R.
(I3) hx, xi ≥ 0 ∀x ∈ R, and hx, xi = 0 ⇔ x = 0.

If V has an inner product, then we call V an inner product space.

1
we may replace R with the field we are using, but then axioms I1 and I2 will change (e.g. in C)
3
Examples

Example
V = Rn , x = (x1 , . . . , xn ), y = (y1 , . . . , yn ). hx, yi := x1 y1 + x2 y2 + · · · + xn yn . (this
is the dot product we covered in MATH 123).

Example
Consider the space C[0, 1] of continuous functions defined on the interval [0, 1] into R.
We define the inner product as:
Z 1
hf, gi = f (t)g(t) dt
0

4
Suppose V is an inner product space. Define the norm of V to be the map || · || → R+
p
defined by ||x|| = hx, xi.
p
From the previous page, if x = (x1 , x2 , . . . , xn ), then ||x|| = x21 + · · · x2n .
 
1
 −2 
If ||x|| = 1, then x is a unit vector. For instance, given v =  , ||v|| = 3, and
 
 2 
0
1
u = 3 v is a unit vector in the same direction as v.

5
Definition
Let V be an inner product space and u, v ∈ V . The distance between u and v is given
by d(u, v) = ||u − v||.

Definition
Let u, v ∈ V , where V is an inner product space. u and v are said to be orthogonal if
hu, vi = 0.

Theorem
(Pythagorean theorem) Let u, v ∈ V , where V is an inner product space. u and v are
orthogonal if and only if ||u + v||2 = ||u||2 + ||v||2 .

6
Definition
Let W be a subspace of V , where V is an inner product space. z ∈ V is orthogonal to
W if z is orthogonal to every element of W . The collection of all vectors orthogonal to
W is a subspace of V , denoted W ⊥ .

Example
In R2 , the canonical vectors i = (1, 0) and j = (0, 1) are orthogonal vectors.

7
Orthogonal Sets
Definition
A set {u1 , . . . up } of vectors in V is orthogonal if each pair of distinct vectors from the
set is orthogonal, i.e hui , uj i = 0 whenever ui 6= uj .

Example     
3 −1 −1/2
Consider the vector space R3 . Try u1 =  1 , u2 =  2  and u3 =  −2 .
     

1 1 7/2

8
Theorem
If S = {u1 , . . . , up } is an orthogonal set of nonzero vectors in V , S is linearly
independent.

Proof.
Consider the equation
0 = c1 u1 + · · · cp up
h0, u1 i = hc1 u1 + · · · cp up , u1 i
= c1 ||u1 ||2
Therefore c1 = 0, since u1 is nonzero. Applying the above method to the other
vectors, we find c1 = c2 = · · · = cp = 0.

9
Definition
An orthogonal basis for a subspace of V is a basis which is an orthogonal set.

Theorem
Let {u1 , . . . up } be an orthogonal basis for W ⊆ V . Then for all y ∈ W ,

y = c1 u1 + · · · cp up , where
hy, uj i
cj = .
huj , uj i

10
Proof.
We have
hy, u1 i = h(c1 u1 + · · · cp up ), u1 i
= c1 hu1 , u1 i
hy, u1 i
∴ c1 = , since hu1 , u1 i =
6 0.
hu1 , u1 i
We can find the other values ck in a similar manner.

Definition
A set {u1 , . . . up } is an orthonormal set if it is an orthogonal set of unit vectors.

11
Orthogonal Projection

Definition
Let V be an inner product space, and suppose {u1 , u2 , . . . , up } is an orthogonal basis
for a subspace W of V . Then the orthogonal projection of v ∈ V onto W is given by

hv, u1 i hv, u2 i hv, up i

v̂ = u1 + u2 + · · · + up .
hu1 , u1 i hu2 , u2 i hup , up i

12
Example (" # " #)
1 −1
Consider V = R2 , with basis B = , .
1 1
" #
4
Notice that B is also an orthogonal set. Consider v = .
5
• Write v as a linear combination of elements of B.
" #
1
• Find the orthogonal projections of v onto the subspace spanned by , and the
1
" #
−1
subspace spanned by
1.

13
14
15
Examples

" # " #
2 4
1 Let y = and u = . Express y as a combination of a vector in span{u}
3 −7
and a vector orthogonal to u.
     
1 −1 −1
Let u1 = 1, u2 =  1 , and y =  4 . Find the orthogonal projection of y
     
2

0 0 3
onto span{u1 , u2 }.

16
17
18
Theorem
Suppose W is a subspace of a vector space V , and suppose that v ∈ V . If v̂ is the
orthogonal projection of v onto W , then v̂ is the closest point in W to v. This means
that
||v − v̂|| < ||v − w||

for all w ∈ W distinct from v̂.

Remark
The above theorem is very useful in least-squares problems, for instance ordinary least
squares problems in regression analysis.

19
The Gram-Schmidt Process
The Gram-Schmidt Process

This is an algorithm which produces an orthogonal or orthonormal basis for any given
nonzero subspace of an inner product space. Given a basis {x1 , x2 , . . . , xp } for a
subspace W of V , we proceed as follows:

v1 = x1
hx2 , v1 i
v2 = x2 − v1
hv1 , v1 i
hx3 , v1 i hx3 , v2 i
v3 = x3 − v1 − v2
hv1 , v1 i hv2 , v2 i

20
The Gram-Schmidt Process

Continuing in this manner, we have

hxp , v1 i hxp , v2 i
vp = xp − v1 − v2 − . . .
hv1 , v1 i hv2 , v2 i
hxp , vp−1 i
− vp−1 .
hvp−1 , vp−1 i

{v1 , . . . vp } is an orthogonal basis for W . Furthermore,

span{v1 , . . . , vk } = span{x1 , . . . , xk }, 1 ≤ k ≤ p.

21
Example
Let us use the Gram-Schmidt process to produce orthogonal bases for the following:
     
1 0 0
 1   1   0 
• x1 =   , x2 =   , x3 = 
   
.

 1   1   1 
1 1 1
   
3 8
• x1 =  0  , x2 =  5 .
  

1 −6

22
23
24
25
26
Orthogonal Matrices

A square matrix A is said to be orthogonal if it satisfies the relation

AAT = I.

This means A−1 = AT .

Remark
1 The columns of an orthogonal matrix form an orthonormal basis (so do the rows).
2 If A is an orthogonal matrix, then det(A) = ±1.
3 If A is an orthogonal matrix, then for x, y ∈ Rn , hAx, Ayi = hx, yi.

27
Example
" #
cos θ − sin θ
1 Consider A = . A is an orthogonal matrix.
sin θ cos θ
 
2 −2 1
1
Consider B = 3 1 2 2 . Determine if B is an orthogonal matrix.

2

2 1 −2

28
29
30
MATH 351: Linear Algebra
Eigenvalues and Eigenvectors

Course Instructors:
Abdullah Abubkar
Kenneth Dadedzi
Ralph A. Twum

First Semester, 2021

Department of Mathematics
University of Ghana
Outline

1 Eigenvalues and Eigenvectors

2 The Jordan Canonical Form

1
Eigenvalues and Eigenvectors
Introduction

Some of the important applications of matrices arise in the study of linear systems of
differential equations.
In such systems we want to find the equilibrium or steady-state solutions, and then
classify such points (MATH 450).
Sometimes we also want to find powers of square matrices An or exponentials exp(A).
We can use the theory of eigenvalues and eigenvectors to address such issues. We can
then generalize these concepts to Jordan forms, weight spaces in representation theory,
root spaces of Lie algebras, etc.

2
Definition
Let A be a square matrix. Suppose there exists a nonzero vector v ∈ Rn such that

Av = λ · v,

where λ is a scalar.
λ is called an eigenvalue of A, and v an eigenvector corresponding to the eigenvalue λ.

Example" # " #
3 −2 2
Let A = and v = . We see that Av = 2v.
1 0 1

3
Finding Eigenvalues and Eigenvectors

Let A be an n × n matrix. Therefore A is a linear transformation from Rn to itself (or

a linear endomorphism). We have

Av = λ · v
(A − λI)v = 0

In order to have nontrivial solutions, (A − λI) must be singular, so that

det(A − λI) = 0.
The equation det(A − λI) = 0 is called the characteristic equation of the matrix A.
The polynomial p(λ) = det(A − λI) is the characteristic polynomial of A.

4
Example
" #
3 −2
1 Consider A = . We have
1 0
" #
3 − λ −2
A − λI =
1 0−λ
3 − λ −2
This gives =0
1 −λ
(3 − λ)(−λ) − (1)(−2) = 0
λ2 − 3λ + 2 = 0
(λ − 1)(λ − 2) = 0

So λ = 1 and λ = 2 are the eigenvalues of A.

5
Example
Let us find the eigenvalues of the following matrices:
" #
5 0
2 A=
2 1
" #
2 −1
3 A=
2 4

6
7
 " #
3 −2
Now let us find eigenvectors. Given A = , we find eigenvectors for each
1 0
eigenvalue.
For λ = 2 we have " #" # " #
3 − 2 −2 v1 0
=
1 0 − 2 v2 0
" #" # " #
1 −2 v1 0
=
1 −2 v2 0
v1 − 2v2 = 0
v1 = 2v2

8
 " #
v1
This means any vector such that v1 = 2v2 is an eigenvector corresponding to
v
" 2# " # " #
2 6 1
λ = 2. So we can have , , , etc. (Find the eigenvectors for the other
1 3 1/2
matrices).
Remark
If λ is an eigenvalue of A, then eigenvectors corresponding to λ are nonunique. If v is
an eigenvector corresponding to λ, then so is rv, for r a scalar:

A(rv) = A(rI)v = (rI)Av

= (rI)λv = (rI)(λI)v
= (rλ)v = λ(rv)

9
Remark
If λ is an eigenvalue of A, the set of eigenvectors corresponding to λ is a subspace of
Rn . This subspace is called the eigenspace of Rn corresponding to λ, which we can
denote by Rnλ . The dimension of Rnλ is called the geometric multiplicity of λ.

Characteristic Polynomials
Let A be a square matrix. As we said before, the characteristic polynomial of A is the
polynomial function p(x) = det(A − xI). Roots of the characteristic polynomial are
the eigenvalues of A. For each root λ of the polynomial, the multiplicity of λ is called
its algebraic multiplicity.

10
Let A be an n × n matrix. The characteristic polynomial of A is given by

a11 − x a12 ··· a1n

a21 a22 − x ··· a2n
p(x) =
··· ··· ··· ···
an1 an2 ··· ann − x

The term with highest degree in x is

(a11 − x) · (a22 − x) · · · · · (ann − x).

The term with degree xn−1 is given by (−x)n−1 (a11 + a22 + · · · + ann ) which is
(−x)n−1 (tr(A)).1
1
Recall that the trace of a square matrix is the sum of the entries on its main diagonal.
11
The constant term in p(x) can be found by setting x = 0, so all terms apart from the
constant term vanish. This gives

p(0) = det(A − 0 · I) = det(A)

So we have
p(x) = (−x)n + (−x)n−1 (tr(A)) + · · · + det(A).

12
Example  
3 6 −8
Find the eigenvalues and eigenvectors of the matrix A = 0 0 6 .
 

0 0 2

13
14
15
Eigenvalues, Determinant and Trace
Let p(x) be the characteristic equation of A. Factoring into eigenvalues, we have

(x − λ1 )(x − λ2 ) · · · (x − λn ) = (−x)n + (−x)n−1 (tr(A))

+ · · · + det(A)

Comparing coefficients, we get

λ1 · λ2 · · · λn = det(A)
λ1 + λ2 + · · · + λn = tr(A)

Eigenvalues of a Triangular Matrix

The eigenvalues of a triangular matrix (or diagonal matrix) are the entries on its main
diagonal.

16
One of the reasons we want to find eigenvalues is to decompose the vector space into
eigenspaces, so that we have a basis consisting of eigenvectors, called an eigenbasis.
The matrix corresponding to such an eigenbasis takes on a particularly simple form,
called the Jordan normal form, also called the Jordan form of the matrix.

17
Definition
Let A and B be square matrices of the same dimension. A is similar to B if there
exists an invertible matrix P such that

A = P BP −1 .

Remark
Similarity is an equivalence relation: A is similar to itself (reflexivity); if A is similar to
B, then B is similar to A (symmetry); and if A is similar to B and B is similar to C,
then A is similar to C.

18
Theorem
If A and B are similar matrices, then they have the same characteristic polynomial.
Hence A and B have the same eigenvalues with the same multiplicities.

Proof.
Suppose A is similar to B. Then we have

A = P BP −1
A − λI = P BP −1 − λI
= P BP −1 − P (λI)P −1
= P (B − λI)P −1
∴ det(A − λI) = det(P ) · det(B − λI) · det(P −1 )
= det(B − λI)

19
Theorem
Suppose v1 , v2 , . . . , vr are eigenvectors that correspond to distinct eigenvalues
λ1 , λ2 , . . . , λr of an n × n matrix A. Then the set {v1 , v2 , . . . , vr } is linearly
independent.

Proof.
The proof is by contradiction. Suppose the set {v1 , v2 , . . . , vr } is linearly dependent.
Let p be the least index such that vp+1 is a linear combination of the preceding linearly
independent vectors. Then there exist coefficients c1 , c2 , . . . , cp such that:

20
Proof.
c1 v1 + · · · + cp vp = vp+1 (1)
A[c1 v1 + · · · + cp vp ] = A(vp+1 )
c1 A(v1 ) + · · · + cp A(vp ) = A(vp+1 )
c1 λ1 v1 + · · · + cp λp vp = λp+1 vp+1 (2)
From(1), we get
c1 λp+1 v1 + · · · + cp λp+1 vp = λp+1 vp+1 (3)
c1 (λ1 − λp+1 )v1 + · · · +cp (λp − λp+1 )vp = 0

This gives us ci (λi − λp+1 ) = 0, and so since λi 6= λp+1 , ci = 0 for all i = 1, 2, . . . , p,

so vp+1 = 0, which is impossible. Thus the set {v1 , v2 , . . . , vn } cannot be
dependent.

21
Diagonalization

 
a11
a22
 
 
Recall that a diagonal matrix D is of the form  ..
.

 . 

ann
The entries aii , i = 1, 2, . . . , n are the eigenvalues of D.
Facts about Diagonal Matrices
Let D be the diagonal matrix above: Then
n
Y
1 det(D) = aii = a11 · a22 · · · · · ann
i=1

22
Facts about Diagonal Matrices
Let D be the diagonal matrix above: Then
n
X
2 trace(D) = aii = a11 + a22 + · · · + ann
i=1
 
ak11
ak22
 
k
 
3 D =

..

 . 

aknn

Definition
A square matrix A is diagonalizable if it is similar to a diagonal matrix. Therefore A is
diagonalizable if

A = P DP −1 , where D is a diagonal matrix.

23
Observe that if A is diagonalizable, then

A = P DP −1
Ak = (P DP −1 )k
= (P DP −1 ) · (P DP −1 ) · · · (P DP −1 ) k times
= P Dk P −1

Example" #
7 2
Let A = . Check that
−4 1
" #" #" #
1 1 5 0 −2 −1
A=
−1 −2 0 3 1 1

(Where do these values come from?) Find A5 .

24
25
Theorem
A square matrix A is diagonalizable if and only if A has n linearly independent
eigenvectors.
Proof.
Suppose A is diagonalizable. Then A = P DP −1 , where P = [v1 v2 · · · vn ]
(v1 , v2 , . . . , vn linearly independent columns) and D = diag(λ1 , λ2 , . . . , λn ). Then

AP = A[v1 v2 ··· vn ]
= [Av1 Av2 · · · Avn ]
 
λ1 0 · · · 0
 0 λ2 · · · 0 
 
PD = P  ..


 . 

0 0 · · · λn

26
Proof.
AP = A[v1 v2 ··· vn ]
= [Av1 Av2 · · · Avn ]
 
λ1 0 · · · 0
 0 λ2 · · · 0 
 
PD = P  ..


 . 

0 0 · · · λn
= [λ1 v1 λ2 v2 ··· λn vn ]

27
Proof.
Since A = P DP −1 ⇔ AP = P D, it follows that Avi = λi vi .
Therefore λi , vi is an eigenpair for A.
Since P is invertible, the columns of P are linearly independent, hence A has linearly
independent eigenvectors.

28
Proof.
Now, suppose that A has linearly independent eigenvectors. We can form the matrix P
with columns the chosen eigenvectors. We have

AP = A[v1 v2 ··· vn ]
= [Av1 Av2 ··· Avn ]
= [λ1 v1 λ2 v2 ··· λ n vn ]
 
λ1 0 ··· 0
 0 λ2 ··· 0
 
=P ..


 . 

0 0 ··· λn
= PD

So A = P DP −1 , and so it is diagonalizable.
29
Example
Diagonalize the following matrices if possible:
" #
2 3
1
4 1
 
4 2 2
2 2 4 2
 

2 2 4
 
2 4 3
3 −4 −6 −3
 

3 3 1

30
31
32
33
34
An n × n matrix with n distinct eigenvalues is diagonalizable.
If the eigenvalues are not distinct, then we have to check the geometric multiplicities:
" #
1 1
Consider the matrix A = .
1 1
Let us find the characteristic polynomial and eigenvalues of A, and hence attempt to
diagonalize the matrix.

35
Eigenvalues and Diagonalization
Suppose A is an n × n matrix with distinct eigenvalues λ1 , λ2 , . . . , λp (p ≤ n):

• For 1 ≤ k ≤ p, dim(Vλk ) ≤ mult(λk ).

• A is diagonalizable ⇔ the sum of the dimensions of the distinct eigenspaces is n.
• If A is diagonalizable and Bk is a basis for the eigenspace corresponding to λk for
each k, then the total collection of the vectors in Bk forms a basis of eigenvectors
for Rn .
 
5 0 0 0
0 5 0 0
For instance consider A =   . We will check if A is diagonalizable.
 
1 4 −3 0 
−1 −2 0 −3

36
37
38
39
40
The Jordan Canonical Form
Introduction

Some matrices cannot be diagonalized. In such cases the algebraic multiplicity of some
of the eigenvalues is greater than the corresponding geometric multiplicity.
However there is a procedure for finding a generalized eigenvector so we can find a
form of the matrix analogous to a diagonal matrix. This form is called the Jordan form
(or the Jordan normal form or Jordan canonical form) of the matrix.

41
Suppose A is a square matrix which is diagonalizable:
 
λ1 0 ··· 0
 0 λ
2 ··· 0 
A = P DP −1 , where D = 
 
 ··· ··· ··· ···


0 ··· ··· λn

We saw that the characteristic polynomial p(x) = det(A − xI), and the eigenvalues
were the roots of the polynomial.

42
If λ is a root of pA (x), then A − λI = 0.
We can consider a similar polynomial function in matrices P (X):
" #
3 −2
Consider A = . λ = 1 and λ = 2 are the eigenvalues of A. Therefore the
1 0
characteristic polynomial of A is pA (x) = (x − 1)(x − 2) = x2 − 3x + 2.
The equivalent polynomial for matrices is P (X) = X 2 − 3X + 2.

What is P(A), where A is the matrix above?

43
  
5 0 0 0
0 5 0 0
Now consider the matrix B =  .
 
1 4 −3 0 
−1 −2 0 −3
The eigenvalues of B are −3 and 5, each with multiplicity 2.
The characteristic polynomial is pB (x) = (x − 5)2 (x + 3)2 . Now consider
fB (x) = (x − 5)(x + 3). What is (B − 5I)(B + 3I)?

44
We have    
0 0 0 0 8 0 0 0
0 0 0 0 0
  8 0 0
·  = 04×4 .
 
4 −8 0

1   1 4 0 0
−1 −2 0 −8 −1 −2 0 0

What is the relation between pB (x) and fB (x)?

45
Definition
Let A be a square matrix. We define the minimal polynomial of A to be the (unique)
monic polynomial of least degree such that f (A) = 0.

Example  
2 1 1
Consider A = 0 2 0 .
 

0 0 2
Notice that 2 is an eigenvalue of A - in fact it is the only eigenvalue. This gives
pA (x) = (x − 2)3 . The minimal polynomial can be (x − 2), (x − 2)2 , or (x − 2)3 .

46
Example
We have
A − 2I 6= 0
(A − 2I)2 = 0
Hence the minimal polynomial of A is fA (x) = (x − 2)2 .

Theorem
Similar matrices have the same minimal polynomial.

Theorem
Cayley-Hamilton Let A be a square matrix. If pA (x) is the characteristic polynomial of
A, then pA (A) = 0. Hence the minimal polynomial divides the characteristic
polynomial of A.

47
Theorem
Let A be a square matrix (over C). A is diagonalizable ⇐⇒ the minimal polynomial
of A splits into a product of n distinct linear factors.

48
  
2 1 1
Let us consider the matrix A = 0 2 0 again. We can find the eigenvectors
 

0 0 2
corresponding to 2:(A −   x2 = −x3 and x1 is free. This gives us two
 2I)v = 0 gives
0 1
eigenvectors: v1 =  1  and v2 = 0.
   

−1 0
We have a problem - the eigenvalue 2 has multiplicity 3, but we have only 2 distinct
eigenvectors. How do we obtain a third vector?

49
Let us consider the equations (A − 2I)w = v1 and (A − 2I)w = v2 to see if we can
get a solution. For the first one, we have
    
0 1 1 w1 0
=
0 0 0 w2   1 
    

0 0 0 w3 −1
This gives: w2 + w3 = 0
0=1
0 = −1

So we don’t get a solution for the first equation.

50
For the second equation, we have
    
0 1 1 w1 1
0 0 0 w2  = 0
    

0 0 0 w3 0
This gives: w2 + w3 = 1
0=0
0=0

51
    
0 1
Solving this, we have w2 = 1 − w3 , and w1 is free, giving the vectors 0 and 0.
   

1 0
 
1
However 0 is already an eigenvector corresponding to 2, so we really get one new
 

0
 
0
vector v3 = 0.
 

1
Generalized Eigenvectors
We have (A − 2I)v3 = v2 , and so (A − 2I)2 v3 = (A − 2I)v2 = 0.
We call v3 a generalized eigenvector of order 2.

52
The vectors v1 , v2 and v3 form a basis for R3 (check!).
   
1 0 0 1 0 0
Let P = 0 0 1 . We have P −1 = 0 1 1. Now evaluate P −1 AP .
   

0 1 −1 0 1 0

53
The Jordan Form

Definition  
λ 1 0 ··· 0 0
 0 λ 1 ··· 0 0
 
 
 0 0 λ ··· 0 0
A k × k Jordan block Jk is a matrix of the form 
· · ·
.
 ··· ··· ··· ··· · · ·

 0 0 0 ··· λ 1
 

0 0 0 ··· 0 λ

54
 " #
λ 1
For instance, a 2 × 2 Jordan block is of the form and a 3 × 3 Jordan block is
0 λ
 
λ 1 0
of the form  0 λ 1 .
 

0 0 λ
Note that in some books the Jordan form is written so the 10 s are on the subdiagonal.

55
Let J be the Jordan form of a square matrix A. Just as in the case if A is
diagonalizable, by considering the equation P −1 AP = J, if we can find a suitable
change of coordinates, then the matrix representation can be changed to (almost) a
diagonal matrix.
Let J be a Jordan block of size k × k, and consider the canonical basis vectors e1 , . . . ek

56
We have    
λ 1 1
 λ 1  0
   
   
Je1 =  λ  · 0 = λe1
 .
   
..
. 1   .. 


λ 0
Je2 = λe2 + e1
..
.
Jek = λek + ek−1

57
Definition
Let A be a square matrix of size n × n. A sequence v1 , v2 , . . . , vr of vectors is called a
Jordan string2 for A if it satisfies the following equations:

Av1 = λv1
Avi = λvi + vi−1 1<i≤r

Jordan Form of a matrix

If we can form a basis which consists of Jordan strings for A, then the matrix
representation of the linear map T : V → V given by T (v) = Av is a matrix with
Jordan blocks (of varying sizes) down the diagonal.

2
also called a Jordan chain
58
This gives
 
Jλ1
 .. 
 . 
J =
 ..


 . 
Jλk
J is called the Jordan normal form of A.
Theorem
Every square matrix is similar to a Jordan form.

59
Example 
3 1 0
Let A = −1 1 0. The only eigenvalue of A is 2, with multiplicity 3. The
 

0 0 2
characteristic polynomial of A is p(x) = (2 − x)3 and the minimal polynomial of A is
m(x) = (x − 2)2 . This means A cannot be diagonalized.
   
1 0
The eigenvectors of A are −1 and 0.
   

0 1

60
Example  
1
Solving the equations Aw = v, where v is an eigenvector, we get a third vector 0,
 

0
   
1 1
such that −1 and 0 form a Jordan chain of length 2.
   

0 0
We can verify
   −1  
1 1 0 2 1 0 1 1 0 3 1 0
−1 0 0  0 2 0  −1 0 0 = −1 1 0
     

0 0 1 0 0 2 0 0 1 0 0 2

61
Find the Jordan form (if possible) of the following matrices
 
1 2 3
1 0 −1 7 
 

0 0 −5
 
5 −1 0 0
9
 −1 0 0
2 
7 −2

0 0
0 0 12 −3

62
  
1 −1 −2 3
0 0 −2 3 
 
3
1 −1
 
0 1
0 0 1 −2

63