Abstract Linear Algebra

Math 350

April 29, 2015

Contents

1 An introduction to vector spaces 2

1.1 Basic definitions & preliminaries . . . . . . . . . . . . . . . . . . 4
1.2 Basic algebraic properties of vector spaces . . . . . . . . . . . . . 6
1.3 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Dimension 10
2.1 Linear combination . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 Zorn’s lemma & the basis extension theorem . . . . . . . . . . . 16

3 Linear transformations 18
3.1 Definition & examples . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2 Rank-nullity theorem . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Vector space isomorphisims . . . . . . . . . . . . . . . . . . . . . 26
3.4 The matrix of a linear transformation . . . . . . . . . . . . . . . 28

4 Complex operators 34
4.1 Operators & polynomials . . . . . . . . . . . . . . . . . . . . . . 34
4.2 Eigenvectors & eigenvalues . . . . . . . . . . . . . . . . . . . . . 36
4.3 Direct sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.4 Generalized eigenvectors . . . . . . . . . . . . . . . . . . . . . . . 49
4.5 The characteristic polynomial . . . . . . . . . . . . . . . . . . . . 54
4.6 Jordan basis theorem . . . . . . . . . . . . . . . . . . . . . . . . . 56

1
Chapter 1

An introduction to vector
spaces

Abstract linear algebra is one of the pillars of modern mathematics. Its theory
is used in every branch of mathematics and its applications can be found all
around our everyday life. Without linear algebra, modern conveniences such
as the Google search algorithm, iPhones, and microprocessors would not exist.
But what is abstract linear algebra? It is the study of vectors and functions on
vectors from an abstract perspective. To explain what we mean by an abstract
perspective, let us jump in and review our familiar notion of vectors. Recall
that a vector of length n is a n × 1 array
 
a1
 a2 
 ..  ,
 
 . 
an

where ai are real numbers, i.e., ai ∈ R. It is also customary to define

  

 a1 

 a 2
  

Rn =  .  | ai ∈ R ,
 

  ..  

 
an
 

which we can think of as the set where all the vectors of length n live. Some of
the usefulness of vectors stems from our ability to draw them (at least those in
R2 or R3 ). Recall that this is done as follows:

2
 z

y
 
  a
a  b 
b
b
x c c
a

a y
b

Basic algebraic operations on vectors correspond nicely with our picture of

vectors. In particular, if we scale a vector v by a number s then in the picture
we either stretch or shrink our arrow.
y

  sb
a
s· b
b a

sa
x

The other familiar thing we can do with vectors is add them. This corre-
sponds to placing the vectors “head-to-tail” as shown in the following picture.

y
c
b

    d
a c
+ b+d
b d

x
a a+c

In summary, our familiar notion of vectors can be captured by the following

description. Vectors of length n live in the set Rn that is equipped with two
operations. The first operation takes any pair of vectors u, v ∈ Rn and gives
us a new vector u + v ∈ Rn . The second operation takes any pair a ∈ R and
v ∈ Rn and gives us a new vector a · v ∈ Rn .
With this summary in mind we now give a definition which generalizes this
familiar notion of a vector. It will be very helpful to read the following in parallel
with the above summary.

3
1.1 Basic definitions & preliminaries
Throughout we let F represent either the rational numbers Q, the real numbers
R or the complex numbers C.
Definition. A vector space over F is a set V along with two operation. The
first operation is called addition, denoted +, which assigns to each pair u, v ∈ V
an element u + v ∈ V . The second operation is called scalar multiplication
which assigns to each pair a ∈ F and v ∈ V an element av ∈ V . Moreover, we
insist that the following properties hold, where u, v, w ∈ V and a, b ∈ F:

• Associativity

u + (v + w) = (u + v) + w and a(bv) = (ab)v.

• Commutativity of +
u + v = v + u.

• Distributivity

a(u + v) = au + av and (a + b)v = av + bv

• Multiplicative Identity
The number 1 ∈ F is such that

1v = v for all v ∈ V.

• Additive Identity & Inverses

There exists an element 0 ∈ V , called an additive identity or a zero,
with the property that

0+v =v for all v ∈ V.

Moreover, for every v ∈ V there exists some u ∈ V , called an inverse of

v, such that u + v = 0.

It is common to refer to the elements of V as vectors and the elements

of F as scalars. Additionally, if V is a vector space over R we call it a real
vector space or an R-vector space. Likewise, a vector space over C is called
a complex vector space or a C-vector space.
Although this definition is intimidating at first, you are more familiar with
these ideas than you might think. In fact, you have been using vector spaces in
your previous math courses without even knowing it! The following examples
aim to convince you of this.

Examples.

4
 1. Rn is a vector space over R under the usual vector addition and scalar
multiplication as discussed in the introduction.
2. Cn , the set of column vectors of length n whose entries are complex num-
bers, is a a vector space over C.
3. Cn is also a vector space over R where addition is standard vector addition
and scalar multiplication is again the standard operation but in this case
we limit our scalars to real numbers only. This is NOT the same vector
space as in the previous example; in fact, it is as different as a line is to a
plane!
4. Let P(F) be the set of all polynomials with coefficients in F. That is

P(F) = {a0 + a1 x + · · · + an xn | n ≥ 0, a0 , . . . , an ∈ F} .

Then P(F) is a vector space over F. In this case our “vectors” are poly-
nomials where addition is the standard addition on polynomials. For ex-
ample, if v = 1 + x + 3x2 and u = x + 7x2 + x5 , then

u + v = (1 + x + 3x2 ) + (x + 7x2 + x5 ) = 1 + 2x + 10x2 + x5 .

Scalar multiplication is defined just as you might think. If v = a0 + a1 x +

· · · + an xn , then

s · v = sa0 + sa1 x + · · · + san xn .

5. Let C(R) be the set of continuous functions

f : R → R.

Then C(R) is a vector space over R where addition and scalar multiplica-
tion is given as follows. For any functions f, g ∈ C(R) we define

(f + g)(x) = f (x) + g(x).

Likewise, for scalar multiplication we define

(s · f )(x) = sf (x).

The reader should check that these definitions satisfy the axioms for a
vector space.
6. Let F be the set of all functions f : R → R. Then the set F is a vector
space over R where addition and scalar multiplication are as given in
Example 5.
You might be curious why we use the term “over” when saying that a vector
space V is over F. The reason for this is due to a useful way to visualize abstract
vector spaces. In particular, we can draw the following picture

5
 V

where our set V is sitting over our scalars F.

1.2 Basic algebraic properties of vector spaces

There are certain algebraic properties that we take for granted in Rn . For
example, the zero vector  
0
 ..  n
 . ∈R
0
is the unique additive identity in Rn . Likewise, in Rn we do not even think
about the fact that −v is the (unique) additive inverse of v. These algebraic
properties are so fundamental that we certainly would like our general vector
spaces to have these same properties as well. As the next several lemmas show,
this is happily the case.
Assume throughout this section that V is a vector space over F.
Lemma 1.1. V has a unique additive identity.
Proof. Assume 0 and 00 are both additive identities in V . To show V has a
unique additive identity we show that 0 = 00 . Playing these two identities off
each other we see that
00 = 0 + 00 = 0,
where the first equality follows as 0 is an identity and the second follows since
00 is also an identity.
An immediate corollary of this lemma is that now we can talk about the
additive identity or the zero of a vector space. To distinguish between zero, the
number in F and the zero the additive identity in V we will often denote the
latter as 0V .
Lemma 1.2. Every element v ∈ V has a unique additive inverse denoted −v.
Proof. Fix v ∈ V . As in the proof of the previous lemma, it will suffice to show
that if u and u0 are both additive inverses of v, then u = u0 . Now consider
u0 = 0V + u0 = (u + v) + u0 = u + (v + u0 ) = u + 0V = u,
where associativity gives us the third equality.

6
Lemma 1.3 (Cancellation Lemma). If u, v, w are vectors in V such that

u + w = v + w, (*)

then u = v

Proof. To show this, add −w to both sides of (*) to obtain (u + w) + −w =

(v + w) + −w. By associativity,

u + (w + −w) = v + (w + −w)

u + 0V = v + 0V
u = v.

Lemma 1.4. For any a ∈ F and v ∈ V , we have

0 · v = 0V

and
a · 0V = 0V .
Proof. The proof of this is similar to the Cancellation Lemma. We leave its
proof to the reader.

The next lemma asserts that −1 · v = −v. A natural reaction to this state-
ment is: Well isn’t this obvious, what is there to prove? Be careful! Remember
v is just an element in an abstract set V endowed with some specific axioms.
From this vantage point, it is not clear that the vector defined by the abstract
rule −1 · v should necessarily be the additive inverse of v.

Lemma 1.5. −1 · v = −v.

Proof. Observe that −1 · v is an additive inverse of v since

v + −1 · v = 1 · v + −1 · v = (1 − 1) · v = 0v = 0V ,

where the last two equalities follow from the distributive law and the previous
lemma respectively. As v has only one additive inverse by Lemma 1.2, then
−1 · v = −v.

1.3 Subspaces
Definition. Let V be a vector space over F. We say that a subset U of V is
a subspace (of V ), provided that U is a vector space over F using the same
operations of addition and scalar multiplication as given on V .

7
 Showing that a given subset U is a subspace of V might at first appear to
involve a lot of checking. Wouldn’t one need to check Associativity, Commuta-
tivity, etc? Fortunately, the answer is no. Think about it, since these properties
hold true for all the vectors in V they certainly also hold true for some of the
vectors in V , i.e., those in U . (The fancy way to say this is that U inherits all
these properties from V .) Instead we need only check the following:
1. 0V ∈ U
2. u + v ∈ U, for all u, v ∈ U (Closure under addition)
3. av ∈ U, for all a ∈ F, and v ∈ U (Closure under scalar multiplica-
tion)
Examples.

1. For any vector space V over F, the sets V and {0V } are both subspaces of
V . The former is called a nonproper subspace while the latter is called
the trivial or zero subspace. Therefore a proper nontrivial subspace of
V is one that is neither V nor {0V }.
2. Consider the real vector space R3 . Fix real numbers a, b, c. Then we claim
that the subset

U = (x, y, z) ∈ R3 | ax + by + cz = 0


is a subspace of R3 . To see this we just need to check the three closure

properties. First, note that 0R3 = (0, 0, 0) ∈ U , since 0 = a0 + b0 + c0. To
see that U is closed under addition let u = (x1 , y1 , z1 ), v = (x2 , y2 , z2 ) ∈ U .
Since

a(x1 +x2 )+b(y1 +y2 )+c(z1 +z2 ) = (ax1 +by1 +cz1 )+(ax2 +by2 +cz2 ) = 0+0 = 0

we see that u + v = (x1 + x2 , y1 + y2 , z1 + z2 ) ∈ U . Lastly, a similiar check

shows that U is closed under scalar multiplication. Let s ∈ R, then

0 = s0 = s(ax1 + by1 + cz1 ) = asx1 + bsy1 + csz1 .

This means that su = (sx1 , sy1 , sz1 ) ∈ U .

3. Recall that P(R) is the vector space over R consisting of all polynomials
whose coefficients are in R. In fact, this vector space is also a subspace
of C(R). To see this note that P(R) ⊂ C(R). Since the zero function
and the zero polynomial are the same function, then 0C(R) ∈ P(R). Since
we already showed that P(R) is a vector space then it is certainly closed
under addition and scalar multiplication, so P(R) is a subspace of C(R).

8
4. This next examples demonstrates that we can have subspaces within sub-
spaces. Consider the subset P ≤n (R) of P(R) consisting of all those poly-
nomials with degree ≤ n. Then, P ≤n (R) is a subspace of P(R). As the
degree of the zero polynomial is (defined to be) −∞, then P ≤n (R). Ad-
ditionally, if u, v ∈ P ≤n (R), then clearly the degree of u + v is ≤ n, so
u + v ∈ P ≤n (R). Likewise P ≤n (R) is certainly closed under scalar multi-
plication. Combining this example with the previous one shows that we
actually have the following sequence of subspaces

P ≤0 (R) ⊂ P ≤1 (R) ⊂ P ≤2 (R) ⊂ · · · ⊂ P(R) ⊂ C(R).

5. The subset D of all differentiable functions in C(R), is a subspace of the R-

vector space C(R). Since the zero function f (x) = 0 is differentiable, and
the sum and scalar multiple of differentiable functions is differentiable, it
follows that D is a subspace.
6. Let U be the set of solutions to the differential equation f 00 (x) = −f 0 (x),
i.e.,
U = {f (x) | f 00 (x) = −f 0 (x)} .
Then U is a subspace D, the space of differentiable functions. To see this,
first note that the zero function is a solution to our differential equation.
Therefore U contains our zero vector. To check the closure properties
let f, g ∈ U . Therefore f 00 (x) = −f 0 (x) and that g 00 (x) = −g(x) and
moreover,

(f + g)00 (x) = f 00 (x) + g 00 (x) = −f 0 (x) + −g 0 (x) = −(f + g)0 (x).

In other words, f + g ∈ U . To check closure under scalar multiplication

let s ∈ R. Now

(s · f )00 (x) = sf 00 (x) = −sf 0 (x) = −(sf )0 (x),

and so s · f ∈ U .

9
Chapter 2

Dimension

2.1 Linear combination

Definition. A linear combination of the vectors v1 , . . . , vm is any vector of
the form
a1 v1 + · · · + am vm ,
where a1 , . . . , am ∈ F. For a nonempty subset S of V , we define
span(S) = {a1 v1 + · · · + am vm | v1 , . . . , vm ∈ S, a1 , . . . , am ∈ F} ,
and call this set the span of S. If S = ∅, we define span(∅) = {0V }. Lastly, if
span(S) = V , we say that S spans V or that S is a spanning set for V .
For example, consider the vector space Rn and let S = {e1 , . . . , en }, where
 
0
 .. 
 . 
 
 1
ei =  ,

 .
 ..


0
that is, the vector whose entries are all 0 except
 the ith, which is 1. Then
a1
n  .. 
R = span(S), since we can express any vector  .  ∈ R as
an
 
a1
 .. 
 .  = a1 e1 + · · · an en .
an
The vectors e1 , . . . , en play a fundamental role in the theory of linear algebra.
As such they are named the standard basis vectors for Rn .

10
 Now consider the vector space of continuous functions C(R). For brevity let
us write the function f (x) = xn as xn and let S = {1, x, x2 , . . .}. Certainly

span(S) = a0 1 + a1 x + a2 x2 + · · · + an xn | n ≥ 0, a0 , . . . , an ∈ R = P(R).


This example raises a subtle point we wish to make explicit. Although our set
S has infinite cardinality, each element in span(S) is a linear combination of a
finite number of vectors in S. We do not allow something like 1 + x + x2 + · · ·
to be an element in span(S). A good reason for this restriction is that, in this
case, such an expression is not defined for |x| ≥ 1, so it could not possibly be
an element of C(R).
Example 3 in Section 1.3 shows that P(R) is a subspace of C(R). The next
lemma provides an alternate way to see this fact where we take S = {1, x, x2 , . . .}
and V = C(R). Its proof is left to the reader.
Lemma 2.1. For any S ⊆ V , we have that span(S) is a subspace of V .
To motivate the next definition, consider the set of vectors from R2 :
     
1 1 3
S= , , .
1 0 2

Since      
a 1 1
=b + (a − b)
b 1 0
 
3
we see that span(S) = R2 . That said, the vector is not needed in order
2
 
3
to span R2 . It is in this sense that is an “unnecessary” or “redundant”
2
 
3
vector in S. The reason this occurs is that is a linear combination of the
2
other two vectors in S. In particular, we have
     
3 1 1
= +2 ,
2 0 1
or        
0 1 1 3
= +2 − .
0 0 1 2
Consequently, the next definition makes precise this idea of “redundant” vectors.
Definition. We say a set S of vectors is linearly dependent if there exists
distinct vectors v1 , . . . , vm ∈ S and scalars a1 , . . . , am ∈ F, not all zero, such
that
a1 v1 + · · · + am vm = 0V .
If S is not linearly dependent we say it is linearly independent.

11
 As the empty set ∅ is a subset of every vector space, it is natural to ask if ∅ is
linearly dependent or linearly independent. The only way for ∅ to be dependent
is if there exists some vectors v1 , . . . , vm in ∅ whose linear combination is 0V .
But we are stopped dead in our tracks since there are NO vectors in ∅. Therefore
∅ cannot be linearly dependent, hence, ∅ is linearly independent.
Lemma 2.2 (Linear Dependence Lemma). If S is a linearly dependent set,
then there exists some element v ∈ S so that
span(S − v) = span(S).
Moreover, if T is a linear independent subset of S, we may choose v ∈ S − T .
Proof. As S is linearly dependent we know there exist distinct vectors
v1 , . . . , vi , vi+1 , . . . , vm
| {z } | {z }
∈T ∈S−T

and scalars a1 , . . . , am , not all zero, such that

a1 v1 + · · · + am vm = 0V .
As T is linearly independent and v1 , . . . , vi are distinct, we cannot have ai+1 =
· · · = am = 0. (Why?) Without loss of generality we may assume that am 6= 0.
At this point choose v = vm and observe that v ∈ / T . Rearranging the above
equation we obtain
 
a1 ai ai+1 am−1
v = vm = − v1 + . . . + vi + vi+1 + . . . + vm−1 ,
am am am am
which implies that v ∈ span(S − v). Moreover, since S − v ⊆ span(S − v), we
see that S ⊂ span(S − v). Lemma 2.1 now implies that
span(S) ⊆ span(S − v) ⊆ span(S),
which yields our desired result.
Lemma 2.3 (Linear Independence Lemma). Let S be linearly independent. If
v ∈ V but not in span(S), then S ∪ {v} is also linearly independent.
Proof. If V − span(S) = ∅, then there is nothing to prove. Otherwise, let v ∈ V
such that v ∈/ span(S) and assume for a contradiction that S ∪ {v} is linearly
dependent. This means that there exists distinct vectors v1 , . . . , vm ∈ S ∪ {v}
and scalars a1 , . . . , am , not all zero, such that a1 v1 + a1 v1 + · · · + am vm = 0V .
First, observe that v = vi for some i and that ai 6= 0. (Why?) Without loss of
generality we may choose i = m. Just like the calculation we performed in the
proof of the Linear Dependence Lemma, we also have
 
a1 am−1
v = vm = − v1 + · · · + vm−1 ∈ span(S),
am am
which contradicts the fact that v ∈
/ span(S). We conclude that S ∪{v} is linearly
independent.

12
2.2 Bases
Definition. A (possibly empty) subset B of V is called a basis provided it is
linearly independent and spans V .
Examples.

1. The set of standard basis vectors e1 , . . . , en are a basis for Rn and Cn .

(This explains their name!)
2. The set {1, x, x2 , . . . , xn } form a basis for P ≤n (F).
3. The infinite set {1, x, x2 , . . .} form a basis for P(F).
4. The emptyset ∅ forms a basis for the trivial vectors space {0V }. This might
seem odd at first but consider the definitions involved. First ∅ was defined
to be linearly independent. Additionally, we defined span(∅) = {0V }.
Therefore ∅ must be a basis for {0V }.
The proof of the next lemma is left to the reader.
Lemma 2.4. The subset B is a basis for V if and only if every vector u ∈ V is
a unique linear combination of the vectors in B.
Theorem 2.5 (Basis Reduction Theorem). Assume S is a finite set of vectors
such that span(S) = V . Then there exists some subset B of S that is a basis for
V.
Proof. If S happens to be linearly independent we are done. On the other hand,
if S is linearly dependent, then, by the Linear Dependence Lemma, there exists
some v ∈ S such that
span(S − v) = span(S)
If S − v is not independent, we may continue to remove vectors until we obtain
a subset B of S which is independent. (Note that sinceS is finite we cannot con-
tinue removing vectors forever, and since ∅ is linearly independent this removal
process must result in an independent set.) Additionally,

span(B) = V,

since the Linear Dependence Lemma guarantees that the subset of S obtained
after each removal spans V . We conclude that B is a basis for V .
Theorem 2.6 (Basis Extension Theorem). Let L be a linearly independent
subset of V . Then there exists a basis B of V such that L ⊂ B.
We postpone the proof of this lemma to Section 2.4.
Corollary 2.7. Every vector space has a basis.
Proof. As the empty set ∅ is a linearly independent subset of any vector space
V , the Basis Extension Theorem implies that V has a basis.

13
2.3 Dimension
Lemma 2.8. If L is any finite independent set and S spans V , then |L| ≤ |S|.
Proof. Of all the sets that span V and have cardinality |S| choose S 0 so that it
maximizes |L ∩ S 0 |. If we can prove that L ⊂ S 0 we are done, since

|L| ≤ |S 0 | = |S|.

For a contradiction, assume L is not a subset of S 0 . Fix some vector u ∈ L − S 0 .

As S 0 spans V and does not contain u, then D = S 0 ∪ {u} is linearly dependent.
Certainly, span(D) = V . Now define the linearly independent subset

T = L ∩ D,

and observe that u ∈ T . By the Linear Dependence Lemma there exists some
v ∈ D − T so that
span(D − v) = span(D) = V.
Observe u 6= v. This immediately yields our contradiction since |D − v| = |S 0 | =
|S| and D − v has one more vector from L (the vector u) than S 0 does. As this
contradicts our choice of S 0 , we conclude that L ⊂ S 0 as needed.
Theorem 2.9. Let V be a vector space with at least one finite basis B. Then
every basis of V has cardinality | B |.

Proof. Fix any other basis B 0 of V . As B 0 spans V , Lemma 2.8, with S = B 0

and L = B, implies that | B | ≤ | B0 |. Our proof will now be complete if we
can show that | B | ≮ | B 0 |. For a contradiction, assume that n = | B | < | B 0 |
and let L be any n + 1 element subset of B 0 and S = B. (As B 0 is a basis, L
is linearly independent.) Lemma 2.8 then implies that n + 1 = |L| ≤ |S| = n,
which is absurd.
Definition. A vector space V is called finite-dimensional if it has a finite
basis B. As all bases in this case have the same cardinality, we call this common
number the dimension of V and denote it by dim(V ).
A beautiful consequence of Theorem 2.9, is that in order to find the dimen-
sion of a given vector space we need only find the cardinality of some basis for
that space. Which basis we choose doesn’t matter!
Examples.

1. The dimension of Rn is n since {e1 , . . . , en } is a basis for this vector space.

2. Recall that {1, x, x2 , . . . , xn } is a basis for P ≤n (R). Therefore dim P ≤n (R) =
n + 1.

14
 3. Consider the vector space C over C. A basis for this space is {1} since
every element in C can be written uniquely as s · 1 where s is a scalar in
C. Therefore, we see that this vector space has dimension 1. We can write
this as dimC (C) = 1, where the subscript denotes that we are considering
C as a vector space over C.
On the other hand, recall that C is also a vector space over R. A basis
for this space is {1, i} since, again, every element in C can be uniquely
expressed as
a·1+b·i
where a, b ∈ R. It now follows that this space has dimension 2. We write
this as dimR (C) = 2.
4. What is the dimension of the trivial vector space {0V }? A basis for this
space is the emptyset ∅, since by definition it is linearly independent and
span(∅) = {0V }. Therefore, this space has dimension |∅| = 0.
We now turn our attention to proving some basic properties about dimension.
Theorem 2.10. Let V be a finite-dimensional vector space. If L is any linearly
independent set in V , then |L| ≤ dim(V ). Moreover, if |L| = dim(V ), then L is
a basis for V .
Proof. By the Basis Extension Theorem, we know that there exists a basis B
such that

L ⊆ B. (*)

This means that |L| ≤ | B | = dim(V ). In the special case that |L| = dim(v) =
| B |, then (∗) implies L = B, i.e., L is a basis.
A useful application of this theorem is that whenever we have a set S of
n + 1 vectors sitting inside an n dimensional space, then we instantly know that
S must be dependent. The following corollary, is another useful consequence of
this theorem.

Corollary 2.11. If V is a finite-dimensional vector space, and U is a subspace,

then dim(U ) ≤ dim(V ).
It might occur to the reader that an analogous statement for spanning sets
should be true. That is, if we have a set S of n − 1 vectors which sits inside an
n-dimensional vectors space V can we conclude that span S 6= V ? As the next
theorem shows, the answer is yes.
Theorem 2.12. Let V be a finite-dimensional vector space. If S is any spanning
set for V , then dim(V ) ≤ |S|. Moreover, if |S| = dim(V ) then S is a basis for
V.

15
 To prove this theorem, we would like to employ similar logic as in the proof
of the previous theorem but with Theorem 2.5 in place of Theorem 2.6. The
problem with this is that S is not necessarily finite. Instead, we may use the
following lemma in place of Theorem 2.6. Both its proof, and the proof of this
lemma are left as exercises for the reader.
Lemma 2.13. Let V be a finite-dimensional vector space. If S is any spanning
set for V , then there exists a subset B of S, which is a basis for V .

2.4 Zorn’s lemma & the basis extension theorem

Definition. Let X be a collection of sets. We say a set B ∈ X is maximal if
there exists no other set A ∈ X such that B ⊂ A. A chain in X is a subset
C ⊆ X such that for any two sets A, B ∈ C either

A⊆B or B ⊆ A.

Lastly, we say C ∈ X is an upper bound for a chain C if A ⊆ C, for all A ∈ C.

Observe that if C is a chain and A1 , . . . , Am ∈ C, then there exists some
1 ≤ k ≤ m, such that
Ak = A1 ∪ A2 ∪ · · · ∪ Am .
This observation follows by a simple induction, which we leave to the reader.
Zorn’s Lemma. Let X be a collection of sets such that every chain C in X has
an upper bound. Then X has a maximal element.
Lemma 2.14. Let V be a vector space and fix a linearly independent subset L.
Let X be the collection of all linearly independent sets in V that contain L. If
B is a maximal element in X, then B is a basis for V .
Proof. By definition of the set X, we know that B is linearly independent. It
only remains to show that span(B) = V . Assume for a contradiction that
it does not. This means there exists some vector v ∈ V − span(B). By the
Linear Independence Theorem, B ∪ {v} is linearly independent and hence must
be an element of X. This contradicts the maximality of B. We conclude that
span(B) = V as desired.
Proof of Theorem 2.6. Let L be a linearly independent subset of V and define
X to be the collection of all linearly independent subsets of V containing L. In
light of Lemma 2.14, it will suffice to prove that X contains a maximal element.
An application of Zorn’s Lemma, assuming its conditions are met, therefore
completes our proof. To show that we can use Zorn’s Lemma, we need to check
that every chain C in X has an upper bound. If C = ∅, then L ∈ X is an upper
bound. Otherwise, we claim that the set
[
C= A,
A∈C

16
is an upper bound for our chain C. Clearly, A ⊂ C, for all A ∈ C. It now
remains to show that C ∈ X, i.e., L ⊂ C and C is independent. As C 6= ∅, then
for any A ∈ C, we have
L ⊆ A ⊆ C.
To show C is independent, assume

a1 v1 + · · · + am vm = 0V ,

for some (distinct) vi ∈ C and a1 ∈ F. By construction of C each vector vi must

be an element of some Ai . As C is a chain the above remark implies that

Ak = A1 ∪ A2 ∪ · · · ∪ Am ,

for some 1 ≤ k ≤ m. Therefore all the vectors v1 , . . . , vm lie inside the linearly
independent set Ak ∈ X. This means our scalars a1 , . . . , am are all zero. We
conclude that C is an independent set.

17
Chapter 3

Linear transformations

In this chapter, we study functions from one vector space to another. So that the
functions of study are linked, in some way, to the operations of vector addition
and scalar multiplication we restrict our attention to a special class of functions
called linear transformations. Throughout this chapter V and W are always
vector spaces over F.

3.1 Definition & examples

Definition. We say a function T : V → W is a linear transformation or a
linear map provided
T (u + v) = T (u) + T (v)
and
T (av) = aT (v) for all v ∈ V, a ∈ F
for all u, v ∈ V and a ∈ F. We denote the set of all such linear transformations,
from V to W , by L(V, W ) .
To simplify notation we often write T v instead of T (v). It is not a coincidence
that this simplified notation is reminiscent of matrix multiplication; we expound
on this in Section 3.4.
Examples.

1. The function T : V → W given by T v = 0W for all v ∈ V is a linear map.

Appropriately, this is called the zero map.
2. The function I : V → V , given by Iv = v, for all v ∈ V is a linear map.
It is called the identity map.
3. Let A be an m × n matrix with real coefficients. Then A : Rn → Rm given
by matrix-vector product is a linear map. In fact, we show in Section 3.4,
that, in some sense, all linear maps arise in this fashion.

18
 4. Recall the vector space P ≤n (R). Then the map T : P ≤n (R) → Rn defined
by  
a0
 a1 
T (a0 + a1 x + · · · an xn ) =  . 
 
 .. 
an
is a linear map.
5. Recall the space of continuous functions C. An example of a linear map
on this space is the function T : C → C given by T f = xf (x).
6. Recall that D is the vector space of all differentiable function f : R → R
and F is the space of all function g : R → R. Define the map ∂ : D → F,
so that ∂f = f 0 . We see that ∂ is a linear map since
∂(f + g) = (f + g)0 = f 0 + g 0 = ∂f + ∂g
and
∂(af ) = (af )0 = af 0 = a∂f.
7. From calculus, we obtain another linear map T : C → R given by
Z 1
Tf = f dx.
0

The reader should convince himself that this is indeed a linear map.
Although the above examples draw from disparate branches of mathematics,
all these maps have the property that they map the zero vector to the zero
vector. As the next lemma shows, this is not a coincidence.
Lemma 3.1. Let T ∈ L(V, W ). Then T (0V ) = 0W .
Proof. To simplify notation let 0 = 0V . Now
T (0) = T (0 + 0) = T (0) + T (0).
Adding −T (0) to both sides yields
T (0) + −T (0) = T (0) + T (0) + −T (0).
Since all these vectors are elements of W , simplifying gives us 0W = T (0).
It is often useful to “string together” existing linear maps to obtain a new
linear map. In particular, let S ∈ L(U, V ) and T ∈ L(V, W ) where U is another
F-vector space. Then the function defined by
ST (v) = S(T v)
is clearly a linear map in L(U, W ). (The reader should verify this!) We say
that T S is the composition or product of S with T . The reader may find the
following figure useful for picturing the product of two linear maps.

19
 S T

v Sv T (Sv)

U V W

There is another important way to combine two existing linear maps to

obtain a third. If S, T ∈ L(V, W ) and a, b ∈ F, then we may define the function

(aS + bT )(v) = aSv + bT v

for any v ∈ V . Again we encourage the reader to check that this function is a
linear map in L(V, W ).
Before closing out this section, we first pause to point out a very important
property of linear maps. First, we need to generalize the concept of a line in
Rn to a line in an abstract vector space. Recall, that any two vector v, u ∈ Rn
define a line via the expression av + u, where a ∈ R. As this definition requires
only vector addition and scalar multiplication we may “lift” it to the abstract
setting. Doing this we have the following definition.
Definition. Fix vectors u, v ∈ V . We define a line in V to be all points of the
form
av + u, where a ∈ F .
Now consider applying a linear map T ∈ L(V, W ) to the line av + u. In
particular, we see that

T (av + u) = aT (v) + T (v).

In words, this means that the points on our line in V map to points on a new
line in W , defined by the vectors T (v), T (u) ∈ W . In short we say that linear
transformations have the property that they map lines to lines.
In light of the preceding lemma, even more is true. Observe that any line
containing 0V is of the form av + 0V . (We think of such lines as analogues to
lines through the origin in Rn .) Lemma 3.1 now implies that such lines are
mapped to lines of the form

T (av + 0V ) = aT (v) + T (0W ) = aT (v) + 0W .

In other words, linear transformations actually map lines through the origin in
V to lines through the origin in W .

20
3.2 Rank-nullity theorem
The aim of this section is to prove the Rank-Nullity Theorem. This theorem
describes a fundamental relationship between linear maps and dimension. An
immediate consequence of this theorem, will be an beautiful proof to the fact
that a homogeneous system of equations with more variables than equations
must have an infinite number of solutions.
We begin with the following definition.

Definition. Let T : V → W be a linear map. We say T is injective if T v = T u

implies that u = v. On the other hand, we say that T is surjective provided
that for every w ∈ W , there exists some v ∈ V such that T v = w. A function
that is both injective and surjective is called bijective.

It might help to think of T as a cannon that shoots shells (elements in V ) at

targets (elements of W )1 . From this perspective there is an easy way to think
about surjectivity and injectivity.
• Surjectivity means that the cannon T hits every element in W .
• Injectivity means that every target in W is hit at most once.

• Bijectivity means that every target is hit exactly once. In this case we can
think of T as “matching up” the elements in T with the elements in W .

u Tu

v Tv

V W

As is always the case in mathematics, it will be beneficial to have more

than one description of a single idea. Our next lemma provides this alternative
description of injectivity and surjectivity.
Definition. Fix T ∈ L(V, W ). Define the null space of T to be

null T = {v ∈ V | T v = 0W }

and the range of T to be

ran T = {T v | v ∈ V } .
1 D. Saracino, A first course in abstract algebra.

21
 Observe that the null space is a subset of V where as the range is a subset
of W .
Lemma 3.2. Let T ∈ L(V, W ).
1. T is surjective if and only if ran T = W .
2. T is injective if and only if null T = {0V }.
Proof. Saying that T is surjective is equivalent to saying that for any w ∈ W
there exists some v ∈ V such that T v = w. In other words,

ran T = {T v | v ∈ V } = W,

as claimed. For the second claim, begin by assume T is injective. This means
that u = v whenever T u = T v. Consequently,

null T = {v ∈ V | T v = 0W = T (0V )} = {0V },

as desired. For the other direction, assume {0V } = null T and consider any two
vectors u, v ∈ V such that T u = T v. To prove T is injective we must show that
u = v. To this end, the linearity of T yields

T (u − v) = 0W .

This means that u − v ∈ null T = {0V }. Hence u − v = 0V or u = v as

desired.
The next lemma, whose proof we leave to the reader, states that our two
sets null T and ran T are no ordinary sets; they are vector spaces in their own
right.
Lemma 3.3. Let T ∈ L(V, W ). Then null(T ) is a subspace of V and ran(T ) is
a subspace of W .
Examples.

1. Consider the linear map T : Rn+1 → P ≤n given by

T (a0 , . . . , an+1 ) = a0 + a1 x + · · · an xn .

Then null T = {(0, . . . , 0)} = {0R2 } and ran T = P ≤n .

2. The T : P → P given by T (f )(x) = f 0 (x) is linear with null T = constant
polynomials and ran T = P. Therefore, T is surjective but not injective.
3. Consider the map T : R2 → R3 given by
 
  x
x
T = y .
y
−(x + y)

22
 6 R3 . In fact,
Then null T = {0R2 }, but ran T =
  
 x 
ran T =  y  ∈ R3 | x + y + z = 0
z
 

which is the equation of a plane in R3 . So this map if injective but not

surjective.
Before reading any further, can you spot a relation among the dimensions
of the domain, range and null space in example 3)?
As the next theorem shows, there is one and an important one at that! The
reader should check that in fact, both examples 1) and 3) do indeed satisfy this
theorem.
Theorem 3.4 (Rank-Nullity). Assume V is finite-dimensional. For any T ∈
L(V, W ),
dim V = dim(null T ) + dim(ran T ).
Proof. As null T is a subspace of V , and hence a vector space in its own right, it
has a basis. Let {e1 , . . . , ek } be such a basis for null T . By the Basis Extension
Theorem (Theorem 2.6), there exists vectors f1 , . . . , fm so that
B = {e1 , . . . ek , f1 , . . . , fm }
is a basis for V . Since
dim V = k + m = dim null T + m,
we must show dim ran T = m. To this end, it suffices to show that S =
{T f1 , . . . , T fm } is a basis for ran T . We do this in two parts.
We first show that span(S) = ran T . This readily follows from the following:
ran T = {T v | v ∈ V }
= {T (a1 e1 + · · · + ak ek + b1 f1 + · · · + bm fm ) | ai , bi ∈ F}
= {a1 T e1 + · · · + ak T ek + b1 T f1 + · · · + bm T fm ) | ai , bi ∈ F}
= {b1 T f1 + · · · + bm T fm ) | bi ∈ F}
= span(S),
where the second equality uses the fact that B is a basis for V and the third
equality follows since e1 , . . . , ek ∈ null T .
We now turn our attention to showing that S is linearly independent. To do
this we must show that if a1 T f1 + · · · + am T fm = 0W , then all the scalars are
zero. By the linearity of T we have
0W = T (a1 f1 + · · · + am fm ),
which means that a1 f1 + · · · + am fm ∈ null T . As e1 , . . . , ek are a basis for null T
it follows that
b1 e1 + · · · + bk ek = a1 f1 + · · · + am fm ,

23
for some scalars bi . Rearranging we see that

0V = −(b1 e1 + · · · + bk ek ) + a1 f1 + · · · + am fm .

The linear independence of B = {e1 , . . . ek , f1 , . . . , fm } forces all the scalars to

be zero. In particular, a1 = a2 = · · · = 0 as needed. As we have shown that S
is independent and spans ran T , we may conclude that it is a basis for ran T as
desired.
To motivate our first corollary recall the following fact about plain old sets.
If X and Y are sets and f : X → Y is injective then |X| ≤ |Y |. On the other
hand if f is surjective, then |X| ≥ |Y |. As dimension measures the “size” of a
vector space, the following is the vector space analogue to this set theory fact.
Corollary 3.5. Let V and W be finite-dimensional vector spaces and let T be
an arbitrary linear map in L(V, W ).
1. If dim V > dim W , then T is not injective.

2. If dim V < dim W , then T is not surjective.

Proof. Fix T ∈ L(V, W ). To prove the first claim, assume dim V > dim W . By
the Rank-Nullity Theorem we see that

dim(null T ) = dim V − dim(ran T ) ≤ dim V − dim W > 0,

where the second inequality follows since ran T is a subspace of W . Conse-

quently, null T is not the trivial space, so by Lemma 3.2 T is not injective.
Likewise, if dim V < dim W , then

dim(ran T ) = dim V − dim(null T ) ≤ dim V < dim W.

Consequently, ran T cannot be all of W , i.e., T is not surjective.

In general mathematical function can be injective without being surjective
and vice versa. For example consider the functions f, g : Z → Z given by
f (n) = 2n and 
n + 1 if n > 0

g(n) = 0 if n = 0 .

n + 2 if n < 0.


Then f is injective but not surjective and g is surjective but not injective since
g(−2) = 0 = g(0). Consequently the next theorem is quiet amazing. It states
that if two vector spaces have the same dimension, then a linear map between
them is surjective if and only if it is injective!

Corollary 3.6. Let V and W be finite-dimensional vector spaces with the same
dimension. For any linear map T ∈ L(V, W ) we have that T is surjective if and
only if T is injective.

24
Proof. By the Rank-Nullity Theorem states we always have

dim V = dim(null T ) + dim(ran T ).

Now, observe that

T is surjective ⇐⇒ ran T = W
⇐⇒ dim(ran T ) = dim W
⇐⇒ dim(ran T ) = dim V
⇐⇒ dim(null T ) = 0
⇐⇒ null T = {0V }
⇐⇒ T is injective,

where the third equivalence is the fact that dim V = dim W and the fourth
equivalence follows from the Rank-Nullity Theorem.

Before closing this section, let demonstrate two beautiful applications of

the Rank-Nullity Theorem. First, consider a homogeneous system of linear
equations

a11 x1 + . . . + a1n xn = 0
a21 x1 + . . . + a2n xn = 0
..
.
am1 x1 + . . . + a1n xn = 0.

A standard result from any elementary linear algebra course is that if this system
has more variable than equations (n > m), then a non-trivial solution to the
system exists, i.e., one other than x1 = · · · xn = 0. We are now in a position to
give an elegant proof of this fact. First, rewrite this system in matrix form as
Ax = 0, where
   
a11 · · · a1n x1
A =  ... .. ..  and x =  ...  .
  
. . 
am1 ··· amn xn

Recall that A : Rn → Rm is a linear map given by matrix vector multiplication.

As n > m, Corollary 3.5 states that T is not injective and hence {0V } ( null T .
Therefore there exists some nonzero x ∈ null T . As x is nonzero and Ax = 0,
we see that our system has a nontrivial solution as claimed.
For our second application, consider a system of (not necessarily homoge-

25
neous) linear equations

a11 x1 + . . . + a1n xn = b1
a21 x1 + . . . + a2n xn = b2
..
.
am1 x1 + . . . + a1n xn = bm .

In matrix form this becomes Ax = b. Another standard result from elementary

linear algebra is that if our system has more equations than unknowns, i.e., n <
m, then there exists some choice of b ∈ Rm so that our system is inconsistent
(has no solutions). To prove this, again think of A : Rn → Rm as a linear map.
Corollary 3.5 tell us that A is not surjective. This means there exists some
b ∈ Rm so that no choice of x ∈ Rn gives Ax = b. In other words, for this b,
our system is inconsistent.

3.3 Vector space isomorphisims

Our next goal is to identify when two vectors spaces are essential the same.
Mathematically, we say they are isomorphic which is latin for “same shape”.
To see what we mean by this, imagine you are given an F-vector space V and you
paint all its elements red to obtain a new space W . Although this new space W
“looks” different (all its vectors are red!), it still has the same algebraic structure
as V .
A more concrete example of isomorphic vector spaces has been in front of
us almost since page one! In fact, it might have already occurred to you that as
vector spaces Rn+1 and P ≤n were strikingly similar. Certainly as sets they are
very different – one is a set of vectors while the other is a set of polynomials!
In terms of their vector space structure this is just a cosmetic difference. To
convince you note that an arbitrary vector in Rn+1 looks like
 
a0
 .. 
 . 
an

while an arbitrary vector in P ≤n looks like a0 + a1 x + · · · + an xn . In either case,

a vector is just a list of n + 1 numbers {a0 , . . . , an }. Moreover, the operation of
addition and scalar multiplication are essentially the same in both spaces too.
For example, addition in Rn+1 is given by
     
a0 b0 a0 + b0
 ..   ..   ..
 . + . =

. 
an bn an + bn

26
which is really no different then addition in P ≤n which looks like

(a0 + a1 x + · · · + an xn ) + (b0 + b1 x + · · · + bn xn )
= (a0 + b0 ) + (a1 + b1 )x + · · · + (an + bn )xn .

. Intuitively, these two spaces are the “same”. With this example in mind
consider the formal definition for vector spaces to be isomorphic.
Definition. We say two vector spaces V and W are isomorphic and write
V ∼= W , if there exists T ∈ L(V, W ) which is both injective and surjective. We
call such a T an isomorphism.
Theorem 3.7. Two finite-dimension vector spaces V and W are isomorphic if
and only if they have the same dimension.
Proof. Assume V and W are isomorphic. This means there exists a linear map
T : V → W that is both surjective and injective. Corollary 3.5 immediately
implies that dim V = dim W . For the reverse direction, let B V = {v1 , . . . , vn }
be a basis for V and B W = {w1 , . . . , wn } be a basis for W . As every vector
v ∈ V can be written (uniquely) as

v = a1 v1 + · · · + an vn

for ai ∈ F, we may define a function T : V → W by

T v = a1 w1 + · · · + an wn .

Observe that the uniqueness of our representation of v implies that T is a well-

defined function. Moreover, a straightforward check reveals that T is indeed a
linear map. It only remains to show that T is an isomorphism. To see that T is
injective, let that v ∈ null T and let bi ∈ F be such that v = b1 v1 + · · · + bn vn .
This means
0W = T v = b1 w1 + · · · + bn wn .
Since B W is an independent set, it follows that all our scalars bi must be 0 and,
in turn, v = 0. This shows that null T = {0V }, i.e., T is injective.
To see that T is also surjective, note that any vector w ∈ W can be written
as
w = c1 w1 + · · · + cm wm ,
for some choice of scalars ci (why?). Now consider the vector c1 v1 +· · ·+cm vm ∈
V and observe that

T (c1 v1 + · · · + cm vm ) = c1 w1 + · · · + cm wm = w.

This shows that T is surjective.

Definition. Let T ∈ L(V, W ). We say T is invertible provided there exists
some S ∈ L(W, V ) so that ST : V → V is the identity map on V and T S :
W → W is the identity map on W . We call S an inverse of T .

27
 As a consequence of the next lemma, we are able to refer to the inverse of
T which we denote by T −1 .
Lemma 3.8. Let T ∈ L(V, W ). If T is invertible, then its inverse is unique.
Proof. Assume S and S 0 are both inverses for T . Then

S = SIW = ST S 0 = IV S 0 = S 0

where IV and IW are the identity maps on V and W respectively.

Lemma 3.9. Let T ∈ L(V, W ). Then T is invertible if and only if T is an
isomorphism.

Proof. Let us first assume that T is invertible. We must prove that T is both
injective and surjective. To see injectivity, let u ∈ null T , then

u = IV u = T −1 T u = T −1 0W = 0V ,

where IV is the identity map on V . We conclude that null T = {0V } and hence
T is injective. To see that T is also surjective fix w ∈ W . Observe that T maps
the vector T −1 w ∈ V onto w since

T (T −1 w) = T T −1 w = IW w = w.

We conclude that T is surjective.

Now assume T : V → W is an isomorphism. As T is both injective and
surjective, then for every w ∈ W there exists exactly one v ∈ V so that T v = w.
Now define the function S : W → V by S(w) = v. We claim T −1 = S. By
definition we have S(T v) = v for all v ∈ V and T S(w) = w for all w ∈ W . It
only remains to show that S is linear. Let w1 , w2 ∈ W and let v1 , v2 be the
unique vectors in V so that T vi = wi . As T is linear then T (v1 + v2 ) = w1 + w2 .
By definition of S we now have

S(w1 + w2 ) = v1 + v2 = S(w1 ) + S(w2 ).

Likewise,
S(aw1 ) = aw1 = aS(w1 ).
We may now conclude that T −1 = S and hence T is invertible.

3.4 The matrix of a linear transformation

In this section we study a striking connection between linear transformations
and matrices. In fact, we will see that linear transformations and matrices are
really two sides of the same coin! Before beginning let us review the basics of

28
matrix multiplication. Let A = [~a1 · · · ~an ] where ~ai ∈ Rm , so that A is an m × n
matrix whose ith column is the vector ~ai . For any ~b ∈ Rn we define
 
b1
A~b := [~a1 · · · ~an ]  ...  = b1~a1 + · · · + bn~an .
 

bn

So A~b is the linear combintion of the columns of A using the coefficients in ~b.
Example. 

  a      
1 4 5  1 4 5
b =a +b +c
2 3 6 2 3 6
c

Moreover, if B = [~b1 · · · ~bk ] where each ~bi ∈ Rn , then we define

AB = A[~b1 · · · ~bk ] = [A~b1 · · · A~bk ],

so that the ith column of AB is A~bi .

With this review under our belt, let us begin our study. As usual, fix finite-
dimensional F-vector spaces V and W with bases B = {v1 , . . . , vn } for V and
C = {w1 , . . . , wm } for W .
Definition. For any v ∈ V , we define
 
b1
[v]B =  ... 
 

bn
where v = b1 v1 + · · · + bn vn . We call this vector the coordinates of v with
respect to the basis B.
The next fact follows directly from the definition of coordinates. We leave
its proof to the reader.
Lemma 3.10 (Linearity of Coordinates). For any u, v ∈ V and a, b ∈ F, we
have
[au + bv]B = a[u]B + b[v]B
Now fix T ∈ L(V, W ) and consider the action of T on some v ∈ V . First let
 
b1
[v]B =  ...  .
 

bn
Now we see that

T v = T (b1 v1 + · · · + bn vn )
= b1 T v1 + · · · + bn T vn .

29
As T vi ∈ W and C is a basis for W we must have

T vi = b1i w1 + · · · + bmi wm

for some scalars bij ∈ F where 1 ≤ i ≤ m and 1 ≤ j ≤ n. Solving for the

coordinates of T v with respect to C we have
By the linearity of coordinates we now have

[T v]C = [b1 T v1 + · · · + bn T vn ]C = b1 [T v1 ]C + · · · + bn [T vn ]C .

If we set  
c1i
[T vi ]C =  ... 
 

cmi
then we see that
     
c11 c12 c1n
[T v]C = b1  ...  + b2  ...  + · · · + bn  ... 
     

cm1 cm2 cmn

  
c11 ··· c1n b1
 .. ..   ..  .
=  . .  . 
cm1 ··· cmn bn

This discussion proves the following theorem.

Theorem 3.11. Fix T ∈ L(V, W ). Then

[T v]C = [T ]CB [v]B ,

where [T ]CB is the matrix whose columns are [T vi ]C . That is

   
c11 · · · c1n c1i
[T ]CB =  ... ..  where [T v ] =  ..  .

.  i C  . 
cm1 · · · cmn cmi

The reader should take note that [T ]CB is an m × n matrix and T is a linear
map from an n-dimensional space V to an m-dimensional space W .
Having developed this connection let consider the following concrete exam-
ple.

Examples.

30
 1. Define the linear map
T : R3 → P ≤2
 
a
by T  b  = (a + b)x + (a + c)x2 . Now let
c
     
 1 1 0 
B =  1 , 0 , 1 
0 1 1
 

be our basis for R3 and take C = {1, x, x2 } as our basis for P ≤2 . As

     
1 1 0
T  1  = 2x+x2 , T  0  = x+2x2 , T  1  = x+x2 ,
0 1 1

we have  
0 0 0
[T ]CB = 2 1 1 .
1 2 1
   
3 1
Next, consider the vector v =  4  ∈ R3 so that [v]B =  2 . By
5 3
Theorem 3.11 we have
    
0 0 0 1 0
[T v]C = 2 1 1  2  =  7  ,
1 2 1 3 8

which is consistent as T v = 7x + 8x2 .

2. For any n-dimensional vector space V , let I : V → V be the identity map.

If B is any basis for V , the reader should check that
 
1 0
[I]B ..
B =
 
. 
0 1

where the matrix on the right has size n × n. For obvious reasons this
matrix is often called the identity matrix and is denoted by In .

In the past, you may have wondering why matrix multiplication is defined
in the way it is. Why not just multiply corresponding entries? Why is this
definition so great? We are now in a position to answer this question! Consider
our next theorem.

31
Theorem 3.12. Let U be another finite-dimensional vector space with basis D.
For any T ∈ L(V, W ) and S ∈ L(W, U ), we have

[ST ]D D C
B = [S]C · [T ]B

where the operation · on the left is the usual matrix multiplication.

In other words, matrix multiplication is precisely the operation needed in

order to obtain the matrix [ST ]D D C
B from the matrices [S]C and [T ]B . Another
way to look at this is that composition of functions on the linear transformation
side corresponds to matrix multiplication on the matrix side. Let us now prove
this pretty result.
Proof. To simplify notation let us denote [T ]CB by [T ] and [S]D
C by [S]. Theo-
rem 3.11 tells us that
[ST vi ]D = [S][T vi ]C .
As ST ∈ L(U, W ), then
h i h i
[ST ]D
B = [ST v1 ]D · · · [ST vn ]D = [S][T v1 ]C · · · [S][T vn ]C = [S][T ],

where the last equality is our definition of matrix multiplication.

We are now in a position to answer our last fundamental question regarding
this correspondence between linear transformations and matrices. Assume B 0
and C 0 are also bases for V and W respectively. For any T ∈ L(V, W ) how are
0
the matrices [T ]CB and [T ]CB0 related to each other?

Theorem 3.13 (Change of Basis).

0 0
[T ]CB0 = [J]CC [T ]CB [I]B
B0 ,

where J and I are the identity maps on V and W respectively.

Proof. Certainly we have the following equality

T = JT I.

Several applications of Theorem 3.12 now yields

0 0 0 0
[T ]CB0 = [JT I]CB0 = [J]CC [T I]CB0 = [J]CC [T ]CB [I]B
B0 .

You might be mislead into thinking that [I]BB0 must be the identity matrix
since I is the identity map plus we know this to be true when B = B 0 (see Ex-
ample 2 above). The following simple example shows that this is NOT generally

32
true. Consider
 the  identity I : R3 → R3 where B 0 is the standard basis
  map
 1 1 0 
and B =  1  ,  0  ,  1  . Since
0 1 1
 

           
1 1 0 1 1 0
1       1 
e1 = 1 + 0 − 1 , e2 = 1  −  0  +  1  ,
2 2
0 1 1 0 1 1

and      
1 1 0
1
e3 = −  1  +  0  +  1 
2
0 1 1
We see that
 
h i 1/2 1/2 −1/2
[I]B
B0 = [e1 ]B [e2 ]B [e3 ]B =  1/2 −1/2 1/2 
−1/2 1/2 1/2

is certainly not the identity matrix. We hope this example dispels any possible
confusion.

33
Chapter 4

Complex operators

In this chapter we study linear transformations that map a vector space V into
itself. Such maps, formally called operators, are the foundation for a rich theory.
This chapter aims to provide an introduction to this theory.

4.1 Operators & polynomials

Definition. We call a linear map T from V to V an operator. If V is a
complex vector space we say T is a complex operator. Likewise, if V is a real
vector space we say T is a real operator. We denote the set of operators L(V )
instead of L(V, V ).
As the domain and range of an operator T ∈ L(V ) is the same, we may com-
pose such function with themselves an arbitrary number of times. In particular,
we write
Ti = T · · · T},
| T {z
i
0
for integers i > 0 and define T = IV to be the identity map on V . If p(x) =
a0 +a1 x+· · · an xn is any polynomial with coefficients in F we define the function
p(T ) : V → V given by

p(T )u = a0 u + a1 T u + · · · + an T n u,

for any u ∈ V . A straightforward check, which we leave to the reader, shows

that p(T ) is an operator. For example consider the operator T : C → C given
by    
a 2a
T = .
b 3b

34
If p(x) = 2 + 3x + x2 , then
         
a a a a a
p(T ) = (2 + 3T + T 2 ) =2 + 3T + T2
b b b b b
     
2a 2a 4a
= +3 +
2b 3b 9b
 
12a
= .
20b

Continuing in this vain we note that if f (x), g(x), and h(x) are polynomials
such that f (x) = g(x)h(x), then f (T ) = g(T )h(T ). We encourage the reader to
check this fact for themselves. Moreover, since f (x) = g(x)h(x) = h(x)g(x) we
see that
g(T )h(T ) = h(T )g(T ). (4.1)
Returning to the example above, observe that p(x) = (1 + x)(2 + x). Com-
puting we obtain
        
a a a 2a
p(T ) = (1 + T )(2 + T ) = (1 + T ) 2 +
b b b 3b
 
4a
= (1 + T )
5b
     
4a 8a 12a
= + = ,
5b 15b 20b

which agrees with our computation above. The reader is encouraged to check
that one gets the same answer by computing
 
a
(2 + T )(1 + T ) .
b

For the remainder of this section we review some basic facts about polyno-
mials that will be needed throughout the remainder of this chapter. We start
with one of the most famous theorems in algebra.
Theorem 4.1. (Fundamental Theorem of Algebra) Every non-constant p(z) ∈
P(C) has at least one root. Consequently, p(z) factors completely into linear
terms as
p(z) = a(z − λ1 ) · · · (z − λn ),
for some c, λi ∈ C.
Recall that for integers a, b, we say a divides b, and write a|b provided there
exists some integer c such that b = ac. We now give an analogous definition for
polynomials.
Definitions. For polynomials p(x), q(x) we say q(x) divides p(x) and write
q(x)|p(x), provided there exists some polynomials s(x) such that p(x) = q(x)s(x).

35
 We define the greatest common divisor or GCD of p(x) and q(x) to be
the monic polynomials of greatest degree, written (p, q), that divides both p
and q. We can easily extend this definition to a finite number of polynomials
p1 , . . . , pk by defining their GCD (p1 , . . . , pk ) to be the monic polynomials of
largest degree that divides each pi .
Lastly, we say p1 , . . . , pk are relatively prime provided their GCD is 1.
For example, observe that x − 1|x3 − 1 since (x − 1)(x2 + x + 1) = x3 − 1. We
also have that that the polynomials (x − a)n and (x − b)m are relatively prime
provided a 6= b and if a = b then their GCD is (x − a)k , where k = min(n, m).
The next lemma, whose proof can be readily found in many algebra books,
will serve as a critical ingredient in Section 4.4.
Lemma 4.2 (Bézout). Let p1 (x), . . . , pk (x) ∈ P(C) with GCD d(x). Then there
exists polynomials, a1 (x), . . . , ak (x) ∈ P(C) such that

a1 (x)p1 (x) + · · · + ak (x)pk (x) = d(x).

4.2 Eigenvectors & eigenvalues

Definition. Let T ∈ L(V ). We say a subspace U of V if T -invariant provided
that T (U ) = {T u | u ∈ U } ⊆ U .
The utility of this definition is that if U is T -invariant, then we can restrict
T to the “smaller” linear map

T : U → U.

A particularly, important T -invariant subspace is given by the following

definition.
Definition. Let T ∈ L(V ). A scalar λ ∈ F is said to be an eigenvalue, if there
is a nonzero vector v ∈ V , such that

T v = λv.

In this case, we say v is an eigenvector for λ and refer to the tuple (λ, v) as
an eigenpair for T .

Observe that if (λ, v) is an eigenpair for T then span(v) is T -invariant and

T : span(v) → span(v)

is given by multiplication by λ. Geometrically, this means that along the line

generated by v, i.e., span(v), T acts by simply stretching the line by a factor of
λ as if the line were made of rubber. To illustrate consider the following picture
where the dashed line is span(v).

36
 V
λv
v

0V

The significance of this is that although T may be wildly complicated in general,

the existence of an eigenpair (λ, v) means that at least in the v direction T acts
in a very simple way – it just stretches the line by a factor of λ.
Examples.

1. For the identity operator, IV , every nonzero vector v is an eigenvector

with corresponding eigenvalue 1. What are the eigenvalues/vectors for
the zero map on V ?
 
2 0
2. Let A = . As an operator A : R2 → R2 we see (by inspection)
0 −1
that      
1 0
2, and −1,
0 1
are its two eigenpairs.
   
x x + 2y
3. Now define the operator T on R2 by T = . Unlike
y 4y − x
the previous example, it is a bit harder to just spot the eigenpairs for this
operator. Instead, let us appeal to some simple algebra. We seek values
x, y and λ such that    
x + 2y x
=λ .
4y − x y
Equating the first and second coordinates we obtain two equations:

x + 2y = λx and 4y − x = λy. (?)

Using the second equation to obtain an expression for x we then plug this
expression into the first equation to obtain (after simplifying)

0 = y(λ2 − 5λ + 6) = y(λ − 3)(λ − 2). (4.2)

Therefore y = 0 or λ = 3, 2. We see from above that when y = 0, x is also

0. Since the zero vector cannot be an eigenvector we are not interested
in this solution. If λ = 2, then solving (?) yields x = 2, y = 1. Hence

37
   
2
2, is an eigenpair for T . Likewise, when λ = 3, we get the
1   
1
eigenpair 3, .
1
4. Let T be the operator on the space of continuous function C(R) given by
Z
T f = f (x) dx.

Now consider the function eλx with λ 6= 0. Since

Z
1
T (eλx ) = eλx dx = eλx
λ
we see that λ1 , eλx is an eigenpair for this operator.

Lemma 4.3. Let T ∈ L(V ) and λ ∈ F. Then the nonzero vectors in null(T −λ)
are precisely the eigenvectors for T corresponding to λ.
Proof. To prove this lemma we must show that 0V 6= v ∈ null(T − λ) if and
only if (λ, v) is an eigenpair for T . This follows directly from the definitions
involved. In particular, we have

0V 6= v ∈ null(T − λ) ⇔ (T − λ)v = 0V and v 6= 0V

⇔ T v − λv = 0V and v 6= 0V
⇔ T v = λv and v 6= 0V
⇔ (λ, v) is an eigenpair for T.

We first remark that a straightforward consequence of this lemma is that λ

is an eigenvalue for T if and only if null(T − λ) is not trivial.
This lemma also suggests that the subspace null(T − λ) is intimately con-
nected to the eigenvectors for λ. As such, it makes sense to formally name this
subspace.
Definition. Let T ∈ L(V ) and λ ∈ F be an eigenvalue for T . Then null(T − λ)
is called the eigenspace of T corresponding to λ.
Theorem 4.4. Let T ∈ L(V ). Suppose λ1 , . . . , λm are distinct eigenvalues of
T with corresponding eigenvectors v1 , . . . , vm . Then the vectors v1 , . . . , vm are
linearly independent.
Proof. Let ai be scalars such that

a1 v1 + · · · + am vm = 0V .

Next define the new operator

Tj = (T − λ1 ) · · · (T − λj−1 )(T − λj+1 ) · · · (T − λm ).

38
As we can write the factors of Tj in any order, it follows that Tj vi = 0V , provided
i 6= j. Next we determine the value of Tj vj . To compute this, first observe that

(T − λk )vj = T vj − λk vj = λj vj − λk vj = (λj − λk )vj .

Using this repeatedly, we obtain

Tj vj = (T − λ1 ) · · · (T − λj−1 )(T − λj+1 ) · · · (T − λm )vj

= (λj − λ1 ) · · · (λj − λj−1 )(λj − λj+1 ) · · · (λj − λm ) vj .
| {z }
Λ

It now follows that

0V = Tj (a1 v1 + · · · + am vm ) = aj Λvj .

As our eigenvalues are distinct, then Λ 6= 0. Dividing through by Λ now yields

0V = aj vj . Therefore aj = 0, since vj is an eigenvector and cannot be the zero
vector. As j was arbitrary, this shows that all our scalars must indeed be zero,
completing our proof.
Corollary 4.5. Any operator on a finite-dimensional vector space V has at
most dim V distinct eigenvalues.
Proof. If V has dimension n, then any set of linearly independent vectors must
have at most n vectors. The result now follows immediately from the preceding
theorem.
What can we say if T has exactly dim V linearly independent eigenvectors.
3
In fact,
we can  saya lot! To
 motivate consider the operator T on R given
a a + 2b
by T  b  =  4b − a . Straightforward calculations show that T has
c 3c
eigenpairs
        
2 1 0
2,  1  , 3,  1  and 3,  0  ,
0 0 1

where we denote the ith vector by vi . In other words, T simply stretches the
lines given by v1 , v2 , and v3 as shown in the following illustration.

39
 z

T v3

v3

y
v2
v1

T v2
x
T v1

Since that these three eigenvectors are linearly independent, they form a
basis      
 2 1 0 
B =  1 , 1 , 0 
0 0 1
 

for R3 . Consequently, T is completely determined by these eigenvectors. More-

over, observe that  
2 0 0
[T ]B = 0 3 0 .
0 0 3
As we have just seen, operators that have “enough” eigenvectors are easy
to understand – they essentially just stretch space. Such operators also have a
very nice description in terms of their matrix representations as described by
our last theorem. We leave the details to the reader.
Definition. Let T be any operator on an F-vector space. We say T is diag-
onalizable provided there exists a basis B for V such that [T ]B is a diagonal
matrix, i.e.,  
λ1
 λ2  0
[T ]B =  ,
 
..
.
0
 
λn
for some scalars λ1 , . . . , λn ∈ F.
Theorem 4.6. Let T ∈ L(V ). Then T is diagonalizable if and only if V has a
basis consisting entirely of eigenvectors for T .

We now turn our attention to the following question: When do operators

have eigenvectors? We would hope that all operators have eigenvectors but, as

40
the next example shows, this is unfortunately not true! To illustrate such an
operator consider T ∈ L(R2 ) given by T (x, y) = (−y, x). If T had eigenvectors,
then we would be able to find values x, y, and λ such that
(−y, x) = T (x, y) = λ(x, y) (?)
Equating the first and second coordinates yields
−y = λx and x = λy.
2
Combining we obtain y(λ + 1) = 0. Since y = 0 cannot be a solution (why?)
and the equation
(λ2 + 1) = 0 (†)
has no real solutions, we conclude that T has no eigenvectors.
As you might already suspect, what if we work over C instead of over R? In
that case, the equation (†) has a solution, namely, λ = ±i. If we indeed do this
by considering T as an operator on C2 then we obtain the eigenpairs:
(i, (i, 1)) and (−i, (−i, 1)) ,
since
T (i, 1) = (−1, i) = i(i, 1) and T (−i, 1) = (−1, −i) = −i(−i, 1).
More generally, since polynomials always have roots over C, then it stands to
reason that any complex operator should have at least one eigenvector. Not only
does the next theorem assert this but its proof relies solely on the Fundamental
Theorem of Algebra!
Theorem 4.7. Every operator T on a non-trivial finite-dimensional complex
vector space V has at least one eigenvalue.
Proof. Set n = dim V and fix any nonzero vector u ∈ V . Now consider the n + 1
vectors
u, T u, . . . , T n u.
As the dimension of V is n, these n + 1 vectors must be dependent and so there
exists scalars ai ∈ C not all zero such that
0V = a0 u + a1 T u + · · · + an T n u
= (a0 + a1 T + · · · + an T n )u
= p(T )u,
where p(z) = a0 + a1 z + · · · + an z n . Let m be the largest index such that
am 6= 0. As u is a nonzero vector we must have 0 < m ≤ n. In particular,
p(z) is a nonconstant polynomial. By the Fundamental Theorem of Algebra
p(z) factors as p(z) = am (z − λ1 )(z − λ2 ) · · · (z − λm ), where λ1 , . . . , λm ∈ C.
Plugging this into the above equation yields,
0V = am (T − λ1 )(T − λ2 ) · · · (T − λm )u.
It now follows that one of the factors (T − λi ) must have a nontrivial null space.
In other words, λi is an eigenvalue for T .

41
 The above theorem only guarantees that a complex operator has at least
one eigenvalue. It does not guarantee that we have lots of distinct eigenvalues,
 
0 1
say dim V of them! To see an example of this, consider the matrix A =
0 0
thought of as an operator A : C2 → C2 . To find its eigenpairs we set up the
usual eigenvector/value equation
     
w z z
=A =λ ,
0 w w

to obtain
 the
 equations
 w = λz and 0 = λw. We see that the only eigenpairs for
z
A are 0, where z 6= 0. Additionally, this example demonstrates that
0
complex operators may not even have dim V linearly independent eigenvectors.
(In this case all the eigenvectors for A are multiplies of (1, 0).) This means,
thanks to Theorem 4.6, that not every complex-operator is diagonalizable.
That said, all is not lost! The next theorem demonstrates that, for the
right choice of basis, every complex operator can be represented by an “upper-
triangular” matrix. We start with a few definitions.
Definition. Let T ∈ L(V ). We say an basis B = {v1 , . . . , vn } for V is T -
triangularizing provided that

T vk ∈ span(v1 , . . . , vk ),

for each k ≤ n.
The next definition and lemma reveal why the term “triangularizing” is used.

Definition. We say a square matrix M is upper triangular provided it is of

the form  
λ1
 λ2  ∗
,
 
 ..
.
0
 
λn
so that the values weakly above the diagonal are arbitrary (indicated by the ∗)
but the values strictly below the diagonal must be zero.

Lemma 4.8. Let T be an operator on V with basis B = {v1 , . . . , vn }. Then the

following are equivalent.
(a) The basis B is T -triangularizing.
(b) For all k ≤ n, the subspaces span(v1 , . . . , vk ) are T -invariant.

(c) The matrix [T ]B is upper triangular.

42
Proof.
Case: (a) ⇒ (b)
Assume B is T -triangularizing and fix k ≤ n. Now for any i ≤ k we have

T vi ∈ span(v1 , . . . , vi ) ⊂ span(v1 , . . . , vk ).

To see that this implies T -invariance, consider an arbitrary u ∈ span(v1 , . . . , vk ).

We know that u = a1 v1 + · · · + ak vk for some scalars ai . Thus

T u = T (a1 v1 + · · · + ak vk ) = a1 T v1 + · · · + ak T vk ∈ span(v1 , . . . , vk ),

where the last step follows since subspaces are closed under linear combinations.
This shows span(v1 , . . . , vk ) is T -invariant.

Case: (b) ⇒ (c)

Assume span(v1 , . . . , vk ) is T -invariant for each k ≤ n. To show that

[T ]B = {[T v1 ]B · · · [T vk ]B · · · [T vn ]B }

is upper triangular, it will suffice to show that the bottom n − k rows of the kth
column [T vk ]B are all zeros. To see this, observe that T vk ∈ span(v1 , . . . , vk ),
as B is T -triangularizing. This means there exists scalars a1 , . . . , ak so that

T vk = a1 v1 + · · · + ak vk + 0vk+1 + · · · + 0vn .

Hence the coordinates of this vector and hence the kth column of our matrix is
 
a1
 .. 
 . 
 
 ak  ← kth row
[T vk ]B = 
  .
 0 

 . 
 .. 
0

As the bottom n − k rows are indeed all zero, we conclude that [T ]B is upper
triangular.

Case: (c) ⇒ (a)

In this case we know that [T ]B is an upper triangular of the form
 
a1,1 a1,2 a1,3 . . . a1,n

 a2,2 a2,3 . . . a2,n 


 a3,3 .

 .. .. 
 . . 
0 an,n

43
We must show that T vk ∈ span(v1 , . . . , vk ). To do this lets consider the coordi-
nates of T vk (with respect to B) first.
 
a1,k
 .. 
 . 
 
 ak,k  ,
[T vk ]B = [T ]B [vk ]B = [T ]B ek =  
 . 
 .. 
0

where the first equality follows from Theorem 3.11 and ek is the kth standard
vector. By definition we thus have

T vk = a1,k vk + · · · + ak,k vk ∈ span(v1 , . . . , vk ).

So B is T -triangularizing as needed.
Theorem 4.9. Let T be a complex operator on V . Then there exists a T -
triangularizing basis B for V .

Proof. We proceed by induction on dim V . The result is clear when dim V = 1

since basis in this case is T -triangularizing.
Now let T be an operator on an n > 1 dimensional space V . By Theorem 4.7,
we know there exists some eigenpair (λ, v1 ) for T . As v1 is nonzero (why?) then
we can extend this single vector to a basis {v1 , u2 , . . . , un } for all of V . Set
U = span(u2 , . . . , un ). Observe dim U = n − 1 and v1 ∈ / U . Next, define the
function S : U → U by
Su = a2 u2 + · · · an un
where T u = a1 v1 + a2 u2 + · · · + an un . A straightforward check shows that S is
indeed an operator. By induction we know that U has a S-triangularizing basis
B U = {v2 , . . . , vn }.
We now claim that
B = {v1 } ∪ {v2 , . . . , vn }
is a T -triangularizing basis for all of V . Certainly, B is a basis for V as it is
linearly independent (why?) and | B | = n = dim V . It only remains to see that
B is T -triangularizing. When k = 1, we see that T v1 = λv1 ∈ span(v1 ), since
(λ, v1 ) is an eigenpair for T .
For k ≥ 2 we observe that as B U is S-triangularizing, then

Svk ∈ span(v2 , . . . , vk )

where T vk = a1 v1 + Svk . So, T vk ∈ span(v1 , . . . , vk ). This concludes our

proof.

44
4.3 Direct sums
To motivate this section consider the vector space R2 thought of as the xy-plane.
From this perspective the two subspaces

X = {(x, 0) | x ∈ R} and Y = {(0, y) | y ∈ R}

of R2 have a more familar description – they are just the x- and y-axes. The
fact that these axes only intersect at the origin is reflected in the fact that

X ∩ Y = {(0, 0)}.

We also know that every vector in R2 can be written (uniquely) as vector in the
x-axis plus a vector in the y-axis. This is mirrored by the fact that

R2 = {u + v | u ∈ X, v ∈ Y } .

It now makes sense to denote the set on the right by X + Y , in which case we
write R2 = X + Y . That is R2 is the “sum” of two of its subspaces. We can
perform a similar decomposition to R3 as well. Redefining X, Y and Z to be
the subspaces given by the x-, y-, and z-axis, then we certainly have

R3 = X + Y + Z.

Our first definition formalizes these ideas.

Definition. Let U1 , . . . , Um be subspaces of V . We define their sum to be the
set
U = U1 + · · · + Um = {u1 + · · · + um | ui ∈ Ui } .
We say U is the direct sum of the Ui provided that the only choice of ui ∈ Ui
that gives
u1 + · · · + um = 0V ,
is ui = 0V . In this case we write,

U = U1 ⊕ · · · ⊕ Um .

A straightforward check shows that U is a subspace of V . As it is clear that

Ui is a subset of U (check!), it now follows that Ui is a subspace of U . There
is also an important analogy that is not to be missed. The subspaces Ui are
behaving a lot like vectors. Saying that U = U1 + · · · + Um is reminiscent of
a spanning set of vectors, while the definition of a direct sum is reminiscent
of linear independence. The next few lemmas make this analogy stronger. We
leave the proof of the first lemma to the reader.
Lemma 4.10. Let U1 , . . . , Um be subspaces of V and consider the space

U = U1 + · · · + Um .

Then, U is a direct sum of the Ui ’s if and only if for every u ∈ U there is a

unique choice of ui ∈ Ui so that vector u = u1 + · · · + um .

45
make this an iff, so that we can create direct sums by
ToDo
partitioning a basis.

Lemma 4.11. Let U1 , . . . , Um be subspaces of V such that

V = U1 ⊕ · · · ⊕ Um .

If B i is a basis for each Ui , then

[ [
B = B1 ··· Bm

is a basis for V . Consequently, dim V = dim U1 + · · · + dim Um .

Proof. To simplify notation we prove the claim when m = 2. As the reader
should check, the same proof works in general. Let U = U1 and W = U2 and
fix bases B U = {u1 , . . . , uk } and B W = {w1 , . . . , wm } for U and W respectively.
A simple computation shows that B spans all of V :

span(B) = {(a1 u1 + · · · + ak uk ) + (b1 w1 + · · · + bm wm ) | ai , bi ∈ F}

= {u + w | u ∈ U, w ∈ W }
=U +W
=V

where the second equality follows since B U spans U and B W spans W . To see
that B is linearly independent, assume ai and bi are such that

(a1 u1 + · · · + ak uk ) + (b1 w1 + · · · + bm wm ) = 0V .

By definition of our sum being direct, we must have

a1 u1 + · · · + ak uk = 0V and b1 w1 + · · · + bm wm = 0V .

By the independence of the ui ’s and the wi ’s we see that all the ai ’s and all the
bi ’s are zero.

Continuing our analogy between direct sums and bases, the next lemma is a
generalization of the Basis Extension Theorem. In fact the main ingredient in
its proof is exactly this theorem.
FIX the wording of this lemma ToDo

Lemma 4.12. Fix vector space V and subspace U . Let S = {v1 , . . . , vk } be a

set of vectors such that U ∩ span(S) = {0V }. Then there exists a subspace W
such that
V =U ⊕W
and S ⊂ W .

46
Proof. Let W 0 = span(S). If V = U ⊕ W 0 we are done. Otherwise, let B U be
a basis for U and B W 0 be one for W 0 . By the previous lemma B U ∪ BW 0 is
a basis for U ⊕ W 0 . Now extend B U ∪ B W 0 to a basis for all of V by adding
vectors v1 , . . . , v` . Then

W = span(B W 0 ∪ {v1 , . . . , v` })

is the desired space.

A nice consequence of this lemma is that if U is a subspace of V , so that
V = U ⊕ W . Simply choose S = {0V } in the lemma!
As we have already seen, direct sums decompose our vector space into
(smaller) vector spaces. Interestingly, we can also use direct sums to decom-
pose operators as well. The one requirement is that the operator must behave
“nicely” with respect to the subspaces involved. Our next definition makes this
precise.
Definition. Let T ∈ L(V ). We say a subspace U of V if T -invariant provided
that T (U ) = {T u | u ∈ U } ⊆ U .
To motive what is to come, let us consider an example. Let T ∈ L(R3 ) be
given by  
x−y x+y
T (x, y, z) = √ , √ , 3z .
2 2
Observe that T decomposes into two parts. The first is a rotation by 45◦ in the
xy-plane and the second is scaling by 3 along the z-axis. To illustrate consider
the following picture.
z

T e3

e3

e1 45◦
T e1 y

Now define the following subspaces of R3

U = {(x, y, 0) | x, y ∈ R} and W = {(0, 0, z) | z ∈ R} .

It is clear from our description of T that U and W are both T -invariant sub-
spaces. Moreover, it is clear that R3 = U ⊕ W . Now consider [T ]B , where

47
B = {e1 , e2 } ∪ {e3 }. That is B is the union of a basis for U and a basis for W .
Computing we see that
 √1
− √12 0

2
[T ]B =  √12 √1
2
0 .
0 0 3
Looking closely, we see this matrix consists of a 2×2 block (encodes the rotation)
and 1 × 1 block (encodes the scaling) along its diagonal and all the other entries
are zero.
The significance of all this, is that if we can decompose our vector space
into the direct sum of T -invariant subspace, then this means that our operator
also decomposes into operators on smaller spaces as seen in this example. We
consider this idea in complete generality. The reader is encouraged to keep our
specific example in mind when reading the general ideas. We begin with an
expected definition.
Definition. We say a square matrix M is block diagonal provided it is of the
form  
 A1 0 
 

A2

 
 
M = ,

 ... 

 
 
Am
 
0
where each of the Ai are square matrices lying along the diagonal of M and all
other entries in M are zero. In the case each block is just a 1 × 1 matrix, we
say M is diagonal.
As suggested by our motivating example, we now have the following lemma.
Lemma 4.13. Assume U1 , . . . , Um are T -invariant subspaces of V such that
V = U1 ⊕ · · · ⊕ Um .
If B i is a basis for Ui , then
 
 [T |U1 ]B1 0 
 
[T |U2 ]B2
 
 
 
[T ]B =  ,

 ... 

 
 
[T |Um ]Bm
 
0

where B = B 1 ∪ · · · ∪ B m . Conversely, if B is any basis for V such that [T ]B is

block diagonal, then V is the direct sum of T -invariant subspaces.

48
 The reader should pause to convince themselves that this lemma is a direct
generalization of Theorem 4.6. In fact, it further strengthens our analogy be-
tween linear independent vectors and direct sums. This should be somewhat
expected since the 1-dimensional space spanned by an eigenvector is just an
invariant subspace. The proof of this lemma is left to the reader.

4.4 Generalized eigenvectors

In Section 4.2, we saw that the (real) operator that rotates R2 by 90◦ fails to
have a single eigenvector. From this we concluded that, in general, not all real
operators decompose into their eigenspaces, i.e., they are not all diagonalizable.
In the same section, we also showed that the complex operator A : C3 → C3 ,
where  
1 1 0
A = 0 1 0 ,
0 0 3
only has eigenspaces
  
1 0
null(A − 1) = span  0  and null(A − 3) = span  0  .
0 1
Consequently, it follows that not all complex operators are diagonalizable either.
Since the final goal of this chapter is to decompose (complex) operators into
invariant subspaces, these examples demonstrate that eigenspaces are the wrong
invariant subspaces to consider. The question now is: Which are the right
invariant subspaces? To help answer this question, let us revisit the operator
A : C3 → C3 from above. In particular, observe that
 2  
0 1 0 0 0 0
null(A − 1)2 = null 0 0 0 = null 0 0 0 .
0 0 2 0 0 4
It now follows that
   
1 0
null(A − 1)2 = span( 0  ,  1  ,
0 0
and moreover, that
null(A − 1)2 ⊕ null(A − 3) = C3 .
This example suggests the following definition.
Definition. Let λ be an eigenvalue of T and define the generalized eigenspace
(corresponding to λ) to be the subspace
G λ = null(T − λ)dim V .
We call 0V 6= v ∈ G λ a generalized eigenvector (corresponding to λ).

49
 Just like eigenspaces, we next show that generalized eigenspaces are also
invariant subspaces.
Lemma 4.14. Let λ be an eigenvalue of T . Then G λ is a T -invariant.
Proof. Fix n = dim V and let u ∈ G λ = null(T − λ)n . To prove invariance, we
must show that T u ∈ null(T − λ)n . Computing, we see that
(T − λ)n T u = T (T − λ)n u = T 0V = 0V ,
where the first equality follows as T commutes with itself. Hence T u ∈ null(T −
λ)n as needed.
As suggested by the above example, our goal now is to prove the following
structure theorem for complex operators.
Theorem 4.15. Let T be an operator on a finite-dimensional complex vector
space V . If the distinct eigenvalues of T are λ1 , . . . , λm , then
V = G λ1 ⊕ · · · ⊕ G λm .
In order to prove this theorem, we will need to establish a few lemmas and a
deeper understanding of generalized eigenvectors. We do this first, postponing
the proof of this theorem to the end of the section. We begin with a technical
lemma.
Lemma 4.16. Assume U is a T -invariant subspace of V with dimension m.
Then
T m (U ) = T m+1 (U ) = T m+2 (U ) = · · · .
Proof. First observe that T i (U ) ⊇ T i+1 (U ). To see this note that
T i+1 u = T i (T u) ∈ T i (U ),
since T u ∈ U as U is T -invariant. This gives us the following sequence of
inclusions
U ⊇ T (U ) ⊇ T 2 (U ) ⊇ · · · .
Observe we must have some i such that T i (U ) = T i+1 (U ). If not then
m = dim U > dim T (U ) > dim T 2 (U ) > · · ·
would be an infinite decreasing sequence of positive integers! Therefore there
must exist a smallest i ≤ m so that T i (U ) = T i+1 (U ). (Why must i ≤ m?)
Now observe that
T i+1 (U ) = T T i (U ) = T T i+1 (U ) = T i+2 (U ).
Doing this again we get
T i+2 (U ) = T T i+1 (U ) = T T i+2 (U ) = T i+3 (U ).
Repeating this argument indefinitely yields
T i (U ) = T i+1 (U ) = T i+2 (U ) = · · ·
and completes the proof.

50
 We now turn our attention to exploring an important connection between
T -triangularizing bases and generalized eigenspaces. To warm-up, let T be a
complex operator on V with T -triangularizing basis B = {v1 , . . . , vn } so that
 
λ1 . . . a1,k ∗
 λ2 . . . a2,k 
.
 
 .. .. 
 . 
[T ]B = 
 .
 λ k



 . ..


0 λn

As suggested by our choice of notation, we claim that every entry on the diagonal
is actually an eigenvalue. To see this it will suffice to show that null(T − λk )
is not trivial. This follows almost immediately. As B is T -triangularizing, we
know that

(T − λk )vi ∈ span(v1 , . . . , vi ) ⊂ span(v1 , . . . , vk−1 )

for all i < k. Additionally,

(T − λk )vk = (a1,k v1 + · · · ak−1,k vk−1 + λk vk ) − λk vk ∈ span(v1 , . . . , vk−1 ).

This means (T − λk ) is a map from the k-dimensional space span(v1 , . . . , vk ) to

the k − 1-dimensional space span(v1 , . . . , vk−1 ). Consequently, it must have a
nontrivial nullspace. (Why?) We now see that all the values on the diagonal of
[T ]B are eigenvalues for T .
In fact, the next lemma states an even deeper connection between eigenvalues
and triangularizing bases.
Lemma 4.17. Fix a complex operator T on the n-dimensional vector space V
and let B = {v1 , . . . , vn } be any T -triangularizing basis for V . Then every scalar
on the diagonal of [T ]B is an eigenvalue for T .
Moreover, if λ1 , . . . , λm are the distinct eigenvalues for T , then λi appears
on the diagonal of [T ]B precisely dim G λi times. Consequently,

dim V = dim G λ1 + · · · + dim G λm .

Proof. As we have already proved the first claim, we concentrate on the second.
To this end, it suffices to prove the following special case:
If S is an arbitrary operator on V with S-triangularizing basis B, then
zero appears on the diagonal of [S]B exactly dim null S n times.
The general result follows from this special case as follows. Let λ be an eigen-
valueIf
d = dim null(T − λi )n = dim G λi ,
| {z }
S

51
then by the special case d is the number of times zero appears on the diagonal
of  
λi
 λi 0 
[S]B = [T − λi ]B = [T ]B − [λi ]B = [T ]B − 
 
. . 
.
0
 
λi
which, we now see, is the same as the number of times λi appears on the diagonal
of [T ]B . This proves the general result.
We now concentrate on proving the special case. For concreteness, set
 
a1,1 a1,2 a1,3 . . . a1,n

 a2,2 a2,3 . . . a2,n 

[S]B = 
 a3,3 ,

 .. .. 
 . . 
0 an,n

and let d be the number of zero entries along the diagonal of [S]B . The number
of nonzero entries along this diagonal is then n − d. The proof will certainly be
complete if we can establish that
dim ran S n = dim S n (V ) = n − d,
since an application of Rank-Nullity shows that
dim null S n = dim V − dim ran S n
= n − (n − d)
= d, the number of zeros on the diagonal of [S]B ,
as needed. To this end set
Vi = span(v1 , . . . , vi ),
for 0 < i, and V0 = {0V }. As Vi ⊆ Vi+1 , for 0 ≤ i, we obtain the following
sequence of inclusions
{0V } ⊆ S n (V1 ) ⊆ S n (V2 ) ⊆ · · · ⊆ S n (Vn ) = S n (V ). (?)
Instead of proving directly that dim S n (V ) = n − d, it will be easier to show
that
dim S n (V ) = # of strict inclusions in (?) = n − d.
At this point, we have reduced the problem to establishing these two equalities.
The first equality will be proved by establishing the claim that
dim S n Vi + 1 = dim S n Vi+1 (1)
provided S n Vi ( S n Vi+1 . Likewise, the second equality will follow from the
claim that
S n Vi ( S n Vi+1 if and only if ai+1,i+1 6= 0. (2)

52
We now turn our attention to proving claims (1) and (2). To prove claim (1),
observe that since
S n (Vi ) = span(S n v1 , . . . , S n vi ),
it follows that the spanning sets, for any two consecutive spaces in (?), differ by
exactly one vector. This proves claim (1).
To prove claim (2), let us first assume that ai+1,i+1 = 0. As we already
know that S n (Vi ) ⊆ S n (Vi+1 ), we need only establish the reverse inclusion. To
this end observe that since ai+1,i+1 = 0, our matrix representation of S implies
that
Svi+1 = a1,i v1 + · · · + ai,i vi ∈ Vi .
Additionally, as B is S-triangularizing, we also have Sv1 , . . . , Svi ∈ Vi . This
implies that
S(Vi+1 ) = span(Sv1 , . . . , Svi+1 ) ⊆ Vi .
Now consider,

S n (Vi+1 ) = S n−1 S(Vi+1 ) ⊆ S n−1 (Vi ) = S n (Vi ),

where the last equality follows from Lemma 4.16 since dim Vi = i < n. It only
remains to consider the case when a = ai+1,i+1 6= 0. In this case, we see that

S n vi+1 = b1 v1 + · · · + bi vi + an vi+1 ,

for some scalars b1 , . . . , bi . As a 6= 0 and the vj are linearly independent, we

see that S n vi+1 ∈
/ span(v1 , . . . , vi ) = Vi . Consequently, S n vi+1 ∈
/ S n (Vi ) ⊆ Vi ,
where the inclusion follows as B is S-triangularizing. So, S Vi ( S n Vi+1 , which
n

completes our proof.

fix working of statement; awk ToDo

Lemma 4.18. Let T be a complex operator whose distinct eigenvalues are

λ1 , . . . , λm . Assume ui ∈ G λi are such that

0V = u1 + · · · + um ,

each ui = 0V . Consequently, if W is the sum of the subspaces G λi , then

W = G λ1 ⊕ · · · ⊕ G λm .

Proof. For brevity set n = dim V and G i = G λi . Assume for a contradiction

that there exists ui ∈ G i , not all zero, so that

0V = u1 + · · · + um . (?)

Without loss of generality let us assume u1 6= 0V . We now argue that we

can further assume that u1 is an eigenvector. To show this observe that since
(T − λ1 )n u1 = 0V , then there must exists a largest k ≥ 0 so that

(T − λ1 )k u1 6= 0V and (T − λ1 )k+1 u1 = 0V

53
This means the (T − λ1 )k u1 is an eigenvector for T . As each of the G i are
T -invariant, and hence (T − λ1 )k -invariant, applying the operator (T − λ1 )k to
both sides of (?), and relabeling the ith vector as ui , allows us to assume u1 is
an eigenvector.
We now model the remainder of our proof on the proof in Theorem 4.4. As
such define the operators

S = (T − λ2 )n · · · (T − λm )n

As in the proof of Theorem 4.4 we have

(
0V if i 6= 1
Sui =
Λu1 6= 0V if i = 1

for some Λ 6= 0 in C. Multiplying (?) through by S now reveals that 0V = Λu1 .

This is a contradiction since Λ 6= 0 and u1 6= 0V .

Proof of Theorem 4.15. It follows from the previous lemma that

G λ1 ⊕ · · · ⊕ G λm ⊆ V.

To complete the proof we must show this is actually an equality. The easiest
way to do this is to show the dimension of the direct sum is n = dim V . Now
consider
m
X
dim(G λ1 ⊕ · · · ⊕ G λm ) = dim(G λi ) = dim V,
i=1

where the second equality follows directly from Lemma 4.17.

4.5 The characteristic polynomial

Throughout this section let T be a complex operator on V with distinct eigen-
values λ1 , . . . , λm . Additionally set di = dim G λi . We then have the following
definition.
Definition. The characteristic polynomial of a complex operator T is the
polynomial
ρT (x) = (x − λ1 )d1 · · · (x − λm )dm .

For example, consider the operator A : C3 → C3 , where

 
1 5 5
A = 0 1 5 .
0 0 3

54
Then by Lemma 4.17, it follows that

ρA (x) = (x − 1)2 (x − 3).

Now let us compute the operator ρA (A). We see that

 2    
0 5 5 −2 5 5 0 0 0
ρA (A) = (A − 1)2 (A − 3) = 0 0 5  0 −2 5 = 0 0 0 .
0 0 2 0 0 0 0 0 0

Our next (famous) theorem states that this always occurs.

Theorem 4.19 (Cayley-Hamilton). Let T be any complex operator, then ρT (T ) =
0.
Proof. Let B = {v1 , . . . , vn } be a T -triangularizing basis for V so that
 
λ1 . . . b1 ∗
 `2 . . . b2 
.
 

 . . . .. 

[T ]B =  .

 `k 

 .. 
 . 
0 `n
Now set Vi = span(v1 , . . . , vi ). First observe that T v1 = `1 v1 or equivalently,

(T − `1 ) : V1 → {0V }.

Next we claim that for any k > 1 we have

(T − `k ) : Vk → Vk−1 .

If i < k, then, as B is T -triangularizing, we have

(T − `k )vi = T vi − `k vi ∈ Vk .

Additionally, we have

(T − `k )vk = (b1 v1 + · · · + bk−1 vk−1 + `k vk ) − `k vk ∈ Vk−1 .

We now have the following sequence of maps

T −`
n T −`n−1 T −`
2 T −`
1
V = Vn −
−−−
→ Vn−1 −−−−−→ ··· −−−→ V1 −−−→ {0V }.

For any u ∈ V we now get

0 = (T − `1 ) · · · (T − `n )u = (T − λ1 )d1 · · · (T − λm )dm u = ρT (T )u,

where the second equality follows from Lemma 4.17. As u ∈ V is arbitrary,

ρT (T ) has to be the zero map as claimed.

55
4.6 Jordan basis theorem
Definition. We say an operator N on V is nilpotent if there exists some
positive integer k such that N k = 0.
Examples. 1. The zero operator is nilpotent.
2. The operator N : C3 → C3 given by N (a, b, c) = (0, a, b) is nilpotent since
N 3 = 0.
3. The matrix operator A : C3 → C3 where
 
0 0 0
A = 1 0 0
0 1 0
is nilpotent as A3 is the zero matrix. Do the nilpotent operators A and
N look similar? They should since they are the same! In fact, you should
convince yourself that
[N ]{e1 ,e2 ,e3 } = A,
so that A is just the matrix representation of N with respect to the stan-
dard basis for C3 . A good way to think about N is that it forces the
vectors e1 , e2 , e3 to “walk the plank”:

N N N
e1 e2 e3 0C3

4. The operator M : C5 → C5 , given by

M (a, b, c, d, e) = (0, a, 0, c, d)
is nilpotent as the reader should check that M 3 = 0. In this case M forces
our standard vectors to walk two planks:

M M
e1 e2 0C3

M M M
e3 e4 e5 0C3

From this we see that span(e1 , e2 ) and span(e3 , e4 , e5 ) are M -invariant

spaces such that C5 = span(e1 , e2 ) ⊕ span(e3 , e4 , e5 ). Consequently,
Lemma 4.13 gives us
 
0 0
 1 0



0
[M ]{e1 ,e2 }∪{e3 ,e4 ,e5 } = 
 0 0 0  .
 0
1 0 0 
0 1 0

56
 5. Let T be any operator with eigenvalue λ. Then (T − λ) restricted to the
generalized eigenspace corresponding to λ is nilpotent.
Definition. Let N be a nilpotent operator on V . We say a basis J is a Jordan
basis with respect to N , provided there exists vectors v1 , . . . , vs ∈ J such that

J = {v1 , N v1 , . . . , N k1 v1 } ∪ · · · ∪ {vs , N vs , . . . , N ks vs }

and N (N ki vi ) = 0V . We say the vectors vi are J -generating vectors.

Definition. Let k ≥ 0. We define a Jordan block of size k + 1 to be the
(k + 1) × (k + 1) matrix
 
0
1 0 
 
J(k) = 
 . .. ,

 1 
 
1 0

so that J(k) has k + 1 zeros down the diagonal and k ones on the off diagonal.

As a warm-up for the proof of our next lemma, assume that N is a nilpotent
operator on V and that J = {v, N v, . . . , N k v} is a basis for V where N k+1 v =
0V . Before continuing, the reader should convince themselves that

[N ]J = J(k).

Lemma 4.20. Let N be a nilpotent operator on V and assume J is a Jordan

basis for V with respect to N . Then
 
 J(k1 ) 0 
 

J(k2 )

 
 
[N ]J = ,

 ... 

 
 
J(ks )
 
0

where k1 , . . . , ks are as defined in our definition of a Jordan basis.

Proof. As

J = {v1 , N v1 , . . . , N k1 v1 } ∪ · · · ∪ {vs , N vs , . . . , N ks vs }

we see that

V = span(v1 , N v1 , . . . , N k1 v1 ) ⊕ · · · ⊕ span(vs , N vs , . . . , N ks vs ).

57
Certainly the ith summand is N -invariant. Moreover, denoting the ith sum-
mand Ui and its basis B i = {vi , N vi , . . . , N ki vi }, we see from our warm-up
that  
N |Ui B = J(ki ).
i

An application of Lemma 4.13 now yields the final result.

Theorem 4.21. Let V be a nontrivial finite-dimensional vector space and let
N be a nilpotent operator on V . Then there exists a Jordan basis for V with
respect to N .
Proof. Begin by setting V0 = {0V } and defining Vi+1 = {v ∈ V | N v ∈ Vi }. A
straightforward argument shows that Vi = null N i and

V0 ⊂ V1 ⊂ · · · ⊂ Vm = V,

for some m. In fact, as V is nontrivial we must have m ≥ 1.

We now aim to prove the following, slightly stronger, result. Given any set
S of linearly independent vectors such that

span(S) ∩ Vm−1 = {0V }, (?)

there exists a Jordan basis J for V such that S is a set of (not necessarily all) J -
generating vectors. We proceed by induction on m. If m = 1, then V = null N ,
and the result follows by the Basis Extension Theorem. Now assume m > 1 and
let S = {v1 , . . . , vk } be such that (?) holds. By Lemma 4.12 we see that there
exists a subset W such that

V = Vm−1 ⊕ W

and S ⊂ W . Extending S, if necessary, we may assume it is a basis for W .

Now observe that S 0 = {N v1 , . . . , N vk } ⊂ Vm−1 . We claim that (?) holds
with S 0 in place of S and Vm−2 in place of Vm−1 . For the moment, let us assume
this to be true. Applying our inductive hypothesis to the vector space Vm−1
and the set S 0 then gives us a Jordan basis J 0 for Vm−1 such that S 0 is a set of
J 0 -generating vectors. It now follows that J = J 0 ∪ S, is a Jordan basis for
all of V and S is a set of J -generating vectors. In light of this, it only remains
to prove our claim.
First we show span(S 0 ) ∩ Vm−2 is trivial. To this end, assume ai are such
that
a1 N v1 + · · · + ak N vk ∈ Vm−2 .
By linearity, we see that N (a1 v1 + · · · + ak vk ) ∈ Vm−2 . Therefore a1 v1 + · · · +
ak vk ∈ Vm−1 ∩ W = {0V }. As the vi are independent, we must have all ai = 0.
We conclude span(S 0 ) ∩ Vm−2 is trivial as needed.
A similar argument shows that S 0 is independent. If the ai are now such
that
0V = a1 N v1 + · · · ak N vk = N (a1 v1 + · · · ak vk ),

58
then we see that

a1 v1 + · · · + ak vk ∈ W ∩ null N ⊂ W ∩ Vm−1 = {0V }.

As the vi are independent, then all the ai ’s are zero. This shows S 0 is a inde-
pendent.

Corollary 4.22. (Jordan Form) Let T be a complex operator on an n-dimensional

space V with distinct eigenvalues λ1 , . . . , λm . Then there exists a basis B for V
so that  
 A1 0 
 

A2

 
 
[T ]B =  ,

 ... 

 
 
Am
 
0
and Ai is some dim G i × dim G i block diagonal matrix whose blocks are of the
form  
λi
 1 λi 
 
 . .. .


 1 
 
1 λi
Proof. By Theorem 4.15 we know that

V = G λi ⊕ · · · ⊕ G λm .

As each G i is T invariant, it will suffice to show, thanks to Lemma 4.13, that

there exists a basis B i for G i so that Ai may be taken to be
 
T |G i B .
i

From Example 5 above, we know that T − λi is nilpotent when restricted to

G i . Theorem 4.21 now provides us with a Jordan basis B i for G i with respect
to (T − λi ). This indeed finishes the proof since Lemma 4.20 tells us that the
matrix      
T |G i Bi = (T − λi )|G i Bi + λi Bi
has the desired form.

59