115B – Linear Algebra

University of California, Los Angeles

Duc Vu

Spring 2021

This is math 115B – Linear Algebra which is the second course of the undergrad
linear algebra at UCLA – continuation of 115A(H). Similar to 115AH, this class is
instructed by Professor Elman, and we meet weekly on MWF from 2:00 pm to 2:50
pm. There is no official textbook used for the class. You can find the previous linear
algebra notes (115AH) with other course notes through my github. Any error in this
note is my responsibility and please email me if you happen to notice it.

Contents

1 Lec 1: Mar 29, 2021 5

1.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Lec 2: Mar 31, 2021 9

2.1 Vector Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Direct Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Lec 3: Apr 2, 2021 13

3.1 Direct Sums (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Quotient Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Lec 4: Apr 5, 2021 17

4.1 Quotient Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5 Lec 5: Apr 7, 2021 19

5.1 Quotient Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.2 Linear Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6 Lec 6: Apr 9, 2021 22

6.1 Linear Transformation (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.2 Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

7 Lec 7: Apr 12, 2021 27

7.1 Projection (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
7.2 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Duc Vu (Spring 2021) Contents

8 Lec 8: Apr 14, 2021 33

8.1 Dual Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

9 Lec 9: Apr 16, 2021 37

9.1 Dual Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
9.2 The Transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
9.3 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

10 Lec 10: Apr 19, 2021 42

10.1 Polynomials (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

11 Lec 11: Apr 21, 2021 45

11.1 Minimal Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
11.2 Algebraic Aside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

12 Lec 12: Apr 23, 2021 50

12.1 Triangularizability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

13 Lec 13: Apr 26, 2021 54

13.1 Triangularizability (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
13.2 Primary Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

14 Lec 14: Apr 28, 2021 57

14.1 Primary Decomposition (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . 57

15 Lec 15: Apr 30, 2021 62

15.1 Primary Decomposition (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . 62
15.2 Jordan Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

16 Lec 16: May 3, 2021 66

16.1 Jordan Blocks (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
16.2 Jordan Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

17 Lec 17: May 5, 2021 70

17.1 Jordan Canonical Form (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . 70

18 Lec 18: May 7, 2021 72

18.1 Jordan Canonical Form (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . 72
18.2 Companion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

19 Lec 19: May 10, 2021 78

19.1 Companion Matrix (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
19.2 Smith Normal Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

20 Lec 20: May 12, 2021 82

20.1 Rational Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

21 Lec 21: May 14, 2021 87

21.1 Rational Canonical Form (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . 87

2
Duc Vu (Spring 2021) List of Theorems

22 Lec 22: May 17, 2021 91

22.1 Inner Product Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

23 Lec 23: May 19, 2021 95

23.1 Inner Product Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . 95

24 Lec 24: May 21, 2021 98

24.1 Inner Product Spaces (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . 98
24.2 Spectral Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

25 Lec 25: May 24, 2021 101

25.1 Spectral Theory (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

26 Lec 26: May 26, 2021 104

26.1 Spectral Theory (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
26.2 Hermitian Addendum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

27 Lec 27: May 28, 2021 106

27.1 Positive (Semi-)Definite Operators . . . . . . . . . . . . . . . . . . . . . . . 106

28 Lec 28: Jun 2, 2021 112

28.1 Positive (Semi-)Definite Operators (Cont’d) . . . . . . . . . . . . . . . . . . 112
28.2 Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

29 Lec 29: Jun 4, 2021 117

29.1 Least Squares (Cont’d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
29.2 Rayleigh Quotient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

30 Additional Materials: Jun 04, 2021 121

30.1 Conditional Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
30.2 Mini-Max . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
30.3 Uniqueness of Smith Normal Form . . . . . . . . . . . . . . . . . . . . . . . 123

List of Theorems
10.12Fundamental Theorem of Arithmetic (Polynomial Case) . . . . . . . . . . . 44
11.6 Cayley-Hamilton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
12.10Fundamental Theorem of Algebra . . . . . . . . . . . . . . . . . . . . . . . . 53
14.2 Primary Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
22.5 Orthogonal Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
22.6 Best Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
24.8 Schur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
25.5 Spectral Theorem for Normal Operators . . . . . . . . . . . . . . . . . . . . 103
26.4 Spectral Theorem for Hermitian Operators . . . . . . . . . . . . . . . . . . 104
27.9 Singular Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
28.5 Singluar Value - Linear Transformation Form . . . . . . . . . . . . . . . . . 114
28.7 Polar Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
30.4 Minimax Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

3
Duc Vu (Spring 2021) List of Definitions

List of Definitions
1.1 Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Ring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Vector Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 Subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.8 Span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.9 Direct Sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1 Independent Subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.6 Complementary Subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.5 T-invariant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
6.9 Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
7.9 Dual Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
8.8 Annihilator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
9.7 Transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
9.11 Row/Column Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
9.13 Polynomial Division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
9.16 Polynomial Degree and Leading Coefficient . . . . . . . . . . . . . . . . . . 41
10.1 Greatest Common Divisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
10.6 Irreducible Polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
12.2 Triangularizability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
12.4 Splits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
12.8 Algebraically Closed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
15.2 Jordan Block Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
15.3 Nilpotent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
16.1 Sequence of Generalized Eigenvectors . . . . . . . . . . . . . . . . . . . . . . 66
16.3 Jordan Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
16.4 Jordan Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
18.5 Companion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
19.2 T-Cyclic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
19.7 Equivalent Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
22.1 Inner Product Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
22.2 Sesquilinear Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
22.8 Adjoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
23.2 Isometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
24.1 Unitary Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
24.4 Unitary Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
25.1 Hermitian(Self-Adjoint) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
27.1 Positive/Negative (Semi-) Definite . . . . . . . . . . . . . . . . . . . . . . . 106
27.8 Pseudodiagonal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
28.1 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4
Duc Vu (Spring 2021) 1 Lec 1: Mar 29, 2021

§1 Lec 1: Mar 29, 2021

§1.1 Ve c t o r S p a c e s
Notation: if ? : A × B → B is a map (= function) write a ? b for ?(a, b), e.g., + : Z × Z → Z
where Z = the integer.

Definition 1.1 (Field) — A set F is called a FIELD under

• Addition: + : F × F → F

• Multiplication: · : F × F → F

if ∀a, b, c ∈ F , we have

A1) (a + b) + c = a + (b + c)

A2) ∃ 0 ∈ F 3 a + 0 = a = 0 + a

A3) A2) holds and ∃x ∈ F 3 a + x = 0 = x + a

A4) a + b = b + a

M1) (a · b) · c = a · (b · c)

M2) A2) holds and ∃1 6= 0 ∈ F s.t. a · 1 = a = 1 · a ( 1 is unique and written 1 or 1F )

M3) M2) holds and ∀0 6= x ∈ F ∃y ∈ F 3 xy = 1 = yx (y is seen to be unique and

written x−1 )

M4) x · y = y · x

D1) a · (b + c) = a · b + a · c

D2) (a + b) · c = a · c + b · c

Example 1.2
Q, R, C are fields as is

F2 := {0, 1} with + : given by

+ 0 1 0 1

0 0 1 0 0 0

1 1 0 1 0 1

5
Duc Vu (Spring 2021) 1 Lec 1: Mar 29, 2021

Fact 1.1. Let p > 0 be a prime number in Z. Then ∃ a field Fpn having pn elements write
|Fpn | = pn ∀n ∈ Z+ .

Definition 1.3 (Ring) — Let R be a set with

• +:R×R→R

• ·:R×R→R

satisfying A1) – A4), M1), M2), D1), D2), then R is called a RING.
A ring is called

i) a commutative ring if it also satisfies M4).

ii) an (integral) domain if it is a commutative ring and satisfies

M 3’) a · b = 0 =⇒ a = 0 or b = 0

(0 = {0} is also called a ring – the only ring with 1 = 0)

Example 1.4 (Proof left as exercises) 1. Z is a domain and not a field.

2. Any field is a domain.

3. Let F be a field
F [t] := {polys coeffs in F }
with usual +, · of polys, is a domain but not a field. So if f ∈ F [t]

f = a0 + a1 t + . . . + an tn

where a0 , . . . , an ∈ F .

4. Q := ab |a, b ∈ Z, b 6= 0 < C (< means ⊂ and 6=) with usual +, · of fractions.



(when does ab = dc ?)

5. If F is a field
 
f
F (t) := |f, g ∈ F [t], g 6= 0 (rational function)
g

with usual +, · of fractions is a field.

6
Duc Vu (Spring 2021) 1 Lec 1: Mar 29, 2021

√  √
Example 1.5√(Cont’d from above) 6. Q[ −1] := α + β −1 ∈ C|α, β ∈ Q < C.
Then Q[ −1] is a field and
√ na √ o
Q( −1) := |a, b ∈ Q[ −1], b 6= 0
b√
= Q[ −1]
na √ o
= |a, b ∈ Z[ −1], b 6= 0
b
√  √
where Z[ √−1] := α + β −1 ∈ C, α, β ∈ Z < C. How to show this? – ratio-
nalize (Z[ −1] is a domain not a field, F [t] < F (t) if F is a field so we have to
be careful).

7. F a field
Mn F := {n × n matrices entries in F }
is a ring under +, · of matrices.
 
1 0
1 Mn F = In = n × n identity matrix  . . . 
 

0 1
 
0 ... 0
0 Mn F = 0 = 0n = n × n zero matrix  ... .. 

.
0 ... 0

is not commutative if n > 1.

In the same way, if R is a ring we have

Mn R = {n × n matrices entries in R}

e.g., if R is a field Mn F [t].

8. Let ∅ =
6 I ⊂ R be a subset, e.g., [α, β] , α < β ∈ R. Then

C(I) = {f : I → R|f continuous}

is a commutative ring and not a domain where

(f u g)(x) := f (x) u g(x)

0(x) = 0
1(x) = x

for all x ∈ I.

Notation: Unless stated otherwise F is always a field.

7
Duc Vu (Spring 2021) 1 Lec 1: Mar 29, 2021

Definition 1.6 (Vector Space) — Let F be a field, V a set. Then V is called a

VECTOR SPACE OVER F write V is a vector space over F under

• + : V × V → V – Addition

• · : F × V → V – Scalar multiplication

if ∀x, y, z ∈ V ∀α, β ∈ F .

1. (x + y) + z = x + (y + z)

2. ∃0 ∈ V 3 x + 0 = x = 0 + x (0 is seen to be unique and written 0 or 0V )

3. 2) holds and ∃v ∈ V 3 x + v = 0 = v + x (v is seen to be unique and written

−x)

4. x + y = y + x

5. 1F · x = x.

6. (α · β) · x = α · (β · x)

7. (α + β) · x = α · x + β · x

8. α · (x + y) = α · x + α · y

Remark 1.7. The usual properties we learned in 115A hold for V a vector space over F , e.g.,
0F V = 0V , general association law,. . .

8
Duc Vu (Spring 2021) 2 Lec 2: Mar 31, 2021

§2 Lec 2: Mar 31, 2021

§2.1 Ve c t o r S p a c e s ( C o nt ’ d )

Example 2.1
The following are vector space over F

1. F m×n := {m × n matrices entries in F }, usual +, scalar multiplication, i.e., if

A ∈ F m×n , let Aij = ij th entry of A. If A, B ∈ F m×n , then

(A + B)ij := Aij + Bij

(αA)ij := αAij ∀α ∈ F

i.e., component-wise operations.

2. F n = F 1×n := {(α1 , . . . , αn ) |αi ∈ F }

3. Let V be a vector space over F , ∅ =

6 S a set. Define

Fcn(S, V ) := {f : S → V | f a fcn}

Then Fcn(S, V ) is a vector space over F ∀f, g ∈ Fcn(S, V ), ∀α ∈ F . For all

x ∈ S,

f + g : x 7→ f (x) + g(x)
αf : x 7→ αf (x)

i.e.

(f + g)(x) = f (x) + g(x)

(αf )(x) = αf (x)

with 0 by 0(x) = 0V ∀x ∈ S.

4. Let R be a ring under +, ·, F a field 3 F ⊆ R with +, · on F induced by +, · on

R and 0F = 0R , 1F = 1R , i.e.

+
|{z} F × F : F × F → F and |{z}
· F ×F :F ×F →F
on R
on R
| {z } | {z }
restrict dom restrict dom

i.e. closed under the restriction of +, · on R to F and also with 0F = 0R and

1F = 1R (we call F a subring of R). Then R is a vector space over F by
restriction of scalar multiplication, i.e., same + on R but scalar multiplication

· F ×R
:F ×R→R

e.g., R ⊆ C and F ⊆ F [t].

9
Duc Vu (Spring 2021) 2 Lec 2: Mar 31, 2021

Example 2.2 (Cont’d from above)

Note: C is a vector space over R by the above but as a vector space over C is different.

5. In 4) if R is also a field (so F ⊆ R is a subfield). Let V be a vector space over

R. Then V is also a vector space over F by restriction of scalars, e.g., Mn C is a
vector space over C so is a vector space over R so is a vector space over Q.

§2.2 Subspaces

Definition 2.3 (Subspace) — Let V be a vector space under +, ·, ∅ =

6 W ⊆ V a subset.
We call W a subspace of V if ∀w1 , w2 ∈ W, ∀α ∈ F ,

αw1 , w1 + w2 ∈ W

with 0W = 0V is a vector space over F under +|W ×W and ·|F ×W i.e., closed under the
operation on V .

Theorem 2.4
Let V be a vector space over F, ∅ =
6 W ⊆ V a subset. Then W is a subspace of V iff
∀α ∈ F , ∀w1 , w2 ∈ W , αw1 + w2 ∈ W .

Example 2.5 1. Let ∅ 6= I ⊆ R, C(I) the commutative ring of continuous function

f : I → R. Then C(I) is a vector space over R and a subspace of Fcn(I, R).

2. F [t] is a vector space over F and n ≥ 0 in Z.

F [t]n := {f | f ∈ F [t], f = 0 or deg f ≤ d}

is a subspace of F [t] (it is not a ring).

Attached is a review of theorems about vector spaces from math 115A.

§2.3 Direct Sums

Problem 2.1. Can you break down an object into simpler pieces? If yes can you do it
uniquely?

Example 2.6
Let n > 1 in Z. Then n is a product of primes unique up to order.

10
Duc Vu (Spring 2021) 2 Lec 2: Mar 31, 2021

Example 2.7
Let V be a finite dimensional inner product space over R (or C) and T : V → V a
hermitian (=self adjoint) operator. Then ∃ an ON basis for V consisting of eigenvectors
for T . In particular, T is diagonalizable. This means

V = ET (λ1 ) ⊥ . . . ⊥ ET (λr ) (*)

ET (λi ) := {v ∈ V | T v = λi v} =
6 0 eigenspace of λi ; λ1 , . . . , λr the distinct eigenvalues
of T . So
T E (λ ) : ET (λi ) → ET (λi )
T i

i.e., ET (λi ) is T-invariant and

T ET (λi )
= λi 1ET (λi )

and (*) is unique up to order.

Goal: Generalize this to V any finite dimensional vector space over F , any F , and T : V → V
linear. We have many problems to overcome in order to get a meaningful result, e.g.,

Problem 2.2. 1. V may not be an inner product space.

2. F 6= R or C is possible.

3. F * is possible, so cannot even define an inner product.

4. V may not have any eigenvalues for T : V → V .

5. If we prove an existence theorem, we may not have a uniqueness one.

We shall show: given V a finite dimensional vector space over F and T : V → V a

linear operator. Then V breaks up uniquely up to order into small T -invariant subspace
that we shall show are completely determined by polys in F [t] arising from T . Motivation:
Generalize the concept of linear independence, Spectral Theorem Decomposition, to see
how pieces are put together (if possible).

11
Duc Vu (Spring 2021) 2 Lec 2: Mar 31, 2021

Definition 2.8 (Span) — Let V be a vector space over F , Wi ⊆ V , i ∈ I – may not

be finite, subspaces. Let
( )
X X X
Wi = Wi := v ∈ V |∃wi ∈ Wi , i ∈ I, almost all wi = 0 3 v = wi
i∈I i∈I i∈I

when almost all zero means only finitely many wi 6= 0. Warning: In a vector space/F
we can only take finite linear combination of vectors. So
! ( )
X [ [
Wi = Span Wi = finite linear combos of vectors in Wi
i∈I i∈I i∈I

e.g., if I is finite, i.e., |I| < ∞, say I = {1, . . . , n} then

X
Wi = W1 + . . . + Wn := {w1 + . . . + wn | wi ∈ Wi ∀i ∈ I}
i∈I

Definition 2.9 (Direct Sum) — Let V be a vector space over F , Wi ⊆ V , i ∈ I,

subspace. Let W ⊆ V beL a subspace. We say that W is the (internal) direct sum of
the Wi , i ∈ I write W = i∈I Wi if
X
∀w ∈ W ∃! wi ∈ Wi almost all 0 3 w = wi
i∈I

e.g., if I = {1, . . . , n}, then

w ∈ W1 ⊕ . . . ⊕ Wn means ∃! wi ∈ Wi 3 w = w1 + . . . + wn

Warning: It may not exist.

12
Duc Vu (Spring 2021) 3 Lec 3: Apr 2, 2021

§3 Lec 3: Apr 2, 2021

§3.1 D i r e c t S u m s ( C o nt ’ d )

Definition 3.1 (Independent Subspace) — Let V be a vector space over F, Wi ⊆ V, i ∈ I

subspaces. We sayP the Wi , i ∈ I, are independent if whenever wi ∈ Wi , i ∈ I, almost
all wi = 0, satisfy wi = 0, then wi = 0 ∀i ∈ I.

Theorem 3.2
Let V be a vector space over F, Wi ⊆ V, i ∈ I subspaces, W ⊆ V a subspace. Then
the following are equivalent:
L
1. W = i∈I Wi
P
2. W = i∈I Wi and ∀i
X
Wi ∩ Wj = 0 := {0}
j∈I\{i}
P
3. W = i∈I Wi and the Wi , i ∈ I, are independent.

L P
Proof. 1) =⇒ 2) Suppose W = i∈I Wi . Certainly, W = i∈I Wi . Fix i and suppose
that X
∃x ∈ Wi ∩ Wj
j∈I\{i}

By definition, ∃wi ∈ Wi , wj ∈ Wj , j ∈ I \ {i} almost all 0 satisfying

X
wi = x = wj
j6=i

So X
0V = 0W = wi − wj
j6=i

But X
0W = 0Wk 0Wk = 0V ∀k ∈ I
I

By uniqueness of 1), wi = 0 so x = 0.
2) =⇒ 3) Let wi ∈ Wi , i ∈ I, almost all zero satisfy
X
wi = 0
i∈I

Suppose that wk 6= 0. Then

X X
wk = − wi ∈ W k ∩ wi = 0,
i∈I\{k} i6=k

13
Duc Vu (Spring 2021) 3 Lec 3: Apr 2, 2021

a contradiction. So wi =P0 ∀i
3) =⇒ 1) Suppose v ∈ i∈I Wi and ∃wi , wi0 ∈ Wi , i ∈ I, almost all 0 3
X X
wi = v = wi0
i∈I i∈I

− wi0 ) = 0, wi − wi0 ∈ Wi ∀i. So

P
Then i∈I (wi

wi − wi0 = 0, i.e., wi = wi0 ∀i

and the wi0 s are unique.
P
Warning: 2) DOES NOT SAY Wi ∩Wj = 0 if i =
6 j. This is too weak. It says Wi ∩ j6=i Wj =
0.

Corollary 3.3

L over F, Wi ⊆ V, i ∈ I subspaces. Suppose I = I1 ∪ I2 with

Let V be a vector space
I1 ∩ I2 = ∅ and V = i∈I Wi . Set
M M
WI1 = Wi and W I2 = Wj
i∈I1 j∈I2

Then
V = W I1 ⊕ W I2

Proof. Left as exercise – Homework.

Notation: Let V be a vector space over F , v ∈ V . Set

F v := {αv|α ∈ F } = Span(v)
if v 6= 0, then F v is the line containing v, i.e., F v is the one dimensional vector space over
F with basis {v}.

Example 3.4
Let V be a vector space over F .

1. If ∅ =
6 S ⊆ V is a subset, then
X
F v = Span(S)
v∈S

the span of S. So

Span S = {all finite linear combos of vectors in S}

2. If ∅ =
6 S is linearly indep. (i.e. meaning every finite nonempty subset of S is
linearly indep.), then M
Span(S) = Fs
s∈S

14
Duc Vu (Spring 2021) 3 Lec 3: Apr 2, 2021

L
Example 3.5 (Cont’d from above) 3. If S is a basis for V , then V = s∈S F s.
P
4. If ∃ a finite set S ⊆ V 3 V = Span(S), then V = s∈S F s and ∃ a subset
B ⊆ S that is a basis for V , i.e., V is a finite dimensional vector space over F
and dim V = dimF V = |B| is indep. of basis B for V .

5. Let V be a vector space over F, W1 , W2 ⊆ V finite dimensional subspaces. Then

W1 + W2 , W1 ∩ W2 are finite dimensional vector space over F and

dim(W1 + W2 ) = dim W1 + dim W2 − dim(W1 ∩ W2 )

So
W1 + W2 = W1 ⊕ W2 ⇐⇒ W1 ∩ W2 = ∅
Warning: be very careful if you wish to generalize this.

Definition 3.6 (Complementary Subspace) — Let V be a finite dimensional vector

space over F, W ⊆ V a subspace if

V = W ⊕ W 0, W 0 ⊆ V a subspace

We call W 0 a complementary subspace of W in V .

Example 3.7
Let B0 be a basis of W . Extend B0 to a basis B for V (even works if V is not finite
dimensional). Then
M
W0 = F v is a complement of W in V
B\B0

Note: W 0 is not the unique complement of W in V – counter-example?

Consequences: Let V be a finite dimensional vector space over F, W1 , . . . , Wn ⊆ V sub-

spaces, Wi 6= 0 ∀i. Then the following are equivalent
1. V = W1 ⊕ . . . ⊕ Wn .
2. If Bi is a basis (resp., ordered basis) for Wi ∀i, then B = B1 ∪ . . . ∪ Bn is a basis
(resp. ordered) – with obvious order – for V .
Proof. Left as exercise (good one)!

Notation: Let V be a vector space over F, B a basis for PV , x ∈ V . Then, ∃!αv ∈ F, v ∈ B,

almost all αv = 0 (i.e., all but finitely many) s.t. x = B αv v. Given x ∈ V ,
X
x= αv v
v∈B

to mean αv is the unique complement of x on v and hence αv = 0 for almost all v ∈ B.

15
Duc Vu (Spring 2021) 3 Lec 3: Apr 2, 2021

§3.2 Q u o t i e nt S p a c e s
Idea: Given a surjective map f : X → Y and “nice”, can we use properties of Y to obtain
properties of X?

Example 3.8
Let V = R3 , W = X − Y plane. Let X = plane parallel to W intersecting the z-axis
at γ.
z
V = R3
γ
v
X

W
x

So

X = {(α, β, γ)|α, β ∈ R}
= {(α, β, 0) + (0, 0, γ)|α, β ∈ R}
= W + γ e3
|{z}
(0,0,1)

Note: X is a vector space over R ⇐⇒ γ = 0 ⇐⇒ W = X (need 0V ). Let v ∈ X. So

v = (x, y, γ) some x, y ∈ R. So
 

 

W + v := (α, β, 0) + (x, y, γ) | α, β ∈ R

| {z } | {z } 

arbitrary fixed

= {(α + x, β + y, γ) | α, β ∈ R}
= W + γe3

It follows if v, v 0 ∈ V , then

W + v = W + v 0 =⇒ v − v 0 ∈ W

Conversely, if v, v 0 ∈ V with X = W + v, then

v 0 ∈ X =⇒ v 0 = w + v some w ∈ W

hence
v0 − v ∈ W
So for arbitrary v, v 0 ∈ V , we have the conclusion W + v = W + v 0 ⇐⇒ v − v 0 ∈ W .
We can also write W + v as v + W .

16
Duc Vu (Spring 2021) 4 Lec 4: Apr 5, 2021

§4 Lec 4: Apr 5, 2021

§4.1 Q u o t i e nt S p a c e s ( C o nt ’ d )
Recall from the last example of the last lecture, we have
[
V = W +v
v∈V

If v, v 0 ∈ V , then
0 6= v 00 ∈ (W + v) ∩ (W + v 0 )
means
W + v − W + v 00 = W + v 0
This means either W + v = W + v 0 or W + v ∩ W + v 0 = ∅, i.e., planes parallel to the
xy-plane partition V into a disjoint unions of planes.
Let
S := {W + v| v ∈ V }
the set of these planes. We make S into a vector space over R as follows: ∀v, v 0 ∈ V, ∀α ∈ R
define

(W + v) + (W + v 0 ) := W + (v + v 0 )
α · (W + v) := W + αv

We must check these two operations are well-defined and we set

0S := W

Then (W + v) + W = W + v = W + (W + v) make S into a vector space over R.

If v ∈ V let γv1 = the k th component of v. Define

S → {(0, 0, γ)| γ ∈ R} → R

by
W + v 7→ (0, 0, γv ) 7→ γ
both maps are bijection and, in fact, linear isomorphism. So

S∼
= {(0, 0, γ)| γ ∈ R} ∼
=R

Note: dim V = 3, dim W = 2, dim S = 1 and we also have a linear transformation

V → S by (α, β, γ) 7→ W + γe3

a surjection.
We can now generalize this.
Construction: Let V be a vector space over F, W ⊆ V a subspace. Define ≡ mod W
called congruent mod W on V as follows: if x, y ∈ V, then

x≡y mod W ⇐⇒ x − y ∈ W ⇐⇒ ∃w ∈ W 3 x = w + y

Then, for all x, y, z ∈ V , ≡ mod W satisfies

17
Duc Vu (Spring 2021) 4 Lec 4: Apr 5, 2021

1. x ≡ x mod W

2. x ≡ y mod W =⇒ y ≡ x mod W

3. x ≡ y mod W and y ≡ z mod W =⇒ x ≡ z mod W

We can conclude that ≡ mod W is an equivalence relation on V .

Notation: For x ∈ V, W ⊆ V , let

x := {y ∈ V | y ≡ x mod W }

We can also write x as [x]W if W is not understood. Also, x ⊆ V is a subset and not an
element of V called a coset of V by W . We have

x = {y ∈ V | y ≡ x mod W }
= {y ∈ V | y = w + x for some w ∈ W }
= {w + x| w ∈ W } = W + x = x + W

Example 4.1
0V = W + 0V = W .

Note: W + x translates every element of W by x. By 2), 3) of ≡ mod W , we have check

y ∈ x = W + x ⇐⇒ x ∈ y = W + y

and
x≡y mod W ⇐⇒ x = y ⇐⇒ W + x = W + y
and check
x ∩ y = ∅ ⇐⇒ (W + x) ∩ (W + y) = ∅ ⇐⇒ x 6≡ y mod W
This means the W + x partition V , i.e.,
[
V = (W + x) with (W + x) ∩ (W + y) = ∅ if x = (W + x) 6= (W + y) = y
V

Let
V := V /W := {x| x ∈ V } = {W + x| x ∈ V }
a collection of subsets of V .

18
Duc Vu (Spring 2021) 5 Lec 5: Apr 7, 2021

§5 Lec 5: Apr 7, 2021

§5.1 Q u o t i e nt S p a c e s ( C o nt ’ d )
Suppose we have W ⊆ V a subspace. For x, y, z, v ∈ V

x≡y mod W (+)

z≡v mod W

Then
(x + z) − (y + v) = (x − y) + (z − v) ∈ W
| {z } | {z }
∈W ∈W

So
x+z mod y + v mod W
and if α ∈ F
αx − αy = α(x − y) ∈ W ∀x, y ∈ V
So
αx ≡ αy mod W
Therefore, V = V /W . If (+) holds, then for all x, y, z, v ∈ V and α ∈ F , we have

x+z =y+v ∈V
αx = αy ∈ V

Notice V = V /W satisfies all the axioms of a vector space with 0V = 0V = {y ∈ V | y ≡ 0 mod W } =

W + 0V = W .
We call V = V /W the Quotient Space of V by W .
We also have a map
– : V → V = V /W by x 7→ x = W + x
which satisfies
–
αv + v 0 7→ αu + v 0 = αv + v 0
for all v, v 0 ∈ V and α ∈ F . Then

dim V = dim ker–

dim V = dim W + dim V /W
dim V /W = dim V − dim W

which is called the codimension of W in V .

19
Duc Vu (Spring 2021) 5 Lec 5: Apr 7, 2021

Proposition 5.1
Let V be a vector space over F , W ⊆ V a subspace, V = V /W . Let B0 be a basis for
W and
B1 = {vi | i ∈ I, vi − vj ∈
/ W if i 6= j}
where vi 6= vj if i 6= j or w + vi 6= w + vj if i 6= j.
Let
C = {vi = W + vi | i ∈ I, vi ∈ B1 }
If C is a basis for V = V /W , then B0 ∪ B1 is a basis for V (compare with the proof
of the Dimension Theorem).

Proof. Hw 2 # 3.

§5.2 L i n e a r Tr a n s f o r m a t i o n
A review of linear of linear transformation can be found here.
Now, we consider
GLn F := {A ∈ Mn F | det A =
6 0}
The elements in GLn F in the ring Mn F are those having a multiplicative inverse. If R is a
commutative ring, determinants are still as before but

GLn R := {A ∈ Mn R| det A is a unit in R}

= A ∈ Mn R| A−1 exists


Example 5.2
Let V be a vector space over F, W ⊆ V a subspace. Recall

V = V /W = {v = W + v| v ∈ V }

a vector space over F s.t. for all v1 , v2 ∈ F and α ∈ F

0V = 0V = W
v1 + v2 = v1 + v2
αv1 = αv1

Then
– : V → V /W = V by v 7→ v = W + v
is an epimorphism with ker– = W .

Recall from 115A(H) that the most important theorem about linear transformation is
Universal Property of Vector Spaces. As a result, we can deduce the following corollary

20
Duc Vu (Spring 2021) 5 Lec 5: Apr 7, 2021

Corollary 5.3
Let V, W be vector space over F with bases B, C respectively. Suppose there exists a
bijection f : B → C , i.e., |B| = |C |. Then V ∼
= W.

Proof. There exists a unique T : V → W 3 T B = f . T is monic by the Monomorphism

Theorem (T takes linearly indep. sets to linearly indep. sets iff it’s monic) and is onto as
W = Span(C ) = Span (f (B)).

21
Duc Vu (Spring 2021) 6 Lec 6: Apr 9, 2021

§6 Lec 6: Apr 9, 2021

§6.1 L i n e a r Tr a n s f o r m a t i o n ( C o nt ’ d )

Theorem 6.1
Let T : V → W be linear. Then ∃X ⊆ V a subspace s.t.

V = ker T ⊕ X with X ∼
= im T

Proof. Let B0 be a basis for ker T . Extend B0 to a basis B for V by the Extension
Theorem. Let B1 = B \ B0 , so B = B0 ∨ B1 (B = B0 ∪ B1 and B0 ∩ B1 = ∅) and let
M
X= Fv
B1
L
As ker T = B0 F v, we have
V = ker T ⊕ X
and we have to show
X∼
= im T

Claim 6.1. T v, v ∈ B1 are linearly indep.

In particular, T v 6= T v 0 if v, v 0 ∈ B1 and v 6= v 0 . Suppose
X
αv T v = 0W , αv ∈ F almost all αv = 0
v∈B

Then  
X X
0W = T  αv v  , i.e. αv v ∈ ker T
v∈B1 B1

Hence X X
αv v = βv v ∈ ker T almost all βv ∈ F = 0
B1 B0
As B1 αv v − B0 βv v = 0 and B = B0 ∪ B1 is linearly indep., αv = 0 ∀v. This proves
P P
the above claim.
Let C = {T v| v ∈ B1 }. By the claim
B1 → C by v 7→ T v is 1 − 1
and onto as C is linearly indep. Lastly, we must show C spans im T . Let w ∈ im T . Then
∃x ∈ V 3 T x = w. Then
   
X X
w = Tx = T  αv v  + T  αv v 
B0 B1
X X X
= αv T v + αv T v = αv T v
B0 B1 B1

lies in span C as needed.

22
Duc Vu (Spring 2021) 6 Lec 6: Apr 9, 2021

Remark 6.2. Note that the proof is essentially the same as the proof of the Dimension
Theorem.

Corollary 6.3 (Dimension Theorem)

If V is a finite dimensional vector space over F , T : V → W linear then

dim V = dim ker T + dim im T

Corollary 6.4
If V is a finite dimensional vector space over F , W ⊆ V a subspace, then

dim V = dim W + dim V /W

Proof. − : V → V /W by v 7→ v = W + v is an epi.

Important Construction: Set

T : V → Z be linear
W = ker T
V = V /W
− : V → V /W by v 7→ v = W + v linear

∀x, y ∈ V we have

x = y ∈ V ⇐⇒ x ≡ y mod W ⇐⇒ x − y ∈ W ⇐⇒ T (x − y) = 0Z

i.e., when W = ker T

x = y ⇐⇒ T x = T y (*)
This means
T : V → Z defined by W + v = v 7→ T v
is well-defined, i.e., via function, since if x = y, then T (x) := T x = T y =: T (y). From (*),

x = y ⇐⇒ T (x) = T (x) = T (y) =: T (y)

so
T : V → Z is also injective
As T is linear, let α ∈ F, x, y ∈ V , then

T (αx + y) = T (αx + y) = T (αx + y)

= αT x + T y = αT (x) + T (y)

as needed. Therefore,
T : V → Z by x 7→ T (x)

23
Duc Vu (Spring 2021) 6 Lec 6: Apr 9, 2021

is a monomorphism, so induces an isomorphism onto im T and we recall im T = im T , so

V ∼
= im T = im T
and we have a commutative diagram

T
V Z

−
T

V / ker T = V

This can also be written as

V T Z

inclusion map
−

T
V / ker T = V im T

Consequence: Any linear transformation T : V → Z induces an isomorphism

T : V / ker T → im T by v = ker T + v 7→ T v
This is called the First Isomorphism Theorem. We also have
V = ker T ⊕ X with X ⊆ V and X ∼
= im T ∼
= V / ker T
This means that all images of linear transformations from V are determined, up to
isomorphism, by V and its subspaces. It also means, if V is a finite dimensional vector
space over F , we can try prove things by induction.

§6.2 Pro jections

Motivation: Let m < n in Z+ and
π : Rn → Rn by (α1 , . . . , αn ) 7→ (α1 , . . . , αn , 0, . . . , 0)
!
Lm
a linear operator onto i=1 Γei where ei = 0, . . . , |{z} 1 ,...,0 .
im

Definition 6.5 (T-invariant) — Let T : V → V be linear, W ⊆ V a subspace. We say

W is T -invariant if T (W ) ⊆ V if this is the case, then the restriction T W of T can be
viewed as a linear operator
T W :W →W

24
Duc Vu (Spring 2021) 6 Lec 6: Apr 9, 2021

Example 6.6
Let T : V → V be linear.

1. ker T and im T are T -invariant.

2. Let λ ∈ F be an eigenvalue of T , i.e., ∃0 6= v ∈ V 3 T v = λv, then any subspace

of the eigenspace
ET (λ) := {v ∈ V | T v = λv}
is T -invariant as T ET (λ)
= λ1ET (λ)

Remark 6.7. Let V be a finite dimensional vector space over F , T : V → V linear. Suppose
that
V = W1 ⊕ . . . ⊕ Wn
with each Wi T -invariant, i = 1, . . . , n and Bi an ordered basis for Wi , i = 1, . . . , n. Let
B = B1 ∪ . . . ∪ Bn be a basis of V ordered in the obvious way.
Then the matrix representation of T in the B basis is
h i 
T W1 0
 B1 
[T ]B = 
 .. 
 . h

i 
0 T Wn
Bn

Example 6.8
Suppose that T : V → V is diagonalizable, i.e., there exists a basis B of eigenvectors
of T for V . Then, T : V → V , M
V = ET (λi )
each ET (λi ) is T -invariant.
T ET (λi )
= λi 1ET (λi )

Goal: Let V be a finite dimensional vector space over F , n = dim V , T : V → V linear.

Then ∃W1 , . . . , Wm ⊆ V all T -invariant subspaces with m = m(T ) with each Wi being as
small as possible with V = W1 ⊕ . . . ⊕ Wm . This is the theory of canonical forms.
Recall: If V is a finite dimensional vector space over F , T : V → V linear, B an ordered
basis for V , then the matrix representation [T ]B is only unique up to similarity, i.e., if C
is an another ordered basis
[T ]C = P [T ]B P −1
where P = [1V ]B,C ∈ GLn F , the change of basis matrix B → C .

Definition 6.9 (Projection) — Let V be a vector space over F , P : V → V linear. We

call P a projection if P 2 = P ◦ P = P .

25
Duc Vu (Spring 2021) 6 Lec 6: Apr 9, 2021

Example 6.10 1. P = 0V or 1V : V → V , V is a vector space over F .

2. An orthogonal projection in 115A.

3. If P is a projection, so is 1V − P .

If T : V → V is linear, then

V = ker T ⊕ X with X ∼
= im T

Lemma 6.11
Let P : V → V be a projection. Then

V = ker P ⊕ im P

Moreover, if v ∈ im P , then
Pv = v
i.e.
P im P
: im P → im P is 1im P
In particular, if V is a finite dimensional vector space over F, B1 an ordered basis for
ker P , B2 an ordered basis for im P , then B = B1 ∪ B2 is an ordered basis for V and
 
0
 .. 
 . 
   
[0]B1 0  0 
[P ]B = = 
0 [1im P ]B2  1 

 . . 
 . 
1

Proof. Let v ∈ V , then v − P v ∈ ker P , since

P (v − P v) = P v − P 2 v = P v − P v = 0

Hence
v = (v − P v) + P v ∈ ker P + im P
ker P ∩ im P = 0 and P im P
= 1im P . Let v ∈ im P. By definition, P w = v for some
w ∈ V . Therefore,
Pv = PPw = Pw = v
Hence
P im P
= 1im P
If v ∈ ker P ∩ im P , then
v = Pv = 0

26
Duc Vu (Spring 2021) 7 Lec 7: Apr 12, 2021

§7 Lec 7: Apr 12, 2021

§7.1 P r o j e c t i o n ( C o nt ’ d )

Lemma 7.1
Let V be a vector space over F , W, X ⊆ V subspaces. Suppose

V =W ⊕X

Then ∃! P : V → V a projection satisfying

W = ker P (*)
X = im P

We say such a P is the projection along W onto X.

Proof. Existence: Let v ∈ V . Then

∃!w ∈ W, x ∈ X 3 v = w + x

Define
P : V → V by v 7→ x
To show P 2 = P , we suppose v ∈ V satisfies v = w + x, for unique w ∈ W , x ∈ X. Then check P
is linear
P v = P w + P x = P x = 1X x = x and well
defined
so
P 2v = P x = x = P v ∀v ∈ V
hence P 2 = P .
Uniqueness: Any P satisfying (*) takes a basis for W to 0 and fix a basis of X. Therefore,
P is unique by the UPVS.

Remark 7.2. Compare the above to the case that V is an inner product space over F , W ⊆ V
is a finite dimensional subspace and P : V → V by v →
7 vW , the orthogonal projection of P
onto W .

Proposition 7.3
Let V be a vector space over F , W, X ⊆ V subspaces s.t. V = W ⊕ X, P : V → V the
projection along W onto X, and T : V → V linear. Then the following are equivalent:

1. W and X are both T −invariant.

2. P T = T P .

27
Duc Vu (Spring 2021) 7 Lec 7: Apr 12, 2021

Proof. 2) =⇒ 1) : W is T -invariant: We have W = ker P , so if w ∈ W , P w = 0. Hence

PTw = TPw = T0 = 0

T w ∈ ker P = W so W is T -invariant.
X is T -invariant, X = im P , P |X = 1X . So if x ∈ X

T x = T P x = P T x ∈ im P = X

So X is T -invariant.
1) =⇒ 2) Let v ∈ V . Then ∃! w ∈ W , x ∈ X s.t.

v =w+x

As P |X = 1X and P |W = 0, so P v = P x. By 1), W and X are T -invariant, so

P T v = P T (w + x) = P T w + P T x
= 0 + Tx = TPx = TPw + TPx = TPv

for all v ∈ V and P T = T P .

Remark 7.4. One can easily generalize from the case

V = W1 ⊕ W2

that we did to the case

V = W1 ⊕ . . . ⊕ Wn
by induction on n as  

V = Wi ⊕ W1 ⊕ . . . ⊕ Ŵi ⊕ . . . ⊕ Wn 
|{z}
omit

Construction: Let
V = W1 ⊕ . . . ⊕ Wn
as above. Define
PWi : V → V
to be the projection along W1 ⊕ . . . ⊕ Ŵi ⊕ . . . ⊕ Wn , i.e.

ker PWi = W1 ⊕ . . . ⊕ Ŵi ⊕ . . . ⊕ Wn

and onto Wi = im PWi as in the above Proposition. Then we have

a) Each PWi is linear (and a projection).

b) ker PWi = W1 ⊕ . . . ⊕ Ŵi ⊕ . . . ⊕ Wn .

c) Wi is PWi -invariant and PWi Wi

= 1Wi . In particular, im PWi = Wi .

d) PWi PWj = δij PWi where

(
1, if i = j
δij =
0, 6 j
if i =

28
Duc Vu (Spring 2021) 7 Lec 7: Apr 12, 2021

e) 1V = PW1 + . . . + PWn .

Moreover, if T : V → V is linear and each Wi is T -invariant, then

T PWi = PWi T, i = 1, . . . , n

Hence

T = T 1V = T (PW1 + . . . PWn ) = T PW1 + . . . + T PWn

= PW1 T + . . . + PWn T

i.e., 1V T = T 1V . This implies

T Wi
: Wi → Wi
is given by
T Wi
= T PWi Wi
or T is determined by what it does to each Wi .

Remark 7.5. Compare this to the case that T is diagonalizable and the Wi are the eigenspaces.

Question 7.1. Let V be a real or complex finite dimensional inner product space, T :
V → V hermitian. What can you replace ⊕ by? What if V is a complex finite dimensional
inner product space and T : V → V is normal.

Exercise 7.1. Suppose V is a vector space over F , P1 , . . . , Pn : V → V linear and satisfy

i) Pi − Pj = δij Pi , i = 1, . . . , n

ii) 1V = P1 + . . . + Pn

iii) Wi = im Pi , i = 1, . . . , n

Then

V = W1 ⊕ . . . ⊕ Wn
Pi = PWi i = 1, . . . , n

§7.2 Dual Spaces

Question 7.2. Let V = R3 , v ∈ V . What is the first question that we should ask about v?

Motivation/Construction: Let V be a vector space over F , B a basis for V . Fix v0 ∈ B.

By the UPVS, ∃! fv0 : V → F linear satisfying
(
1 if v0 = v
fvv0 (v) = = δv,v0 ∀v ∈ B
0 if v0 6= v

29
Duc Vu (Spring 2021) 7 Lec 7: Apr 12, 2021

Example 7.6
Let En = {e1 , . . . , en } be the standard basis for Rn and in the above e1 = v0 . . . Then

fe1 : Rn → R satisfies

If v = (α1 , . . . , αn ) in Rn
n
X
v= αi ei
i=1
so
n
!
X
fe1 (v) = fe1 αi ei
i=1
n
X n
X
= αi fe1 (ei ) = αi δii = α1
i=1 i=1

this first coordinate of v.

Notation: If A ⊆ B are sets, we write A < B if A 6= B.

As v0 6= 0,
0 < im fv0 ⊆ F is a subspace
Notice dimF F = 1, so dim im fv0 ≤ dim F = 1 and

dim im fv0 = 1, i.e. im f0 = F

So fv0 : V → F is a surjective linear transformation. Since this is true for all v0 ∈ B, for
each v ∈ B, ∃! fv : V → F s.t.
(
1 if v = v 0
fv (v 0 ) = δv,v0 = ∀v 0 ∈ B
0 if v 6= v 0

Now suppose that x ∈ V , then

X
∃! αv ∈ F, v ∈ B, almost all 0 s.t. x = αv v
B

Hence
!
X X
fv0 (x) = fv0 αv v = αv fv0 (v)
v∈B B
X
= αv δv,v0 = αv0
B

30
Duc Vu (Spring 2021) 7 Lec 7: Apr 12, 2021

Example 7.7
B = En standard basis for Rn
(
1 if ei = ej
fei (ej ) = δei ,ej = δi,j =
0 6 ej
if ei =

Then if v = (α1 , . . . , αn ) ∈ Rn = V . Then

fei (v) = fei (α1 , . . . , αn ) = αi

So we observe in the above that if x ∈ V , then

X
x= fv (x)v
B

We call fv the coordinate function on v relative to B.

Example 7.8
inner product space over R, B = {v1 , . . . , vn } an
Let V be a finite dimensional P
orthonormal basis. Then if x = B αi vi , then

αi = hx, vi i

Take
X X
hx, vi i = h αj vj , vi i = αj hvj , vi i
X X
= αj δij kvi k2 = αj δij = αi

i.e. the linear map

fvi := h, vi i : V → R by x 7→ hx, vi i
is the coordinate function on vectors relative to B.

Definition 7.9 (Dual Space) — Let V be a vector space over F . A linear transformation
f : V → F is called a linear functional. Set

V ∗ := L(V, F ) := {f : V → F |f is linear}

is called the dual space of V .

31
Duc Vu (Spring 2021) 7 Lec 7: Apr 12, 2021

Proposition 7.10
Let V, W be a vector space over F . Then

L(V, W ) := {T : V → W |T linear}

is a vector space over F . Moreover, if V, W are finite dimensional vector spaces over F

dim L(V, W ) = dim V dim W

In particular, if V is a finite dimensional vector space over F , then so is V ∗ and

dim V = dim V ∗

so
V ∼
=V∗

Proof. 115A.

Example 7.11
Let V be a vector space over F . Then the following are linear functionals

1. 0 : V → F

2. Let 0 6= v0 ∈ V then {v0 } is a basis for F v0 . Therefore, {v0 } extends to a basis

B for V . Let f v0 ∈ V ∗ be the coordinate function for V on v0 relative to B.
Then f v0 ∈ B ∗ := {f v|v ∈ B}.

32
Duc Vu (Spring 2021) 8 Lec 8: Apr 14, 2021

§8 Lec 8: Apr 14, 2021

§8.1 D u a l S p a c e s ( C o nt ’ d )

Example 8.1 (Cont’d from Lec 7) 3. trace: Mn F → F by

n
X
A 7→ Aii
i=1

4. α < β ∈ R, then
Z β
I : C [α, β] → R by f 7→ f
α

5. Fix γ ∈ [α, β] , α < β ∈ R. Then the evaluation map at γ

eγ : C [α, β] → R by f 7→ f (γ)

Lemma 8.2
Let V be a vector space over F , B a basis for V ,

B ∗ := {f v0 : V → F | coordinate function on v0 relative to B}

so
f v0 (v) = δv0 ,v ∀v ∈ B
the set of coordinate functions relative to B. Then B ∗ ⊆ V ∗ is linearly indep.

Proof. Suppose
X
0 = 0V ∗ = βvf v, βv ∈ F almost all 0
v∈B

We need to show βv = 0 ∀v ∈ B. Evaluation at v0 ∈ B yields

!
X X
0 = 0V ∗ (v0 ) = βvf v (v0 ) = βvf v(v0 )
B
X
= βvfv,v0 = βv0
B

So βv = 0 ∀v ∈ B and the lemma follows.

33
Duc Vu (Spring 2021) 8 Lec 8: Apr 14, 2021

Corollary 8.3
Let V be a vector space over F with basis B. Then the linear transformation

DB : V → V ∗ induced by B → B ∗ by v 7→ f v

is a monomorphism.
In particular, if V is a finite dimensional vector space over F , then B ∗ is a basis for
V ∗ and
DB : V → V ∗ is an isomorphism

Proof. By the Monomorphism Theorem, DB is monic in view of he lemma if V is a finite

dimensional vectors space over F , then
dim V = dim V ∗
so V ∼
= V ∗ by the Isomorphism Theorem.

Remark 8.4. 1. If V = R∞
f := {(α1 , α2 , . . .) |αi ∈ R almost all 0}, then by HW1 # 4,

DE∞ : V → V ∗ is not an isomorphism

2. DB : V → V ∗ in the corollary depends on B. There exists no monomorphism V → V ∗

that does not depend on a choice of basis. However, there exists a “nice” monomorphism,
i.e., defined independent of basis.

L : V → (V ∗ )∗ =: V ∗∗

V ∗∗ is called the double dual of V . We now construct it.

Lemma 8.5
Let V be a vector space over F , v ∈ V . Then

Lv : V ∗ → F by f 7→ Lv (f ) := f (v)

the evaluation map at v is linear, i.e.

Lv ∈ V ∗∗

Proof. For all f, g ∈ V ∗ , α ∈ F

Lv (αf + g) = (αf + g)(v) = αf (v) + g(v) = αLv f + Lv g

Theorem 8.6
The “natural” map
L : V → V ∗∗ by v 7→ L(v) := Lv
is a monomorphism.

34
Duc Vu (Spring 2021) 8 Lec 8: Apr 14, 2021

Proof. L is linear: Let v, w ∈ V, α ∈ F . Then for all f ∈ V ∗ , as V ∗∗ = (V ∗ )∗

L(αv + w)(f ) = Lαv+w (f ) = f (αv + w)

= αf (v) + f (w) = αLv f + Lw f = (αLv + Lw ) (f )
= (αL(v) + L(w)) (f )

So
L(αv + w) = αL(v) + L(w)
L is monic. Suppose v 6= 0. To show Lv = L(v) 6= 0. By example 2,

∃0 6= f ∈ V ∗ 3 f (v) 6= 0

So
Lv f = f (v) 6= 0
so Lv = L(v) 6= 0 and L is monic.

Corollary 8.7
If V is a finite dimensional vector space over F , then L : V → V ∗∗ is a natural
isomorphism.

Proof. dim V = dim V ∗ = dim V ∗∗ and the Isomorphism Theorem.

Identification: Let V be a finite dimensional vector space over F . Then ∀v, w ∈ V

1. v = w ⇐⇒ Lv = Lw

2. ∀f ∈ V ∗ f (v) = f (w) ⇐⇒ Lv f = Lw f

Moreover, if W is also a finite dimensional vector space over F , then if T : V → W is linear,

∃! T̃ : V ∗∗ → W ∗∗ linear and if T̃ : V ∗∗ → W ∗∗ ∃! T : V → W linear. In other words, V
and V ∗∗ can be identified by
v ↔ Lv
because
Lv (f ) = f (v) ∀v ∈ V ∀f ∈ V ∗
Construction: Let V be a finite dimensional vector space over F with basis B = {v1 , . . . , vn }.
Then
B ∗ := {f1 , . . . , fn }
defined by
fi (vj ) = δij ∀i, j
i.e., fi is the coordinate function on vi relative to B. Since

Lvi (fj ) = fj (vi ) = δij ∀i, j

Lvi ∈ V ∗∗
B ∗∗ := {Lv1 , . . . , Lvn }

35
Duc Vu (Spring 2021) 8 Lec 8: Apr 14, 2021

Pn Pn
is the dual basis of B ∗ for V ∗∗ . So we have if x = i=1 αi vi ∈V, g = i=1 βi fi ∈ V ∗.
n
X n
X
x= αi vi = fi (x)vi
i=1 i=1
Xn Xn n
X
g= βi fi = Lvi (g)fi = g(vi )fi
i=1 i=1 i=1

i.e.
n
X
x= fi (x)vi ∀x ∈ V
i=1
Xn
g= g(vi )fi ∀g ∈ V ∗
i=1

Motivation: Let V be an inner product space over R, ∅ = 6 S ⊆ V a subset. What is S ⊥ ?

Note: ∀v ∈ V , h, vi : V → R by x 7→ hx, vi is a linear functional. To generalize this to an
arbitrary vector space over F , we define the following.

Definition 8.8 (Annihilator) — Let V be a vector space over F , ∅ =

6 S ⊆ V a subset.
Define the annihilator of S to be

S ◦ := {f ∈ V ∗ | f (x) = 0 ∀x ∈ S}
= {f ∈ V ∗ | f |S = 0} ⊆ V ∗

Remark 8.9. Many people write hv, f i for f (v) in the above even though f ∈
/ v.

36
Duc Vu (Spring 2021) 9 Lec 9: Apr 16, 2021

§9 Lec 9: Apr 16, 2021

§9.1 D u a l S p a c e s ( C o nt ’ d )

Lemma 9.1
Let V be a vector space over F , ∅ =
6 S ⊆ V a subset. Then

1. S ◦ ⊆ V ∗ is a subspace.

2. If V is a finite dimensional vector space over F and we identify V as V ∗∗ (by

v ↔ Lv ), then S ⊆ S ◦◦ := (S ◦ )◦ .

Proof. 1. For all f, g ∈ S ◦ , α ∈ F , we have

(αf + g)(x) = αf (x) + g(x) = 0 ∀x ∈ S

Hence αf + g ∈ S ◦ and S ◦ ⊆ V ∗ is a subspace.

2. Let x ∈ S. Then ∀f ∈ S ◦ , we have

0 = f (x) = Lx f, so Lx ∈ (S ◦ )◦ = S ◦◦

Theorem 9.2
Let V be a finite dimensional vector space over F , S ⊆ V a subspace. Then

dim V = dim S + dim S ◦

Proof. Let B0 = {v1 , . . . , vk } be a basis for S. Extend this to

B = {v1 , . . . , vn } a basis for V

B0 = {f1 , . . . , fn } the dual basis of B

Claim 9.1. C := {fk+1 , . . . , fn } is a basis for S ◦ .

If we show this, the theorem follows. Let f ∈ S ◦ . Then
n
X n
X
f= Lvi (f )fi = f (vi )fi
i=1 i=1
Xk n
X n
X
= f (vi )fi + f (vi )fi = f (vi )fi
i=1 i=k+1 i=k+1

lies in span C so C spans. As C ⊆ B ∗ which is linearly indep., so is C . This proves the

claim.

37
Duc Vu (Spring 2021) 9 Lec 9: Apr 16, 2021

Corollary 9.3
Let V be a finite dimensional vector space over F , S ⊆ V a subspace. Then S = S ◦◦ .

Proof. As S ⊆ S ◦◦ , it suffices to show dim S = dim S ◦◦ . By the theorem, we have

dim V = dim S + dim S ◦
dim V ∗ = dim S ◦ + dim S ◦◦
where dim V = dim V ∗ . So dim S = dim S ◦◦ .

Remark 9.4. If V is an inner product space over R, compare all this to ∅ =

6 S ⊆ V a subset
and S ⊥ , S ⊥⊥ .

§9.2 T h e Tr a n s p o s e
Construction: Fix T : V → W linear. For every S : W → X, we have a composition
S ◦ T : V → X is linear
So T :→ W linear induces a map
T ? : L(W, X) → L(V, X)
by
S 7→ S ◦ T

Proposition 9.5
Let V, W, X be vector spaces over F , T : V → W linear. Then

T ? : L(W, X) → L(V, X)

is linear.

Proof. Let S1 , S2 ∈ L(W, X), α ∈ F . Then

T ? (αS1 + S2 ) = (αS1 + S2 ) ◦ T
= αS1 ◦ T + S2 ◦ T = αT ? S1 + T ? S2

Corollary 9.6
Let T : V → W be linear. Then

T ∗ : W ∗ → V ∗ by f 7→ f ◦ T

is linear.

Proof. Let X = F in the proposition.

38
Duc Vu (Spring 2021) 9 Lec 9: Apr 16, 2021

Definition 9.7 (Transpose) — Let T : V → W be linear. The linear map T ? : W ∗ →

V ∗ in the corollary is called the transpose of T and denoted by T > .

Note: The transpose “turns thing around”

T
V −→ W
T>
V ∗ ←− W ∗

Lemma 9.8
Let T : V → W be linear. Then

ker T > = (im T)◦ ∈ W ∗

Proof. g ∈ ker T > ⇐⇒ T > g = 0 ⇐⇒ (T > g)(v) = 0 ∀v ∈ V ⇐⇒ (g ◦ T )(v) = 0

∀v ∈ V ⇐⇒ g(T v) = 0 ∀v ∈ V ⇐⇒ g ∈ (im T )◦ .

Theorem 9.9
Let V, W be finite dimensional vector space over F , T : V → W linear. Then

dim im T = dim im T >

Proof. Consider:

dim W ∗ = dim ker T > + dim im T >

dim W = dim im T + dim(im T )◦

Notice that dim W ∗ = dim W . By the lemma, dim im T = dim im T > .

Computation: Let V, W be finite dimensional vector space over F .

B, B ∗ ordered dual bases for V, V ∗

C , C ∗ ordered dual bases for W, W ∗

Suppose

B = {v1 , . . . , vn } , B ∗ = {f1 , . . . , fn }
fi (vj ) = δij ∀i, j

So

C = {w1 , . . . , wn } , C ∗ = {g1 , . . . , gn }
gi (wj ) = δij ∀i, j

39
Duc Vu (Spring 2021) 9 Lec 9: Apr 16, 2021

Let h i
A = [T ]B,C , B = T>
C ∗ ,B ∗

be the matrix representation of T, T > in the ordered bases B, C and C ∗ , C ∗ respectively.

By definition of A and B, we have
m
X
T vk = Aik wi k = 1, . . . , n
i=1
n
X
T > gj = Bij fi j = 1, . . . , m
i=1

So
Bkj = Ajk ∀j, k
So we just proved. . .

Theorem 9.10
Let V, W be finite dimensional vector space over F , T : V → W linear, B, B ∗ ordered
dual bases for V, V ∗ and C , C ∗ ordered dual bases for W, W ∗ . Then
h i  >
T> = [T ]B,C
C ∗ ,B ∗

Definition 9.11 (Row/Column Rank) — Let A ∈ F m×n . The row (column) rank of A
is the dimension of the span of the rows (columns) of A.

We know if A ∈ F m×n , we can view

A : F n×1 → F m×1 by v 7→ A · v

a linear transformation and the matrix representation of A is

A = [A]En,1 ,Em,1

where En,1 , Em,1 are the standard bases for F n×1 and F m×1 respectively.

Corollary 9.12
Let A ∈ F m×n . Then
row rank A = column rank A
and we call this common number the rank of A.

§9.3 Po l y n o m i a l s

40
Duc Vu (Spring 2021) 9 Lec 9: Apr 16, 2021

Definition 9.13 (Polynomial Division) — Let f, g ∈ F [t], f 6= 0. We say that f divides

g ∈ F [t] write f |g if ∃h ∈ F [t] s.t. g = f h, i.e. g is multiple of f , e.g. t + 1|t2 − 1.

Lemma 9.14
If f |g and f |h in F [t], then f |gk + hl in F [t] for all k, l ∈ F [t].

Proof. By definition,
g = f g1 , h = f h1 , g1 , h1 ∈ F [t]
So
gk + hl = f g1 k + f h1 l = f (g1 k + h1 l)
in F [t].

Remark 9.15. If f |g ∈ F [t] and 0 6= a ∈ F , then af |g and f |ag.

Definition 9.16 (Polynomial Degree and Leading Coefficient) — Let

0 6= f = atn + an−1 tn−1 + . . . + a1 t + a0 ∈ F [t]

with a, a0 , . . . , an−1 ∈ F and a 6= 0. We call n the degree of f write deg f = n and a

the leading coefficient of F write lead f = a. If a = 1, we say f is monic.

We can define the degree of 0 ∈ F [t] to be the symbol −∞ or just do not define it at
all.
Remark 9.17. Let f, g ∈ F [t] \ {0}. Then

lead(f g) = lead(f ) · lead(g) 6= 0 ∈ F

So
deg(f g) = deg f + deg g

41
 Duc Vu (Spring 2021) 10 Lec 10: Apr 19, 2021

§10 Lec 10: Apr 19, 2021

§10.1 Po l y n o m i a l s ( C o nt ’ d )
Division Algorithm: Let 0 6= f ∈ F [t], g ∈ F [t]. Then

∃! q, r ∈ F [t]

satisfying
g = fq + r with r = 0 or deg r < deg f

Definition 10.1 (Greatest Common Divisor) — Let f, g ∈ F [t] \ {0}. We say d in F [t]
is a gcd (greatest common divisor) of f, g if

i) d is monic.

ii) d|f and d|g in F [t].

iii) if e|f and e|g in F [t], then e|d in F [t].

Remark 10.2. If a gcd of f, g exists, then it is unique.

Remark 10.3. If d = 1 is a gcd of f, g ∈ F [t], we say that f, g are relatively bear.

Remark 10.4. Compare the above with analogous in Z.

Theorem 10.5
Let f, g ∈ F [t] \ {0}. Then a gcd of f, g exists and is unique write gcd(f,g) for the gcd
of f, g. Moreover, we have an equation

d = f k + gl ∈ F [t] for some k, l ∈ F [t] (?)

Proof. The existence and (?) follow from the Euclidean Algorithm. Let f, g ∈ F [t] \ {0}.
Then iteration of the Division Algorithm produces equations in F [t], if f + g ∈ F [t],

g = q1 f + r1 deg r1 < deg f

f = q2 r1 + r2 deg r2 < deg r1
..
.
rn−3 = qn−1 rn−2 + rn−1 deg rn−1 < deg rn−2
rn−2 = qn rn−1 + rn deg rn−1 < deg rn
rn−1 = qn+1 + rn

42
Duc Vu (Spring 2021) 10 Lec 10: Apr 19, 2021

where rn is the remainder of least degree (rn 6= 0).

This must stop in ≤ deg f steps. Plugging from the bottom up and using the lemma shows

rn = f k + gl ∈ F [t]

and if e|r1 → e|r2 → . . . → e|rn then (lead rn )−1 rn is the gcd of f and g in F [t] if a = lead f

a−1 rn = a−1 f k + a−1 gl

Definition 10.6 (Irreducible Polynomial) — f ∈ F [t] \ F is called irreducible if there

does not exist g, h ∈ F [t] 3 f = gh with deg g, deg h < deg f . Equivalently, if

f = gh ∈ F [t], then 0 6= g ∈ F or 0 6= h ∈ F

Example 10.7
If f ∈ F [t], deg f = 1, then f is irreducible.

Remark 10.8. If f, g ∈ F [t] \ F with f irreducible, then either f and g are relatively prime
or f |g since only a, af, 0 6= a ∈ F can divide f .

Lemma 10.9 (Euclid)

Let f ∈ F [t] be irreducible and f |gh in F [t]. Then f |g or f |h.

Proof. Suppose f × g where × means does not divide. Then f and g are relatively prime.
By the Euclidean Algorithm, there exists an equation

1 = f k + gl ∈ F [t]

Hence
h = f hk + ghl ∈ F [t]
As f |f hk and f |ghl in F [t], f |h by the lemma.

Remark 10.10. In Z the analog of an irreducible element is called a prime element.

Remark 10.11. Euclid’s lemma is the key idea. The “correct” generalization of “prime” is the
conclusion of Euclid’s lemma. This generalization is profound as, in general, there is difference
between the two conditions “irreducible” and “prime”, although not for Z or F [t].

We know that any positive integer is a product of positive primes unique up to order n. If
we allow n < 0 such is unique up to ±1.

43
Duc Vu (Spring 2021) 10 Lec 10: Apr 19, 2021

Theorem 10.12 (Fundamental Theorem of Arithmetic (Polynomial Case))

Let g ∈ F [t] \ F . Then there exists uniquely a ∈ F , r ∈ Z+ , p1 , . . . , pr ∈ F [t] distinct
monic irreducible polynomial, e1 , . . . , er ∈ Z+ s.t. we have a factorization

g = ape11 . . . perr

unique up to order.

Proof. (Sketch) Existence: We induct on n = deg g ≥ 1. If g is irreducible, a, (lead g)−1 g, a =

lead g work. If g is reducible,

g = f h ∈ F [t], 1 < deg f, deg h < deg g

By induction, f, h have factorization hence we’re done as g = f h.

Uniqueness: We induct on n = deg g ≥ 1. If

ape11 . . . perr = g = bq1f1 . . . qsfs

with pi , qi monic irreducible, a, b ∈ F, ei , fj ∈ Z+ for all i, j, deg q1 ≥ 1, so deg q1 × a. By

Euclid’s lemma
qi |pj for some j
Changing notation, we may assume that j = 1. As p1 is irreducible p1 = q1 and by (M 30 )

g0 := ape11 −1 pe22 . . . perr = bq1f1 −1 q2f2 . . . qsfs

As deg g0 < deg g, induction yields

r = s, e1 − 1 = f1 − 1, ei = fi , i > 1, a = b = lead g0 , pi = qi ∀i, ei = fi ∀i

Remark 10.13. Applying the Euclidean Algorithm is relatively fast to compute, (for f |g
takes ≤ deg f steps to get a gcd). Factoring into the irreducible is not.

44
 Duc Vu (Spring 2021) 11 Lec 11: Apr 21, 2021

§11 Lec 11: Apr 21, 2021

§11.1 M i n i m a l Po l y n o m i a l s
We use the following theorem from 115A, Matrix Theory Theorem.

Remark 11.1. Let T : V → V be linear. If f = an tn + . . . + a1 t + a0 ∈ F [t], we can plug T

in for t to get
f (T ) = an T n + . . . + a1 T + a0 1V ∈ L(V, V )
More precisely
eT : F [t] → L(V, V ) by t 7→ T
i
ai T i is a ring homomorphism. Since we have
P P
i.e. f = ai t 7→ f (T ) =

T n = T ◦| .{z
. . ◦} T, n≥0
n

Can we use the remark if V is a finite dimensional vector space over F ?

Lemma 11.2
Let V be a finite dimensional vector space over F , f, g, h ∈ F [t], B an ordered basis
for V , T : V → V linear. Then

1. [g(T )]B = g ([T ]B )

2. If f = gh ∈ F [t], then
f (T ) = g(T )h(T )

Pn
Proof. • By MTT, if g = i=0 ai t
i
∈ F [t], then
" n # n
X X
i
ai T i B
 
[g(T )]B = ai T =
i=0 B i=0
X
= ai [T ]iB = g ([T ]B )

• Left as exercise.

Lemma 11.3
Let V be a finite dimensional vector space over F , T : V → V linear. Then ∃q ∈
F [t]\{0} 3 q(T ) = 0 and if a = lead q, then q0 := a−1 q is moinc and satisfies q0 (T ) = 0

q ∈ ker eT := {f ∈ F [t]| f (T ) = 0}

Proof. Let n = dim V . By MTT

dim L(V, V ) = dim Mn F = n2 < ∞

45
Duc Vu (Spring 2021) 11 Lec 11: Apr 21, 2021

So
2
1V , T, T 2 , . . . , T n ∈ L(V, V )
are linearly dependent. So ∃a0 , . . . , an2 ∈ F not all 0 s.t.
n 2
X
ai T i = 0
i=0

Pn2 i
Then q = i=0 ai t works.

Theorem 11.4
Let V be a finite dimensional vector space over F , T : V → V linear. Then ∃!0 6=
qT ∈ F [t] monic called the minimal polynomial of T having the following properties:

1. qT (T ) = 0

2. If g ∈ F [t] satisfies g(T ) = 0, then qT |g ∈ F [t]. In particular, if 0 6= g ∈ F [t]

satisfies g(T ) = 0, then deg g ≥ deg qT and if deg g = deg qT , then g = (lead g)qT

Proof. By the lemma, ∃0 6= q ∈ F [t] monic s.t. q(T ) = 0. Among all such q, choose one
with deg q minimal.
Claim 11.1. q works.
Let g 6= 0 in F [t] satisfy g(T ) = 0. To show q|g ∈ F [t]. Write g = qh + r in F [t] with
r = 0 or deg r < deg q. Then

0 = g(T ) = q(T )hh(T ) + r(T ) = r(T )

If r 6= 0, then r0 = (lead r)−1 r is a monic poly satisfying r0 (T ) = 0, deg r0 < deg q,

contradicting the minimality of deg q. So r0 = 0 and q|g ∈ F [t]. If q 0 also satisfies 1) and
2), then
q|q 0 and q 0 |q ∈ F [t] both monic so q = q 0
The last statement follows as if

h, g ∈ F [t], g|h, h 6= 0, then deg h ≥ deg q

Corollary 11.5
Let V be a finite dimensional vector space over F , B an ordered basis for V1 and
T : V → V linear. Then
qT = q[T ]B
In particular, if A, B ∈ Mn F are similar write A ∼ B. Then

qA = qB

Proof. qT = q[T ]B by MTT and the first lemma.

46
Duc Vu (Spring 2021) 11 Lec 11: Apr 21, 2021

Note:By the theorem, if V is a finite dimensional vector space over F g ∈ F [t] g = 6 0, and
deg g < deg qT , then q(T ) 6= 0.
Goal: Let V be a finite dimensional vector space over F , B an ordered basis of V , T : V → V
linear. Call
tI − [T ]B the characteristics matrix of T relative to B
Recall the characteristics polynomial fT of T is defined to be

fT := f[T ]B = det (tI − [T ]B ) ∈ F [t]

We want to show fT satisfies the

Theorem 11.6 (Cayley-Hamilton)

If V is a finite dimensional vector space over F , T : V → V linear, then

qT |fT , hence fT (T ) = 0

In particular, deg qT ≤ deg fT .

Remark 11.7. 1. There exists a determinant proof of this – essentially Cramer’s rule.
2. A priori we only know deg qT ≤ n2 , where n = dim V .
3. fT is independent of B depends on properties of det : Mn F [t] → F [t]

det (tI − A) = det P (tI − A) P −1

= det tI − P AP −1

for each P ∈ GLn F

Proposition 11.8
Let V be a finite dimensional vector space over F , T : V → V linear. Then qT and fT
have the same roots in F , the eigenvalues of T .

Proof. Let λ be a root of qT . To show λ is an eigenvalue of T , i.e., a root of fT . As λ is a

root of qT , using the Division Algorithm that

qT = (t − λ)h ∈ F [t]

So
0 = qT (T ) = (T − λ1V )h(T )
As
0 ≤ deg h < deg qT , we have h(T ) 6= 0
Since h(T ) 6= 0∃0 6= v ∈ V s.t.
w = h(T )v 6= 0
Then
0 = qT (T )v = (T − λ1V ) h(T )v = (T − λ1V ) w

47
Duc Vu (Spring 2021) 11 Lec 11: Apr 21, 2021

So 0 6= w ∈ ET (λ) and λ is an eigenvalue of T .

Conversely, suppose λ is a root of fT so an eigenvalue of T . Let 0 6= v ∈ ET (λ). Then
t − λ ∈ F [t] satisfies (T − λ)w = 0 for all w ∈ F v, i.e. it is the minimal poly of
T |F v : F v → F v. But qT (T ) = 0 on V so t − λ|qT by the definition that t − λ is the minimal
poly of T |F v .

§11.2 Algebraic Aside

Let V be a finite dimensional vector space over F , T : V → V linear. Te minimality poly
qT of T is algebraically more interesting than fT . Recall we have a ring homomorphism

eT : F [t] → L(V, V )

given by X X
ai ti 7→ ai T i
so eT is not only a linear transformation but a ring homomorphism, i.e., it also follows that

(f g)(T ) = f (T )g(T ) ∀f, g ∈ F [t]

We know that
dimF F [t] = ∞
which has {1, t, . . . , tn , . . .} is a basis for F [t] and

dimF L(V, V ) = (dim V )2 < ∞

by MTT. So
0 < ker eT := {f ∈ F [t]|eT f = f (T ) = 0}
is a vector space over F and a subspace of F [t]. This induces a linear transformation

eT : V / ker eT → im eT = F [T ]

which is an isomorphism. If V = V / ker T , we have

X  X  X
i
eT ai ti = eT ai t i = ai T
X i X
= ai T = ai T i

Check that eT is also a ring isomorphism onto im eT . By definition, if f (T ) = 0, f ∈ F [t],

then
qT |f ∈ F [t]
It follows that
ker eT = {qt g|g ∈ F [t]} ⊆ F [t]
called an ideal in the ring F [t].
The first isomorphism of rings gives rise to ker eT whit quotient isomorphic to F [t] ⊆ L(V, V ).
So we are at a higher level of algebra. Then this allows us to view F [t] as acting on V , i.e.
there exists a map
F [t] × V → V (*)

48
Duc Vu (Spring 2021) 11 Lec 11: Apr 21, 2021

by

f · v := f (T )v
qT (T ) = 0

This turns V into what is called an F [t]-module, i.e., V via (*) satisfies the axioms of a
vector space over F but the scalars F [t] are now a ring rather than only a field.

49
 Duc Vu (Spring 2021) 12 Lec 12: Apr 23, 2021

§12 Lec 12: Apr 23, 2021

§12.1 Tr i a n g u l a r i z a b i l i ty

Proposition 12.1
Let V be a finite dimensional vector space over F , T : V → V linear, W ⊆ V a
T -invariant subspace. Then T induces a linear transformation

T : V /W → V /W by T (v) := T (v)

where v = W + v, V = V /W and

qT |qT ∈ F [t]

Proof. By the hw, we need only to prove that

qT |qT ∈ F [t]

But also by the hw,

qT (T ) = qT (T )
As qT (T ) = 0,
0 = qT (T ) = qT (T )
so
qT |qT
by the defining property of qT .

Definition 12.2 (Triangularizability) — Let V be a finite dimensional vector space

over F , T : V → V linear. We say T is triangularizable if ∃ an ordered basis B for V
s.t. A = [T ]B satisfies Aij = 0 ∀i < j, i.e.
 
∗ 0
A =  . . .  is lower triangular (*)
 

∗ ∗

Note: If B = {v1 , . . . , vn } in (*) and C = {vn , vn−1 , . . . , v1 }, then

 
∗ ∗
 ..
[T ]C =   is upper triangular

.
0 ∗

Hence, by Change of Basis Theorem,

[T ]B ∼ [T ]C

50
Duc Vu (Spring 2021) 12 Lec 12: Apr 23, 2021

Remark 12.3. Suppose V is a finite dimensional vector space over F , dim V = n, T : V → V

linear, B an ordered basis for V , A = [T ]B is triangular (upper or lower). Then

fT = (t − A11 ) . . . (t − Ann ) ∈ F [t]

and A11 , . . . , Ann are all the eigenvalues of T (not necessarily distinct) and hence roots of qT .

Definition 12.4 (Splits) — We say g ∈ F [t] \ F splits in F [t] if g is a product of linear

polys in F [t], i.e.,
g = (lead g)(t − α1 ) . . . (t − αn ) ∈ F [t]

Example 12.5
If V is a finite dimensional vector space over F , T : V → V linear and T is triangular-
izable, then fTsplits in F [t].
0 1
Note: ∈ M2 R is not triangularizable as it has no eigenvalues.
−1 0

Theorem 12.6
Let V be a finite dimensional vector space over F , T : V → V linear. Then T is
triangularizable if and only if qT splits in F [t].

Proof. “ =⇒ ” We induct on n = dim V .

n = 1 : It’s obvious.
n > 1 : We proceed by induction: let λ be a root of qT in F (qT splits in F [t]). Then λ is a
root of qT hence an eigenvalue of T . Let 0 6= vn ∈ ET (λ), so W = F vn is T -invariant. By
the Proposition, T induces a linear map
T : V /W → V /W by v 7→ T (v)
and
qT |qT ∈ F [t]
We also know that
W = ker(− : V → V /W ) by v 7→ v
and
dim V /W = dim V − dim W = n − 1
as − : v → v is epic. Since qT splits in F [t] and qT |qT in F [t], qT also splits in F [t] by
Fundamental Theorem of Algebra. Thus, by induction,
∃v1 , . . . , vn−i ∈ V 3 C = {v1 , . . . , v n−1 }
 
is an ordered basis for V = V /W with A = T C is lower triangular, i.e., Aij = 0 if
i < j ≤ n − 1. Thus
n−1
X
T vj = Aij v i , 1≤j ≤n−1
i=j

51
Duc Vu (Spring 2021) 12 Lec 12: Apr 23, 2021

hence
n−1
X n−1
X
0 = T vj − Aij v i = T vj − Aij vi
i=j i=j

1 ≤ j ≤ n − 1 in V = V /W . Therefore,
n−1
X
T vj − Aij vi ∈ ker− = W = F vn
i=j

by definition as W = ker− : V → V /W .
In particular, ∃Anj ∈ F , 1 ≤ j ≤ n − 1 satisfying
n−1
X
T vj − Aij vi = Anj vn
i=j

So
n
X
T vj = Aij vn 1≤j ≤n−1
i=j

By choice, Aij = 0, i < j ≤ n − 1 and

T vn = λvn

By hw 2 # 3, B = {v1 , . . . , vn } is an ordered basis for V and

   
T C 0
 .. 
[T ]B = 
 .
 0
An1 . . . An,n−1 λ

which is lower triangular, as needed. “ =⇒ ” Let B = {v1 , . . . , vn } be an ordered basis

for V . A = [T ]B is lower triangular. Then
n
Y
fT = (t − Aii ) splits in F [t]
i=1

A11 , . . . , Ann are the (not necessarily distinct) eigenvalues of T and hence roots of qT .
Let λi = Aii , i = 1, . . . , n. We have
n
X n
X
T vj = Aij vi = λj vj + Aij vi , 1≤j ≤n−1
i=1 i=j+1

T vn = λn vn

So
n
X
(T − λj 1V )vj = Aij vi ∈ Span (vj+1 , . . . , vn ) ∀1 ≤ j ≤ n − 1 (*)
i=j+1

Now
(T − λn 1V )vn = 0

52
Duc Vu (Spring 2021) 12 Lec 12: Apr 23, 2021

So
(T − λn 1V )vn−1 ∈ Span(vn ) by (∗)
This implies
(T − λn 1V )(T − λn−1 1V )vn−1 = 0
By induction, we may assume that

(T − λn 1V ) . . . (T − λj 1V )vj = 0

So by (*),
(T − λn 1V ) . . . (T − λj 1V )(T − λj−1 1V )vj−1 = 0
Therefore,
fT (T )vi = (T − λn 1V ) . . . (T − λi 1V )vi = 0
for i = 1, . . . , n. As B is a basis for V , fT (T ) = 0. Thus qT |fT ∈ F [t]. In particular, qT
splits in F [t].

Corollary 12.7
Let V be a finite dimensional vector space over F , T : V → V a triangularizable linear
operator. Then
qT |fT ∈ F [t]
In particular,
fT (T ) = 0

Definition 12.8 (Algebraically Closed) — A field F is called algebraically closed if

every f ∈ F [t] \ F splits in F [t]. Equivalently, f ∈ F [t] \ F has a root in F .

Corollary 12.9 (Cayley-Hamilton – Special Case)

Let F be algebraically closed, V a finite dimensional vector space over F , T : V → V
linear. Then

1. T is triangularizable.

2. qT |fT

3. fT (T ) = 0

Theorem 12.10 (Fundamental Theorem of Algebra)

(FTA) C is algebraically closed.

Proof. It’s assumed (proven in 132 – Complex Analysis or 110C – Algebra).

53
 Duc Vu (Spring 2021) 13 Lec 13: Apr 26, 2021

§13 Lec 13: Apr 26, 2021

§13.1 Tr i a n g u l a r i z a b i l i ty ( C o nt ’ d )
Remark 13.1. Let V be a finite dimensional vector space over F , T : V → V linear, B an
ordered basis for V , A = [T ]B . So qA = qT and fA = fT .

Let n = dim V . Given a field F , ∃F̃ an algebraically closed field satisfying F ⊆ F̃ is a

subfield. Then
A ∈ Mn F ⊆ Mn F̃
So by the corollary,
fA (A)v = 0 ∀v ∈ F̃ n×1
where we view A : F̃ n×1 → F̃ n×1 linear. Then

fA (A)v = 0 ∀v ∈ F n×1 ⊆ F̃ n×1

viewing
A : F n×1 → F n×1 linear
Thus,
fA (A) = 0
Hence fT (T ) = 0 and qT = qA |fA = fT . So qT |fT in F [t]. Thus, if we knew such an F̃
exists in general, we would have proven the Cayley-Hamilton Theorem in general, i.e., if V
is a finite dimensional vector space over F and T : V → V linear, then

qT |fT ∈ F [t]
fT (T ) = 0

This is, in fact, true (and proven in Math 110C). Of course, assuming FTA, this proves
Cayley-Hamilton for all fields F ⊆ C.

Remark 13.2. The symmetric matrices

   
0 1 2 1
∈ M2 F2 and ∈ M2 F 5
1 0 1 3

are both triangularizable, but not diagonalizable.

§13.2 Primary Decomposition

Algebraic Motivation: Let f ∈ F [t] \ F be monic. Write

f = pe11 . . . perr , p1 , . . . , pr distinct monic

irreducible polys in F [t], ei > 0 ∀i. Set

f
q= = pe11 . . . pei i . . . perr
pei i

54
Duc Vu (Spring 2021) 13 Lec 13: Apr 26, 2021

Then pi , qi are relatively prime so there exists an equation

1 = pei i ki + qi gi ∈ F [t], i = 1, . . . , n (*)

if we plug a linear operator T : V → V into (*), we get

1V = pei i (T )k1 (T ) + qi (T )gi (T ) ∀i

Linear Algebra Motivation: Let V be a finite dimensional vector space over F , T : V → V

linear. Suppose
V = W1 ⊕ W2 , W1 , W2 ⊆ V subspaces
with W1 , W2 both T -invariant.
Let Bi be an ordered basis for Wi , i = 1, 2 and B = B1 ∪ B2 an ordered basis for V . Then
 
[T |W1 ]B1 0
[T ]B =
0 [T |W2 ]B2

Let PWi : V → V be the projection onto Wi along Wj , j 6= i. Then we know

1V = PW1 + PW2
PWi PWj = δij PWj
PWi T = T PWi , i = 1, 2
T = T PW1 + T PW2 = T |W1 + T |W2

By hw 4 # 6
qT = lcm (qT |W1 , qT |W2 )
This easily extends to more blocks.

Lemma 13.3
Let f ∈ F [t], T : V → V linear. Then ker f (T ) is T -invariant.

Proof. If v ∈ ker f (T ), to show T v ∈ ker f (T ). But

f (T ) T v = T f (T )v = 0

so this is immediate.

Lemma 13.4
Let g, h ∈ F [t] \ F be relatively prime. Set f = gh ∈ F [t]. Suppose T : V → V is
linear and f (T ) = 0. Then

ker g(T ) and ker h(T ) are T -invariant

subspaces of V and
V = ker g(T ) ⊕ ker h(T ) (+)

55
Duc Vu (Spring 2021) 13 Lec 13: Apr 26, 2021

Proof. By the lemma we just proved, we need only show (+). Since g, h are relatively
prime, there exists equation
1 = gk + hl ∈ F [t]
Hence
1V = g(T )k(T ) = h(T )l(T )
as linear operators on V i.e. ∀v ∈ V

v = g(T )k(T )v + h(T )l(T )v (*)

Since f (T ) = 0 we have

0 = f (T )k(T )v = h(T )g(T )k(T )v

Therefore,
g(T )k(T )v ∈ ker h(T )
and
0 = f (T )l(T )v = g(T )h(T )l(T )v
so
h(T )l(T )v ∈ ker g(T )
It follows by (*), ∀v ∈ V

v = g(T )k(T )v + h(T )l(T )v ∈ ker h(T ) + ker g(T )

where
V = ker g(T ) + ker h(T )
By (*), if v ∈ ker g(T ) ∩ ker h(T ), then

v = g(T )k(T )v + h(T )l(T )v = 0

Hence
V = ker g(T ) ⊕ ker h(T )
as needed.

56
 Duc Vu (Spring 2021) 14 Lec 14: Apr 28, 2021

§14 Lec 14: Apr 28, 2021

§14.1 P r i m a r y D e c o m p o s i t i o n ( C o nt ’ d )

Proposition 14.1
Let V be a finite dimensional vector space over F , T : V → V linear, g, h ∈ F [t] \ F
monic and relatively prime. Suppose that

qT = gh ∈ F [t]

Then ker g(T ) and ker h(T ) are T -invariant.

V = ker g(T ) ⊕ ker h(T )

and
g = qT ker g(T )
and h = qT ker h(T )

Proof. By the last lemma in last lecture, we need only prove the last statement. By
definition, we have
g(T ) ker g(T ) = 0 and h(T ) ker h(T ) = 0
So by definition,
qT ker q(T )
|g and qT ker h(T )
|h ∈ F [t]
As g and h are relatively prime, by the FTA, so are
qT ker g(T )
and qT ker h(T )

Therefore, we have
 
f := lcm qT ,q
ker g(T ) T ker h(T )

= qT
ker q(T )qT
ker h(T )

Since
V = ker g(T ) ⊕ ker h(T )
f (T )v = 0 ∀v ∈ V
Hence
qT |f ∈ F [t]
By (+) and FTA
f |gh = qT
As both f and qT are monic,
f = qT
Applying FTA again, we conclude that
g = qT ker g(T )
and h = qT ker h(T )

We now generalize the proposition to an important result that decomposes a finite dimen-
sional vector space over F relative to a linear operator T : V → V .

57
Duc Vu (Spring 2021) 14 Lec 14: Apr 28, 2021

Theorem 14.2 (Primary Decomposition)

Let V be a finite dimensional vector space over F , T : V → V linear, and qT =
pe11 . . . perr , with p1 , . . . , pr distinct monic irreducible polys in F [t], e1 , . . . , er ∈ Z+ .
Then there exists a direct sum decomposition of V into subspaces W1 , . . . , Wr

V = W1 ⊕ . . . ⊕ Wr (*)

satisfying all of the following:

i) Each Wi is T -invariant, i = 1, . . . , r

ii) qT |Wi = pei i , i = 1, . . . , r

iii) qT = ri=1 pei i = ri=1 qT |W

Q Q
i

iv) If Bi is an ordered basis for Wi , i = 1, . . . , r, B = B1 ∪ . . . ∪ Br is an ordered

basis for V with  
[T |W1 ]B1 0
[T ]B = 
 .. 
. 
0 [T |Wr ]

Moreover, any direct sum decomposition (*) of V satisfying i), ii), iii) is uniquely
determined by T and the p1 , . . . , pr up to order. If in addition, this is the case, then

Wi = ker pei i (T ) i = 1, . . . , r

Proof. We induct on r.

• r = 1 is immediate

• r > 1 By TFA, pe11 and g = pe22 . . . perr are relatively prime, so by the Proposition

V = W1 ⊕ V 1

where

W1 = ker pe11 (T ) and W1 is T -invariant

V1 = ker g(T ) and V1 is T -invariant
qT |W1 = pe11 qT |V1 = pe22 . . . perr

Let
T1 = T |V1 : V1 → V1
By induction on r, we may assume all of the following:

V1 = W2 ⊕ . . . ⊕ Wr
Wi = ker pei i (T1 ) and is T1 -invariant
qT1 |W = pei i for i = 2, . . . , r
i

58
Duc Vu (Spring 2021) 14 Lec 14: Apr 28, 2021

Note:
r
X
ker pei i (T1 ) ∩ ker pj (T1 ) = 0 ∀i > 0
j=2
j6=i

Claim 14.1. Let 2 ≤ i ≤ r. Then

ker pei i (T ) = ker pei i (T1 )

Let v ∈ ker pei i (T ), i > 1. So

pei i (T )v = 0
Hence
r
e
Y
0= pj j (T )v = g(T )v,
j=2

i.e.,
v ∈ ker g(T ) = V1
So
T v = T |V1 v = T1 v
and
0 = pei i (T )v = pei i (T1 )v
as needed.
Let v ∈ ker pei i (T1 ), i > 1. By definition, v ∈ V1 , so

0 = pei i (T1 )v = pei i (T |V1 ) v

= pei i (T )|V1 v = pei i (T )v

This proves the claim.

The existence of (*), i), ii), iii) nad Wi = ker pei i (T ), i = 1, . . . , r, now follow. More-
over, i) and (*) yield iv).

Uniqueness: Suppose that

V = W1 ⊕ . . . ⊕ Wr
satisfies i), ii), iii). If we show

Wi = ker pei i (T ), i = 1, . . . , r

the result will follow. It suffices to do the case i = 1. Let

V1 = W2 ⊕ . . . ⊕ Wr
V = W1 ⊕ V 1

As each Wi is T -invariant and V1 is T -invariant. As before

pe11 and g = pe22 . . . perr

59
Duc Vu (Spring 2021) 14 Lec 14: Apr 28, 2021

and relatively prime by FTA. So by hw 4 # 6

 
qT = lcm qT |V1 , qT |V1

It follows that
qT |V1 = pe22 . . . perr = g
Moreover, we have an equation

1 = pe11 k + gl ∈ F [t]

So
1V = pe11 (T )k(T ) + g(T )l(T ) (+)

Claim 14.2. W1 = ker pe11 (T ) and hence we are done.

Since
qT |W1 = pe11
We have
pe11 (T )v = 0 ∀v ∈ W1
Hence
W1 ⊆ ker pe11 (T )
To finish, we must know
ker pe11 (T ) ⊆ W1
Let
v ∈ ker pe11 (T ) ⊆ V = W1 ⊕ V1
So ∃!w1 ∈ W1 , v1 ∈ V1 s.t.
v = w1 + v1
Since W1 ⊆ ker pe11 (T ),
pe11 (T )W1 = 0
By assumption, pe11 (T )v = 0, so
pe11 (T )v1 = 0
As V1 = W2 ⊕ . . . ⊕ Wr

pei i = qT |W , i = 2, . . . , r by (ii)
i

We have
pe22 (T ) . . . perr (T )v1 = 0
Hence by (+)

v1 = 1V v1 = pe11 (T )k(T )v1 + pe22 (T ) . . . perr (T )l(T )v1 = 0

Therefore,
v = w1 + v 1 = w1 ∈ W 1
and it follows that ker pe11 (T ) ⊆ W1 as needed.

60
Duc Vu (Spring 2021) 14 Lec 14: Apr 28, 2021

Recall: Let V be a finite dimensional vector space over F , T : V → V linear is called

diagonalizable if there exists an ordered basis B for V consisting of eigenvectors of T . By
hw 2 # 2, this is equivalent to M
V = ET (λ)
λ

61
 Duc Vu (Spring 2021) 15 Lec 15: Apr 30, 2021

§15 Lec 15: Apr 30, 2021

§15.1 P r i m a r y D e c o m p o s i t i o n ( C o nt ’ d )
Recall: Let V be a finite dimensional vector space over F , T : V → V linear is called
diagonalizable if there exists an ordered basis B for V consisting of eigenvectors of T . By
hw 2 # 2, this is equivalent to M
V = ET (λ)
λ

Theorem 15.1
Let V be a finite dimensional vector space over F , T : V → V linear. Then T is
diagonalizable iff qT splits in F [t] and has no repeated roots in F . If this is the case,
then
Yr
qT = (t − λi ), λ1 , . . . , λr the distinct roots of qT
i=1

Proof. “ ⇐= ” qT = ri=1 (t − λi ), λ1 , . . . , λr the distinct roots of qT . Let Vi =

Q
ker(T − λi 1V ) = ET (λi ), i = 1, . . . , r. Then by the Primary Decomposition Theorem,

V = V1 ⊕ . . . ⊕ Vr

SO T is diagonalizable.
“ =⇒ ” Let B = {v1 , . . . , vn } be an ordered basis for V consisting of eigenvectors of T
with λi the eigenvalue of vi and ordered s.t.

λ1 , . . . , λr are the distinct eigenvalues of T

For each j, 1 ≤ j ≤ n, we have

(T − λi 1V ) vj = T vj − λi vj = (λj − λi ) vj , j = 1, . . . , n

So
r
Y
(T − λi 1V ) vj = 0 for j = 1, . . . , n
i=1
i.e.,
r
Y
(T − λi 1V ) vanishes on a basis for V
i=1
hence vanishes on all of V . It follows that
r
Y
qT | (t − λi ) ∈ F [t]
i=1

In particular, qT splits in F [t] and has no multiple roots in F by FTA. As every eigenvalue
of T is a root of fT , we have

t − λi |qT , i = 1, . . . , r

62
Duc Vu (Spring 2021) 15 Lec 15: Apr 30, 2021

using fT and qT have the same roots. Therefore,

r
Y
qT = (t − λi ) ∈ F [t]
i=1

§15.2 J o r d a n B l o ck s

Definition 15.2 (Jordan Block Matrix) — J ∈ Mn F is called a Jordan block matrix

of eigenvalue λ of size n if
 
λ 0
1 λ 
 
J = Jn (λ) := 
 1  ∈ Mn F

 . . 
 . λ
0 1

Note: fJn (λ) = det (tI − Jn (λ)) = (t − λ)n ∈ F [t], so splits with just one root of multiplic-
ity.

Definition 15.3 (Nilpotent) — T : V → V linear is called nilpotent if qT = tm , some

m, i.e., ∃M ∈ Z+ 3 T M = 0.

Example 15.4
J = Jn (0) is nilpotent and has qJ = tm for some m. In fact, qJ = tn – why?
In fact, let A ∈ Mn F , A : F n×1 → F n×1 linear with A ∼ N with

N = Jn (λ− λIn = Jn (0)

Then as N is nilpotent and

A = P N P −1 , some P ∈ GLn F,

we have

n
An = P N P −1 = P N P −1 P N P −1 . . . P N P −1 = P N n P −1 = 0

So A is nilpotent. Now N is nilpotent.

If S = {e1 , . . . , en } is the standard basis for F n×1

N ei = ei+1 , i≤n−1
N en = 0
N 2 ei = N − N ei = ei+2 , i≤n−2
..
.

63
Duc Vu (Spring 2021) 15 Lec 15: Apr 30, 2021

Example 15.5 (Cont’d from above)

In any case, we have
)
dim im N r = n − r
if r ≤ n
dim ker N r = r
)
dim im N r = 0
if r > n
dim im ker N r = n

Lemma 15.6
Let J = Jn (λ) ∈ Mn F . Then

1. λ is the only eigenvalue of J.

2. dim EJ (λ) = 1

3. tJ = qJ = (t − λ)n

4. fJ (J) = 0

Proof. Let
N = J − λI ∈ Mn F
the characteristics matrix of J
 
0 ... 0
 .. .. 
N n−1 = . . ∈ Mn F

0 0
1 0 ... 0

is not the zero matrix, but

Nn = 0
So
qT |(t − λ)n and qJ 6 |(t − λ)n−1
It follows that qJ = (t − λ)n = fJ . This shows 3) and 4). By the computation,

dim ker N = 1

and
ker N = ET (λ)
This gives 2) as fT = (t − λ)n , 1) is clear.

Remark 15.7. Jn (λ) has only a line as an eigenspace, so among triangulariazable operator
away from being diagonalizable when n ≥ 1.

64
Duc Vu (Spring 2021) 15 Lec 15: Apr 30, 2021

Proposition 15.8
Let A ∈ Mn F be triangularizable. Suppose fA = (t − λ)n for some λ ∈ F . Then A is
diagonalizable iff qA = (t − λ) iff A = λI.

Proof. If qA = t − λ, then A = λI as

F n×1 = ker (A − λI)

The converse is immediate.

Computation: Let V be a finite dimensional vector space over F , dim V = n, T : V → V

linear. Suppose there exists B = {v1 , . . . , vn } an ordered basis for V satisfying

[T ]B = Jn (λ)

Then by definition

T v1 = λv1 + v2 i.e. (T − λ1V )v1 = v2

T v2 = λv2 + v3 i.e. (T − λ1V )v2 = v3
..
. (+)
T vn−1 = λvn−1 + vn i.e. (T − λ1V ) vn−1 = vn
T vn = λvn

So
Eλ (λ) = F vn
v1 , . . . , vn−1 are not eigenvectors, but do satisfy

(T − λ1V )vi = vi+1 i = 1, . . . , n − 1

n−i
(T − λ1V ) vi = vn , an eigenvector

So we can compute v1 , . . . , vn−1 from the eigenvalue vn .

65
 Duc Vu (Spring 2021) 16 Lec 16: May 3, 2021

§16 L e c 1 6 : M ay 3 , 2 0 2 1

§16.1 J o r d a n B l o ck s ( C o nt ’ d )

Definition 16.1 (Sequence of Generalized Eigenvectors) — Let T : V → V be linear,

0 6= vn ∈ ET (λ). We say v1 , . . . , vn is an (ordered) sequence of generalized eigenvectors
of eigenvalue λ of length n if (+) above holds, i.e.,

(T − λ1V )vi = vi+1 , i = 1, . . . , n − 1

(T − λ1V )vn = 0

We let

gn (λ) = gn (vn , λ) := {v1 , . . . , vn }

= v1 , (T − λ1V )n−1 v1


be an ordered sequence of generalized eigenvectors for T of length n relative to λ.

Note: We should really write

gn (vn , λ, v1 , . . . , vn−1 )

Lemma 16.2
Let V be a vector space over F , T : V → V linear, 0 6= vn ∈ ET (λ), v1 , . . . , vn an
ordered sequence of generalized eigenvectors of T of length n, gn (λ) = {v1 , . . . , vn }.
Then

1. gn (λ) is linearly independent.

2. If V is a finite dimensional vector space over F , dim V = n, then

i) gn (λ) is an ordered basis for V

ii) [T ]gn (λ) = Jn (λ)

Proof. 1. We have seen that (∗) implies

(T − λ1V )n−i vi = vn i<n

(T − λ1 − V )vn = 0

So
(T − λ1V )k vi = 0 ∀k > n − i
Suppose
α1 v1 + . . . + αn vn = 0, αi ∈ F not all 0
Choose the least k s.t. αk 6= 0. Then

0 = (T − λ1V )n−k (αk vk + . . . + αn vn ) = αk vn

66
Duc Vu (Spring 2021) 16 Lec 16: May 3, 2021

As vn 6= 0, αk = 0, a contradiction.
So 1) follows and 1) → 2).

Definition 16.3 (Jordan Canonical Form) — A ∈ Mn F is called a matrix in Jordan

canonical form (JCF) if A has the block form
 
Jr1 (λ1 ) 0
A=
 .. 
. 
0 Jrm (λm )

λ1 , . . . , λm not necessarily distinct.

Definition 16.4 (Jordan Basis) — Let V be a finite dimensional vector space over F ,
T : V → V linear. An ordered basis B for V is called a Jordan basis (if it exists) for
V relative to T if B is the union

gr1 (v1,r1 , λ1 ) ∪ . . . ∪ grm (vm,rm , λm ) (?)

where grj (vj,rj , λj ) is an ordered sequence of generalized eigenvectors of T relative to

λj ending at eigenvector vj,rj . The λ1 , . . . , λm need not be distinct.

Proposition 16.5
Let V be a finite dimensional vector space over F , T : V → V linear. Then V has a
Jordan basis relative to T ⇐⇒ T has a matrix representation in Jordan canonical
form (JCF).

Proof. Let wi = gri (vi,ri , λi ) in (?). The only thing to show is: Wi is T -invariant, but this
follows from our computation.

Conclusion: Let T : V → V be linear with V having a Jordan basis relative to T . Gathering

all the Jordan blocks with the same eigenvalues together and ordering these into increasing
size, we can write such a Jordan basis as follows:

λ1 , . . . , λm the distinct eigenvalues of T

B = gr11 (v11 , λ1 ) ∪ . . . ∪ gr1 ,n1 (v1,n1 , λ1 )

∪ gr21 (v21 , λ2 ) ∪ . . . ∪ gr2,n2 (v2,n2 , λ2 )
..
.
∪ grm ,1 (vm,1 , λm ) ∪ . . . ∪ grm ,nm (vm,rm , λm )

with
ri1 ≤ ri2 ≤ . . . ≤ ri ni , 1≤i≤m

67
Duc Vu (Spring 2021) 16 Lec 16: May 3, 2021

e.g.  
1 0
0 1   
  J1 (1)
 1 0 
   J1 (1) 
[T ]B = 1 1 = 
   J2 (1) 
 0 2 0 0
  J3 (2)
 1 2 0
0 1 2
Let
Wij = Span gri ,j (vij , λi ) ∀i, j
These are all T -invariant. We have
Y
fT = (t − λi )rij
i,j

and
Y
qT = lcm ((t − λi )rij |j = 1, . . . , ni )
i
Y
= (t − λi )rini
i

So
qT |fT and fT (T ) = 0
Also
qT |Wij = fT |Wij = (t − λi )rij
for all 1 ≤ j ≤ nj , 1 ≤ i ≤ m. There are called the elementary divisors of T

V = W11 ⊕ . . . ⊕ W1,n1 ⊕ . . . ⊕ Wm1 ⊕ . . . ⊕ Wmnm

Now let Pij be the projection onto Wij along

W11 ⊕ . . . ⊕ W
d ij ⊕ . . . ⊕ Wm,nm
|{z}
omit

Then
(
Pjl if i = k and j = l
Pij Pkl = δik δjl Pjl =
0 otherwise
1V = P11 + . . . + Pmnm
T Pij = Pij T
T = T P11 + . . . + T Pmnm = T W11
+ ... + T Wmnm

Abusing notation
λ1 , . . . , λm are the distinct eigenvalues of T
Let
Wi = Wi1 ⊕ . . . ⊕ Wini i = 1, . . . , m

68
Duc Vu (Spring 2021) 16 Lec 16: May 3, 2021

As ri1 ≤ . . . ≤ rini ,

(T − λi 1V )rini Wij
= 0, 1 ≤ j ≤ ni
(T − λi 1V )ri ni −1 Wij
6= 0

showing
qT |Wi = (t − λi )rini
So
V = W1 ⊕ . . . ⊕ Wm
is the unique primary decomposition of V relative to T .
Note: The Jordan canonical form of T above is completely determined by the elementary
divisors of T .

§16.2 J o r d a n C a n o n i c a l Fo r m

Theorem 16.6
Let V be a finite dimensional vector space over F , T : V → V linear. Suppose that
qT splits in F [t]. Then there exists a Jordan basis B for V relative to T . Moreover,
[T ]B is unique up to the order of the Jordan blocks. In addition, all such matrix
representations are similar.

Proof. Reduction 1: We may assume that

qT = (t − λ)r

Suppose that
qT = (t − λ1 )r1 . . . (t − λm )rm ∈ F [t]
λ1 , . . . , λm distinct. Set

Wi = ker (T − λi 1V )ri , i = 1, . . . , m

By the Primary Decomposition Theorem,

V = W1 ⊕ . . . ⊕ Wm

Wi is T -invariant, i = 1, . . . , n

qT |W = (t − λi )ri , i = 1, . . . , m
i

So we need only find a Jordan basis for each Wi .

69
 Duc Vu (Spring 2021) 17 Lec 17: May 5, 2021

§17 L e c 1 7 : M ay 5 , 2 0 2 1

§17.1 J o r d a n C a n o n i c a l Fo r m ( C o nt ’ d )
Proof. (Cont’d from Lec 16) Reduction 2: We may assume that qT = tr , i.e., λ = 0.
Suppose that we have proven the case for λ = 0. Let S = T − λ1V , T as in Reduction 1.
Then
S r = (T − λ1V )r = 0 and S r−1 = (T − λ1V )r−1 6= 0
Therefore,
qS = t r
if B is a Jordan basis for V relative to S, then

[S]B = [T ]B − λI

is a JCF with diagonal entries 0. Hence

[T ]B = [S]B + λI

is a JCF with diagonal entries λ and B is also a Jordan basis for V relative to T . Reduction 2
now follows easily. We turn to
Existence: We have reduced to the case

qT = t r , i.e., T r = 0, T r−1 6= 0

In particular, T is nilpotent. We induct on dim V .

• dim V = 1 is immediate.

• dim V > 1: T is singular, so 0 < ker T , as λ = 0 is an eigenvalue. Since V is a finite

dimensional vector space over F , by the Dimension Theorem, T is not onto, i.e.,

im T < V

As im T is T -invariant, we can (and do) view

T im T
: im T → im T linear

As T r = 0, certainly (T |im T )r = 0, so

T im T
is also nilpotent

and
qT im T
qT ∈ F [t]
since

qT T im T
= 0 = qT (T )
So qT |im T splits in F [t] and

q = ts , for some s ≤ r
T
im T

70
Duc Vu (Spring 2021) 17 Lec 17: May 5, 2021

by FTA. By induction on dim V , im T has a Jordan basis relative to T |im T . So

im T = W1 ⊕ . . . ⊕ Wm , some m
with each Wi being T |im T − (hence T −) invariant and Wi has a basis of an ordered
sequence of generalized eigenvectors for T |Wi , hence for T |im T and T ,
gri (0) = wi , T wi , . . . , T ri −1 wi , ri ≥ 1


Thus we have
T ri wi = 0, i = 1, . . . , m
ri
q =t , i = 1, . . . , m
T
Wi

Since wi ∈ Wi ⊆ im T ,
∃vi ∈ V 3 T vi = wi , i = 1, . . . , m
So we also have
T ri +1 vi = T ri T vi = T ri wi = 0
and
T ri vi = T ri −1 T vi = T ri −1 wi 6= 0
Therefore, vi , T vi , . . . , T ri vi is an ordered sequence of generalized eigenvalues for T
in V , and, in particular, linearly independent. For each i = 1, . . . , m, let
Vi = Span {vi , T vi , . . . , T ri vi }
So
 
Xri 
Vi = αj T j vi |αj ∈ F
 
j=0

= {f (T )vi |f ∈ F [t], f = 0 or deg f ≤ ri }

= F [T ]Vi
Since each Vi is spanned by an ordered sequence of generalized eigenvectors for T ,
each Vi is T -invariant, i = 1, . . . , m.
Note: If f ∈ F [t] and f (T )wi = 0, then f (T ) = 0 in Wi and similarly if f ∈ F [t] and
f (T )vi = 0, then f (T ) = 0 on Vi as f (T )wi = 0 implies
0 = T j f (T )wi = f (T )T j wi = 0 ∀i
Set
V 0 = V1 + . . . + Vm
Each Vi is T -invariant, so V 0 is T -invariant.
Claim 17.1. V 0 = V1 ⊕ . . . ⊕ Vm

In particular,
B0 = {v1 , T v1 , . . . , T ri v1 , . . . , vm , T vm , . . . , T rm vm }
is a basis for V 0 .

71
 Duc Vu (Spring 2021) 18 Lec 18: May 7, 2021

§18 L e c 1 8 : M ay 7 , 2 0 2 1

§18.1 J o r d a n C a n o n i c a l Fo r m ( C o nt ’ d )
Proof. (Cont’d) Suppose ui ∈ Vi , i = 1, . . . , m satisfies

u1 + . . . + um = 0 (1)

To show ui = 0, i = 1, . . . , m. As ui ∈ Vi , ∃fi ∈ F [t] 3

ui = fi (T )vi

where we let fi = 0 if ui = 0. So (1) becomes

fi (T )v1 + . . . + fm (T )vm = 0 (2)

Since T f (T ) = f (T )T ∀f ∈ F [t] and

wi = T vi i = 1, . . . , m

taking T of (2) yields

f1 (T )w1 + . . . + fm (T )wm = 0
As the T -invariant Wi satisfying

W1 + . . . + Wm = W1 ⊕ . . . ⊕ Wm (*)

We have
fi (T )wi = 0, i = 1, . . . , m
Hence
fi (T ) = 0 on Wi , i = 1, . . . , m
Thus
tri = q fi ∈ F [t], i = 1, . . . , m
T
Wi

In particular, since ri ≥ 1 ∀i, we can write

fi = tgi ∈ F [t], i = 1, . . . , m
deg gi < deg fi , i = 1, . . . , m if fi 6= 0

Since
fi (T ) = T gi (T ) = gi (T )T
and
wi = T vi , i = 1, . . . , m
(2) now becomes
g1 (T )w1 + . . . + gm (T )wm = 0 (3)
Since each Wi is T -invariant, by (*)

gi (T )wi = 0, hence gi (T ) = 0 on Wi

72
Duc Vu (Spring 2021) 18 Lec 18: May 7, 2021

for i = 1, . . . , m by the definition of Wi . Therefore, for each i, i = 1, . . . , m

tri = q gi ∈ F [t]
T
Wi

In particular, we can write

gi = tri hi ∈ F [t], i = 1, . . . , m

So
fi = tri +1 hi ∈ F [t], i = 1, . . . , m
Thus we have
ui = fi (T )vi = hi (T )T ri +1 vi = 0, i = 1, . . . , m
This establishes claim 1. As

wi = T vi ∈ Wi , i = 1, . . . , m

We have

T V 0 = T V1 ⊕ . . . ⊕ T Vm
= W1 ⊕ . . . ⊕ Wm = T V (?)

since each Wi , Vi is T -invariant and

T Vi = Wi , i = 1, . . . , m

Therefore,
T V0
=T V1
+ ... + T Vm

Claim 18.1. V = ker T + V 0

Let v ∈ V . Since
TV 0 = TV
by (?), we have ∀v ∈ V
∃v 0 ∈ V 0 3 T v 0 = T v,
so
v − v 0 ∈ ker T
and
v = v 0 + w some w ∈ ker T
i.e.
v ∈ V 0 + ker T
as needed.
Now by construction, we have a Jordan basis B0 for the T -invariant subspace V 0 relative
to T |V 0 . Let
C = {u1 , . . . , uk } be a basis for ker T = ET (0)
Modifying the Toss In Theorem, we get a basis for V as follows. If u1 ∈ / Span B0 , let
B1 = B0 ∪{u1 }. Otherwise, let B1 = B0 . If u2 ∈
/ Span B1 , let B2 = B1 ∪{u2 }. Otherwise,

73
Duc Vu (Spring 2021) 18 Lec 18: May 7, 2021

let B2 = B1 . In either case, B2 is a linearly independent set. Continuing in this way, since
B0 ∪ C spans V , we get a spanning set of V

B = B0 ∪ {uj1 , . . . , ujr } ⊆ V

with
Tuji = 0
for some uji constructed above, 1 ≤ i ≤ s.
Using claim 1, we have

V = V 0 ⊕ Span {uj1 , . . . , ujs }

= V1 ⊕ . . . ⊕ Vm ⊕ F uj1 ⊕ . . . ⊕ F ujs

and [T ]B is in Jordan canonical form. This proves existence.

Note: F uji are the g1 (uji , 0) and the uji are eigenvectors that cannot be extended to
gi (vi , 0) of longer length.
Uniqueness: By reduction 1) and 2), we have

qT = tr , T r = 0, T r−1 6= 0

Let C be an ordered basis for V . Then by MTT

mj = dim im T j = rank T j C = rank [T ]jC

 
(*)

Let B be any Jordan basis for V relative to T , say

 
Jr1 (0) 0
[T ]B = 
 .. 
. 
0 Jrm (0)

the corresponding Jordan canonical form. Prior computation showed for each i, 1 ≤ i ≤ m,
(
rank Jrji (0) = ri − j
if j < ri
dim ker Jrji (0) = j

and (
rank Jrji (0) = 0
if j ≥ ri
dim ker Jrji (0) = ri
Clearly, for each i,  j 
Jr1 (0)
[T ]jB =
 .. 
. 
Jrjm (0)
as [T ]B is in block form. So by (*),
m
mj = rank [T ]jB =
X
rank Jrji (0)
i=1

74
Duc Vu (Spring 2021) 18 Lec 18: May 7, 2021

It follows that we have

mj−1 − mj = rank [T ]j−1 j
B − rank [T ]B
= # of l × l Jordan blocks Jl (0) in (+) with l ≥ j
We also have, in the same way,
mj − mj+1 = rank [T ]jB − rank [T ]j+1
B
= # of l × l Jordan blocks Jl (0) in (+) with l ≥ j + 1
Consequently, there are precisely
(mj−1 − mj ) − (mj − mj+1 ) = mj−1 − 2mj + mj+1
which equals the number of l × l Jordan blocks Jl (0) in (+) with l = j. This number is
independent of B as it is
rank T j−1 − 2 rank T j + rank T j+1
Thus, [T ]B is unique up to order of the Jordan blocks. This proves uniqueness.
If B 0 is another Jordan basis, then
[T ]B0 ∼ [T ]B
by the Change of Basis Theorem. This finishes the proof (phewww. . . such a long
proof !)

Corollary 18.1
Let A ∈ Mn F . If qA ∈ F [t] splits in F [t], then A is similar to a matrix in JCF unique
up to the order of the Jordan blocks.

Corollary 18.2
Let F be an algebraically closed field, e.g., F = C. Then every A ∈ Mn F is similar
to a matrix in JCF unique up to the order of the Jordan blocks and for every V , a
finite dimensional vector space over F , and T : V → V linear, V has a Jordan basis
relative to T . Moreover, the Jordan blocks of [T ]B are completely determined by the
elementary divisors (minimal polys) that correspond to the Jordan blocks.

Theorem 18.3
Let F be an algebraically closed field, e.g., F = C, A, B ∈ Mn F . Then, the following
are equivalent

1. A ∼ B

2. A and B have the same JCF (up to block order)

3. A and B have the same elementary divisors counted with multiplicities.

75
Duc Vu (Spring 2021) 18 Lec 18: May 7, 2021

Corollary 18.4
Let F be an algebraically closed field. Then A ∼ A> .

Proof. For any B ∈ Mn F , qB = qB > .

§18.2 Companion Matrix

Definition 18.5 (Companion Matrix) — Let g = tn + an−1 tn−1 + . . . + a1 t + a0 ∈ F [t],

n ≥ 1. The matrix  
0 0 ... 0 a0
1 0 0 a1 
 
 .. .. 
0 1 . . 
C(g) := 
 .. .. .. ..


. . . .
 
 0 an−2 
0 0 ... 1 an−1
is called the companion matrix of g.

Example 18.6
C (tn ) = Jn (0).

Note: If f, g ∈ F [t] are monic, then

f = g ⇐⇒ C(f ) = C(g)

Lemma 18.7
Let g ∈ F [t] \ F be moinc. Then
fC(g) = g

Proof. Let g = tn + an−1 tn−1 + . . . + a0 ∈ F [t] \ F . We induct on n, using properties about

determinants.

• n = 1 is immediate

• n > 1 Expanding on the determinant

 
t 0 ... 0 a0

−1 t .. 
 . 

fC(g) = det (tI − C(g)) = det  0 −1
 ..
.
 .. ..
 

 . 0 .

0 . . . . . . −1 t + an−1

76
Duc Vu (Spring 2021) 18 Lec 18: May 7, 2021

along the top row and induction yields

t tn−1 + an−1 tn−2 + . . . + a1 + (−1)n−1 a0 (−1)n−1 = g

Lemma 18.8
Let g ∈ F [t] \ F be monic. Then

qC(g) = fC(g) = g

In particular,
fC(g) (C(g)) = 0

77
 Duc Vu (Spring 2021) 19 Lec 19: May 10, 2021

§19 L e c 1 9 : M ay 1 0 , 2 0 2 1

§19.1 C o m p a n i o n M a t r i x ( C o nt ’ d )
Remark 19.1. If C is a companion matrix in Mn F , viewing

C : F n×1 → F n×1 linear,

then
B = e1 , Ce1 , . . . , C n−1 e1


is a basis for F n×1 and

(n−1 )
X
n×1 i
F = αi C ei |αi ∈ F
i=0
= F [C]e1 := {f (C)e1 |f ∈ F [t]}

Definition 19.2 (T-Cyclic) — Let V be a vector space over F , T : V → V linear. We

say v ∈ V is a T -cyclic vector for V and V is T -cyclic if

V = Span {v, T v, . . . , T n v, . . .} = F [T ]v

Warning: Let T : V → V be linear. It is rare that V is T -cyclic. However, if v ∈ V , then

F [t]v ⊆ V is a T -invariant subspace and F [T ]v is T -cyclic. So T -cyclic subspace generalize
the notion of a line in V .

Proposition 19.3
Let V be a finite dimensional vector space over F , n = dim V , T : V → V linear.
Suppose there exists a T -cyclic vector v for V , i.e., V = F [T ]v. Then all of the
following are true

i) B = v, T v, . . . , T n−1 v is an ordered basis for V



ii) [T ]B = C(fT )

iii) fT = qT

Proof. i) As dim V = n, the set {v, T v, . . . , T n v} must be linearly independent. Let

j ≤ n be the first positive integer s.t.

T j v ∈ Span v, T v, . . . , T j−1 v


say
T j v = αj−1 T j−1 v + αj−2 T j−2 v + . . . + α1 T v + α0 v (*)
for α0 , . . . , αj−1 ∈ F . Take T of (*), to get

T j+1 v = αj−1 T j v + αj−2 T j−1 v + . . . + α1 T 2 v + α0 T v

78
Duc Vu (Spring 2021) 19 Lec 19: May 10, 2021

which lies in Span(v, T v, . . . , T j−1 v) by (*). Iterating this process shows

T N v ∈ Span v, T v, . . . , T j−1 v

∀N ≥ j

It follows that
v = F [T ]v = Span v, T v, . . . , T j−1 v


So
n = dim V ≤ j, hence n = j
This proves i).
ii) The computation proving i) shows

B = v, T v, . . . , T n−1 v


is an ordered basis for V . As

[T ]B = [T v]B T 2 v B . . . T n−2 v B [T n v]B

   

 
0 0 0 ∗
 .. 
1 0 . ∗
=

.. .. 

0 1 . .
0 0 ... 1 ∗
it is a companion matrix, hence must be C(fT ) and by the lemma, we have proven
ii).
iii) fT = f[T ]B = q[T ]B = qT as [T ]B = C(fT ).

Example 19.4
Let V be a finite dimensional vector space over F , dim V = n, T : V → V linear s.t.
there exists an ordered basis B with

[T ]B = Jn (λ)

Set S = T − λ1V : V → V linear. Then ∃v ∈ V 3

B = v, Sv, . . . , S n−1 v


So v s an S-cyclic vector and

V = F [S]v

Fact 19.1. If A ∈ Mr F [t], C ∈ Ms F [t], B ∈ F [t]r×s , then

 
A B
det = det A det C
O C

where X
det D = sgn σD1σ(1) . . . Dnσ(n)

79
Duc Vu (Spring 2021) 19 Lec 19: May 10, 2021

§19.2 S m i t h N o r m a l Fo r m
We say that A ∈ F [t]m×n is in Smith Normal Form (SNF) if A is the zero matrix or if A is
the matrix of the form  
q1 0 . . .
 0 q2 
 
 .. . . 
.
 . 


 q r



 0 

 .. 
 . 
0
with q1 |q2 |q3 | . . . |qr in F [t] and all monic, i.e., there exists a positive integer r satisfying
r ≤ min(m, n) and q1 |q2 |q3 | . . . |qr monic in F [t] s.t. Aii = qi for 1 ≤ i ≤ r and Aij = 0
otherwise.
We generalize Gaussian elimination, i.e., row (and column) reduction for matrices with
entries in F to matrices with entries in F [t]. The only difference arises because most
elements of F [t] do not have multiplicative inverses.
Let A ∈ Mn (F [t]). We say that A is an elementary matrix of
i) Type I: if there exists λ ∈ F [t] and l 6= k s.t.

1 if i = j

Aij = λ if (i, j) = (k, l)

0 otherwise


ii) Type II: If there exists k 6= l s.t.


1 if i = j 6= l or i = j 6= k



0 if i = j = l or i = j = k
Aij =


 1 if (k, l) = (i, j) or (k, l) = (j, i)

0 otherwise

iii) Type III: If there exists a 0 6= u ∈ F and l s.t.


1 if i = j 6= l

Aij = u if i = j = l

0 otherwise


Remark 19.5. Let A ∈ F [t]m×n . Multiplying A on the left (respectively right) by a suitable
size elementary matrix of
a) Type I is equivalent to adding a multiple of a row (respectively column) of A to another
row (respectively column) of A.
b) Type II is equivalent to interchanging two rows (respectively columns) of A.
c) Type III is equivalent to multiplying a row (respectively column) of A by an element in
F [t] having a multiplicative inverse.

80
Duc Vu (Spring 2021) 19 Lec 19: May 10, 2021

Remark 19.6. 1. All elementary matrices are invertible.

2. The definition of elementary matrices of Types I and II is exactly the same as that given
when defined over a field.
3. The elementary matrices of Type III have a restriction. The u’s appearing in the
definition are precisely the elements in F [t] having a multiplicative inverse. The reason
for this is so that the elementary matrices of Type III are invertible.

Let
GLn (F [t]) := {A|A is invertible}
Warning: A matrix in Mn (F [t]) having det(A) 6= 0 may no longer be invertible, i.e.,
have an inverse. What is true is that GLn (F [t]) = {A|0 6= det(A) ∈ F }, equivalently
GLn (F [t]) consists of those matrices whose determinant have a multiplicative inverse in
F [t].

Definition 19.7 (Equivalent Matrix) — Let A, B ∈ F [t]m×n . We say that A is

equivalent to B and write A ≈ B if there exist matrices P ∈ GLm (F [t]) and Q ∈
GLn (F [t]) s.t. B = P AQ.

Theorem 19.8
Let A ∈ F [t]m×n . Then A is equivalent to a matrix in Smith Normal Form. Moreover,
there exist matrices P ∈ GLm (F [t]) and Q ∈ GLn (F [t]), each a product of matrices
of Type I, Type II, Type III, s.t. P AQ is in SNF.

Remark 19.9. The SNF derived by this algorithm is, in fact, unique. In particular, the monic
polynomials q1 |q2 |q3 | . . . |qr arising in the SNF of a matrix A are unique and are called the
invariant factor of A. This is proven using results about determinant.

81
 Duc Vu (Spring 2021) 20 Lec 20: May 12, 2021

§20 L e c 2 0 : M ay 1 2 , 2 0 2 1

§20.1 R a t i o n a l C a n o n i c a l Fo r m
If A, B ∈ F [t]m×n then A ≈ B if and only if they have the same SNF if and only if they
have the same invariant factors. So what good is the NSF relative to linear operators on
finite dimensional vector spaces?
Let A, B ∈ Mn (F ). Then A ∼ B if and only if tI − A ≈ tI − B in Mn (F [t]) and this is
completely determined by the SNF hence the invariant factors of tI − A and tI − B. Now
the SNF of tI − A may have some of its invariant factors 1, and we shall drop these.
Let V be a finite dimensional vector space over F with B an ordered basis. Let T : V → V
be a linear operator. If one computes the SNF of tI − [T ]B , it will have the form
 
1 0 ... ... 0
0 1 0
 
 .. . . .. 
.
 . .

 q 1



 q2 

 .. .. .
. . .. 
0 ... ... qr

with q1 |q1 | . . . |qr are all the monic polynomials in F [t] \ F . These are called the invariant
factors of T . They are uniquely determined by T . The main theorem is that there exists
an ordered basis B for V s.t.
 
C(q1 ) 0 ... 0
 0 C(q2 ) . . . 0 
[T ]B =  .
 
. . . .. 
 . . . 
0 . . . C(qr )
and this matrix representation is unique. This is called the rational canonical form or RCF
of T . Moreover, the minimal polynomial qt of T is qr . The algorithm computes this as
well as all invariant factors of T . The characteristic polynomial fT of T is the product
of q1 . . . qr . This works over any field F , even if qT does not split. The basis B gives a
 subspaces V = W1 ⊕ . . . ⊕ Wr where fT |Wi = qT |Wi = qi
decomposition of V into T -invariant
and if dim(Wi ) = ni then Bi = vi , T vi , . . . , T ni −1 vi is a basis for Wi .
Let V be a finite dimensional vector space over F with B an ordered basis. Let T : V → V
be a linear operator. Suppose that qT splits over F . Then we know that there exists a
Jordan canonical form of T .
Question 20.1. How do we compute it?
We use the Smith Normal Form of tI −[T ]B to compute the invariant factors q1 |q1 | . . . |qr
of T just as one does to compute the RCF of T . We then factor each qi . Suppose this
factorization is
qi = (t − λ1 )r1 . . . (t − λm )rm
in F [t] with λ1 , . . . , λm distinct. Note that qi+1 has this as a factor so it has the form

qi+1 = (t − λ1 )s1 . . . (t − λm )sm . . . (t − λm+k )sm+k

82
Duc Vu (Spring 2021) 20 Lec 20: May 12, 2021

with si ≥ ri for each 1 ≤ i ≤ m and m + 1, . . . , m + k ≥ 0 with λ1 , . . . , λm+k distinct. Then

the totality of all the (t − λi )rj , including repetition if they occur in different qi ’s give all
the elementary divisors of T . So to get the JCF of T we take for each qi as factored above
the block matrix  
Jr1 (λ1 ) 0 . . . 0
 .. .. .. 
 . . . 
0 . . . Jrm (λm )
and replace C(qi ) by it in the RCF, i.e., we take all the Jordan blocks Jr (λ) associated to
each and every factor of the form (t − λ)r in each and every invariant factor qi determined
by the SNF and form a matrix out of all such blocks. This is the JCF which is unique only
up to block order.
Let V be a finite dimensional vector space over F , T : V → V linear, v ∈ V . Then as
before, if v ∈ V
F [t]v = {f (T )v|f ∈ F [t]} ⊆ V
the T -cyclic subspace of V generated by v and satisfies

nv := dim F [T ]v ≤ dim V

and has ordered basis

Bv := v, T v, . . . , T nv −1 v


As F [T ]v is T -invariant,  
 
T |F [T ]v Bv = C fT |F [T ]v

and
qT |F [T ]v = fT |F [T ]v
We want to show that V can be decomposed as a direct sum of T -cyclic subspaces of V .
The SNF of the characteristic matrix

tI − [T ]C

C is an ordered basis for V , which gives rise to invariants of T

q1 | . . . |qr ∈ F [t] (*)

q1 6= 1, qi monic for all i.

Note: The SNF of (+) has no 0’s on the diagonal asfT 6= 0. We want to show there exists
an ordered basis B for V with all the following properties

i) V = W1 ⊕ . . . ⊕ Wr , ni = dim Wi , i = 1, . . . , r

ii) Wi is T -invariant, i = 1, . . . , r

iii) Wi = F [T ]vi are T -cyclic, Wi = ker qT |W (T |Wi )

i

iv) qi = qT |W = fT |W , i = 1, . . . , r with qi as in (*)

i i

v) qT = qr

vi) fT = q1 . . . qr = qT |W1 . . . qT |Wr

83
Duc Vu (Spring 2021) 20 Lec 20: May 12, 2021

vii) Bvi = vi , T vi , . . . , T ni −1 vi is an ordered basis for Wi , i = 1, . . . , r



viii) B = B1 ∪ . . . ∪ Br is an ordered basis for V satisfying

 
C(q1 ) 0
[T ]B = 
 .. 
. 
0 C(qr )
called the rational canonical form of T and it is unique.
The uniqueness follows from the uniqueness of SNF. From the definition of equivalent
matrix, we have the following remark

Remark 20.1. If A ∈ Mn F [t] is in SNF, then

A ∈ GLn F [t] ⇐⇒ A = I

since  
q1 0
 .. 

 . 


 qr 


 0 

..
0 .
means 0 . . . 0 · q1 . . . qr ∈ F \ {0} if there are any 0’s on the diagonal, which is inseparable.

Lemma 20.2
Let g ∈ F [t] \ F be monic of degree n. Then
 
1 0
 .. 
It − C(q) ≈ 
 . 

 1 
0 q

Corollary 20.3
Let V be a finite dimensional vector space over F , T : V → V linear q1 | . . . |qr the
invariants of T in F [t]. Then
 
C(q1 ) 0
tI − 
 .. 
. 
0 C(qr )

where dim V = ri=1 deg qi

P

Certainly, if there exists an ordered basis B for V a finite dimensional vector space over F ,
T : V → V linear s.t. [T ]B is in RCF, then everything in goal falls out. So by the above,
the goal will follow if we prove the following

84
Duc Vu (Spring 2021) 20 Lec 20: May 12, 2021

Theorem 20.4
Let A0 , B0 ∈ Mn F , A = tI − A0 and B = tI − B0 in Mn F [t], the corresponding
characteristic matrices. Then the following are equivalent

i) A0 ∼ B0 (i.e. A0 and B0 are similar)

ii) A ∼ B (i.e., A and B are equivalent)

iii) A and B have the same SNF.

We need two preliminary lemmas.

Lemma 20.5
Let A ≈ B in Mn F [t]. Then ∃P, Q ∈ GLm F [t] each products of elementary matrices
s.t. A = P BQ.

Proof. P ∈ GLn F [t] iff its SNF = I which we get using elementary matrices.

For the second lemma, we need the “division algorithm” by “linear polys” in Mn F [t]. If we
were in F [t], we know if f, g ∈ F [t], f 6= 0,

g = f q + r ∈ F [t] with r = 0 or deg r < deg f

So if f = t − a, r ∈ F , i.e., r = g(a) by plugging in a into (*). But for matrices,

AQ + R 6= QA + R

but the same argument to get (*) for polys, will give a right and left remainder.
Notation: Let Ai ∈ Mr F , i = 0, . . . , n and let

An tn + An−1 tn−1 + . . . + A0

denote
An (tn I) + . . . + A0 I ∈ Mn F [t]
So if
A = (αij )
then
Atn = (αij tn )
i.e., two matrix polynomials are the same iff all their corresponding entries are equal, i.e.,

(Mn F )[t] = Mr (F [t])

85
Duc Vu (Spring 2021) 20 Lec 20: May 12, 2021

Lemma 20.6
Let A0 ∈ Mn F , A = tI − A0 ∈ Mn F [t] and

0 6= P = P (t) ∈ Mn F [t]

Then there exist matrices M, N ∈ Mn F [t] and R, S ∈ Mn F satisfying

i) P = AM + R

ii) P = N A + S

86
 Duc Vu (Spring 2021) 21 Lec 21: May 14, 2021

§21 L e c 2 1 : M ay 1 4 , 2 0 2 1

§21.1 R a t i o n a l C a n o n i c a l Fo r m ( C o nt ’ d )
Recall from last lecture,

Lemma 21.1
Let A0 ∈ Mn F , A = tI − A0 ∈ Mn F [t] and

0 6= P = P (t) ∈ Mn F [t]

Then there exist matrices M, N ∈ Mn F [t] and R, S ∈ Mn F satisfying

i) P = AM + R

ii) P = N A + S

Proof. i) Let
m = max deg Plk , Plk 6= 0
l,k

and ∀i, j let (

lead Pij if deg Pij = m
αij =
0 if Pij = 0 or deg Pij < m
So
Pij = αij tm + lower terms in t ∈ F [t]
Let αij ∈ Mn F and let

Pm−1 = (αij )tm−1 = αij tm−1

Every entry in

APm−1 = (tI − A0 ) (αij )tm−1

= (αij )tm − A0 (αij )tm−1

has deg = m or is zero and the tm -coefficient of (APm−1 )ij is αij . Thus, P − APm−1
has polynomial entries of degree at most m − 1 (or = 0). Apply the same argument
to P − APm−1 (replacing m by m − 1 in (*)) to produce a matrix Pm−2 in Mn F [t]
s.t. all the polynomial entries in (P − APm−1 ) − APm−2 have degree at most m − 2
(or = 0). Continuing this way, we construct matrices Pm−3 , . . . , P0 satisfying if

M := Pm−1 + Pm−2 + . . . + P0

then
R := P − AM
has only constant entries, i.e., R ∈ Mn F . So

P = AM + R

as needed.

87
Duc Vu (Spring 2021) 21 Lec 21: May 14, 2021

ii) This can be proven in an analogous way.

Theorem 21.2
Let A0 , B0 ∈ Mn F , A = tI − A0 , B = tI − B0 in Mn F [t]. Then

A ≈ B ∈ Mn F [t] ⇐⇒ A0 ∼ B0 ∈ Mn F

Proof. “ ⇐= ” If
B0 = P A0 P −1 , P ∈ GLn F,
then
P (tI − A0 ) P −1 = P tP −1 − P A0 P −1 = tI − B0 = B
So B = P AP −1 and B ≈ A.
“ =⇒ ” Suppose there exist P1 , Q1 ∈ GLn F [t], hence each a product of elementary matrices
by Lemma 20.5, satisfying

B = tB − B0 = P1 AQ1 = P1 (tI − A0 ) Q1

Applying Lemma 21.1, we can write

i) P1 = BP2 + R, P2 ∈ Mn F [t], R ∈ Mn F

ii) Q1 = Q2 B + S, Q2 ∈ Mn F [t], S ∈ Mn F

Since B = P1 AQ1 , P1 , Q1 ∈ GLn F [t], we also have

iii) P1 A = BQ−1

iv) AQ1 = P1−1 B

Thus, we have
i)
B = P1 AQ1 = (BP2 + R)AQ1 = BP2 AQ1 + RAQ1
iv) ii)
= BP2 P1−1 B + RAQ1 = BP2 P1−1 B + RA(Q2 B + S)
= BP2 P1−1 B + RAQ2 B + RAS

i.e., we have

v) B = BP2 P1−1 B + RAQ2 B + RAS

By i)
R = P1 − BP2
Plugging this into RAQ2 B, yields
i)
RAQ2 B = (P1 − BP2 )AQ2 B = P1 AQ2 B − BP2 AQ2 B
iii)
= BQ−1
 −1 
1 Q2 B − BP2 AQ2 B = B Q1 Q2 − P2 AQ2 B

i.e.

88
Duc Vu (Spring 2021) 21 Lec 21: May 14, 2021

vi) RAQ2 B = B Q−1

 
1 Q2 − P2 AQ2 B

Plug vi) into v) to get

v)
B = BP2 P1−1 B + RAQ2 B + RAS
vi)
= BP2 P1−1 B + B Q−1
 
1 Q2 − P2 AQ2 B + RAS
= B P2 P1−1 + Q−1
 
1 Q2 − P2 AQ2 B + RAS

Let
T = P2 P1−1 + Q−1
1 Q2 − P2 AQ2

Then

vii) B = BT B + RAS ∈ Mn F [t]

We next look at the degree of the poly entries of these matrices.

viii) Every entry of B = tI − B0 is zero or has deg ≤ 1 and every entry of RAS =
R(tI − A0 )S has is zero or has deg ≤ 1.

Question 21.1. What about BT B?

Let T = Tm tm + Tm−1 tm−1 + . . . + T0 with T0 , . . . , Tm ∈ Mn F . Then

BT B = (tI − B0 ) Tm tm + Tm−1 tm−1 + . . . + T0 (tI − B0 )

= Tm tm+2 + lower terms in t

Comparing coefficients of the matrix of polys BT B = B − RAS using vii), viii) shows

Tm = 0

Hence
T =0
So vii) becomes

tI − B0 = B = BT B + RAS = RAS = R(tI − A0 )S

= RST + RA0 S (*)

comparing coefficients of the poly matrices in (*) shows

I = RS
B0 = RA0 S

i.e., B0 = RA0 S = RA0 R−1 .

89
Duc Vu (Spring 2021) 21 Lec 21: May 14, 2021

Theorem 21.3
Let A0 , B0 ∈ Mn F , A = tI − A0 , B = tI − B0 in Mn F [t]. Then the following are
equivalent

i) A0 ∼ B0

ii) A ≈ B

iii) A and B have the same SNF.

iv) A0 and B0 have the same invariant factors.

In particular, if V is a finite dimensional vector space over F , T : V → V linear, q1 | . . . |qr

the invariants of T , then

V = ker q1 (T ) ⊕ . . . ⊕ ker qn (T )
qr = qT
fT = q1 . . . qr
Qr
Note: If qi = j=1 (t − λi )ej is an invariant factor, then
 
Je1 (λ1 ) 0
C(qi ) ∼ 
 .. 
. 
0 Jer (λr )

Corollary 21.4
Let A, B ∈ Mn F , F ⊆ K a subfield. Then A ∼ B in Mn F iff A ∼ B in Mn K.

90
 Duc Vu (Spring 2021) 22 Lec 22: May 17, 2021

§22 L e c 2 2 : M ay 1 7 , 2 0 2 1

§22.1 Inner Product Spaces

√ √
Notation: − : C → C by α + β −1 7→ α − β −1 ∀α, β ∈ R is called the complex
conjugation. If F ⊆ C, set
F := {α| α ∈ F }
is a field, e.g., F = F if F ⊆ R.

Definition 22.1 (Inner Product Space) — Let F ⊆ C satisfy F = F , V a vector space

over F . Then V is called an inner product space over F relative to

h, i = h, iV : V × V → F

satisfies

1. pv : V → F by pv (w) := hw, vi is linear for all v ∈ V , i.e., pv ∈ V ∗

2. hv, wi = hv, wi for all v, w ∈ V

3. kvk2 := hv, vi ∈ R ∩ F for all v ∈ V and kvk2 ≥ 0 in R and = 0 iff v = 0 (*)

Let V be an inner product space over F . Then,

1. If v ∈ V satisfies hw, vi = 0 for all w ∈ V , then v = 0.

2. Let v1 , v2 ∈ V \ {0},
hv2 , v1 i
w= v1
kv1 k2
is called the orthogonal projection of v2 on v1 and v = v2 − w is orthogonal to w, i.e.
hv, wi = 0, write v ⊥ w.

Definition 22.2 (Sesquilinear Map) — A map f : V → W of inner product space over

F is called sesquilinear if v1 , v2 ∈ V , α ∈ F

f (v1 + αv2 ) = f (v1 ) + αf (v2 )

Let V † := {f : V → F | f sesquilinear} a vector space over F .

Example 22.3
If F ⊆ R, then any sesquilinear map is linear and V † = V ∗ .

91
Duc Vu (Spring 2021) 22 Lec 22: May 17, 2021

Remark 22.4. Let V be an inner product space over F .

1. p : V → V ∗ by v 7→ pv is sesquilinear.

p(αv1 + v2 )(w) = hw, αv1 + v2 i

= αhw, v2 i + hw, v1 i = αp(v1 ) + p(v2 )

for all α ∈ F , v1 , v2 , w ∈ V . Also, we can deduce that p is an injection and if V is finite

dimensional, then p is a bijection.
2. If v ∈ V , let λv : V → F by w 7→ hv, wi, i.e., λv (w) = hv, wi. Then λv is sesquilinear.
Moreover,
λ : V → V † by v 7→ λv
is linear. As hv, wi = 0 for all w → v = 0, λ is injective hence monic. If V is finite
dimensional then λ is an isomorphism.
3. If f : V → W is sesquilinear, it is called a sesquilinear isomorphism if it is bijective and
f −1 is sesquilinear. Then f is a sesquilinear isomorphism iff f is bijective.

Let V be an inner product space over F .

p
1. If v ∈ V , kvk := kvk2 ≥ 0 is called the length of v.

2. Length and ∠ make sense in V by the Cauchy – Schwarz inequality

|hv, wi| ≤ kvkkwk ∀v, w ∈ V

and V is a metric space by distances from v, w := d(v, w) := kv − wk as the triangle

inequality
kv + wk ≤ kvk + kwk
holds for all v, w ∈ W .

3. Gram – Schmidt: If W ⊆ V is a finite dimensional subspaces, then ∃ an orthogonal

basis for W
B = {w1 , . . . , wn } , i.e. hwi , wj i = 0 if i 6= j
and if kwi k ∈ F ∀i, then ∃ an orthonormal basis
 
w1 wn
C = ,...,
kw1 k kwn k

4. In 3), if v ∈ V let B = {w1 , . . . , wn } be an orthogonal basis for W . Set

n n
X hv, wi i X wi
vw := wi = hv, iwi
kwi k2 kwi k2
i=1 i=1

hv,wi i
Then, the wi -coordinate of vw is kwi k2
∈ F . Hence

fi = p wi :V →F
kwi k2

is the corresponding coordinate function, so B ∗ = {f1 , . . . , fn } is the dual basis of B.

92
Duc Vu (Spring 2021) 22 Lec 22: May 17, 2021

6 S ⊆ V be a subset. The orthogonal complement S ⊥ of S is defined by

5. Let ∅ =

S ⊥ := {x ∈ V | x ⊥ s ∀s ∈ S} ⊆ V

a subspace.
Note: The sesquilinear map
p : V → V ∗ by v 7→ pv
induces an injective sesquilinear map

p S⊥
: S⊥ → S◦

and we have
S ⊆ S ⊥⊥ := (S ⊥ )⊥
If S is a subspace, S ∩ S ⊥ = 0 so

S + S⊥ = S ⊕ S⊥

write
S + S⊥ = S ⊥ S⊥
called an orthogonal direct sum and if V is finite dimensional then

S = S ⊥⊥

e.g., if v ∈ V , then
ker pv = (F v)⊥
so
V = F v ⊥ (F v)⊥
More generally, we have the following crucial result.

Theorem 22.5 (Orthogonal Decomposition)

Let V be an inner product space over F , S ⊆ V a finite dimensional subspace. Then

V = S ⊥ S⊥

i.e., if v ∈ V
∃!s ∈ S, s⊥ ∈ S ⊥ 3 v = s + s⊥
In particular, s = vS . If V is finite dimensional, then

dim V = dim S + dim S ⊥

Theorem 22.6 (Best Approximation)

Let V be an inner product space over F , S ⊆ V a finite dimensional subspace, v ∈ V .
Then vS ∈ Sis the best approximation to v in S, i.e., for all s ∈ S

kv − vS k ≤ kv − sk with equality iff s = vS

93
Duc Vu (Spring 2021) 22 Lec 22: May 17, 2021

Remark 22.7. More generally, if V is an inner product space over F ,

V = W1 ⊕ . . . ⊕ Wn

with
wi ⊥ wj ∀wi ∈ Wi , wj ∈ Wj , i 6= j
We call V an orthogonal direct sum or orthogonal decomposition of V .

By the Orthogonal Decomposition Theorem,

V = Wi ⊥ Wi⊥

and
Wi⊥ = W1 ⊥ . . . Ŵi ⊥ . . . ⊥ Wn
|{z}
omit

Let Pi : V → V be the projection along

Wi⊥ = W1 ⊥ . . . ⊥ Ŵi ⊥ . . . ⊥ Wn

onto Wi . Then we have

ker Pi = Wi⊥
im Pi = Wi
Pi Pj = δij Pj ∀i, j
1V = P1 + . . . + Pn

The Pi are called orthogonal projections. As Wi ⊆ V is finite dimensional in the above,

Pi (v) = vWi

So
v = vW1 + . . . + vWn
is a unique decomposition of v relative to (*).

Definition 22.8 (Adjoint) — Let V, W be inner product spaces over F , T : V → W

linear. A linear transformation T ∗ : W → V is called the adjoint of T if

hT v, wiW = hv, T ∗ wiV ∀v ∈ V ∀w ∈ W

Theorem 22.9
Let V, W be finite dimensional inner product space over F , T : V → W linear. Then
the adjoint T ∗ : W → V exists.

94
 Duc Vu (Spring 2021) 23 Lec 23: May 19, 2021

§23 L e c 2 3 : M ay 1 9 , 2 0 2 1

§23.1 I n n e r P r o d u c t S p a c e s ( C o nt ’ d )

Corollary 23.1
Let V, W be finite dimensional inner product space over F , T : V → W linear. Then

T = T ∗∗ := (T ∗ )∗

and
hT ∗ w, viV = hw, T viW ∀w ∈ W ∀v ∈ V

Proof. We have

hT v, wiW = hv, T ∗ wiV = hT ∗ w, viV

= hw, T ∗∗ viW = hT ∗∗ v, wiW

which completes the proof.

Definition 23.2 (Isometry) — Let V, W be inner product space over F , T : V → W

linear. Then T is called an isometry (or isomorphism of inner product space over F ) if

1. T is an isomorphism of vector space over F

2. T preserves inner products, i.e.,

hT v, T v 0 iW = hv, v 0 iV ∀v, v 0 ∈ V

Remark 23.3. Let T : V → W linear of inner product space over F . If T preserves inner
products, then T is monic.

T v = 0 ⇐⇒ kT vk = 0 ⇐⇒ hT v, T vi = 0 ⇐⇒ hv, vi = 0

95
Duc Vu (Spring 2021) 23 Lec 23: May 19, 2021

Theorem 23.4
Let V, W be finite dimensional inner product space over F with dim V = dim W and
T : V → W linear. Then the following are equivalent

1. T preserves inner product.

2. T is an isometry.

3. If B = {v1 , . . . , vn } is an orthogonal basis for V , then C = {T v1 , . . . , T vn } is an

orthogonal basis for W and

kT vi k = kvi k i = 1, . . . , n

4. ∃ an orthogonal basis B = {v1 , . . . , vn } for V s.t. C = {T v1 , . . . , T vn } is an

orthogonal basis for W with kT vi k = kvi k i = 1, . . . , n.

Proof. 1) =⇒ 2) T is monic by the remark above, so an isomorphism by the Isomorphism

theorem.
2) =⇒ 3) By the Isomorphism theorem, C is a basis for W and C is orthogonal with
kvi k = kT vi k for all i.
3) =⇒ 4) is immediate.
4) =⇒ 1) By thePIsomorphism theorem, T is an isomorphism of vector space over F . If
n Pn
x, y ∈ V , let x = i=1 αi vi , y = i=1 βi vi , then
X X
hx, yi = αi βj hvi , vj i = αi βj δij kvi k2
i,j i,j
X X
2
= αi βj δij kT vi k = αi βj δij hT vi , T vj i
i,j i,j

= hT x, T yi

Corollary 23.5
Let V, W be finite dimensional inner product space over F both having orthonormal
basis. Then V is isometric to W if and only if dim V = dim W .

Proof. Apply UPVS and the theorem above.

Theorem 23.6
Let V, W be inner product space over F , T : V → W linear. Then T preserves inner
products iff T preserves lengths, i.e., kT vkW = kvkV for all v ∈ V .

Proof. “ =⇒ ” The result is immediate.

96
Duc Vu (Spring 2021) 23 Lec 23: May 19, 2021

“ ⇐= ” Let x, y ∈ V and
√
hx, yiV = α + β −1
√
hT x, T yiW = γ + δ −1

for α, β, γ, δ ∈ R. We notice that

2α = 2γ =⇒ α = γ
√
So we are done if F ⊆ R. Suppose F * R, then there exists 0 6= µ ∈ R s.t. µ −1 ∈ F .
Then
√ √ √
hx, −1µyiV = − −1µhx, yiV = −µ −1α + βµ
√ √ √
hT x, −1µT yiW = − −1µhT x, T yiW = −µ −1γ + δµ

Analogous to (*),
βµ = δµ, so β = δ
Hence hx, yiV = hT x, T yiW .

97
 Duc Vu (Spring 2021) 24 Lec 24: May 21, 2021

§24 L e c 2 4 : M ay 2 1 , 2 0 2 1

§24.1 I n n e r P r o d u c t S p a c e s ( C o nt ’ d )

Definition 24.1 (Unitary Operator) — Let V be an inner product space over F ,

T : V → V linear. We call T a unitary operator if T is an isometry. If F ⊆ R, such a
T is called an orthogonal operator.

Proposition 24.2
Let V be an inner product space over F , T : V → V linear. Suppose that T ∗ exists.
Then, T is an isometry if and only if T ∗ = T −1 , i.e., T T ∗ = 1V = T ∗ T .

Proof. “ =⇒ ” As T is an isomorphism of vector space over F , T −1 : V → V exists and is

linear. As T preserves inner products, for all x, y ∈ V

hT x, yi = hT x, 1V yi = hT x, T T −1 yi = hx, T −1 yi

It follows that T ∗ = T −1 by uniqueness.

“ ⇐= ” As T ∗ T = 1V = T T ∗ , T is invertible with T −1 = T ∗ , so T is an isomorphism.
Since
hT x, T yi = hx, T ∗ T yi = hx, yi
for all x, y ∈ V . T preserves inner products.

Remark 24.3. Let V be a finite dimensional inner product space over F , T : V → V linear.
1. T is monic iff T is epic iff T is an iso of vector space over F .
2. T is unitary ⇐⇒ T ∗ T = 1V ⇐⇒ T T ∗ = 1V
3. T is unitary ⇐⇒ T ∗ is unitary as T ∗∗ = T

Definition 24.4 (Unitary Matrix) — Let F ⊆ C, F = F . We say A ∈ Mn F is unitary

if A∗ A = I. Equivalently, AA∗ = I. Let

Un F := {A ∈ GLn F | AA∗ = I}

If F ⊆ R, we say A ∈ Mn F is orthogonal if A> A = I. Equivalently, AA> = I. Let

n o
On F := A ∈ GLn F | AA> = I

98
Duc Vu (Spring 2021) 24 Lec 24: May 21, 2021

Remark 24.5. 1. Let F ⊆ C, F = F , F n×1 , F 1×n inner product space over F via the
dot product. If A ∈ Mn F , then

A = [A]sn ,1 : F n×1 → F n×1

linear and sn,1 the ordered standard basis. Then A is unitary iff
i) The columns of A form an ordered orthonormal basis for F n×1
ii) The rows of A form an ordered orthonormal basis for F 1×n
2. If T : V → V is linear, V an inner product space over F with dim V = n, B, C ordered
orthonormal bases for V , then T is unitary iff [T ]B,C is unitary.

§24.2 Spectral Theory

Lemma 24.6
Let V be an inner product space over F , T : V → V linear, W ⊆ V a subspace.
Suppose that T ∗ exists. Then the following is true: If W is T -invariant, then W ⊥ is
T ∗ -invariant.

Proof. Let v ∈ W ⊥ , w ∈ W , then

hw, T ∗ vi = hT w, vi = 0

Lemma 24.7
Let V be a finite dimensional inner product space over F , T : V → V linear. Then
the following is true: If λ is an eigenvalue of T , then λ is an eigenvalue of T ∗ .

Proof. Let S = T − λ1V : V → V linear. Then

S ∗ = T ∗ − λ1V : V → V linear

Then ∀w ∈ V ,
0 = h0, wi = hSv, wi = hv, S ∗ wi
Hence v ⊥ im S ∗ and v ∈
/ im S ∗ as v 6= 0. By the Dimension Theorem,

0 < ker S ∗ , ET ∗ (λ) 6= 0

Theorem 24.8 (Schur)

Let V be a finite dimensional inner product space over F with F = R or C and
T : V → V linear. Suppose that fT splits in F [t]. Then, there exists an ordered
orthonormal basis B for V s.t. [T ]B is upper triangular.

99
Duc Vu (Spring 2021) 24 Lec 24: May 21, 2021

Proof. We induct on n = dim V .

n = 1 is immediate.
n > 1. By the 2nd lemma, ∃λ ∈ F and 0 6= v ∈ ET ∗ (λ). By the Orthogonal Decomposition
Theorem,
V = F v ⊥ (F v)⊥
and
dim(F v)⊥ = dim V − dim F v = n − 1
F v is T ∗ -invariant, hence (F v)⊥ is T ∗∗ = T -invariant. Let C0 be an ordered basis for
(F v)⊥ . Then C = C0 ∪ {v0 } is an ordered basis for V and we have
h i 
T (F v)⊥ ∗
 C0 

 ∗  
[T ]C =  .
. 

 .  
 ∗ 
0 [T v0 ]C

By expansion,
f fT ∈ F [t]
T
(F v)⊥

hence f ∈ F [t] splits. By induction, there exists an orthonormal basis B0 =

T
(F v)⊥ h i n o
{v1 , . . . , vn−1 } for (F v)⊥ s.t. T (F v)⊥ is upper triangular. Then B = B0 ∪ kvkv
is
B0
an orthonormal basis for V s.t. [T ]B is upper triangular.

100
 Duc Vu (Spring 2021) 25 Lec 25: May 24, 2021

§25 L e c 2 5 : M ay 2 4 , 2 0 2 1

§25.1 S p e c t r a l T h e o r y ( C o nt ’ d )

Definition 25.1 (Hermitian(Self-Adjoint)) — Let V be an inner product space over F ,

T : V → V linear. Suppose that T ∗ exists. We say that T is normal

T T ∗ = T ∗T

and is Hermitian if T = T ∗ , i.e.

hT v, wi = hv, T wi ∀v, w ∈ V

Note: If T is Hermitian, T ∗ exists automatically and T is normal.

Lemma 25.2
Let V be an inner product space over F , λ ∈ F , 0 6= v ∈ V , T : V → V a normal
operator. Then
v ∈ ET (λ) ⇐⇒ v ∈ ET ∗ (λ)

Proof. Let S = T − λ1V , then S ∗ = T ∗ − λ1V . It follows that

SS ∗ = S ∗ S, i.e. S is normal
Then
kSvk2 = hSv, Svi = hv, S ∗ Svi
= hv, SS ∗ vi = hS ∗ v, S ∗ vi
= kS ∗ vk2
So
v ∈ ET (λ) ⇐⇒ Sv = 0 ⇐⇒ S ∗ v = 0 ⇐⇒ v ∈ ET ∗ (λ)

Corollary 25.3
Let V be an inner product space over F , T : V → V normal, λ 6= µ eigenvalue of T .
Then, ET (λ) and ET (µ) are orthogonal. In particular,
X 1
ET (λ) = ET (λ)
λ
λ

Proof. Let 0 6= v ∈ ET (λ), 0 6= w ∈ ET (µ). Then by the lemma, w ∈ ET ∗ (µ) and

λhv, wi = hλv, wi = hT v, wi = hv, T ∗ wi
= hv, µwi = µhv, wi
As λ 6= µ, we obtain hv, wi = 0.

101
Duc Vu (Spring 2021) 25 Lec 25: May 24, 2021

Proposition 25.4
Let V be a finite dimensional inner product space over F , F = R or C, T : V → V
linear, B an ordered orthonormal basis for V s.t. [T ]B is upper triangular. Then, T
is normal if and only if [T ]B is diagonal.

Proof. “ ⇐= ” If  
λ1 0
[T ]B =
 .. 
. 
0 λn
then  
λ1 0
[T ∗ ]B = [T ]∗B =
 .. 
. 
0 λn
So
|λ1 |2
 
0
[T T ∗ ]B = [T ]B [T ∗ ]B = 
 .. 
. 
0 |λn |2
= [T ∗ ]B [T ]B
= [T ∗ T ]B

Hence, T T ∗ = T ∗ T by the Matrix Theory Theorem.

“ =⇒ ” Let B = {v1 , . . . , vn } be an orthonormal basis for V s.t. A = [T ]B is upper
triangular. By the lemma,

T v1 = A11 v1 and T ∗ v1 = A11 v1

By definition,
n
X n
X
T ∗ v1 = (A∗ )i1 vi = A1i vi
i=1 i=1
So
A1i = 0 ∀i > 1
Hence,
A1i = 0 ∀i > 1
In particular,
A12 = 0
By the lemma,
T v2 = A22 v2 , hence T ∗ v2 = A22 v2
The same argument shows A2i = 0, i 6= 2, i.e.,

A2i = 0, i 6= 2

Continuing this process, we conclude A is diagonal.

102
Duc Vu (Spring 2021) 25 Lec 25: May 24, 2021

Theorem 25.5 (Spectral Theorem for Normal Operators)

Let V be a finite dimensional inner product space over C, T : V → V linear. Then
T is normal if and only if there exists an orthonormal basis B for V consisting of
eigenvectors of T . In particular, if T is normal, then T is diagonalizable.

Proof. This follows immediately by Schur’s theorem, FTA, and the above proposition.

Remark 25.6. Let V be a finite dimensional inner product space over R, T : V → V linear.
Suppose that fT ∈ R[t] splits. Then T is normal iff ∃ an orthonormal basis B for V consisting
of eigenvectors for T .
By Schur’s theorem, T is triangularizable via an orthonormal basis for V . The same result
follows by the proposition in the case F = R.

Spectral Decomposition and Resolution for Normal Operators:

Let V be a finite dimensional inner product space over F , F = R or C, T : V → V
linear s.t. fT splits. So T is normal. Let λ1 , . . . , λr be all the distinct eigenvalues of T in
F , C an orthonormal basis for V . We know
v ∈ ET (λi ) ⇐⇒ v ∈ ET ∗ (λi ) ∀i (+)
Let Pi : V → V be the orthogonal projection along ET (λi )⊥ for i = 1, . . . , r omit at ith
onto ET (λi ).
By (+), Pi : V → V is also the orthogonal projection along ET ∗ (λi )⊥ onto ET ∗ (λi ).
This is a unique decomposition
PET (λi ) = Pi = PET∗ (λi ) ∀i
T Pi = Pi T and T ∗ Pi = Pi T ∗ ∀i
1V = P1 + . . . + Pr
Pi Pj = δij Pi ∀i
T = λ1 P1 + . . . + λr Pr
T ∗ = λ 1 P1 + . . . + λ r Pr
Let Bi be an ordered orthonormal basis for ET (λi ), so B = B1 ∪ . . . ∪ Br is an ordered
orthonormal basis for V with [T ]B and [T ∗ ]B is diagonal.
Let Q = [1V ]B,C . Then Q is unitary as it takes an orthonormal basis to an orthonormal
basis, hence
Q −1 = Q ∗
[T ]B = Q ∗ [T ]C Q
[T ∗ ]B = Q ∗ [T ∗ ]C Q

Theorem 25.7
Let V be a finite dimensional inner product space over F , F = R or C, T : V → V
linear with fT ∈ F [t] splits. Then, T is normal if and only if ∃g ∈ F [t] s.t. T ∗ = g(T ).

103
 D u c Vu ( S p r i n g 2 0 2 1 ) 26 L e c 2 6 : M ay 2 6 , 2 0 2 1

§26 L e c 2 6 : M ay 2 6 , 2 0 2 1

§26.1 S p e c t r a l T h e o r y ( C o nt ’ d )

Remark 26.1. A rotation Tθ : R2 → R2 by ∠θ, 0 < θ < 2π, θ 6= π has no eigenvalues, but is
normal (with R2 an inner product space over R via the dot product) as it is unitary.

Lemma 26.2
Let V be an inner product space over F , T : V → V hermitian. If λ is an eigenvalue
of T , then λ ∈ F ∩ R.

Proof. Let 0 6= v ∈ ET (λ). Then

λkvk2 = λhv, vi = hλv, vi = hT v, vi
= hv, T ∗ vi = hv, T vi = hv, λvi
= λhv, vi = λkvk2
As kvk =
6 0, λ = λ, so it’s real.

Lemma 26.3
Let V be a finite dimensional inner product space over F with F = R or C, T : V → V
hermitian. Then fT ∈ F [t] splits in F [t].

Proof. By previous result, we can assume that F = R. Let B be an orthonormal basis for
V . Then
A := [T ]B = [T ∗ ]B = [T ]∗B = A∗
in Mn R ⊆ Mn C, n = dim V . As
A : Cn×1 → Cn×1 is Hermitian
fA splits with real roots by Lemma 26.2. (and FTA), i.e.,
Y
fA = (t − λi ) ∈ C[t], λi ∈ R ∀i
Q
So fT = fA = (t − λi ) ∈ R[t] splits.

Theorem 26.4 (Spectral Theorem for Hermitian Operators)

Let V be a finite dimensional inner product space over F , F = R or C, T : V → V
hermitian. Then, there exists an orthonormal basis for V of eigenvectors of T and all
all eigenvalues are real.

Proof. If F = C, the result follows by Lemma 26.2 as T is normal. So we may assume

F = R. As fT ∈ R[t] splits by Lemma 26.3, there exists an orthonormal basis B for V s.t.
[T ]B is upper triangular by Schur’s Theorem. As T is normal, it is diagonalizable. The
result follows by Lemma 26.2.

104
Duc Vu (Spring 2021) 26 Lec 26: May 26, 2021

§26.2 Hermitian Addendum

Theorem 26.5
If 0 6= V is a finite dimensional inner product space over R, T : V → V hermitian,
then T has an eigenvalue.

The proof in Axler’s book is very nice, and he does not use determinant theory. He uses
the following arguments

1. If V is a finite dimensional vector space over F , T : V → V linear, then there exists

q ∈ F [t] monic s.t. q(T ) = 0

2. If 0 6= q ∈ R[t], then there exists a factorization

q = β(t − λ1 )e1 . . . (t − λr )er q1f1 . . . qsfs

in R[t] with qi monic irreducible quadratic polynomials in R[t].

This follows by the FTA.

Lemma 26.6
Let q = t2 + bt + c in R[t], b2 < 4c, i.e., q is an irreducible monic quadratic polynomial
in R[t]. If V is a finite dimensional inner product space over R and T : V → V is
Hermitian, then q(T ) is an isomorphism.

Proof. It suffices to show q(T ) is a monomorphism by the Isomorphism Theorem. So it

suffices to show if 0 6= v ∈ V , then q(T )v 6= 0. We have

hq(T )v, vi = hT 2 v, vi + bhT v, vi + chv, vi

= hT v, T vi + bhT v, vi + chv, vi
= kT vk2 + bhT v, vi + ckvk2
≥ kT vk2 − |b|kT vkkvk + ckvk2
|b|kvk 2 b2
   
= kT vk − + c− kvk2 > 0
2 4

So q(T )v 6= 0.

Proof. (of Theorem) Let q ∈ R[t] in 2) satisfy q(T ) = 0. So

0 = q(T ) = (T − λ1 1V )e1 . . . (T − λr 1V )er q1 (T )f1 . . . qs (T )fs

As all the qi (T ) are isomorphism, at least one of the (T − λi 1V ) is not injective, i.e., λi is
an eigenvalue.

105
 Duc Vu (Spring 2021) 27 Lec 27: May 28, 2021

§27 L e c 2 7 : M ay 2 8 , 2 0 2 1

§27.1 Po s i t i ve ( S e m i - ) D e f i n i t e O p e r a t o r s
Let V be a finite dimensional inner product space over F , where F = R or C, T : V → V
hermitian, B = {v1 , . . . , vn } an orthonormal basis of eigenvectors of T , i.e.,
T vi = λi vi , i = 1, . . . , n
So λi ∈ R, i = 1, . . . , n. Suppose v ∈ V . Then
Xn
v= αi vi , αi ∈ F ∀i
i=1
and
Xn n
X
hT v, vi = h T (αi vi ), αj vj i
i=1 j=1
Xn n
X
=h λi αi vi , αj vj i
i=1 j=1
Xn
= λi αi αj hvi , vj i (*)
i,j=1
X n
= λi αi αj δij
i,j=1
X n
= λi |αi |2
i=1

Definition 27.1 (Positive/Negative (Semi-) Definite) — Let V be a finite dimensional

inner product space over F , F = R or C, T : V → V hermitian. We say that T is
positive or positive definite if

hT v, vi > 0 ∀0 6= v ∈ V

and positive semi-definite if

hT v, vi ≥ 0 ∀0 6= v ∈ V

We can define T as negative (semi-) definite similarly.

It follows from (*) that we have

Proposition 27.2
Let V be a finite dimensional inner product space over F , F = R or C, T : V → V
hermitian. Then T is positive semi-definite (respectively positive) if and only if all
eigenvalues of T are non-negative (respectively positive).

106
Duc Vu (Spring 2021) 27 Lec 27: May 28, 2021

Question 27.1. What does this say about the 2nd derivative test for C 2 function, f : S → R
at a critical point in the interior of S?

Theorem 27.3
Let V be a finite dimensional inner product space over F , F = R or C, T : V → V
hermitian. Then T is non-negative (respectively positive) iff ∃S : V → V non-negative
s.t.
T = S2
i.e., T has a square root (respectively, and S is invertible).

Proof. “ =⇒ ” Let B = {v1 , . . . , vn } be an ordered orthonormal basis for V of eigenvectors

of T
T vi = λi vi , λi ≥ 0 ∈ R, i = 1, . . . , n
Then ∃µi ∈ R, µi ≥ 0 s.t. λi = µ2i , i = 1, . . . , n. Let
√   
λ1 0 µ1 0
B=
 .. =
  .. 
. .
√

0 λn 0 µn

So
B 2 = [T ]B
By MTT, ∃S : V → V linear s.t. [S]B = B. So

[T ]B = B 2 = [S]2B = S 2 B
 

Hence T = S 2 by MTT. As B is orthonormal, µi ∈ R for all i

[S ∗ ]B = [S]∗B = B ∗ = B = [S]B

Thus, S = S ∗ by MTT; so hermitian if λi > 0∀i, det B 6= 0, so B ∈ GLn F .

“ ⇐= ” Let B be an ordered orthonormal basis for V of eigenvectors for S. Then
 
µ1 0
[S]B = 
 ..  , µi ≥ 0 ∈ R and

.
0 µn
µ21
 
0
[T ]B = S 2 B = 
   .. 
. 
0 µ2n

is diagonal. Therefore, B is also an orthonormal basis for V of eigenvectors of T . As µ2i ≥ 0

(> 0 if S is invertible), T is non-negative (respectively positive if S is invertible).

107
Duc Vu (Spring 2021) 27 Lec 27: May 28, 2021

Theorem 27.4
Let V be a finite dimensional inner product space over F , F = R or C and T : V → V
hermitian. Suppose that T is non-negative. Then T has a unique square root S, i.e.,
S : V → V non-negative s.t. S 2 = T .

Proof. Let S 2 = T , S : V → V non-negative. The Spectral Theorem gives unique

orthogonal decompositions

V = ET (λ1 ) ⊥ . . . ⊥ ET (λr )
T = λ1 Pλ1 + . . . + λr Pλr
Pλi Pλj = δij Pλi Pλj , ∀i, j
1 V = P λ1 + . . . + P λr

and we also have

V = ES (µ1 ) ⊥ . . . ⊥ ES (µs ), µi ≥ 0, i = 1, . . . , s
S = µ1 Pµ1 + . . . + µs Pµs
Pµi Pµj = δij Pµi , ∀i, j
1V = Pµ1 + . . . + Pµs

In particular,

S 2 = (µ1 Pµ1 + . . . + µs Pµs )(µ1 Pµ1 + . . . + µs Pµs )

= µ21 Pµ1 + . . . + µ2s Pµs

As T = S 2 ,
µ21 Pµ1 + . . . + µ2s Pµs = λ1 Pλ1 + . . . + λr µr
So by uniqueness, we must have s = r and changing the order if necessary

µ2i = λi , Pµi = Pλi , ∀i

Lemma 27.5
Let V, W be finite dimensional inner product space over F , F = R or C, T : V → W
linear. Then T ∗ T : V → V is hermitian and non-negative.

Remark 27.6. If in the definition of positive operator, etc, we omit V being finite dimensional
but assume T ∗ exists, then we would still have T ∗ T hermitian.

Proof. Let x, y ∈ V . Then

hx, (T ∗ T )∗ yiV = hT ∗ T x, yiV = hT x, T yiW = hx, T ∗ T yiV

Since this is true for all x, y

(T ∗ T )∗ = (T ∗ T ∗∗ )∗ = T ∗ T

108
Duc Vu (Spring 2021) 27 Lec 27: May 28, 2021

is hermitian, hence has real eigenvalues. Let λ be an eigenvalue of T ∗ T , 0 6= v ∈ V s.t.

T ∗ T v = λv. Then

λkvk2V = λhv, viV = hλv, viV = hT ∗ T v, viV

= hT v, T viW = kT vk2W ≥ 0

So
kT vk2W
λ= ≥0
kvk2V
as kvk2V 6= 0.

Corollary 27.7
Let V be a finite dimensional inner product space over F , F = R or C, T : V → V linear.
Then T is non-negative (respectively positive) iff ∃S : V → V linear (respectively an
isomorphism) s.t. T = S ∗ S.

Proof. Use the theorem and lemma presented above.

Notation:

• F = R or C, A ∈ F m×n

• A(i) = the ith column of A

• A = A(1) . . . A(m)
 

• h, i = the dot product on F N for any N ≥ 1

• UN (F ) = U ∈ GLN F | U ∗ = U −1


Definition 27.8 (Pseudodiagonal) — Let D ∈ F m×n . We call D pseudodiagonal if

Dij = 0 ∀i 6= j, i.e., only Dii can have non-zero entries.

109
Duc Vu (Spring 2021) 27 Lec 27: May 28, 2021

Theorem 27.9 (Singular Value)

Let F = R or C, A ∈ F m×n . Then ∃U ∈ Un (F ), X ∈ Um (F ) s.t.
 
µ1 0
 .. 
 . 
∗  ∈ F m×n
 
X AU = D =   µr 

 0 

..
0 .

is a pseudodiagonal matrix satisfying

µ1 ≥ . . . ≥ µ r > 0

and
r = rank(A)

Proof. By the lemma, A∗ A ∈ Mn F is hermitian and has non-negative eigenvalues. Let

λ1 , . . . , λr be the positive eigenvalues ordered s.t.

λ1 ≥ . . . ≥ λr > 0

By the Spectral Theorem for Hermitian Operators, ∃U ∈ Un F s.t.

 
λ1 0
 .. 
 . 
 
∗ ∗ ∗
 λr 
(AU ) (AU ) = U A AU =  
 0 

 . . 
 . 
0 0

in Mn F . Let C = AU ∈ F m×n . Then

C ∗ C = (AU )∗ (AU ) ∈ Mn F

Write
λi = µ2i , µi > 0, 1≤i≤r
So
µ1 ≥ . . . ≥ µ r > 0
Set  
µ1 0
 .. 
 . 
 
 µr 
 ∈ Mn F
B=

 0 

 .. 
 . 
0 0

110
Duc Vu (Spring 2021) 27 Lec 27: May 28, 2021

If i > r1 let λi = 0. Then, we have

X X
λi δij = (C ∗ C)ij = (C ∗ )il Clj = Cli Clj
l l
X
= Clj Cli = hC (j) , C (i) i
l

Hence
C = C (1) . . . C (r) 0 . . . 0
 

We continue with the proof in the next lecture.

111
 Duc Vu (Spring 2021) 28 Lec 28: Jun 2, 2021

§28 Lec 28: Jun 2, 2021

§28.1 Po s i t i ve ( S e m i - ) D e f i n i t e O p e r a t o r s ( C o nt ’ d )
Proof. (Cont’d) Recall, we have proven so far
C = C (1) . . . C (r) 0 . . . 0
 

and thus C (1) , . . . , C (r) is an orthogonal set in F m×1 . As C (i) 6= 0, i = 1, . . . , r,



C (1) , . . . , C (r) are linearly independent. In particular,

rank C = r
We also have
kC (i) k2 = hC (i) , C (i) i = λi = µ2i
for i = 1, . . . , m. As U is invertible,
rank A = rank AU = rank C = r
So rank A = r as needed.
Now let
1 (i)
X (i) := C , i = 1, . . . , r
µi
Then X (1) , . . . , X (r) is an orthonormal set. Extend this to an orthonormal basis B =

 (1)
X , . . . , X (m) . Then
X = X (1) . . . X (m) = [1F m×1 ]Sm,1 ,B
 

Since both Sm,1 and B are orthonormal bases, X ∈ Um (F ). Let D be the pseudo-diagonal
matrix  
µ1 0
 .. 
 . 
 ∈ F m×n
 
D :=  µr 

 0 

..
0 .
as in the statement of the theorem. Then
 
µ1
 .. 

 . 

 (1) (m)

XD = X ... X 
 µr 


 0 

..
.
= µ1 X (1) . . . µr X (r) 0 . . . 0
 

= C = AU
Hence
X ∗ AU = D
as needed.

112
Duc Vu (Spring 2021) 28 Lec 28: Jun 2, 2021

Definition 28.1 (Singular Value Decomposition) — Let A ∈ F m×n , F = R or C.

A = XDU ∗ , U ∈ Un F, X ∈ Um F
 
µ1 0
 .. 
 . 
 ∈ F m×n
 
D=  µr  (*)

 0 

..
0 .
µ1 ≥ . . . ≥ µ r > 0 ∈ R

Then (*) is called a singular value decomposition (SVD) of A, µ1 , . . . , µr are the

singular values of A, D is the pseudo-diagonal matrix of A.

Note: Let A = XDU ∗ be an SVD of A. Then

1. The singular values of A are the (positive) square roots of the positive eigenvalues of
A∗ A.
2. The columns of X form an orthonormal basis for F m×1 of eigenvectors of AA∗ .
3. The columns of U form an orthonormal basis for F n×1 of eigenvectors of A∗ A.

Corollary 28.2
The singular values of A ∈ F m×n , F = R or C are unique (including multiplicity) up
to order.

Proof. Let A = XDU ∗ be a SVD of A, X ∈ Um F , U ∈ Un F . Then

A∗ A = (XDU ∗ )∗ (XDU ∗ ) = U D∗ X ∗ XDU ∗ = U D∗ DU ∗
as X ∗ X = I. So  2 
α11
..
A∗ A ∼ D ∗ D = 
 
 .  ∈ Mn F

..
.
2 , . . . , as A∗ A.
have the same eigenvalues α11

Remark 28.3. An SVD of A ∈ F m×n , F = R or C may not be unique.

Corollary 28.4
The singular values of A ∈ F m×n , F = R or C are the same as the singular values of
A∗ ∈ F n×m .

Proof. (XDU ∗ )∗ = U D∗ X ∗ and D, D∗ have the same non-zero diagonal eigenvalues.

The abstract version of the singular value theorem is

113
Duc Vu (Spring 2021) 28 Lec 28: Jun 2, 2021

Theorem 28.5 (Singluar Value - Linear Transformation Form)

Let F = R or C, V a finite dimensional inner product space over F and T : V → W
linear of rank r. Then there exists orthonormal basis

B = {v1 , . . . , vn } for V
C = {w1 , . . . , wm } for W
µ1 ≥ . . . ≥ µ r > 0 ∈ R

satisfying (
µi wi , i = 1, . . . , r
T vi =
0, i>r

Conversely, suppose the above conditions are all satisfied. Then vi is an eigenvector for
T ∗ T with eigenvalue µ2i for i = 1, . . . , r and eigenvalue 0 for i = r + 1, . . . , n. In particular,
µ1 , . . . , µr are uniquely determined.

Proof. Left as exercise.

Remark 28.6. So we see for an arbitrary linear transformation T : V → W of finite dimensional

inner product space over F , F = R or C, singular values can be viewed as a substitute for
eigenvalues.

When F = R or C and A ∈ Mn√F , we get a generalization of the polar representation of

eigenvalues z ∈ C where z = re −1θ .

Theorem 28.7 (Polar Decomposition)

Let F = R or C, A ∈ Mn F . Then there exists Ũ ∈ Un F , N ∈ Mn F hermitian with all
its eigenvalues real and non-negative satisfying

A = Ũ N
√
here N ↔ r, Ũ ↔ e −1θ for n = 1.

Proof. In the singular value theorem, we have m = n. Let A = XDU ∗ be an SVD,

X, U ∈ Un F . We have D = D∗ is hermitian with non-negative eigenvalues. So

A = XDU ∗ = X(U ∗ U )DU ∗ = (XU ∗ )(U DU ∗ )

Since
(XU ∗ )∗ (XU ∗ ) = U X ∗ XU ∗ = U U ∗ = I
XU ∗ ∈ Un F also. Let Ũ = XU ∗ ∈ Un F , N = U DU ∗ which completes the proof.

§28.2 Least Squares

We give an application of SVD

114
Duc Vu (Spring 2021) 28 Lec 28: Jun 2, 2021

Problem 28.1. Let F = R or C, V a finite dimensional inner product space over F ,

W ⊆ V a subspace. Let
PW : V → V by v 7→ vW
be the orthogonal projection of V onto W . By the Approximation Theorem, vW is the best
approximation of v ∈ V onto W . Now let X be another finite dimensional inner product
space over F and T : X → V linear with W = T (X) = im T . Let v ∈ V and x ∈ X. We
call

i) x a best approximation to v via T if

T x = vW = PW (v)

ii) x an optimal approximation to v via T if it is a best approximation to v via T and

kxk is minimal among all best approximation to v via T .

Find an optimal approximation.

Solution:
hx, T ∗ yiX = hT x, yiV ,
we have
W ⊥ = (im T )⊥ = ker T ∗
Since
v − vW ∈ W ⊥ = (im T )⊥ (by the OR Decomposition Theorem)
and
T ∗ v = T ∗ vW
So if x is a best approximation of v via T , then

T ∗T x = T ∗v (*)

i.e., x is also a solution to T ∗ T x = T ∗ v. Conversely, if (*) holds, then

T x − v ∈ ker T ∗ = (im T )⊥ = W ⊥

In particular,

vW = PW v = PW (T x − (T x − v))
= PW (T x) − PW (T x − v)
= Tx + 0 = Tx

Conclusion: x is a best approximation to v via T if and only if T ∗ T x = T ∗ v.

Claim 28.1. Suppose that T is monic. Then

T ∗ T : X → X is an isomorphism

and
PW = T (T ∗ T )−1 T ∗ : V → V (+)

115
Duc Vu (Spring 2021) 28 Lec 28: Jun 2, 2021

Suppose that x ∈ X satisfies T ∗ T x = 0. Then

0 = hT ∗ T x, xiX = hT x, T xiV = kT xk2V (?)

Therefore, T x = 0. But T is monic, so x = 0. Hence T ∗ T : V → V is monic hence an

isomorphism. We now show (+) holds.
Let v ∈ V . Since T ∗ T is an isomorphism, there exists x ∈ X s.t.

T ∗T x = T ∗v (??)

and

T (T ∗ T )−1 T ∗ v = T (T ∗ T )−1 T ∗ T x
= T x = vW = PW (v)

showing (+). This proves the claim and also shows that the x in (??) is a best approximation
to v via T .

116
 Duc Vu (Spring 2021) 29 Lec 29: Jun 4, 2021

§29 Lec 29: Jun 4, 2021

§29.1 L e a s t S q u a r e s ( C o nt ’ d )
Claim 29.1. Let v ∈ V . Then ∃!x ∈ X an optimal approximation to v via T . Moreover,
this x is characterized by

PY (x) = 0 where Y = ker T ∗ T

Let x, x0 be two best approximation to v via T . Then,

T ∗ T x = T ∗ v = T ∗ T x0

Therefore,
x − x0 ∈ ker T ∗ T =: Y
It follows if x is a best approximation to v via T , then any other is of the form x + y, y ∈ Y .
We also have for such x + y

PY (x + y) = PY (x) + PY (y) = PY (x) + y

Let x00 = x − PY (x). Then

PY (x00 ) = PY (x) − PY2 (x) = 0, i.e., x00 ⊥ Y

So
kx00 + yk2 = kx00 k2 + kyk2 ≥ kx00 k2 ∀y ∈ Y
by the Pythagorean Theorem. Hence, x00
= PY ⊥ (x) is the unique optimal approximation.
This proves the claim above.
Let A = T : F n×1 → F m×1 , A ∈ F m×n , v ∈ F m×1 with F = R or C. Let
 
µ1
 .. 
 . 
∗  ∈ F m×n
 
A = XDU , D =   µr 

 0 

..
.

and
µ1 ≥ . . . ≥ µ r > 0 ∈ R
be an SVD. Let’s define
µ−1
 
1
 .. 
 . 
†  ∈ F n×m
µ−1
 
D := 
 r 

 0 

..
.

Then
A† := U D† X ∗ ∈ F n×m
is called the Moore-Penrose generalized pseudoinverse of A. Then the following are true

117
Duc Vu (Spring 2021) 29 Lec 29: Jun 4, 2021

i) rank(A) = rank(A† )

ii) A> v is an optimal approximation in F n×1 to v via A and is unique.

iii) If rank(A) = n, then

A† = (A∗ A)−1 A∗

Proof. i) rank(A) = rank(D) = rank(D† ) = rank(A† ) as X, U are invertible.

ii) Case 1: A = D, i.e., X, U are the appropriate identity matrices. Let W = im A,

U = ker D† D, W = span {ei ∈ Sm,1 |Dii 6= 0}
If v ∈ F m×1 , then  
vW = PW (v) = DD† v = D D† v

So D† v is a best approximation to v relative to D. As

U = ker D† D = Span {ej ∈ Sn,1 |Djj = 0}

and we have
D† v ∈ Span {ej ∈ Sn,1 |Djj 6= 0} = Y ⊥ ,
and PY D† v = 0.

D† v is optimal approximation to v relative to D

Case 2: A = XDU ∗ in general. X, U are unitary, so they preserve dot products, so

z is an optimal approximation to v relative to A = AU U ∗ if and only if U ∗ z is an
optimal approximation to v relative to AU (*). We also have

kAz − vk = kXDU ∗ z − vk = kX ∗ (XDU ∗ z − v) k

= kDU ∗ z − X ∗ vk

So (*) is true iff U ∗ z is an optimal approximation to X ∗ v relative to D. By case 1,

D† X ∗ v is an optimal approximation to X ∗ v relative to D. As A† = U D† X ∗
     
SVD
D D† X ∗ v = (X ∗ AU ) D† X ∗ v = X ∗ A A† v

Therefore, A† v is the optimal approximation to X ∗ v relative to X ∗ A. Thus, as X ∗ is

an isometry, A† v is the optimal approximation to v relative to A.

iii) This follows as in (ii) for if rank(A) = n, then (A∗ A)−1 A∗ v is the unique optimal
best approximation to Az = v.

Warning: In general, (AB)† 6= B † A† .

Let A ∈ F m×n , F = R or C. Solve

AX = B for X ∈ F n×1

for X ∈ F n×1 . As A can be inconsistent, we want an optimal approximation to a

solution.

118
Duc Vu (Spring 2021) 29 Lec 29: Jun 4, 2021

Example 29.1
Let F = R or C. Given data (x1 , y1 ), . . . , (xn , yn ) in F 2 , find the best line relative to
this data, i.e., find
y = λx + b, λ = slope
Let    
x1 1   y1
 .. ..  , λ
A= . X= , Y =  ... 
 
. b
xn 1 yn
To solve AX = Y , we want the optimal solution
   
x1 1   y1
 .. ..  λ  .. 
 . . = . 
b
xn 1 yn

Let W = im A. To find the optimal approximation to AX = YW , X = A† Y works.

But rank(A) = 2 is most probable

X = (A∗ A)−1 A∗ Y

§29.2 R ay l e i g h Q u o t i e nt
Let F = R or C, A ∈ Mn F . The euclidean norm of A is defined by

kAvk
kAk := max
06=v∈F n×1 kvk

If A ∈ Mn F is hermitian, then the Rayleigh Quotient of A

R(v) = RA (v) : F n×1 \ {0} → R

is defined by
hAv, vi
R(v) :=
kvk2
Rayleigh quotients are used to approximate eigenvalues of hermitian A ∈ Mn F .

Theorem 29.2
Let F = R or C, A ∈ Mn F hermitian. Then,

i) maxv6=0 R(v) is the largest eigenvalue of A.

ii) minv6=0 R(v) is the smallest eigenvalue of A.

Proof. By the Spectral Theorem, ∃ an orthonormal basis {v1 , . . . , vn } of eigenvectors for A

with Avi = λvi , i = 1, . . . , n. We may assume

λ1 ≥ . . . ≥ λn ∈ R

119
Duc Vu (Spring 2021) 29 Lec 29: Jun 4, 2021

Pn
i) Let v ∈ F n×1 and v = i=1 αi vi , αi ∈ F , i = 1, . . . , n. Then
n n
hAv, vi X X
R(v) = 2
=h αi λ i vi , αj vj i/kvk2
kvk
i=1 j=1
Pn Pn
i,j=1 i i j ij i j i
hv
λ α α δ , v 2
i=1 λi − |αi |
= =
kvk2 kvk2
By the Pythagorean Theorem
n
X
|αi |2 = kvk2
i=1

So Pn 2
i=1 λ1 |αi | λ1 kvk2
R(v) ≤ = = λ1
kvk2 kvk2
ii) Prove similarly.

Corollary 29.3
Let F = R or C, A ∈ Mn F . Then kAk < ∞. Moreover, if µ is the largest singular
value of A, then
kAk = µ

Proof. Consider:
kAvk2 hAv, Avi hA∗ Av, vi
0≤ = =
kvk2 kvk2 kvk2
for all v 6= 0. Since A∗ A is non-negative, the result follows.

We know that the singular value of A ∈ F m×n are the same as for A∗ ∈ F n×m if F = R or
C. Therefore,

Corollary 29.4
Let A ∈ GLn F, F = R or C, µ the smallest singular value of A. Then
1
kA−1 k = √
µ

Proof. If B ∈ GLn F has an eigenvalue λ 6= 0, 0 6= v ∈ EB (λ), then

1
Bv = λv, so v = B −1 v
λ
Hence if
µ1 ≥ . . . ≥ µn > 0
are the singular values of A,
µn ≥ . . . ≥ µ1 > 0
are the singular values of A−1 as (A−1 )∗ A−1 = (AA∗ )−1 .

120
 Duc Vu (Spring 2021) 30 Additional Materials: Jun 04, 2021

§30 Additional Materials: Jun 04, 2021

§30.1 C o n d i t i o n a l N u mb e r
Let F = R or C, A ∈ GLn F , b 6= 0 in F n×1 . Suppose Ax = b.

Problem 30.1. What happens if we modify x a bit, i.e., by δx ∈ F n×1 . Then we get a
new equation
A(x + δx) = b + δb, δb ∈ F n×1
and we would like to understand the variance in b.

Since A is linear,
A(x + δx) = b + A(δx)
i.e.
A(δx) = δb or δx = A−1 (δb)
and we know, therefore, that

kbk = kAxk ≤ kAk · kxk

kδk = kA−1 (δb)k = kA−1 k · kδbk

Therefore,

1 kAk
≤ as kxk =
6 0 (b 6= 0)
kxk kbk
kδxk kA−1 kkδbk kAk kδbk
=⇒ ≤ · = kAkkA−1 k
kxk 1 kbk kbk

Similarly,
1 kδbk kδxk
−1
≤
kAkkA k kbk kxk
We call the number kAkkA−1 k the Conditional Number of A and denote it cond(A).

Theorem 30.1
Let F = R or C, A ∈ GLn F , b 6= 0 in F n×1 . Then
kδbk kδxk
1. 1
cond(A) kbk ≤ kxk ≤ cond(A) kδbk
kbk

2. Let µ1 ≥ . . . ≥ µr > 0 be the singular values of A. Then

µ1
cond(A) =
µn

Proof. 1. from the computation above.

2. follows over computation on the Rayleigh function.

121
Duc Vu (Spring 2021) 30 Additional Materials: Jun 04, 2021

Remark 30.2. From the theorem,

1. If cond(A) is close to one, then a small relative error in b forces a small relative error in
x.
2. If cond(A) is large, even a small relative error in x may cause a relatively large error in
b.

Remark 30.3. If there is an error SA of A, things would get more complicated. For example,
A + δA may no longer be invertible.

There exist conditions that can control this. For example, if A + SA ∈ GLn F , F = R or C,
it is true that
kδxk kδAk
≤ cond(A)
kx + δxk kAk
One almost never computes cond(A), as error arises trying to compute it as we need to
compute the singular values. However, in some cases, remarkable estimates can be found.

§30.2 Mini-Max
Let F = R or C, A ∈ Mn F . We want a method to compute its eigenvalues if A is hermitian.
Since A is hermitian, by the Spectral Theorem,
 
λ1 0
U ∗ AU = 
 ..  , U ∈ Un F

.
0 λn
where A = [A]Sn,1 .
B = {v1 , . . . , vn } is an ordered orthonormal basis of eigenvectors for V = F n×1 satisfying
Avi = λi vi
So
vi = the ith column of U ∗
We let the order be s.t.
λ1 ≥ . . . ≥ λn
As (F v1 )⊥ is A-invariant, A (F v1 )⊥
has maximum eigenvalue λ2 obtained from v2 , i.e.,

max RA (x) = λn−1

x∈(F v1 )⊥

is obtained from x = v2 . The constraint is

hx, v1 i = 0
We can obtain λn−1 without knowing v1 or λ1 . Let x ∈ V be constrained by hx, zi = 0,
some z 6= 0. Let y = U ∗ x. Then hx, zi = 0 is equivalent to hy, wi = 0 where w = U z.
Computation shows the Rayleigh quotient RU for U satisfies
max
y
RU (y) ≤ λn
hy,wi=0

y
max RU (y) ≥ λn−1
hy,wi=0

122
Duc Vu (Spring 2021) 30 Additional Materials: Jun 04, 2021

So
min max
y
RU (y) ≥ λn−1
w6=0
hy,wi=0

gives an upper and lower bound for RU (y). Let

 
  y1
y1 y2 
 
 .. 
y =  . , ỹ =  0 
 
 .. 
yn .
0

with hỹ, wi = 0. In addition, computation shows,

RU (ỹ) = λ2

Let w = e1 . Then
max RU (y) = λ2
y
hy,e1 i

So
min max
y
RU (y) = λ2
w6=0
hy,wi=0

and
min max
y
RU (y) = λ3
w1 ,w2 6=0
hy,w1 i=0
hy,w2 i=0

Proceed inductively.

Theorem 30.4 (Minimax Principle)

Let F = R or C, A ∈ Mn F hermitian with eigenvalues

λ1 ≥ . . . ≥ λn

Then
min max RA (x) = λk
z1 ,...,zk 6=0 hx,z1 i=0
..
.
hx,zk i=0

Remark 30.5. The minimax principle is also formulated by

min max RA (x) = λn−j , j = 1, . . . , n

Vj x∈Vj

where Vj denotes an arbitrary subspace of dim j.

§30.3 U n i q u e n e s s o f S m i t h N o r m a l Fo r m
Consult Professor Elman’s notes.

123