I

roses and P W lens in

I
I

I
1 . - 'q 1

E
u E
|
I

I n `\. lr. *
| l.
J
| .
| h |
\l \.
J 5 l,..,. -
'u 5 I ,. 1

near Algebra

John M. Erdman
Exercises and Problems in

Linear Algebra
This page intentionally left blank
 Exercises and Problems in
Linear Algebra

John M. Erdman
Portland State University, USA

World Scientific
NEW JERSEY • LONDON • SINGAPORE . BEIJING . SHANGHAI • HONG KONG • TAIPEI . CHENNAI . TOKYO
Published by
World Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
USA 0/jice: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601
UK 0/§€ce.° 57 Shelton Street, Covent Garden, London WC2H 9HE

Library of Congress Cataloging-in-Publication Data

Names: Erdman, John M., 1935- author.
Title: Exercises and problems in linear algebra / John M. Erdman, Portland State University.
Description: New Jersey : World Scientific, [2021] | Includes bibliographical references and index.
Identifiers: LCCN 2020038857 (print) | LCCN 2020038858 (ebook) |
ISBN 9789811220401 (hardcover) | ISBN 9789811221071 (paperback) |
ISBN 9789811220418 (ebook) I ISBN 9789811220425 (ebook other)
Subjects: LCSH: Algebras, Linear--Problems, exercises, etc.
Classification: LCC QA179 .E73 2021 (print) | LCC QA179 (ebook) | DDC 512/.5--dc23
LC record available at https://lccn.loc.gov/2020038857
LC ebook record available at https://lccn.loc.gov/2020038858

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library.

Copyright © 2021 by World Scientific Publishing Co. Pte. Ltd.

All rights reserved. This book, or parts thereof may not be reproduced in any form or by any means,
electronic or mechanical, including photocopying, recording or any information storage and retrieval
system now known or to be invented, without written permission from the publisher

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance
Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy
is not required from the publisher.

For any available suppl momentary material, please visit

https://www.worldscicntific.com/worldscibooks/10.l 142/1 l 830#t=suppl

Desk Editor: Soh Jing Wen

Typeset by Stallion Press

Email: [email protected]

Printed in Singapore
 CONTENTS

Preface xi

Part 1. Matrices and Linear Equations 1

Chapter l. Arithmetic of Matrices 3

l.l. Background . . . . . . . . . . . 3
l.2.Exercises................. 4
l.3.Problems................. 7
1.4. Answers to Odd Numbered Exercises . . l0

Chapter 2. Elementary Matrices, Determinants II

2.1. Background II
2.2.Exercises................... 14
2.3.Problems.................. 21
2.4. Answers to Odd Numbered Exercises . . . 22

Chapter 3. Vector Geometry in R" 25

3.1. Background . . . . . . . . . . . . 25
3.2.Exercises................... 26
3.3.Problems................. 28
3.4. Answers to Odd Numbered Exercises . . . 29

V
vi Exercises and Problems in Linear Algebra

Part 2. Vector Spaces 31

Chapter 4. Vector Spaces 33

4. l . Background 33
4.2 .Exercises..................... 34
4.3 .Problems................... 38
44 Answers to Odd Numbered Exercises . . . . . 38

Chapter 5. Subspaces 39
5. l. Background 39
5.2.Exercises........................ 41
5 . 3 . P r o b l e m s . . . . . . . . . . . . . . . . . . . . . . . 46
5 4 Answers to Odd Numbered Exercises . . . . . . . . 47

Chapter 6. Linear Independence 49

6.1. Background . . . . . . . . . . . . 49
6.2.Exercises.................. 50
6.3.Problems................. 53
6.4. Answers to Odd Numbered Exercises . . 54

Chapter 7. Basis for a Vector Space 55

7.1. Background 55
7.2. Exerclses . .. . .. . 56
7.3.Problems.................. 57
7.4. Answers to Odd Numbered Exercises . . . 57

Part 3. Linear Maps Between Vector Spaces 59

Chapter 8. Linearity 61
8. l. Background 61
8.2.Exercises....................... 63
8 . 3 . P r o b l e m s . . . . . . . . . . . . . . . . . . . . . 68
8 4 Answers to Odd Numbered Exercises . . . . . . 73

Chapter 9. Linear Maps Between Euclidean Spaces 75

9.1. Background . . . 75
9.2. Exercises . . . . 76
 Contents vii

Problems 79
Answers to Odd-Numbered Exercises . . . . . 80

Chapter 10. Projection Operators 83

l0.l.Background 83
10.2.Exercises.................... 84
l0.3.Problems 85
10.4. Answers to Odd Numbered Exercises . . . . 86

Part 4. Spectral Theory of Vector Spaces 87

Chapter I I . Eigenvalues and Eigenvectors 89

ll.l.Background.................. 89
11.2. Exerclses . . . . . . . . 90
ll.3.Problems 91
11.4. Answers to Odd-Numbered Exercises . . . . 92

Chapter 12. Diagonalization of Matrices 93

l2.l.Background 93
12.2. Exerclses . . . . . . . . 95
l2.3.Problems................... 98
12.4. Answers to Odd Numbered Exercises . . . . 99

Chapter 13. Spectral Theorem for Vector Spaces 101

l3.l.Background................ 101
l3.2.Exercises.................. 102
l3.3.Problem.................. 105
13.4. Answers to Odd-Numbered Exercises . . 105

Chapter 14. Some Applications of the Spectral Theorem 107

l4.1.Background.................. 107
l4.2.Exercises.................... 108
l4.3.Problems 112
14.4. Answers to Odd-Numbered Exercises . . . . 114
viii Exercises and Problems in Linear Algebra

Chapter 15. Every Operator is Diagonalizable Plus Nilpotent 115

l5.1.Background.................. 115
l5.2.Exercises.................... 116
l5.3.Problems 122
15.4. Answers to Odd-Numbered Exercises . . . . 123

Part 5. The Geometry of Inner Product Spaces 125

Chapter 16. Complex Arithmetic 127

l6.l.Background............... 127
l6.2.Exercises.................. 127
l6.3.Problems 129
16.4. Answers to Odd-Numbered Exercises . . 129

Chapter 17. Real and Complex Inner Product Spaces 131

l7.1.Background.................. 131
l7.2.Exercises.................... 133
l7.3.Problems 136
17.4. Answers to Odd-Numbered Exercises . . . . 137

Chapter 18. Orthonormal Sets of Vectors 139

l8.l.Background............... 139
l8.2.Exercises.................. 140
l8.3.Problems 142
18.4. Answers to Odd-Numbered Exercises . . 143

Chapter 19. Quadratic Forms 145

l9.l.Background 145
l9.2.Exercises.................... 146
19.3.Problem.................... 147
19.4. Answers to Odd Numbered Exercises . . . . 148

Chapter 20 Optimization 149

20.1. Background 149
202 E x e r c i s e s . . . . . . . . . . . . . . . . . 149
 Contents ix

20.3.Prob1ems 151
20.4. Answers to Odd-Numbered Exercises . . . . . . . . .. 151

Part 6. Adjoint Operators 153

Chapter 21. Adjoints and Transposes 155

21.1. Background . . . . . . . . . . . . 155
21.2. Exerclses . . . . .. 156
21.3.Prob1ems................ .. 157
21.4. Answers to Odd Numbered Exercises . . . 159

Chapter 22. The Four Fundamental Subspaces 161

22.l.Background.................. 161
22.2. Exerclses . . . . . . . . 162
22.3.Prob1ems 167
22.4. Answers to Odd-Numbered Exercises . . . . 171

Chapter 23. Orthogonal Projections 173

23.1.Background............... 173
23.2.Exercises.................. 174
23.3.Prob1ems 177
23.4. Answers to Odd-Numbered Exercises . . 178

Chapter 24. Least Squares Approximation 181

24.l.Background.................. 181
24.2. Exercises . . . . . . . . 181
24.3.Prob1ems 183
24.4. Answers to Odd-Numbered Exercises . . . . 183

Part 7. Spectral Theory of Inner Product Spaces 185

Chapter 25. Spectral Theorem for Real Inner Product Spaces 187
25.1. Background.................. 187
25.2. Exercises................... 188
25.3. P r o b l e m . . . . . . . . . . . . . . . . . . . . 190
25.4. Answers to the Odd-Numbered Exercises . . 190
X Eazereises and Problems in Linear Algebra

Chapter 26. Spectral Theorem for Complex Inner Product Spaces 191
26.1. Background................ 191
26.2. Exercises.................. 192
26.3. Problems 196
26.4. Answers to Odd-Numbered Exercises . . 196

Bibliography 199

Index 201
 PREFACE

This collection of exercises and problems may be used in many ways. I used
it as a source of discussion topics in an activity-oriented course in linear
algebra. The discussions were based principally on student presentations of
solutions to the items included in this study guide and on the difficulties
they encountered while trying to solve them. There were no assigned texts.
Students were free to choose their own sources of information and were
encouraged to consult books, papers, and websites whose writing style they
found congenial, whose emphasis matched their interests, and whose prices
Ht their budgets.
The exercises, I hope, will be both interesting and helpful to an average
student. Some are fairly routine calculations, while others require serious
thought. The easy-to-grade fill-in-the-blank format makes them suitable for
instructors to use in quizzes and assigned homework. The answers, given
for all odd-numbered exercises, should be useful for self-directed learners.
Most of the problems were designed to illustrate some issues that I
regard as needing emphasis. Some of them are somewhat open-ended and
are suitable for projects in classes where groups of students work together.
For those instructors who favor the lecture method or are forced into it
by having classes so large that class discussions or group projects are only
remote fantasies, the problems herein may provide excellent topics for pre-
sentation. As an added convenience for instructors the LaTeX code for all
the exercises and problems will be made available on the World Scientific
website. This should be helpful since some of these items are complicated
and/or lengthy and would be a nuisance to retype for use on exams or
homework assignments.

xi
xii Exercises and Problems in Linear Algebra

In each chapter there is a short introductory background section that

lists the topics on which the following exercises and problems are based.
Since notation and terminology in linear algebra are not entirely uniform
throughout the literature, these background sections are intended to Hx
notation and provide "official" definitions and statements of important the-
orems for the material that follows. It seems clear that students in a class
communicate better if they are encouraged to use agreed-upon notations
and terminology.
One problem that I find with many introductory linear algebra texts
is their conflation of material on vector spaces with that on inner product
spaces. The subjects differ in both notation and terminology. Terms such as
"direct sum" and "projection" mean one thing in one context and something
quite different in the other. In the present study guide I try to keep the
algebraic business about vector spaces (where words like "length", "angle" ,
"distance", and "perpendicular" make no sense) separate from the more
geometric material on inner products. In this study guide vector spaces
are dealt with in parts 2-4, while inner product spaces are the subject of
parts 5-7.
Among the dozens and dozens of linear algebra texts that have appeared,
two that were written before "dumbing down" of textbooks became fash-
ionable are especially notable, in my opinion, for the clarity of their authors'
mathematical vision: Paul Halmos's Finite-Dimensional Vector Spaces [4]
and Hoffman and Kunze's Linear Algebra 161. Some students, especially
mathematically inclined ones, love these books, but others find them chal-
lenging. For students trying seriously to learn the subject, I recommend
examining them. For those more interested in applications, both Elemen-
tary Linear Algebra: Applications Version 111 by Howard Anton and Chris
Rorres and Linear Algebra and its Applications 191 by Gilbert Strang are
loaded with applications. Students who find the level of many of the cur-
rent beginning linear algebra texts depressingly pedestrian and the endless
routine computations irritating, may enjoy reading some more advanced
texts. Two excellent ones are Steven Roman's Advanced Linear Algebra 181
and William C. Brown's A Second Coarse in Linear Algebra 121.
Concerning the material in these notes, I make no claims of original-
ity. While I have dreamed up many of the items included here, there are
many others which are standard linear algebra exercises that can be traced
back, in one form or another, through generations of linear algebra texts,
 Preface xiii

making any serious attempt at proper attribution quite futile. If anyone

feels slighted, please contact me.
There will surely be errors. I will be delighted to receive corrections,
suggestions, or criticism at

[email protected]
This page intentionally left blank
 Part 1

MATRICES AND LINEAR EQUATIONS

This page intentionally left blank
 Chapter 1

ARITI-Il\/IETIC OF la/IATRICES

1.1 Background

Topics: addition, scalar multiplication, multiplication of matrices, inverse

of a nonsingular matrix, Gaussian elimination.

Definition 1.1.1.Two square matrices A and B of the same size are said
to COMMUTE if AB = BA.

Definition 1.1.2. If A and B are square matrices of the same size, then
the COMMUTATOR (or LIE BRACKET) of A and B, denoted by [A, B], is
defined by

[A»B] = A B - B A .

Notation 1.1.3. If A is an m X n matrix (that is, a matrix with m rows and

n columns), then the element in the it row and the nth column is denoted
'al
by (bit . The matrix A itself may be denoted by [ a ] _1j=1 or, more simply,
'

by [did]. In light of this notation it is reasonable to refer to the index i in the

expression (L as the ROW INDEX and to call j the COLUMN INDEX. When
we speak of the "value of a matrix A at (i, j)," we mean the entry in the
th
row and nth column of A. Thus, for example,

1 4
3
A
7
5 01

is a 4 X 2 matrix and a31 = 7.

3
4 Exercises and Problems in Linear Algebra

Notation 1.1.4. The n o n identity matrix is denoted by In or, more

frequently, just by I.

Definition 1.1.5. A matrix A = [Um] is UPPER TRIANGULAR if (L =0

whenever i > j .

Definition 1.1.6. The TRACE of a square matrix A, denoted by tr A, is

the sum of the diagonal entries of the matrix. That is, if A = [Ge] is an
n X n matrix, then
'al

t1°A
II U/jj
j 1

Definition 1.1.7. The TRANSPOSE of an m X n matrix A = [ a ] is the

matrix At = [ a ] obtained by interchanging the rows and columns of A.
The matrix A is SYMMETRIC if At = A.

Proposition 1.1.8. If A is an m X 77, matrix and B is an 77, X p matrix,

then (A8g = BtAt.

1.2 Exercises

1 0 2 1 2
0 3 -1 3 1
(1) Let A 7 B 7 and C
3 0 2
1 1101 2 4 1
3 0

(a) Does the matrix D ABC exist? If so, then $34

(b) Does the matrix E BAC exist? If so, then 622

(c) Does the matrix F BCA exist? If so, then f43

(d) Does the matrix G ACB exist? If so, then 931

(e) Does the matrix H CAB exist? If SO, then h21

(f) Does the matrix J CBA exist? If SO, then .j13

 Arithmetic of Matrices 5

[
1
0 am and C = AB. Evaluate the following.

(a) A37 (b) B63

IcI B138 (d) C42

Note: If M is a matrix My is the product of p copies of M.

(3) Let A
1 1
,J
31 . Find numbers C and d such that A2 = -I.

Answer: c = and d =
(4) Let A and B be symmetric n X n-matrices. Then [A,B] = [B,X],
where X = .
(5) Let A, B, and C be 77, X n matrices. Then [A, B]C+ B[A, C] = [X, Y],
where X = and Y = .
1/3
(6) Let A A
Find numbers c and d such that A2 = 0. Answer:
C and d
1 3 2
(7) Consider the matrix a 6 2 where a is a real number.
0 9 5

(a) For what value of a will a row interchange be required during

Gaussian elimination? Answer: a = .
(b) For what value of a is the matrix singular? Answer: a =

1 0 -1 2 1 2

(8) Let A
0
2
3 1 -1
4 0 3 7
B: 3
0
-1
-2 7 C l3 2 0 5
1 7

- 3 1 - 1 2 4 1
and M i 3A3 - 5(Bc)2. Then W14 and
W41
6 Exercises and Problems in Linear Algebra

kg) If A is an n X n matrix and it satisfies the equation A3 -

4A2 + 3A - 51" 0, then A is nonsingular and its inverse is

(10) Let A, B, and C be n X n matrices. Then HA78]7C] -|- HB, CLA] -I-

HC, AL B] = X, where X

(II) Let A, B, and C be n X 77, matrices. Then [A, C] -|- [B, C] = [X, Y],
where X = and Y = .

1 0 0 0
1
4 1 0 0
(12) Find the inverse of 1 1 . Answer:
3 3
1 0
1 1 1
2 2 2
1

(13) Suppose that A and B are symmetric n X n matrices. In this exercise

we prove that AB is symmetric if and only if A commutes with B.
Below are portions of the proof. Fill in the missing steps and the
missing reasons. Choose reasons from the following list.

(Hl) Hypothesis that A and B are symmetric.

($2) Hypothesis that AB is symmetric.
($3) Hypothesis that A commutes with B.
(Dll Definition of commutes.
(D2l Definition of symmetric.
(T) Proposition 1.1.8.

Proof. Suppose that AB is symmetric. Then

AB (reason: (H2) and

BW (reason: )
(reason: (D2) and

So A commutes with B (reason: .

 Arithmetic of Matrices 7

Conversely, suppose that A commutes with B. Then

(A83t i
(reason: (T))
= BA (reason: and
(reason: and
Thus AB is symmetric (reason:

1.3 Problems

(1) Let A be a square matrix. Prove that if A2 is invertible, then so is A.

Hint. Our assumption is that there exists a matrix B such that
A28 = BAR = I _
We want to show that there exists a matrix C such that
AC i CA i I.
Now to start with, you ought to find it fairly easy to show that there
are matrices L and R such that
LA = AR = I . (*)
A matrix L is a LEFT INVERSE of the matrix A if LA = I; and R is
a RIGHT INVERSE of A if AR = I. Thus the problem boils down to
determining whether A can have a left inverse and a right inverse that
are deferent. (Clearly, if it turns out that they must be the same, then
the C we are seeking is their common value.) So try to prove that if
(*) holds, then L = R.

(2) Anton speaks French and German; Geraldine speaks English, French
and Italian, James speaks English, Italian, and Spanish, Lauren speaks
all the languages the others speak except French, and no one speaks
any other language. Make a matrix A = [ a ] with rows representing
the four people mentioned and columns representing the languages they
speak. Put all = 1 if person i speaks language j and as = 0 otherwise.
Explain the significance of the matrices AAt and AtA.

(3) Portland Fast Foods (PFF), which produces 138 food products all made
from 87 basic ingredients, wants to set up a simple data structure from
8 Exercises and Problems in Linear Algebra

which they can quickly extract answers to the following questions:

(a) How many ingredients does a given product contain?
(b) A given pair of ingredients are used together in how many products?
(c) How many ingredients do two given products have in common?
(d) In how many products is a given ingredient used?
In particular, PFF wants to set up a single table in such a way that:
(i) the answer to any of the above questions can be extracted easily
and quickly (matrix arithmetic permitted, of course); and
(ii) if one of the 87 ingredients is added to or deleted from a product,
only a single entry in the table needs to be changed.
Is this possible? Explain.

(4) Prove Proposition 1.1.8.

(5) Let A and B be 2 X 2 matrices.

(a) Prove that if the trace of A is 0, then A2 is a scalar multiple of the
identity matrix.
(b) Prove that the square of the commutator of A and B commutes
with every 2 X 2 matrix C. Hint. What can you say about the trace
of [A, B1?
(c) Prove that the commutator of A and B can never be a nonzero
multiple of the identity matrix.
(6) The matrices that represent rotations of the any-plane are

A(0)
cost el
- sin
sin 0 cos 0 .

(a) Let X be the vector (-1, 1), 0 = 371/4, and y be A(H) acting on X
(that is, y = A(@lxtl. Make a sketch showing x, y, and 0.
(b) Verify that A(01)A(02) = A(01 -|- 92). Discuss what this means
geometrically.
(c) What is the product of A(0) times A(-9)? Discuss what this means
geometrically.
(d) Two sheets of graph paper are attached at the origin and rotated
in such a way that the point (1, 0) on the upper sheet lies directly
over the point (-5/13, 12/13) on the lower sheet. What point on
the lower sheet lies directly below (6, 4) on the upper one?
 Arithmetic of Matrices 9

(Y) Let
0 a 02
as a.l

0 0 a a2 as
G2

<c
0 0 0 a

II
0 0 0 0 a
0 0 0 0 0
The goal of this problem is to develop a "calculus" for the matrix A. To
start, recall (or look up) the power series expansion for l 1 I . Now see
if this formula works for the 77, X m matrix A by first computing (1-A)-1
directly and then computing the power series expansion substituting A
for as. (Explain why there are no convergence difficulties for the series
when we use this particular matrix A.) Next try to define ln(I + A ) and
eA by means of appropriate series. Do you get what you expect when
you compute en(I+A)? Do formulas like eAeA = e2A hold? What about
other familiar properties of the exponential and logarithmic functions?
Try some trigonometry with A. Use series to define sin, cos, tan,
arctan, and so on. Do things like tan(arctan(A)) produce the expected
results? Check some of the more obvious trigonometric identities.
(What do you get for sin2 A + cos2 A - I? Is cos(2A) the same as
cos2 A - sin2 A?)
A relationship between the exponential and trigonometric functions is
given by the famous formula eia: = cos x -|- i sin cc. Does this hold for A?
Do you think there are other matrices for which the same results
might hold? Which ones?
(8) (a) Give an example of two symmetric matrices whose product is not
symmetric.
Hint. Matrices containing only 0's and l's will suffice.
(b) Now suppose that A and B are symmetric n X n matrices. Prove
that AB is symmetric if and only if A commutes with B.
Hint. To prove that a statement P holds "if and only if" a state-
ment Q holds you must first show that P implies Q and then show
that Q implies P. In the current problem, there are 4 conditions to be
considered:
(i) At = A (A is symmetric),
(ii) Bt = B (B is symmetric),
(iii) (A8g = AB (AB is symmetric), and
(iv) AB = BA (A commutes with B).
10 Eazercises and Problems in Linear Algebra

Recall also the fact given in

(v) Proposition 1.1.8.
The first task is to derive (iv) from (i), (ii), (iii), and (v). Then try to
derive (iii) from (i), (ii), (iv), and (v).

1.4 Answers to Odd-Numbered Exercises

(l) (a) Yes, 142

(b) no, -
(c) yes, -45
(d) no, -
(e) yes, -37
lf no, -
(3) -6, -1
(5) A, BC
(Y) (a) 2
(b) -4

(9) EW - PA + 31%)
(II) A -|- B, C
(13) (AB);, D2, T , BA, H1, D1, BtAt, H1, D2, AB, H3, D1, D2 (Note: the
order of H1 and D2 and the order of H3 and D1 may be reversed.)
 Chapter 2

ELEMENTARY la/IATRICES;
DETERMINANTS

2.1 Background

Topics: elementary (reduction) matrices, determinants, Gauss-Jordan

reduction.
The following definition says that we often regard the effect of multi-
plying a matrix M on the left by another matrix A as the action of A
on M .

Definition 2.1.1. We say that the matrix A ACTS ON the matrix M

0 1
to produce the matrix N if N = AM. For example the matrix
l 0 [ 1
acts on any 2 X 2 matrix by interchanging (swapping) its rows because

{ H 1
0 l Cl b
l 0 C d
C

{
Cl
d
b 1
Definition 2.1.2. We will say that an operation (sometimes called scaling)
that multiplies a row of a matrix (or an equation) by a nonzero con-
stant is a ROW OPERATION OF TYPE I. An operation (sometimes called
swapping) that interchanges two rows of a matrix (or two equations) is a
ROW OPERATION OF TYPE II. And an operation (sometimes called pivoting)
that adds a multiple of one row of a matrix to another row (or adds a mul-
tiple of one equation to another) is a ROW OPERATION OF TYPE III.

11
12 Eazercises and Problems in Linear Algebra

Notation 2.1.3. We adopt the following notation for elementary matrices

that implement type I row operations. Let A be a matrix having n rows.
For any real number T go 0 denote by Mj(r) the n X n matrix that acts on
A by multiplying its jth row by r. (See exercise 1.)

Notation 2.1.4. We use the following notation for elementary matrices

that implement type II row operations. (See Definition 2.l.2.) Let A be a
matrix having n rows. Denote by P the n X n matrix that acts on A by
interchanging its ith and jth rows. (See exercise 2.)

Notation 2.1.5. And we use the following notation for elementary matri-
ces that implement type III row operations. (See Definition 2.l.2.) Let A
be a matrix having n rows. For any real number r denote by E (T) the
n X n matrix that acts on A by adding T' times the jth row of A to the i t h
row. (See exercise 3.)

Definition 2.1.6. If a matrix B can be produced from a matrix A

by a sequence of elementary row operations, then A and B are ROW
EQUIVALENT.

Some Facts about Determinants

Proposition 2.1.7. Let n 6 N and M n x n be the collection of all n X n

matrices. There is exactly one function

det : M nxn ->R:A»->detA

which satisfies

(a) det In = 1.
(b) If A 6 Mnxn and A' is the matrix obtained by interchanging two rows
of A, then det A' = - det A.
(c) If A 6 la/Inxn, c 6 R , and A' is the matrix obtained by multiplying each
element in one row of A by the number e, then det A' = edet A.
ld) IfA 6 M n o n , e 6 R , and A' is the matrix obtained from A by multi-
plying one row of A by e and adding it to another row of A (that is,
choose i and j between 1 and n with i go j and replace ask by ask -l-eaik
for 1 < k _< n), then det A' = det A.

Definition 2.1.8. The unique function det : M n x n -> R described above

is the n X n DETERMINANT FUNCTION.
 Elementary Matrices; Determinants 13

Proposition 2.1.9. If A = [a] for a 6 R (that is, if A 6 M1X 1), then

detA = a; i A 6 2X27 then

det A = a11a22 - 012021.

Proposition 2.1.10. I f A , B 6 M 'nX'r17 then det(AB) = (det A)(det B) .

Proposition 2.1.11. If A 6 M n x n , then det At = detA. (An obvious

corollary of this: in conditions (b), (c), and (d) of Proposition 2.1.7 the
word "columns " may be substituted for the word "rows".)

Definition 2.1.12. Let A be an n X n matrix. The MINOR of the element

ask, denoted by Mjk, is the determinant of the (n - 1) X (n - 1) matrix
which results from the deletion of the J-to row and kth column of A. The
COFACTOR of the element ask , denoted by Cock is defined by

I-1lJ+Im j e .
Cock :=
Proposition 2.1.13. IfA 6 M n x n and 1 _<j _<n, then
TL

det A I GMC# .
k=1

This is the (LAPLACE) EXPANSION of the determinant along the jth 'I"O'l1}.

In light of 2.1.11, it is clear that expansion along columns works as well

as expansion along rows. That is,
TL

det A tljk(jk
j 1

for any k between 1 and n. This is the (LAPLACE) EXPANSION of the deter-
minant along the nth column.

Proposition 2.1.14. An n X 77, matrix A is 'invertible if and only if

detA go 0. If A is invertible, then

A-1 = (deAl-let

where C = [Cock] is the matrix of eofaetors of elements of A.

Definition 2.1.15. A square matrix is SINGULAR if its determinant is

zero. Otherwise it is NONSINGULAR.
14 Eazercises and Problems in Linear Algebra

2.2 Exercises

(1) Let A be a matrix with 4 rows. The matrix M3(4) that multiplies

the 3rd row of A by 4 is . (See 2.1.3.)

(2) Let A be a matrix with 4 rows. The matrix P24 that interchanges

the 2nd and 4th rows of A is . (See 2.1.4.)

(3) Let A be a matrix with 4 rows. The matrix E23 (-2) that adds -2 times

l 1
l
the 3rd row of A to the 2nd row is • (See 2.1.5.)

(4) Let A be the 4 X 4 elementary matrix E43(-6). Then A11

and A-9

(5) Let B be the elementary 4 X 4 matrix P24. Then B-9

and B10

(6) Let C be the elementary 4 X 4 matrix M g ( - 2 ) . Then C4

and C 3

1 2 3
0 1 1
(7) Let A and B I P23E34(-2lM3l-2lE42ll)P14A.
2 1 0
1 2 -3
Then 623 and 632
 Elementary Matrices; Determinants 15

(8) Elementary row operations are performed on a 4 X 4 matrix A to bring

it to upper triangular form. The result is

1 2 0
0 -1 0 1
P24E43 M3(5)E42(2)E31(1)E21(3)A
0 0 2
0 0 0 10
Then the determinant of A is
(9) The system of equations:

2y+3z=7
x+y-z=-2
-cv-l-y-52:0

is solved by applying Gauss-Jordan reduction to the augmented

0 2 3 7
coefficient matrix A 1 1 -1 -2 Give the names of the
-1 1 -5 0
elementary 3 X 3 matrices Xl, . . . ,X8 that implement the following
reduction.
1 1 1 2 1 1 1 2
X1> X2>
A 0 2 3 7 0 2 3 7
1 1 5 0 0 2 6 2

1 1 1 2 1 1 -1
X3> X4>
0 2 3 7 0 2 3 7
0 0 9 9 0 0 1 1

A
1 1 -1 2 1 1
X5> X6>
0 2 0 4 0 1 0 2
0 0 1 1 0 0 1 1

A A A A
1 1 0 -1 1 0 0 3
X7> X8>
0 1 0 2 0 1 0 2
0 0 1 1 0 0 1 1

Answer: XI = 7 X2 7 X3 7 X4
7 X5 7 X6 7 X7 7

X8
16 Eazercises and Problems in Linear Algebra

(10) Solve the following equation for .CU:

3 7 0 6 2
2 0 1 8 0 0
3 4 8 3 1 2

co
det Answer : as

II
27 6 5 0 0 3
3 33 0 2 1 1
1 0 1 3 4 0

1]
(11) Let A
0
l
0 •

2 4 Find A-1 using the technique of augmenting

0
LE
2 3 l
A by the identity matrix I and performing Gauss-Jordan reduction
on the augmented matrix. The reduction can be accomplished by the
application of five elementary 3 X 3 matrices. Find elementary matrices
Xl, X2, and X3 such that A-1 I X3E13(-3)X2M2(l/2)X1I.
(a) The required matrices are XI PM where 'i X2
' I

E i k ( - 2 ) where j = and k and X3

7
E12 la")
when e
T'

(be And then A-1

1 t t2 t3
t 1 t t2
(12) det to t 1 t l1 - a(t))P where a(t) and

t3 t2 t 1
p= .
(13) Evaluate each of the following determinants.

69 39 49
(21) det
57 32 37 _
34 4 4
11 1 1
1 0 1 1
1-1 2 0
(b) det 2
-1 3 1
4 17 0 -5
 Elementary Matrices; Determinants 17

13 3 8 6
0 0 4 0
(c) det
1 0 7 2
3 0 2 0

5 4 -2 3
5 7 -1 8
(14) Let M be the matrix
5 7 6 10
5 7 1 9
(a) The determinant of M can be expressed as the constant 5 times
3 1 5
the determinant of the single 3 X 3 matrix 3
3
(b) The determinant of this 3 X 3 matrix can be expressed as the
constant 3 times the determinant of the single 2 X 2 matrix
V j>ll

(c) The determinant of this 2 X 2 matrix is

(d) Thus the determinant of M is
1 2 5 7 10
1 2 3 6 7
(15) Find the determinant of the matrix 1 1 3 5 5 • Answer:
1 1 2 4 5
1 1 1 1 1

(16) Find the determinants of the following matrices.

-73 78 24 -73 78 24
<c

92 66 25 and B 92 66 25
II

-80 37 10 -80 37 10.01

Hint. Use a calculator (thoughtfully). Answer: det A and

det B = .
(17) Find the determinant of the following matrix.

283 5 or 347.86 X 101583

3136 56 5 cos(2.7402)
6776 121 11 5
2464 44 4 2

Hint. Do not use a calculator. Answer:

18 Eazercises and Problems in Linear Algebra

1 1
0 2 0 2
1 1
0 0 2 2
(18) Let A 1 1
. We find A-1 using elementary row
2
0 2
0
1 1
1 0 2 2

operations to convert the 4 X 8 matrix [A IN] to the matrix

114 A-11.
•

Give the names of the elementary 4 X 4 matrices Xl, . . . , X 1 1 that

implement the following Gauss-Jordan reduction and Iill in the missing
matrix entries.
on-»

1
0 0 1 0 0 0
I
I

1 1
0 0 2 2 0 1 0 0
1 1
2 0 2 0 0 0 1 0
1 1
1 0 2 2 0 0 0

1 1
1 0 2 2

1 1
XI 0 0 2 2
>
1 1
2 0 2 0
1
N II H

0 0
I

1 1
1 0 2 2

1 1
X2 0 0 2 2
>
3 1
0 0 4 4
NJII-\

1
0 0
I
I

1 1
1 0 2 2
NJII-\

1
0 0
I
I

X3 2
>
3 1
0 0 4 4

1 1
0 0 2 2
Elementary Matrices; Determinants 19

1 1
1 0 2 2

X4 0 1 0
>
3 1
0 0 4 4

1 1
_0 0 2 2

1 1
1 0 2 2

X5 0 1 0
1
1 3

1 1
2 2

1 1
2 2

X6 0 1 0
>
1
0 0 1 3

1
_0 0 0 3

1 1
1 0

la
2 2

X7
>
1 0
I
0 0 1 0
1
0 0 0 3

1 1
1 0 2 2

X8 0 1 0
>

0 0 1 0

_0 0 0 1
20 Eazercises and Problems in Linear Algebra

1 1
1 0 2 2

X9 0 1 0 0
>

0 0 1 0

_0 0 0 1

1
1 0 2 0

X10 0 1 0 0
>

0 0 1 0

0 0 0 1

1 0 0 0

X11 0 1 0 0
>
0 0 1 0
0 0 1

Answer: X1 = 7 X2 7 X3 7 X4
7 X5 7 X6 7 X7 7

X9
II

7 X10 7 X11

(19) The matrix

1 1 1
1
2 3 4
1 1 1 1
2 3 4 5
m
II

1 1 1 1
3 4 5 6
1 1 1 1
-4 5 6 7-

is the 4 X 4 HILBERT MATRIX. Use Gauss-Jordan reduction to

compute K = H-1. Then K44 is (exactly) . Now, cre-
ate a new matrix H' by replacing each entry in H by its approx-
Imation to 3 decimal places. (For example, replace E1 by 0.167.)
 Elementary Matrices; Determinants 21

Use Gauss-Jordan reduction again to find the inverse K' of H'. Then
KQ4 is .
(20) Suppose that A is a square matrix with determinant 7. Then
(a) det(P24A) = .
(b) d€tlE23l-4lAl =
(C) det(m3(2)A) 1

2.3 Problems

(1) For this problem assume that we know the following: If X is an m X m

matrix, if Y is an m X n matrix and if 0 and I are zero and identity
matrices of appropriate sizes, then det
Let A be an m X n matrix and B be an n
X
0
X
11
Y
= det X.
m matrix. Prove carefully
that

det det AB .

0 I
Hint. Consider the product
B I ] [B "] I •

(2) Let A and B be n X n-matrices. Your good friend Fred R. Dirndl believes
that

det E in det(A + B) det(A - B).

He offers the following argument to support this claim:

det E in det(A2 - BQ)

det[(A -|- B)(A - B)]

det(A + B) det(A - B).

(a) Comment (helpfully) on his "proof". In particular, explain care-

fully why each of the three steps in his "proof" is correct or incor-
rect. (That is, provide a proof or a counterexample to each step.)
(b) Is the result he is trying to prove actually true?
22 Eazercises and Problems in Linear Algebra

Hint: Consider the product

I
0 A
B
BH
(3) Let J: be a fixed real number that is not an integer multiple of 7T. For
A -|- B
0
0
I ]
each natural number n let ATL = [ask] be the n X n-matrix defined by

0, for lj -kl > 1

Clack 1, for lj - / | 1
2 cos so, for j = k.

-|- 1 .CU
Show that det An = sin ( n . Hint. For each integer n let Dn
sln .CU
det ATL and prove that

Dn+2 - 2DTL+1 cosec + Dn = 0.

(Use mathematical induction.)

2.4 Answers to Odd-Numbered Exercises

1 0 0 0
0 1 0 0
(l) 0 0 4 0
0 0 0 1
1 0 0 0
0 1 0
(3) 0 0 1 0
0 0 0 1

1 0 0 0 1 0 0 0
0 0 0 1 0 1 0 0
(5) 0 0 1 0
7
0 0 1 0
0 1 0 0 0 0 0 1
(Y) -8, -1

(9) P12, E31(1), E32(-1), m g ( - 5 ) , E23(-3)» my(%), E13(1)» E12(-ll

(11) (a) 3, 2, 3, -2
r 1 _1

(be
1 0 OJ
 Elementary Matrices; Determinants 23

(13) 100, 0, -72

(15) -10
(17) 6
(10) 2800, -1329.909
This page intentionally left blank
 Chapter 3

n
VECTOR GEOMETRY IN

3.1 Background

Topics: inner (dot) products, cross products, lines and planes in 3-space,
norm of a vector, angle between vectors, the law of sines.
Notation 3.1.1. There are many more or less standard notations for the
inner product (or dot product) of two vectors X and y. The two that we
will use interchangeably in these exercises are X y and (x, y).

Definition 3.1.2. If X is a vector in Rn then the NORM (or LENGTH) of

7

X is defined by

IIxII i V//<X7X)°

Definition 3.1.3. Let X and y be nonzero vectors in Rn. Then &(x, y),
the ANGLE between X and y, is defined by

arccos
(x, y)
<(x, y)
IIxII llyll
Theorem 3.1.4 (Cauchy-Schwarz inequality) . I x and y CI/I"€ vectors
in R" 7 then

l(x,y)I S IIxII llyll~

(We will often refer to this just as the Schwarz inequality.)
Definition 3.1.5. If X = (a:1,a:2,a:3l and y (y1,y2,y3) are vectors in
R3, then their CROSS PRQDUCT, denoted by X X y, is the vector (:l22y3 -

33392, $391 - 33193, l31Y2 - Sliz Yll-

25
26 Eazercises and Problems in Linear Algebra

3.2 Exercises

(1) The angle between the vectors (1,0, -l,3) and (1, \/§, 3, -go in R4 is
air where Cl •i

(2) Find the angle H between the vectors X (3,-1, 1,0,2,1) and y
(2, -1,0, \/5, 2, 1) in R6. Answer: H =
kg) If CL17 ,an > 0, then

( ) I"
TL

j 1
a J-
k=1
as
) > 112.

The proof of this is obvious from the Cauchy-Schwarz inequality when

we choose the vectors X and y as follows:
X and y = .
(4) Find all real numbers a such that the angle between the vectors 2i -I-
2j -|- (Q - 2)k and Zi -|- (Q - Qlj -|- 2k is g. Answer: a and

(5) Which of the angles (if any) of triangle ABC, with A = (1, -2,0),
B = (2, 1, -2), and C = (6, -1, -3), is a right angle? Answer: the
angle at vertex
(6) The hydrogen atoms of a methane molecule (CH4) are located at
(0,0,0), (1,1,0), (0,1,1), and (1,0,1) while the carbon atom is at
8
(5, 5, Find the cosine of the angle H between two rays starting
at the carbon atom and going to different hydrogen atoms.
Answer: casH =
(7) If a, b, c, d, e, f G R , then

lad+be+cfl S \ a2-l-62-l-c2 \ d2 -I- 62 -l- f2.

The proof of this inequality is obvious since this is just the
Cauchy-Schwarz inequality where an = l , , ) and y =
( 7 7 )~
(8) The volume of the parallelepiped generated by the three vectors i -I-
2j-k,j+k,and3i-j-l-2kis •
 Vector Geometry in R" 27

(9) The equations of the line containing the points (3, -1, 4) and (7, 9, 10)
are
at - 3 _ y- j _ - k
Z
2 - b - c
where b i 7 C i 7 i
7 and k i •

(10) The equations of the line containing the points (5, 2, -1) and (9, -4, 1)
are
.CU h _y-2 Z k
a - -3 c
where Cl ,C 7 h = ands 7

(11) Find the equations of the line containing the point (l, 0, -1) that is
zz:-4 _ 2 y - 3 _ 3 z - 7
parallel to the line
h 2 - 51 ` 6 '
Answer:
33 y J
b =
Z+
4 where a = b
CL 7 h 7

, and _7 = .
(12) The equation of the plane containing the points (0, -1, 1), (1, 0, 2), and
(3,0,1)isa:+by+cz=dwhereb= ,C ,andd= .
(13) The equation of the plane which passes through the points (0, -1, -1 7

(5,0,1),and (4,-1,0)is as:-l-by+cz= 1 wheres= ,b= 7

and C i •

(14) The angle between the planes 41: + 4 2 - 16 = 0 and -2x -|- 2y - 13 = 0
is n' where a = and b =

(15) Suppose that u e R3 is a vector that lies in the first quadrant of the
my-plane and has length 3 and that v e R3 is a vector that lies along
the positive z-axis and has length 5. Then
(21) 1111 X VII i ;
(b) the as-coordinate of u X v is 0 (choose <, >, or =);
(c) the y-coordinate of u X v is 0 (choose <, >, or =); and
(d) the z-coordinate of u X v is 0 (choose <, >, or =).
(16) Suppose that u and v are vectors in a vector space both of length 2v'2
and that the length of u - v is also QI. Then llu + vll =
and the angle between u and v is
1 1
(17) If a = 3 i + 4 j + l 2 k , b = 3 i + 4 j - l 2 k , a n d c = i 12 j +- 16
k 7 then
c ,a X b .
28 Eazercises and Problems in Linear Algebra

3.3 Problems

(1) Show that if a, b, c > 0, then go + Tb + c) 2 < as + §b2 + £@2

(2) Show that if al, . 7 ana w 1 , . . . , w n > 0 and 2kz1 wk = 1, then
2 in
<_
(~ )
2
Clkwk Cly W k .
k 1 k=1

(3) Prove that if (a1,a2, . . .) is a sequence of real numbers such that the
OO OO
2 . 1
series cl converges, then the serles k cl converges absolutely.
k=1 k=1
You may find the following steps helpful in organizing your solution.
(i) First of all, make sure that you recall the difference between a
OO

sequence of numbers (el, C2 7 . . .) and an infinite series co,

k 1
(ii) The key to this problem is an important theorem from third term
Calculus:

A nondecreasing sequence of real numbers converges

if and only if it is bounded. (*)

(Make sure that you know the meanings of all the terms used
here.)
(iii) The hypothesis of the result we are trying to prove is that the
OO

series as:2 converges. What, exactly, does this mean?

'al
2
(iv) For eakczhl natural number n let in as Rephrase (iii) in
k 1
terms of the sequence (b'fl).
(v) Is the sequence (b'fl) nondecreasing?
(vi) What, then, does I say about the sequence (b,,)?
(viii For each natural number n let Cn =
n 1
p. What do we know Z
k=1
about the sequence (eTL) from third term Calculus? What does
(*) say about the sequence (en)?
OO
1
(viii) The conclusion we are trying to prove is that the series Clk
converges absolutely. What does this mean? k 1k
 Vector Geometry in R" 29

(ix) For each natural number n let s'fl =

n 1
-l0kl~
k
Z
Rephrase (viii) in
terms of the sequence (sTL). k=1
(x) Explain how for each n we may regard the number sTL as the dot
product of two vectors in R" .
(xi) Apply the Cauchy-Schwarz inequality to the dot product in (x) .
Use (vi) and (vii) to establish that the sequence (sTL) is bounded
above.
(xii) Use (*) one last time keeping in mind what you said in (ix) .
(4) Let a and b be vectors in R3. Without using the components of a and
b prove that Ha X b 112 = uau2llbu2 - (a,b)2 •

(5) Let a, b, and C be vectors in IR3.

(a) Prove that if a + b + c = 0, then a x b = b X C o C X a.
(b) Use part (a) to prove the law of sines.

3.4 Answers to Odd-Numbered Exercises

(l)

(3) w V5-2» • 7
W)
1
7 7

(5) B
(7) a, b, c, d, e, f
(9) 5, 3, -1, 4
(11) 4, 5, 1, 0
(13) 1, 3, -4
(15) (a) 15
(be >
(c) <
(d) 1

(17) -18
This page intentionally left blank
 Part 2

VECTOR SPACES
This page intentionally left blank
 Chapter 4

VECTOR SPACES

4.1 Background

Topics: real and complex vector spaces, vectors, scalars.

CAUTION. In the following definition IF may be taken to be an arbitrary

field. In this study guide, however, we will deal exclusively with two cases
only, II" = R (the field of real numbers) and IF = (C (the field of complex
numbers, which, if you are not already familiar with it, is introduced in
Chapter 16).

Definition 4.1.1. A VECTOR SPACE is a set V together with operations

of addition and scalar multiplication that satisfy the following axioms:

(1) i x , y e V, thenx-I-y e V;
(2) (x + y) + z = X -|- (y + z) for every x, y, z e V (associativity) ;
(3) there exists 0 e V such that X -|- 0 = X for every X e V (existence of
additive identity) ;
(4) for every X e V there exists -x e V such that x-I-(-x) 0 (existence
of additive inverses);
(5) X -|- y = y -|- X for every x, y e V (commutativity);
(6) i f d € I F " a n d x € V , t h e n a x € V ;
(7) a(x -|- y) = dx + ay for every a e IF and every x, y e V;
(8) (a + B)X = ax -|- Bx for every a, B e IF and every X e V ;
(9) (aB)X = d(BX) for every a, B e IF and every X e V; and
(10) 1 X = X for every X e V.

33
34 Eazercises and Problems in Linear Algebra

When IF = R we speak of V as a REAL VECTOR SPACE and when IF (I

we call it a COMPLEX VECTOR SPACE.

4.2 Exercises

(1) Let V be the set of all real numbers. Define an operation of "addition"
by
.CU EEI y = the maximum of Hz: and y
for all My y 6 V. Define an operation of "scalar multiplication" by
a .CU Z ax
fora11a€Randa:EV.
oder the operations and the set is not a vector space. he
vector space axioms (see 4.1.1 (1)-(l0)) that fail to hold are 7

7 , and .
(2) Let V be the set of all real numbers 33 such that ;U> 0. Define an
operation of "addition" by
as EEI y = :cy -|- l
for all My y 6 V. Define an operation of "scalar multiplication" by
a as Z 02;c

for all a € lR and a: E V.

oder the operations and the set (is/is not) a
vector space. If it is not, list all the vector space axioms (see 4.1.1
(1)-(l0)) that fail to hold. Answer: The axioms that are not satisfied
are

(3) Let V be lli2.2, the set of all ordered pairs (33, y) of real numbers. Define
an operation of "addition" by
('u,,'u)EEI(a:,y) = (u+zl3+ 1, ' u + y + l )
for all (u, u) and (az, y) in V. Define an operation of "scalar multiplica-
tion" by
a (co, y) Z (acc, a y )
for all a 6 R and (so, y) 6 V.
oder the operations and the set is not a vector space. he
vector space axioms (see 4.1.1 (1)-(l0)) that fail to hold are
and .
 Vector Spaces 35

(4) Let V be R2, the set of all ordered pairs (33, y) of real numbers. Define
an operation of "addition" by

Cu»vl EEI Qr,y) (u + ' v 0)

for all (u, u) and (az, y) in V. Define an operation of "scalar multiplica-
tion" by
a (co, y) Z (acc, a y )

for all a 6 R and (so, y) 6 V.

oder the operations and the set is not a vector space. he
vector space axioms (see 4.1.1 (1)-(l0)) that fail to hold are 7

, and
(5) Let V be the set of all n X n matrices of real numbers. Define an
operation of "addition" by
l
A EEI B Z -lAB + BA)
2
for all A, B 6 V. Define an operation of "scalar multiplication" by
a 0

for a11 a € R and A € V.

oder the operations and the set is not a vector space. he
vector space axioms (see 4.1.1 (1)-(l0)) that fail to hold are 7

, and .
(6) Below are portions of proofs of four results about vector spaces which
establish the fact that multiplying a vector Hz: by the scalar -l produces
-as, the additive inverse of az. Fill in the missing steps and the missing
reasons. Choose reasons from the following list.

( H) Hypothesis
(l)-(10) Vector space axioms, see 4.1.1
(PA) Proposition A
(PB) Proposition B
(PC) Proposition C
(RN) Property of the Real Numbers

Proposition 4.2.1 (A). A vector Hz: in a vector space V has at most one
additive inverse. That is, if y and Z are vectors in V such that X -|- y = 0
andx-I-z = 0, t h e n y = z .
36 Eazercises and Problems in Linear Algebra

Proof. Suppose that X + y = 0 and X + Z 0. Then

y (reason: )
y+(x+zl (reason: )
(reason: (2))
(x+yl+ Z (reason:
(reason: ( HII
(reason: (5))
(reason: .
N
II

Proposition 4.2.2 (B). If X E V where V is a vector space and X -|- X = X7

then X = 0.

Proof. If X €Vandx-|- X x, then

X :X-|- 0 (reason:
(reason: (4))
(x + x) + (-x) (reason:
(reason: ( HII
0 (reason: l.
Proposition 4.2.3 (C). If an is a vector in a vector space V, then Ox = 0.

Proof. If X Q V , then
Ox = (0 -|- 0) X (reason:
(reason: (8))
Thus Ox = 0 (reason:

Proposition 4.2.4 (D). If at is a vector in a vector space V, then (-1)x

is -x, the additive inverse of X.

Proof. If X 6 V, then
x + (-1) (reason: (10))
= (l -|- (-l)) - x (reason: )
= 0•X (reason: )
0 (reason: >.
It then follows immediately from that ( - l ) - x X.
 Vector Spaces 37

(7) In this exercise we prove that multiplying the zero vector by an arbi-
trary scalar produces the zero vector. For each step of the proof give
the appropriate reason. Choose reasons from the following list.

(1)-(10) Vector space axioms 4.1.1.

(PB) Proposition 4.2.2
(RN) Property of the Real Numbers

Proposition 4.2.5 (E). If 0 is the zero vector in a vector space and a is

a scalar, then a - 0Z0.

Proof. Let 0 be the zero vector of some vector space. Then for every
scalar a

a 0 a ( 0 + )0 (reason: )
a - 0 -I-0~ 0 (reason: )

It then follows immediately from that a 0 0.

(8) In this exercise we prove that the product of a scalar and a vector is
zero if and only if either the scalar or the vector is zero. After each
step of the proof give the appropriate reason. Choose reasons from the
following list.

(H) Hypothesis.
(l)-(10) Vector space axioms 4.1.1.
(PC), (PE) Propositions 4.2.3 and 4.2.5, respectively.
(RN) Property of the Real Numbers.

Proposition 4.2.6. Suppose that X is a vector and a is a scalar. Then

ax = 0 if and only if a = 0 o r = 0.

Proof. We have already shown in and that if a 0

or X = 0, then ax = 0.
To prove the converse we suppose that ax = 0 and that or 96 0; and
we prove that X = 0. This conclusion results from the following easy
38 Eazercises and Problems in Linear Algebra

calculation:
-x (reason:
\

x (reasons: and

l
(0~ x) (reason:
a
l
0 (reason:
a
0 (reason: .

4.3 Problems

(1) Prove that if V is a vector space, then its additive identity is unique.
That is, show that if 0 and 0 are vectors in V such that X -I- 0 = X for
a 1 1 a : € V a n d x - l - 0 = X for all J: 6 V, then 0 0.
(2) Let V be the set of all real numbers 33 such that .CU > 0. Define an
operation of "addition" by
as y= ivy

for all My y 6 V. Define an operation of "scalar multiplication" by

a
a as 33

fora11a€Randa:EV.
Prove that under the operations and the set is a vector space.
(3) With the usual operations of addition and scalar multiplication the set
of all n X n matrices of real numbers is a vector space: in particular,
all the vector space axioms (see 4.1.1 (1)-(l0)) are satisfied. Explain
clearly why the set of all nonsingular n X n matrices of real numbers is
not a vector space under these same operations.

4.4 Answers to Odd-Numbered Exercises

(1) 3, 4, 7, 8
(3) 7, 8
(5) 2, 4, 10
(7) 3, 7, PB
 Chapter 5

SUBSPACES

5.1 Background

Topics: subspaces of a vector space

Definition 5.1.1. A nonempty subset of M of a vector space V is a

SUBSPACE of V if it is closed under addition and scalar multiplication.
(That is: if X and y belong to M, so does X + y; and if X belongs to M and
a e R, then ax belongs to M . )

Notation 5.1.2. We use the notation M 5 V to indicate that M is a

subspace of a vector space V.

Notation 5.1.3. Here are some frequently encountered families of

functions:

jE' = .7:[a, b] = {f : f is a real valued function on the interval [a, b]}

(5.l.l)
P 77[a,b] = {p: P is a polynomial function on [a, b]} (5.l.2)

7D4 7>4[a, be = {p 6 77: the degree of p is less than 4} (51.3)

QUO Q4[CL, 6] = {p e 77: the degree of p is equal to 4} (51.4)

C C[a,b] Z {f e .F : f is continuous} (51.5)
D D[a, 6] Z {f e .F : f is differentiable} (51.6)
IC lC[a,b] = { f Q F = f is a constant function} (5.l.7)

39
40 Eazercises and Problems in Linear Algebra

B B[a, 6] = {f 6 F: f is bounded} (51.8)

.7 = 7[a,b] = {f 6 I; f is integrable} (51.9)

(A function f 6 .F is BOUNDED if there exists a number M > 0 such that

lf(a:)l g M for all as in [0, b]. It is (RIEMANN) INTEGRABLE if it is bounded
and Tb f(a3) do exists.)

Definition 5.1.4. If A and B are subsets of a vector space then the SUM
of A and B, denoted by A + B, is defined by

A-I-B:: {a+ b : a 6 A and b 6 B}.

Definition 5.1.5. Let M and N be subspaces of a vector space V. If

M VW N = {0} and M + N = V, then V is the (INTERNAL) DIRECT SUM of
M and N. In this case we write

V M ® N.

In this case the subspaces M and N are COMPLEMENTARY and each is the
COMPLEMENT of the other.

Definition 5.1.6. Let V and W be vector spaces. If addition and scalar

multiplication are defined on the Cartesian product V X W by

(u/w) + (Hay) := (u + a:,w + y)

and

a('u,wl := ( l a w )

for all u, x 6 V, all w, y 6 W, and all a 6 IE", then V X W becomes a

vector space. (This is called the PRODUCT or (EXTERNAL) DIRECT SUM of
V and W. It is frequently denoted by V ® W.)

Notice that the same notation is used for internal and external direct
sums. Some authors make a notational distinction by using ®¢ and @€ (or
® and Egg to denote these two concepts. Others regard it as somewhat
pedantic on the grounds that context should make it clear whether or not
the summands are currently being regarded as subspaces of some larger
vector space. The matter becomes even worse when, in inner product spaces,
we encounter 'orthogonal direct sums' (see Definition 18.1.6 in Chapter 18).
 Subspaces 41

Definition 5.1.7. Let M be a subspace of a vector space V. Define an

equivalence relation on V by

;Ur\J y if and only if y LUEM.

For each x 6 V let [as] be the equivalence class containing x. Let V/M be
the set of all equivalence classes of elements of V. For [as] and [y] in V/M
define

[13] + [y] :z [as + y]

and for a 6 R and [as] 6 V/M define

0[a:' := [0;c].

Under these operations V/M becomes a vector space. It is the QUOTIENT

SPACE of V by M. The notation V/M is usually read "V mod M " .

Note that the preceding "definition" contains many statements of fact.

These must be verified. (See Problem 6.)

5.2 Exercises

(1) One of the following is a subspace of R3. Which one?

The set of points (az, y, z) in R3 such that
(a) a z - I - Z y - 3 z = 4 .
(be x 2 1 = y =
(c) a z - I - y + z = 0 a n d : - y - I - z = l.
(dl a : = - z a n d a : = z .
(e) 1:2 -|- y2 = z.
(f) % Z y53.

Answer: l ) is a subspace of R3.

(2) The smallest subspace of R3 containing the vectors (2, -3, -3) and
(0, 3, 2) is the plane whose equation is aa3+by+6z = 0 where a = ,
and b = .
(3) The smallest subspace of R3 containing the vectors (0, -3,6) and
(0, l , -2) is the line whose equations are an = a and Z by where
Cl , and b = .
42 Eazercises and Problems in Linear Algebra

(4) Let ROO denote the vector space of all sequences of real numbers.
(Addition and scalar multiplication are defined coordinatewise.) In
each of the following a subset of ROO is described. Write yes if the set
is a subspace of ROO and no if it is not.

(a) Sequences that have infinitely many zeros (for example,

( l , l , 0 , l , l , 0 , l , l , 0 , . . . ) ) . Answer: .
(b) Sequences that are eventually zero. (A sequence (azk) is eventually
zero if there is an index No such that be'fl = 0 whenever n > f i g . )
Answer: .
(c) Sequences that are absolutely summable. (A sequence (wk) is abso-
lutely summable if Ek=1lmkl < oo.) Answer: .
(d) Bounded sequences. (A sequence (act) is bounded if there is a pos-
itive number M such that l5UI=cl < M for every Le.) Answer: .
(e) Decreasing sequences. (A sequence (wk) is decreasing if a:ni+1 < In
for each n.) Answer: .
(f) Convergent sequences. Answer: .
(g) Arithmetic progressions. (A sequence (act) is arithmetic if it is
of the form (a,a + k , a + 2k,a + 3k, . . .) for some constant je.)
Answer: .
(h) Geometric progressions. (A sequence (ivkl is geometric if it is of
the form (a, kg, k2a, k3a, . . .l for some constant ac.) Answer: .
(5) Let M and N be subspaces of a vector space V. Consider the following
subsets of V.

(a) M VW N. (A vector V belongs to M O N if it belongs to both M

and N.)
(b) M U N. (A vector V belongs to M U N if it belongs to either M
or N.)
(c) M -|- N. (A vector V belongs to M -|- N if there are vectors m e M
and n e N such that V = In -|- n.)
(d) M - N. (A vector V belongs to M - N if there are vectors m e M
and n e N such that V = In - n.)

Which of (a)-(d) are subspaces of V?

Answer:
 Subspaces 43

(6) For a fixed interval [a, b], which sets of functions in the list 5.1.3 are
vector subspaces of which?
Answer:
YI

YI

YI

YI

YI

YI

YI
(7) Let M be the plane ; z : + y + z = 0 and N be the line x = y = z in
R3. The purpose of this exercise is to confirm that R3 = M ® N. This
requires establishing three things: (i) M and N are subspaces of R3
(which is very easy and which we omit); (ii) R3 = M + N; and (iii)
M a n = {0}
(a) To show that R3 = M + N we need R3 Q M + N and M + N Q fig.
Since M C $3 and N C R3, it is clear that M + N C R3. So all that
is required is to show that R3 C M + N. That is, given a vector
X = (a31,a32,;c3) in R3 we must find vectors m = (m1,m2,m3) in
M and n = (n1,n2,n3) in N such that X = m + n. Find two such
vectors.
l
Answer: m ( 7 7

3 )
l
and n 3 ( 7 7

l.
(b) The last thing to verify is that M O N = {0}; that is, that the
only vector M and N have in common is the zero vector. Suppose
that a vector X = (;c1,;c2,a:3) belongs to both M and N. Since
X e M it must satisfy the equation

£61-l'£U2-l'£C3=0. (l)

Since 336 N it must satisfy the equations

1131 SEQ and (2)

£182 S133- (3)

Solve the system of equations (1)-(3).

Answer: X = l , , ).
(8) Let C = C[-l, l] be the vector space of all continuous real valued
functions on the interval 1-1, l]. A function f in C is EVEN if f(-as) =
f(a:) for all as e 1-1, l]; it is ODD if f (-zzz) = -f(a:) for all as e 1-1, l].
44 Eazercises and Problems in Linear Algebra

Let CO = {f 6 C: f is odd} and Ce = {f 6 C: f is even}. To show

that C = CO ® Ce we need to show 3 things.
(i) CO and Ce are subspaces of C. This is quite simple: let's do just one
part of the proof. We will show that CO is closed under addition.
After each step of the following proof indicate the justification
for that step. Make your choices from the following list.

(Al Arithmetic of real numbers.

(DA) Definition of addition of functions.
(DE) Definition of "even function" .
(DO) Definition of "odd function" .
( H) Hypothesis (that is, our assumptions or suppositions).

Proof. Let f , g 6 Co. Then

(f + go(-SUI Z f(-12) + g(-Lu) (reason:

= - f ( a : ) -|- (-g(a:)) (reason: and
= -(f(a:) -|- g(a:)) (reason:
= - ( f + g)(a:). (reason:

Thus f + g 6 Co. (reason l.

(ii) CO FW Ce = { 0 } (where 0 is the constant function on [-l, l] whose
value is zero). Again choose from the reasons listed in part (i) to
justify the given proof.

Proof. Suppose f 6 CO VW Ce. Then for each x e [-l, l]

f(;c) = f(-as) (reason: )

= -f(a:). (reason: )

Thus f(a:) = 0 for every Hz: e [-1, l]; that is, f 0. (reason:
).

(iii) C = CO -|- Ce- To verify this we must show that every continuous
function f on [-1, l] can be written as the sum of an odd function
j and an even function k. It turns out that the functions j and k
can be written as linear combinations of the given function f and
 Subspaces 45

the function g defined by g(a:) = f(-at) for - l < as < l. What

are the appropriate coefficients?

Answer: .7 g
k g.

(9) Let M be the line .CU = y= Z and N be the line .CU y= 2 in R3.

(a) The line M is the set of all scalar multiples of the vector (l, 7

.
(b) The line N is the set of all scalar multiples of the vector 1, 7

.
(c) The set M + N is (geometrically speaking) a in R3; its
equation is as: -|- by -|- Z = 0 where a = and b

(10) Let M be the plane a s - y - I - z = 0 and N b e the plane a i - I - Z y - z = 0

in R3. State in one short sentence how you know that R3 is not the
direct sum of M and N.
Answer:

( I I ) Let M be the plane Za: - 3y+4z-|- l = 0 and N be the line x4 - 2Y - 3 Z

in R3. State in one short sentence how you know that R3 is not the
direct sum of M and N .
Answer :

(12) Let M be the plane a:+y+z = 0 and N be the line ac- l = y = z + 2

in R3. State in one short sentence how you know that R3 is not the
direct sum of M and N .
Answer :
x
(13) Let M be the line as = y = Z and N be the line -
4
= = § in R3. g
State in one short sentence how you know that R3 is not the direct
sum of M and N.
Answer:

(14) Let M be the plane J: -|- y + Z = 0 and N be the line x = - y = 32.

The purpose of this exercise is to see (in two different ways) that R3
is not the direct sum of M and N.
46 Eazercises and Problems in Linear Algebra

(a) If R3 were equal to M ® N, then M O N would contain only the

zero vector. Show that this is not the case by finding a nonzero
vector X in R3 that belongs to M O N.
Answer: X = l , , II.
(b) If R3 were equal to M ® N, then, in particular, we would have
R3 = M -|- N. Since both M and N are subsets of R3, it is clear
that M -|- N C R3. Show that the reverse inclusion R3 C M -|- N
is not correct by finding a vector X e R3 that cannot be written
in the form Ill -|- n where Ill e M and n e N.
Answer: X = (-6, 8, a) is such a vector provided that a go .
(c) We have seen in part (b) that M -l-N ye R3. Then what is M4-N?
Answer: M -|- N= .

5.3 Problems

(1) Let M and N be subspaces of a vector space V. Consider the following

subsets of V .

(a) M VW N. (A vector V belongs to M O N if it belongs to both M

and n.)
(b) M U N. (A vector V belongs to M U N if it belongs to either M
or N.l
(c) M -|- N. (A vector V belongs to M + N if there are vectors 111 e M
and n e N such that V = In -|- n.)
(d) M - N. (A vector V belongs to M - N if there are vectors 111 e M
and n e N such that V = in - n.)
For each of the sets (a)-(d) above, either prove that it is a subspace
of V or give a counterexample to show that it need not be a subspace
of V .
(2) Let C = C[0, l] be the family of continuous real valued functions on the
interval [0, l]. Define

f1(tl=t and f2(i)=t4

 Subspaces 47

for 0 < t < l. Let M be the set of all functions of the form all -|- W2
where a, B e R. And let N be the set of all functions g in C that satisfy
1 1
tg(t) it = 0 and t49(t) dt = 0.
0 0

Is C the direct sum of M and N? (Give a careful proof of your claim and
illustrate it with an example. What does your result say, for instance,
about the function h defined by h(t) = t2 for 0 < t < l.)
(3) Let V be a vector space.
(a) Let .M be a family of subspaces of V. Prove that the intersection
H M of this family is itself a subspace of V.
(b) Let A be a set of vectors in V. Explain carefully why it makes sense
to say that the intersection of the family of all subspaces containing
A is "the smallest subspace of V that contains A" .
(c) Prove that, for A as in part (b), the set of all sums of scalar multiples
of vectors in A is the smallest subspace of V that contains A.

up-»
1
(4) In R3 let M be the line as = y = z, N be the line an

<2
AZ,&Hd
II
I
L = M -|- N. Give a careful proof that L = M ® N.
(5) Let V be a vector space and suppose that V = M ® N. Show that
for every V e V there exist unique vectors Ill e M and n e N such
that V = Ill -|- n. Hint. It should be clear that the only thing you have
to establish is the uniqueness of the vectors Ill and n. To this end,
suppose that a vector v in V can be written as 1111 + 111 and it can also
be written as 1112 -|- 112 where 1111, 1112 e M and no, 112 e N. Prove that
1111 = 1112 and 111 = 112.

(6) Verify the assertions made in definition 5.1.7. In particular, show that
is an equivalence relation, that addition and scalar multiplication
of the set of equivalence classes are well defined, and that under these
operations V/M is a vector space.

5.4 Answers to Odd-Numbered Exercises

(l) (dl
(3) 0, -2
48 Eazercises and Problems in Linear Algebra

(5) (al, IcI, and (d)

iv) (3) 2581 - £62 - £1337 -£U1 -|- 21132 - $1337 -.CE1 - .SEQ -1- 251337 £131 -|- £62 -|- 3237
331 +332 -l-$83, £61 +5132 +5$37
(b) 0, 0, 0
(9) (a) 1, l
(b) 2, 3
(c) plane, l, -2
(II) M is not a subspace of R3.
(13) M -|- N is a plane, not all of R3.
 Chapter 6

LINEAR INDEPENDENCE

6.1 Background

Topics: linear combinations, span, linear dependence and independence.

Remark 6.1.1. Some authors of linear algebra texts make it appear as

if the terms linear dependence and linear independence, span, and basis
pertain only to finite sets of vectors. This is extremely misleading. The
expressions should make sense for arbitrary sets. In particular, do not be
misled into believing that a basis for a vector space must be a finite set of
vectors (or a sequence of vectors). While it is true that in most elementary
linear algebra courses the emphasis is on the study of finite dimensional
vector spaces, bases for vector spaces may be very large indeed. I recom-
mend the following definitions.

Definition 6.1.2. Recall that a vector y is a LINEAR COMBINATION

of distinct vectors Xl, . . . 7 X'fl if there exist scalars a l , . . . , a n such that
y = E k z l a k x k . Note: linear combinations are finite sums. The linear com-
bination E k z l Ozkxk is TRIVIAL if all the coefficients a l , . . . , aTL are zero. If
at least one as is different from zero, the linear combination is NONTRIVIAL.

Example 6.1.3. In R2 the vector (8,2) is a linear combination of the

vectors (l, l ) and (l, -1) because (8,2) = 5(l, II -l- 3(l, -1).
Example 6.1.4. In R3 the vector (l, 2, 3) is not a linear combination of
the vectors (l, l,0) and (l, - l , 0 ) .

Definition 6.1.5. Suppose that A is a subset (finite or not) of a vector

space V. The SPAN of A is the set of all linear combinations of elements of A.

49
50 Eazercises and Problems in Linear Algebra

Another way of saying the same thing: the SPAN of A is the smallest sub-
space of V which contains A. (That these characterizations are equivalent
is not completely obvious. Proof is required. See Problem 3 in Chapter 5.)
We denote the span of A by span A. If U = span A, we say that A SPANS U
or that U is SPANNED BY A.

Example 6.1.6. For each n = 0, 1,2, . . . define a function pn on R

by p ,,(a:) in
. Let 77 be the set of polynomial functions on R. It
is a subspace of the vector space of continuous functions on R. Then
7? = span{p0,p1,p2 . . .}. The exponential function exp, whose value at
x is ex, is not in the span of the set { p 07 p 17 p 2 . . .}.

Definition 6.1.7. A subset A (finite or not) of a vector space is LINEARLY

DEPENDENT if the zero vector 0 can be written as a nontrivial linear combi-
nation of elements of A; that is, if there exist distinct vectors x l , . . . , xn e A
and scalars d l , . . . , an, not all zero, such that E k z l a k x k = 0. A subset
of a vector space is LINEARLY INDEPENDENT if it is not linearly dependent.

Technically, it is a set of vectors that is linearly dependent or indepen-

dent. Nevertheless, these terms are frequently used as if they were properties
of the vectors themselves. For instance, if S = { X l 7 . . . ,x,,} is a finite set of
vectors in a vector space, you may see the assertions "the set S is linearly
independent" and "the vectors X l , . . . 7 X'fl are linearly independent" used
interchangeably.

Example 6.1.8. The (vectors going from the origin to) points on the unit
1 \ \
circle in R2 are linearly dependent. Reason: If X = (1, 0), y = / 2 7 2 7

and Z 577 , t h e n + y + ( - l ) z = 0.
Example 6.1.9. For each n = 0, 1,2, . . . define a function pTL on R by
p = x". Then the set {p0, pa, PA7 . . .} is a linearly independent subset
of the vector space of continuous functions on R.

6.2 Exercises

(l) Show that in the space R3 the vectors X = (1, l,0), y = (0, l , 2), and
Z = (3, l, -4) are linearly dependent by finding scalars a and B such

that ax + By + z 0.
Answer: a Z 7 5
 Linear Independence 51

(2) Let (l, ,0,0), X i (1, 0,1,0), y (0,0,l,l),and Z (0,1,0,1).

(a) We can show that {w, x,y,z} is not a spanning set for R4 by
finding a vector u in R4 such that u ¢ span{w, x, y, z}. One such
vector is u = (l, 2, 3, a) where a is any number except .
(b) Show that {w, X 7 Y 7 z} is a linearly dependent set of vectors by
finding scalars Ag 'Y 7 and 6 such that aw -|- X -|- 'by -|- dz = 0.
Answer: a Z ,/"y ,5= .
(c) Show that {w, x, y, z} is a linearly dependent set by writing Z as
a linear combination of w, x, and y. Answer: Z = w -|-
X -|- y.

kg) Let p(a:) = 1:2 -|- 256 - 3, q(;v) = 21:2 - 3:13 + 4, and T°(x) aa:2 l. The
set {p, q, r} is linearly dependent if a = .

(4) Show that in the vector space R3 the vectors X = (l,2, - l ) , y =

(3, l , 1), and z = (5, -5, 7) are linearly dependent by finding scalars a
and B such that ax -|- By + Z 0.
Answer: a 7 5
(5) Let f1(a3) = s i n , f2(a:) = cos(a: -|- 7rr/6), and f3(;z:) = sin(a: - 7r/4) for
0 < as < 2'rr. Show that {f1, f2, f3} is linearly dependent by finding
constants a and B such that all - 2f2 - Bf3 = 0.
Answer: a and 5
(6) In the space C[0,tr] let f , g, h, and j be the vectors defined by

f(=vl l

900) .CU

Mac) COS 33

2 Hz:
CoCcI COS
2

for 0 < x < 7T. Show that f , g, h, and j are linearly dependent by
writing j as a linear combination of f , g, and h.
Answer: j = f -|- g -|- h.
52 Eazercises and Problems in Linear Algebra

(7) Let u = (A,l,0), V ( lA, , )l , and w = (0, l,A). Find all values of
A which make {u, V7 w} a linearly dependent subset of R3. Answer:

(8) Let u = (l,0,-2), V : (l,2,A), and w = ( 2 , l , - l ) . Find all values

of A which make {u, v, w} a linearly dependent subset of R3. Answer:

(9) Let p(a:) = 333-a22+2a:+3, q(;c) = 3x34-51:2 -Hz:-l, My) = x3+2x+2,

and s(a3) = 7333 + 0a:2 -1- 5. The set {p, q, r, s} is linearly dependent if
Cl

(10) In the space C[0, or] define the vectors f , g, and h by

f(=v) 33

9(12l sin x
Mac) COS .CU

for 0 < x < 7T. We show that f , g, and h are linearly independent.
This is accomplished by showing that if of + Hg + 'oh = 0, then
a = B = by = 0. So we start by supposing that if -|- B9 -|- 'yh = 0.7
that is,

as:+Bsina:JI-tycosa: 0 (l)

for all 336 [0, or].

(a) We see that by must be zero by setting 33 in equation (1).
Now differentiate (1) to obtain

a+Bc:osx 0 (2)

for all as 6 [0, 'ii].

(b) We see that a must be zero by setting 33 in equation (2).
Differentiate (2) to obtain

-Bsinas 0 (3)

for all as 6 [0, 'rr1.

(c) We conclude that 5 = 0 by setting .CU in (3)~
 Linear Independence 53

6.3 Problems

(1) In the space C[0, l] define the vectors f , g, and h by

f(=vl Z .CU

900) = 6"

Mac) = 6-x

for 0 < as < l . Use the definition of linear independence to show that
the functions f , g, and h are linearly independent.
(2) Let a, b, and e be distinct real numbers. Use the definition of linear
independence to give a careful proof that the vectors (l, l , l), (a, b, c),
and (a2, 62, c2) form a linearly independent subset of R3.
(3) Let {u, v, w} be a linearly independent set in a vector space V. Use
the definition of linear independence to give a careful proof that the
set {u -|- v, u -|- w, V + w} is linearly independent in V.
(4) You are the leader of an engineering group in the company you work
for and have a routine computation that has to be done repeatedly. At
your disposal is an intern, Kim, a beginning high school student, who is
bright but has had no advanced mathematics. In particular, Kim knows
nothing about vectors or matrices.
Here is the computation that is needed. Three vectors, a, b, and C are
specified in R5. (Denote their span by M.) Also specified is a (sometimes
long) list of other vectors S = { V I 7 V27 . . . ,v,,} in R5. The problem is
to
(l) determine which of the vectors in S belong to M, and
(2) for each vector Vi e S which does belong to M
find constants QUO 5, and 'Y such that Vi = as + 5 b + 7 C.
Kim has access to Computer Algebra System (Maple, or a similar pro-
gram) with a Linear Algebra package. Write a simple and efficient algo-
rithm (that is, a set of instructions) which will allow Kim to carry out
the desired computation repeatedly. The algorithm should be simple in
the sense that it uses only the most basic linear algebra commands (for
example, Matrix, Vector, Transpose, RowRedacedEchelonForm, etc. in
Maple). Remember, you must tell Kim everything: how to set up the
appropriate matrices, what operations to perform on them, and how to
interpret the results. The algorithm should be as efficient as you can
54 Eazercises and Problems in Linear Algebra

make it. For example, it would certainly not be efficient for Kim to
retype the coordinates of a, b, and C for each new Vi .
Include in your write-up an actual printout showing how your algo-
rithm works in some special case of your own invention. (For this exam-
ple, the set S need contain only 5 or 6 vectors, some in U, some not.)

(5) The point of this problem is not just to get a correct answer to (a)-(c)
below using tools you may have learned elsewhere, but to give a care-
ful explanation of how to apply the linear algebra techniques you have
already encountered to solve this problem in a systematic fashion. For
background you may wish to read a bit about networks and Kirchho# 's
laws (see, for example, [5] Topic: Analyzing Networks, pages 72-77 or
111 Electrical Networks, pages 538-542).
Consider an electrical network having four nodes A, B, C, and
D connected by six branches 1, . . . ,6. Branch 1 connects A and B,
branch 2 connects B and D; branch 3 connects C and B; branch 4
connects C and D; branch 5 connects A and C, and branch 6 connects
A and D.
The current in branch k is In, where k = 1, . . . , 6. There is a 17 volt
battery in branch 1 producing the current 11 that flows from A to B.
In branches 2, 4, and 5 there are 0.5 ohm resistors, and in branches 1,
3, and 6 there are 1 ohm resistors.
(a) Find the current in each branch. (Explain any minus signs that
occur in your answer.)
(b) Find the voltage drop across each branch.
(c) Let pn be the potential at node n = A, B, C, D. The voltage drop
across the branch connecting node j to node k is the difference in
the potentials at nodes j and k. Suppose the network is grounded
at D (so that PD = 0). Find the potential at the other nodes.

6.4 Answers to Odd-Numbered Exercises

(1) -3, 2

(3) 7
(5) W- 1, W
iv) -/5, 0, \/5
(9) -3
 Chapter 7

BASIS FOR A VECTOR SPACE

7.1 Background

Topics: basis, dimension.

Definition 7.1.1. A set B (finite or not) of vectors in a vector space V

is a BASIS for V if it is linearly independent and spans V.
Example 7.1.2. The vectors e1 = (l,0,0), e2 = (0, l,0), and e3 = (0,0, l )
constitute a basis for the vector space R3.
Example 7.1.3. More generally, consider the vector space R" of all
n-tuples of real numbers. For each natural number k between l and n
let ek be the vector that is l in the nth-coordinate and 0 in all the others.
Then the set {e1,e2, . . . ,e"} is a basis for R". It is called the STANDARD
BAsis for R" .
Example 7.1.4. For each n 0, 1,2, . . . define a function pTL on R by
p = as". Then the set {p0, p 1 , p 27 . . .} is a basis for the vector space 7?
of polynomial functions on R.
Two important facts of linear algebra are that regardless of the size of
the space every vector space has a basis and that every subspace has a
complement.

Theorem 7.1.5. Let B be a linearly independent set of vectors in a vector

space V . Then there exists a set C of vectors in V such that B U C is a basis
for V .

Corollary 7.1.6. Every vector space has a basis.

55
56 Eazercises and Problems in Linear Algebra

Corollary 7.1.7. Let V be a vector space. If M -< V , then there exists

N -< V such that M ® N = V .

The next theorem says that any two bases for a vector space are the
same size.

Theorem 7.1.8. If B and C are bases for the same vector space, then
there is a one-to-one correspondence from B onto C.

Definition 7.1.9. A vector space V is FINITE DIMENSIONAL if it has a

finite basis. Its DIMENSION (denoted by dim V) is the number of elements
in the basis. If V does not have a finite basis it is INFINITE DIMENSIONAL.

Remark 7.1.10. Although most elementary texts contain proofs of Theo-

rem 7.1.5 (or its corollaries) and Theorem 7.1.8 for finite dimensional spaces,
it is difficult to find in them proofs that hold for infinite dimensional spaces.
The reason for this is that such proofs require a set theoretic axiom called
Zo1°n's Lemma, which many instructors feel is a topic not appropriate for
beginning courses. (See the discussion in 171, p. 238, ,gal

Theorem 7.1.11. If M and N are #nite dimensional subspaces of a vector

space, then M -|- N is #nite dimensional and

dim(]M -I-N) = diM+dimN-dim(MVWNl.

7.2 Exercises

(l) Let u = (2,0,-1), V = (3,1,0), and w = ( l , - l , c ) where C 6 R. The

set u , v , w} is a basis for R3 provided that c is not equal to .
(2) Let u = (1, -l,3), V = (l,0, l), and w = (1, 2, c) where C E R. The set
u 7 V ) w} is a basis for R3 provided that c is not equal to .
(3) The dimension of 9)"l2 X 2 , the vector space of all 2 X 2 matrices of real
numbers is .
(4) The dimension of 32, the vector space of all 2 X 2 matrices of real
numbers with zero trace is .
(5) The dimension of the vector space of all real valued polynomial func-
tions on R of degree 4 or less is .
(6) In R4 let M be the subspace spanned by the vectors (l, l , l,0) and
(0, -4, l,5) and let N be the subspace spanned by (0, -2, l , 2 ) and
 Basis for a Vector Space 57

(l, -l, l,3). One vector that belongs to both M and N is (l, ,
7 ). The dimension of M O N is and the dimension of
M+Nis

7.3 Problems

(l) Exhibit a basis for SJT2 $ 2 , the vector space of all 2 X 2 matrices of real
numbers.
(2) Exhibit a basis for 32, the vector space of all 2 X 2 matrices of real
numbers with zero trace.
(3) Exhibit a basis for 63, the vector space of all symmetric 3 X 3 matrices
of real numbers.

(4) Let II be the set of all matrices of real numbers of the form

and QI be the set of all real matrices of the form Exhibit a

basis for II, for 'IL for II -|- 'IL and for II VW 'II
(5) Prove that the vectors ( l , l , 0 ) , (l,2,3), and ( 2 , - l , 5 ) form a basis
for R3.
(6) Let V be a vector space and A be a linearly independent subset of V.
Prove that A is a basis for V if and only if it is a maximal linearly
independent subset of V. (If A is a linearly independent subset of V we
say that it is a MAXIMAL linearly independent set if the addition of any
vector at all to A will result in a set that is not linearly independent.)
(7) Let V be a vector space and A a subset of V that spans V. Prove that
A is a basis for V if and only if it is a minimal spanning set. (If A is
a set which spans V we say that it is a MINIMAL spanning set if the
removal of any vector at all from A will result in a set which does not
span V.)

7.4 Answers to Odd-Numbered Exercises

(1) -2
(3) 4
(5) 5
This page intentionally left blank
 Part 3

LINEAR MAPS BETWEEN

VECTOR SPACES
This page intentionally left blank
 Chapter 8

LINEARITY

8.1 Background

Topics: linear maps between vector spaces, kernel, nullspace, nullity, range,
rank, isomorphism.

Definition 8.1.1. A function f : A -> B is ONE-TO-ONE (or INJECTIVE)

if 'LL = 'U in A whenever f ('u,l = f(ul in B.

Definition 8.1.2. A function f : A -> B is ONTO (or SURJECTIVE) if for

every b E B there exists a e A such that b = f(a).

Definition 8.1.3. A function f : A -> B is a ONE-T0-ONE CORRESPON-

DENCE (or BIJECTIVE) if it is both injective and surjective (one-to-one and
onto).

Definition 8.1.4. A map T : V -> W between vector spaces is LINEAR if

T(x+y) =Tx+Ty for Allx, y eV (8.l.l)

and

T ( a x ) = aT for all X 6 V and a 6 IF. (8.l.2)

Here IF = R if V and W are real vector spaces and IF = (C if they are

complex vector spaces.
A scalar valued linear map on a vector space V is a LINEAR FUNCTIONAL.

61
62 Eazercises and Problems in Linear Algebra

A linear map is frequently called a LINEAR TRANSFORMATION, and, in

case the domain and codomain are the same, it is often called a (LINEAR)
OPERATOR. The family of all linear transformations from V into W is
denoted by £(V, W). We shorten £(V, V) to £(V).
Two oddities of notation concerning linear transformations deserve com-
ment. First, the value of T at X is usually written Tx rather than T ( x ) .
Naturally the parentheses are used whenever their omission would create
ambiguity. For example, in (8.l.l) above Tx -l- y is not an acceptable sub-
stitute for T(X + y). Second, the symbol for composition of two linear
transformations is ordinarily omitted. If S e £(U, V) and T e £(V, W),
then the composite of T and S is denoted by TS (rather than by T o S). As
a consequence of this convention when T e £(V) the linear operator T o T
is written as T2, T o T o T as T3, and so on.
For future reference here are two obvious properties of a linear map.

Proposition 8.1.5. If T : V -> W is a linear map between vector spaces,

then T ( 0 ) = 0.

Proposition 8.1.6. If T : V -> W is a linear map between vector spaces,

then T ( - X l = - T X for every X 6 V .

You should prove these propositions if (and only if) it is not immediately
obvious to you how to do so.

Definition 8.1.7. Let T: V -> W be a linear transformation between

vector spaces. Then her T, the KERNEL of T, is defined to be the set of all
X in V such that Tx = 0. The kernel of T is also called the nullspace of T.
If V is finite dimensional, the dimension of the kernel of T is the NULLITY
of T.
Also, ran T, the RANGE of T, is the set of all y in W such that y = Tx
for some X in V. If the range of T is finite dimensional, its dimension is the
RANK of T .

Notation 8.1.8. Let V be a vector space. We denote the IDENTITY MAP

on V (that is, the map X +-> X from V into itself) by Iv, or just I.

The following fundamental result is proved in most linear algebra texts.

Theorem 8.1.9. If T : V -> W is a linear map between finite dimensional

vector spaces, then

rank(T) + nullity(T) = dim V.

 Linearity 63

(For an example of a proof, see [4], p. 90.)

Definition 8.1.10. Let T : V -> W and S: W -> V be linear maps. If

ST = Iv, then T is a RIGHT INVERSE for S and S is a LEFT INVERSE for T.
The mapping T is INVERTIBLE (or is an ISOMORPHISM) if there exists a
linear transformation, which we denote by T-1 mapping W into V, such
that

T T - 1 = IW and T-1T = Iv.

The vector spaces V and W are ISOMORPHIC if there exists an isomorphism

T from V to W. We write V 2 W to indicate that V and W are isomorphic.

8.2 Exercises

(1) Define T: R3 -> R4 by

Tx (121 a;'37 XI + SU2)l"3-$2,931 2582

for all X = (a:1,a:2,;c3) in R3.

(a) Then T(1, -2,3) Z( 7 7l. 7

(b) Find a vector X E R3 such that Tx = (8, 9, -5, 0).

Answer: X = l 7 ). 7

(2) Define T: R4 -> R3 by

Tx (2131 + 1:3 + 134,131 - 2582 $37562 - 5123 -|- 334)

for all X (~U17~U27a;.37a;.4l in R4_

(a) Then T ( 2 , l , - l , 3 ) = ( 7 7 l.
(b) Find a vector X E R4 such that Tx = (3, -1, -3).
Answer: X = l 7 7 7 l.
(3) Let T be the linear map from R3 to R3 defined by

T(:c, y , 2> (a:+2y- 27 2x+3y+ 27 4a2+7y- Z •

The kernel of T is (geometrically) a whose equation(s) is(are)

7

and the range of T is geometrically a whose equation(s)

is(are) .
64 Eazercises and Problems in Linear Algebra

(4) Let T: R3 -> R3 be the linear transformation whose action on the

standard basis vectors of R3 is

T(1,0, 0)

T ( 0 , 1, 0)

T(0,0, 1) = (2, -3,4).

Then T ( 5 , l , - l ) = ( , , l. The kernel of T is the

whose equation is at + ay -|- bz = 0 where a = and
b . The range of T is the whose equations are
as y Z
-=- = - where= and wafered= .
2 C d

(5) Let 77 be the vector space of all polynomial functions on 12. with real
coefficients. Define linear transformations T, D : 77 -> 77 by

(DpW) Z p'(:cl

and

(TPXH2) = 32p(;6)

for all 33 6 IR.

(a) Let p(x) = 1:3 -7zu2+5a:+6 for all x E R. Then ((D-l-T)(p))(a3) =

$4-aa:3-I-ba:2-ba:-I-cwherea= ,b= ,and= .
(b) Let p be as in (a). Then (DTp)(a3) = a;u3 - 6562 + ex -|- 6 where
a= , b= , and C = .
(c) Let p be as in (a). Then (TDp)(a3) = a333 - 6332 -|- ca: where a =
, b= , and C = .
(d) Evaluate (and simplify) the commutator [D, T] :: DT - TD.
Answer: 1D,T1 = .
(e) Find a number p such that lTD)P = TPDP -|- TD. Answer: p =

(6) Let C = C[a, b] be the vector space of all continuous real valued func:-
tions on the interval 1a, b] and C1 = C11a, b] be the vector space of all
continuously differentiable real valued functions on 1a, bl. (Recall that
a function is CONTINUOUSLY DIFFERENTIABLE if it has a derivative
 Linearity 65

and the derivative is continuous.) Let D : C1 -> C be the linear trans-

formation defined by

Df f'
and let T : C -> C1 be the linear transformation defined by
x

(To)(a:) f(t) dt

for all f 6 C and as 6 [a,b].

(a) Compute (and simplify) (DTf)(a3). Answer:
(be Compute (and simplify) (TDf)(a3). Answer:
(c) The kernel of T is .
(d) The range of T is {g e C1 : }.
(7) In this exercise we prove that a linear transformation T: V -> W
between two vector spaces is one-to-one if and only if its kernel con-
tains only the zero vector. After each step of the proof give the appro-
priate reason. Choose reasons from the following list.

(DKI Definition of "kernel" .

(DL) Definition of "linear" .
(DO) Definition of "one-to-one" .
(H) Hypothesis.
(Pa) Proposition 8.1.5.
(Pb) Proposition 8.1.6.
(VA) Vector space arithmetic (consequences of vector space axioms,
definition of subtraction of vectors, etc.)

Proof. Suppose that T is one-to-one. We show that kerT = { 0 al.

Since Ov e kerT (reason: and 7 we need only show that

kerT C {0v}5 that is, we show that if X e kerT, then X = 0 V - So let

X e kerT. Then Tx = 0w (reason: and ) and T 0 V = 0 W
(reason: l. From this we conclude that X = 0 V (reason:
and
Now we prove the converse. Suppose that her T = {0v}. We wish to
show that T is one-to-one. Let x, y 6 V and suppose that Tx = T y .
66 Eazercises and Problems in Linear Algebra

Then

T(X - y) = T(X + (-y)) (reason:

= Tx + T(-y) (reason: )
= Tx + (-Ty) (reason: )
= Tx - Ty (reason:
0W (reason: and

Then X - y 6 kerT (reason: l. So X - y = 0 V (reason: 7

that is, X = y (reason: l. Thus T is one-to-one (reason:

and

(8) Let C1(]R) be the vector space of all functions defined on the real line
R that have continuous derivatives at each point of R and C (R) be
the vector space of continuous functions on R. Define the function
T: C1(Rl -> COR) by

(To)(tl Z f '(t) + 3f(t)

for every t 6 R. (Notice that T is a linear map.) The kernel of T is

the set of all scalar multiples of the function g where g(t) =
for each t. Thus the kernel of the linear map T is the solution space
of the differential equation .
(9) Let C2(lR) be the vector space of all functions defined on the real line
R that have continuous second derivatives at each point of l and C(R)
2.

be the vector space of continuous functions on R. Define the function

T: C2(R) -> COR) by

lTd)(tl Z f " ( t ) + f(t)

for every t 6 IR. (Notice that T is a linear map.) Assume that

the kernel of T is two dimensional. Then kerT = span{g, h} where
g(t) = and h(t) = for all t. Thus the ker-
nel of the linear map T is the solution space of the differential
equation .
 Linearity 67

(10) Deine a function k on the unit square [0, l] X [0, l] by

Xv for 0 <56 <y< l

k(a:, y)
y, for 0 < y <;z:< l

Deine an integral operator K on the vector space C [0, l] of continuous

real valued functions on [0, l] by

1
(Kf)(;v) I<(;v»ylf(yl do
0

for 0 < x g l. Find the function Kf when f is the function defined

byf(a:)=a:2 f o r 0 § a : § l .
Answer: (Kfl(a:) = .
( I I ) Let T : R3 -> R3: X +-> (131 -|- 3562 - 2£c3,£31 - 4:c3,a:1 + 6132).

(a) The kernel of T is a in R3 given by the

equation(s)
(b) The range of T is a in R3 given by the
equation(s)

(12) Let T : R2 -> R3: ( a y ) +-> (Za: - 3y,;c + Zy + l , 5:13 - Zy). State in one
short sentence how you know that T is not a linear transformation.
Answer:

(13) Let a = ( l , 0 , 0 , 0 l , b : l , l,0,0), c : l , l , l,0), and d = (l, l , l, l).

Suppose that T: R4 -> R7 is a mapping such that T ( a ) = T ( b ) =
T ( c ) = T ( d ) = 0 and that T(3,-l9,7,-8) = l , l , l , - 3 , 6 , 2 , 5 ) .
State in a short sentence or two how you know that T is not a linear
transformation.
Answer: •

(14) Suppose that T : R3 -> R3 is a mapping (not identically zero) whose

range is contained in the paraboloid Z = 5132 -|- y2. State in a short
sentence or two how you know that T is not a linear transformation.
Answer:
68 Eazercises and Problems in Linear Algebra

(15) Let T : R2 -> R4: (az, y) +-> (Za:-3y, zz:-7y, :c-l-Zy-I-l, 533-Zy). State in
one short sentence how you know that T is not a linear transformation.
Answer:
(16) Let a = ( l , l , 0 ) and b = ( 0 , l , l ) , and c = ( l , 2 , l ) . Suppose that
T: R3 -> R5 is a mapping such that T(a) = T ( b ) = 0 and that
T ( c ) = (l, -3,6,2,5). State in a short sentence or two how you know
that T is not a linear transformation.
Answer :
(17) Suppose that T : R2 -> R2 is a mapping (not identically zero) such that
T(l, l ) = (3, - 6 ) and T ( - 2 , -2) = (-6,3). State in a short sentence
or two how you know that T is not a linear transformation.
Answer:

8.3 Problems

(1) Let T : V -> W be a linear transformation between vector spaces and

let N be a subspace of W. Define T - ( N ) := {v e V : Tv e N}. Prove
that T - ( N ) is a subspace of V.
(2) Prove that a linear transformation T : R3 -> R2 cannot be one-to-one
and that a linear transformation S : R2 -> R3 cannot be onto. Gener-
alize these assertions.
(3) Prove that one-to-one linear transformations preserve linear indepen-
dence. That is: Let T : V -> W be a one-to-one linear transformation
between vector spaces and {XI7 XI7 . . . ,xn} be a linearly independent
subset of V. Prove that {TX1, TX27 . . . ,Txt,} is a linearly independent
subset of W . Hint. To prove that the vectors Txt, T X 2 7 . . . ,Txn are lin-
early independent, it must be shown that the only linear combination
of these vectors which equals zero is the trivial linear combination. So
suppose that E k z l as,Txk, = 0 and prove that every do must be zero.
Use the result proved in exercise 7.
(4) The goal of this problem is to understand and write up an introduction
to invertible linear transformations. Your write-up should explain with
spectacular clarity the basic facts about invertible linear transforma-
tions. Include answers to the following questions giving complete
proofs or counterexamples as required. (But don't number things in
your report to correspond with the items that follow.)
 Linearity 69

(a) If a linear transformation has a right inverse must it have a left

inverse?
(b) If a linear transformation has a left inverse must it have a right
inverse?
(c) If a linear transformation has both a left and a right inverse, must
it be invertible? (That is, must the left and right inverse be the
same?)
(d) If a linear transformation T has a unique right inverse is T neces-
sarily invertible? Hint. Consider ST + S - I, where S is a unique
right inverse for T.
(e) What is the relationship between a linear transformation being one-
to-one and onto and being invertible?
(f) Let {vi, . . . ,v,,} be a linearly independent set of vectors in V. What
condition should a linear transformation T : V -> W satisfy so that
{TV1, . . . ,Tv.,,} is a linearly independent subset of W?
(g) Let {u1, . . . ,u,,} be a basis for a subspace U of V. What condi-
tions should a linear transformation T : V -> W satisfy so that
{Tu1, . . . , Tu.,,} is a basis for the subspace T(U)?
(h) Suppose the vectors V I 7 . . . , v'fl span the vector space V and
T: V -> W is a linear transformation. If {Tv1, . . . ,TvV,} is a basis
for W what can you conclude about the vectors V I , . . . ,v,,? What
can you conclude about the linear transformation T?
(i) Is it true that two finite dimensional vector spaces are isomorphic
if and only if they have the same dimension?
(5) A sequence of vector spaces and linear maps

> Vn 1
Jn _ > V" jn+1
>
- '>Vn,-l-1

is said to be EXACT AT Vn if ran j'fl = kerjn+1° A sequence is EXACT if

it is exact at each of its constituent vector spaces. A sequence of vector
spaces and linear maps of the form

k
0 >U V -> W >0 (l)

is a SHORT EXACT SEQUENCE. (Here 0 denotes the trivial 0-dimensional

vector space, and the unlabeled arrows are the obvious linear maps.)

(a) The sequence (1) is exact at U if and only if j is injective.

(b) The sequence (1) is exact at W if and only if k is surjective.
70 Eazercises and Problems in Linear Algebra

(c) Let U and V be vector spaces. Then the following sequence is short
exact:

0 >U
L1 _ >
U
®
V
'/T2 _> V > 0.

The indicated linear maps are defined by

$1 : U - > U ® V:a+->(a,0)

and

W2 : U ® V->V:(a,b)+->b.

(d) Suppose a < b. Let /C be the family of constant functions on the

interval 1a, b1, C1 be the family of all continuously differentiable
functions on 1a, bl, and C be the family of all continuous functions
on 1a, b]. Specify linear maps j and k so that the following sequence
is short exact:

0 > /C C1 k
-> C > 0.
(e) Let C be the family of all continuous functions on the interval [0, 2].
Let E1 be the mapping from C into l defined by E1(f) = f ( l ) . 2.

(The functional E1 is called "evaluation at l".)

Find a subspace .F of C such that the following sequence is short
exact.

L _ > - E1
0 > jE' C >]R > 0.
(f) Suppose that the following sequence of finite dimensional vector
spaces and linear maps is exact.

>V, .fn fn f2 ->

0 > Vn 1 ' 1 > VI -f1-> Vo >0

Show that
'al

(-1)"'d1m(vk) = 0.
k=0
 Linearity 71

Definition 8.3.1. It is frequently useful to think of functions as

arrows in diagrams. For example, the situation f : U -> X, g : X -> V,
h : U -> W, j: W -> V may be represented by the following diagram.
f
U X

h g

w j
V

The diagram is said to COMMUTE (or to be a COMMUTATIVE DIAGRAM)

if j o h = g o f.

(g) Suppose that in the following diagram of vector spaces and linear
maps
k
0 U V > W 0
I
f g lh
v
0 U/ > V/ > w' 0
-I Ki
.7

the rows are exact and the left square commutes. Then there exists
a unique linear map h : W -> W' that makes the right square
commute.
In parts (h)-(k) consider the diagram
j k
0 > U > V >W > 0

f g h

0 > U/ >V' W I
> 0
-I k/ >
.7

where the TOWS are exact and the squares commute.

(h) If g is surjective, so is h.
(i) If f is surjective and g is injective, then h is injective.
(j) If f and h are surjective, so is g.
(k) If f and h are injective, so is g.
(6) Let V and W be vector spaces. Prove that (under the usual pointwise
operations of addition and scalar multiplication) £(V, W) is a vector
space.
72 Eazercises and Problems in Linear Algebra

(7) Let M be a subspace of a vector space V and V/M be the quotient

space defined in Definition 5.1.7 in Chapter 5. Then the map

7r:V->V/M:;c»->[cc]

is linear and is called the QUOTIENT MAP.

Remark 8.3.2. Imagine a vector space V of large dimension to which

no coordinates have been assigned. Let M be an arbitrary subspace of V.
Does M have a complementary subspace? Well, yes, in general, many of
them. But how would we go about choosing one? Of course we could start
by working with bases, and putting in coordinates, and writing equations,
and making some arbitrary choices to specify some complementary space.
But all that seems rather unnatural involving a lot of wasted energy. A bet-
ter solution is to relax the demand that the "complement" be literally a
subspace of V, and replace it with something very simple that for most pur-
poses works just as well. That object is the quotient space V / M . No fuss,
no muss, no bother. That is the point of the next problem.

~ ~
(8) If M and N are complementary subspaces of a vector space V, then
N = V / M . (So V = M ® V l M . ) Hint. Consider the map n -> [n] for
every n e N.
The following result is called the fundamental quotient theorem, or
the first 'isomorphism theorem, for vector spaces.
(9) Prove the following theorem and its corollary.

Theorem 8.3.3. Let V and W be vector spaces and M -< V . If T 6

£(V, W ) and kerT D M , then there exists a unique T 6 £(V/M, W ) that
makes the following diagram commute.

V \

'ii

V/M W
T

Furthermore, if is injective if and only if her T M ; and if is subjective if

and only if Tis.
 Linearity 73

Corollary 8.3.4. If T : V -> W is a linear map between vector spaces,

then

ranT%V/kerT.

8.4 Answers to Odd-Numbered Exercises

(1) (a) -2, -l, 5, 5

(b) 6, 3, -2

(3) line, "5 _ y

Za plane, 251: -|- y Z 0
3
(5) (21) 7, 8, 5
(b) 4, 21, 10
(c) 3, 14, 5
(d) I
(Q) 2
(7) Pa, DK, H, DK, Pa, H, DO, VA, DL, Pb, VA, H, VA, DK, H, VA,
Do, H
(9) sint, cost, y" -|- y = 0

( I I ) (21) line, Z I
2
_
Z

(be plane, 2:13 - y - z = 0

(13) R4 is the span of a, b, e, and d, all of which T takes to 0; so were T
linear, its range would contain only 0.
(15) T does not map 0 to 0.
(17) If T were linear, then T ( - 2 , -2) would be -2T(l, l ) = -2(3, -6) =
(-6, 12).
This page intentionally left blank
 Chapter 9

LINEAR MAPS BETWEEN

EUCLIDEAN SPACES

9.1 Background

Topics: linear mappings between finite dimensional spaces, a matrix as a

linear map, the representation of a linear map as a matrix.

Proposition 9.1.1. Let T 6 £(V, W ) where V is an n-dimensional vector

space and W is an in-dimensional vector space and let {e1, e2, . .. , eTL} be a
basis for V . Define an m X n-matrix [T] whose kth column (1 < k < no is
the column vector Ten. Then for each at 6 V we have

TCU [T]56~
The displayed equation above requires a little interpretation. The left
side is T evaluated at as; the right side is an m X n matrix multiplied by an
n X l matrix (that is, a column vector). Then the asserted equality can be
thought of as identifying
(l) two vectors in RM ,
(2) two m-tuples of real numbers, or
(3) two column vectors of length m (that is, two m X l matrices) .

If we wished to distinguish rigorously between column vectors and row

vectors and also wished to identify m-tuples with row vectors, then the

75
76 Eazercises and Problems in Linear Algebra

equation in the preceding proposition would have to read

TCU i ([T](=v¢))¢

To avoid the extra notation in these notes we will not make this distinction.
In an equation interpret a vector as a row vector or as a column vector in
any way that makes sense.

Definition 9.1.2. If V and W are Finite dimensional vector spaces with

bases and T e £(V, W), then the matrix [T] in the preceding proposi-
tion is the MATRIX REPRESENTATION of T. It is also called the STANDARD
MATRIX for T. The RANK of an m X n matrix is the rank of the operator
on T: R" -> RM it represents (with respect to the standard bases on R"
and Run),

9.2 Exercises

(1) Let T: R4 -> R3 be defined by

Tx = (al - 3563 + 134, 21131 -|- S132 -|- £123 -|- 5134, 3582 - 41133 + 7134)
for every X = (al , SEQ, $37 1U4l 6 R4. (The map T is linear, but you need
not prove this.)

l 1
(a) Find [T]. Answer:

m
(b) Find T(1, -2, 1,3). Answer:
•

(c) Find ( m m -2, 1,3)t))t. Answer:

(d) Find her T. Answer: kerT = span{
(e) Find ranT. Answer: ranT =

(2) Let T: R3 -> R4 be defined by

Tx (131 - 3cu3,a31 + 132 - 6;c3,;u2 - 3a33,a31 - 3a:3 l

for every X = (5131, $62, k g ) 6 R3. (The map T is linear, but you need not
prove this.) Then
 Linear Maps Between Euclidean Spaces 77

(a) [T]

(b) T(3,-2,4) =
(c) kerT = span{
(d) ranT = span{

(3) Let 7% be the vector space of all polynomial functions on R with degree
strictly less than n. The usual basis for 7% is the set of polynomials
1,t,t2,t3, . . . , t 7 n - 1 . Define T: 773 -> 775 by

Tf(=vl
/" /"
0 0
pltl dt

for all x , u € R .

(a) Then the matrix representation of the linear map T with respect

to the usual bases for 773 and 775 is

(b) The kernel of T is

(c) The range of T is span{ .

(4) Let 774 be the vector space of polynomials of degree strictly less than 4.
Consider the linear transformation D2 : 774 -> 774: f +-> f " .
78 Eazercises and Problems in Linear Algebra

(a) Then the matrix representation [192] of D2 (with respect to the usual

basis {1, t,t2, t3} for 774) is given by [132]

(b) her D2 span{

(c) ranD2 span{

(5) Let 774 be the vector space of polynomials of degree strictly less than 4
and T : 774 -> 775 be the linear transformation defined by (Tp)(t) =
(2 -|- 3t)p(t) for every p e 774 and t e R. Then the matrix representation
of T (with respect to the usual basis {1, t, t2, t3} for 774) is given by

[T]

(6) Let T: 2.3 -> R3 be the linear transformation whose standard matrix
V1 1 0 '|
|
is 0 1 1 We know that T is not onto because the only vectors
•

._1 0 -1
(u, u, w) that are in the range of T are those that satisfy the relation
u-I-av-I-bw=0wherea= andb= .
(7) Let T be the linear map from R3 to R3 defined by

T(;z:,y,z) = (3:13 + 2 y + 27 :c+ 32, -y-I-4z).

(a) The matrix representation of T is given by

[T]

(b) The range of T is (geometrically speaking) a whose

equation is .
 Linear Maps Between Euclidean Spaces 79

9.3 Problems

(1) Define T: R3 -> R2 by

Tx = (131 -|- 25132 - 333,332 + 333)

for all X = (a:1,a:2,a:3) in R3.
(a) Explain how to find 1T1, the matrix representation for T.
(be Explain how to use [T] to find T(x) when x = (-1, 2, -1).
(cz) Explain how to use [T] to find a vector x in R3 such that Tx
(0, 1).
Carry out the computations you describe.
(2) Let 77 be the vector space of all polynomial functions on R with real
coefficients. Define linear transformations T, D : 77 -> 77 by

(DpW) i p'(:cl

and

(TPXH2) = 332p(;v)

for all x 6 IR.

Explain carefully how to find matrix representations for the linear
transformations D -l- T, DT, and TD (with respect to the usual basis
{l,t,t2} for the space of polynomials of degree two or less). Carry
out the computations you describe. Use the resulting matrices to find
((D + T)(p))(x)» DTp)(a:l, and (TDp)l5ul where p(a7l i 3332 + ex 3
for all x e IR.
(3) Define T: R2 -> R3 by

TX= (121 $132,132 - $ 1 , $ 1 + $ 2 )

for all X = (131,132) in R2.

(a) Explain carefully how to find 1T1, the matrix representation for T.
(b) How do we use [T] to find T(1, -2)?
(c) Are there any nonzero vectors X in R2 such that Tx = 0 .7 Explain.
(d) Under what conditions is a vector (1u,,v,w) in the range of T?
Explain.
(4) Let c1(10, 11) be the vector space of all functions defined on the interval
[0, 11 that have continuous derivatives at each point and C([0, 11) be
80 Eazercises and Problems in Linear Algebra

the vector space of continuous functions on [0, l]. Define a function

T: c1([0, 11) -> c([0, 11) by

(To) f(t)dt+f'(.¢)
0

for every x 6 [0, II.

(a) Prove that the function T is linear.
(b) Let f(a3) = s i n : and g(a:) = cost: for all Hz: e [0, l]. Explain why
one of these functions belongs to the kernel of T while the other
does not.
(5) Let 774 be the vector space of polynomials of degree strictly less than 4.
Consider the linear transformation D2 : 774 -> 774: f +-> f". Explain
carefully how to find [021, the matrix representation of D2 (with respect
to the usual basis {1, t, t2, t3} for PA). Then explain how to use 11321 to
find her D2 and ran D2 .

9.4 Answers to Odd-Numbered Exercises

(l) (a)
0

n 3 -4
12
1 1
'1

|-
1
1
7
(be (1,4, 11)
(c) (1,4, II) (or [1 4 11])
(dl (1,-9,2,5)
(G) R3
'Q 0 Q-
0 0 0
1
(3) (a) 2 0 0
1
0 6 0
1
0 llllllllll

12-

(be {0}
low 32
3
7 1/83 7 $84
 Linear Maps Between Euclidean Spaces 81

2 0 0 0
3 2 0 0
(5) 0 3 2 0
0 0 3 2
0 0 0 3

1 1
3 2
(7) (a) 1 0 3
0 -1 4
(b) plane, 'LL - 31) + 2w = 0
This page intentionally left blank
 Chapter 10

PROJECTION OPERATORS

10.1 Background

Topics: projections along one subspace onto another.

Definition 10.1.1. Let V be a vector space and suppose that V = MGBN.
We know that for each V e V there exist unique vectors In e M and n e N
such that V = In -|- n (see Problem 5 in Chapter 5). Define a function
EMN : V -> V by E M N U = n. The function EMN is the PROJECTION OF
V ALONG M ONTO N. (Frequently we write E for EMN. But keep in mind
that E depends on both M and N . )
Theorem 10.1.2. Let V be a vector space and suppose that V = M ® N .
If E is the projection of V along M onto N , then
(i) E is linear;
(ii) E2 = E (that is, E is IDEMPOTENT);
(iii) ran E = N , and
(iv) her E i M.
Theorem 10.1.3. Let V be a vector space and suppose that E : V -> V is
a function which satisfies
(i) E is linear, and
(ii) E2 = E .
Then
V = her E ® ran E
and E is the projection of V along herE onto rarE.

83
84 Eazercises and Problems in Linear Algebra

Theorem 10.1.4. Let V be a vector space and suppose that V = M ® N .

If E is the projection of V along M onto N , then I - E is the projection of
V along N onto M.

10.2 Exercises

(1) Let M be the line y = 21: and N be the y-axis in R2. Then

[EMN] and [ENM]

L
C a
a ]
where a = ,b= , and C .
(2) Let P be the plane in R3 whose equation is Hz: - Z = 0 and L be the line
whose equations are y = 0 and as = - Z . Let E be the projection of R3
along L onto P and F be the projection of R3 along P onto L. Then

a b a a b
[E] b C b and [F] b b
a b a a b

where a 7 b , and C

(3) Let P be the plane in R3 whose equation is x -|- 2y - z = 0 and L be

Z
the line whose equations are § = y = 5- Let E be the projection of R3
along L onto P and F be the projection of R3 along P onto L. Then

a -b C 3d 36 -3d
1 1
[E] = -d d d and [F] d e d
3 3
Cl - 2d -b + 2d C + 2d 2d 26 2d

where a 7 b ,C 7 d , and e
(4) Let P be the plane in R3 whose equation is an - y - 22 = 0 and L be the
line whose equations are as = 0 and y = - Z . Let E be the projection
of R3 along L onto P and F be the projection of R3 along P onto L.
Then

Cl b b b b b
[E] a C C and [F] a Cl c
Cl a Cl -a a c

where a 7 b 7 and C
 Projection Operators 85

(5) Let E be the projection of R3 along the z-axis onto the plane 3:6 - y -I-
22 = 0 and let F be the projection of R3 along the plane 31:-y+22 = 0
onto the z-axis.

(a) Then [E]

(b) Where does F take the point (4,5, 1)? Answer: ( 7 7

).
(6) Let M be the y-axis and N be the plane an + y - 2z = 0 in IR3.
(a) Then the projection EMN of R3 along M onto N is

(b) The projection ENM takes the vector (3, 2, 1) to l 7 7

10.3 Problems

(1) Let E be a projection on a vector space. Show that a vector X belongs to

the range of E if and only if E x = X . Hint. Recall (from Theorems 10.1.2
and 10.1.3) that a projection is a linear map E such that E2 = E.
(2) Prove Theorem 10.1.2.
(3) Prove Theorem 10.1.3.
(4) Prove Theorem 10.1.4. Hint. Use Theorem 10.1.3.
(5) Let P be the plane in R3 whose equation is Hz: - z = 0 and L be the line
whose equations are y = 0 and as = Explain carefully how to find
the matrix representation of the operator ELP, that is, the projection
of R3 along L onto P. Carry out the computation you describe.
(6) Let L be the line in R3 whose equations are an = y and Z = 0, and let P
be the plane whose equation is as - Z = 0. Explain carefully how to find
the matrix representation of the operator ELP, that is, the projection
of R3 along L onto P. Carry out the computation you describe.
(7) Let P be the plane in R3 whose equation is x - 3y + z = 0 and L be
the line whose equations are Hz: = -2y = - Z . Explain carefully how to
86 Eazercises and Problems in Linear Algebra

find the matrix representation of the operator ELP of R3 along L onto

P and the projection EPL of R3 along P onto L.
(8) Prove that a linear transformation between vector spaces has a left
inverse if and only if it is injective.
(9) Prove that a linear transformation between vector spaces has a right
inverse if and only if it is surjective.

10.4 Answers to Odd-Numbered Exercises

(1) 0, 2, 1
(3) 0, 6, 3, 1, 2
' 1 0 0
(5) (a)
I 03 1
1
2
0
n

9
(b) 0, 0, 2
 Part 4

SPECTRAL THEORY OF
VECTOR SPACES
This page intentionally left blank
 Chapter 11

EIGENVALUES AND
EIGENVECTORS

11.1 Background

Topics: characteristic polynomial, eigenvalues, eigenvectors, eigenspaces,

algebraic multiplicity and geometric multiplicity of an eigenvalue.
Definition 11.1.1. A number A is an EIGENVALUE of an operator T on
a vector space V if Ker(AIv - T) contains a nonzero vector. Any such
vector is an EIGENVECTOR of T associated with A and Ker(AIv - T) is
the EIGENSPACE of T associated with A. The set of all eigenvalues of the
operator T is its (POINT) SPECTRUM and is denoted by 0(T).
If M is an n X n matrix, then det(AIn - M) (where In is the n X n
identity matrix) is a polynomial in A of degree n. This is the CHARACTER-
ISTIC POLYNOMIAL of M. A standard way of computing the eigenvalues
of an operator T on a finite dimensional vector space is to find the zeros
of the characteristic polynomial of its matrix representation. It is an easy
consequence of the multiplicative property of the determinant function (see
Proposition 2.1.10) that the characteristic polynomial of an operator T on
a vector space V is independent of the basis chosen for V and hence of the
particular matrix representation of T that is used.
Theorem 11.1.2 (Spectral Mapping Theorem). If T is an operator
on a #nite dimensional vector space and p is a polynomial, then

(t(p(T)l i p((t(T))~
That is, if 0(T) = { 1 7 .,As9J}, then o(p(T)) = {p(A1), .. 7 p(I<l}~

89
90 Eazercises and Problems in Linear Algebra

11.2 Exercises

'1 1 4
(1) If A
l.

eigenvector ( , 1, 1 ), the eigenvalue

.I
u 7 then the eigenvalue has corresponding

has corresponding eigen-

vector ( 7 4, 1), and the eigenvalue has corresponding eigen-
vector ( 7 2, 1).

I
0 0 2
(2) Let A 0 2 0
q n n

(a) The eigenvalues of A are , , and

(b) The matrix A has a one-dimensional eigenspace.
It is the span of the vector (1, , ).
(c) The matrix A has a two-dimensional eigenspace.
It is the span of the vectors (1, 0, ) and (0, 1,

l
0 1 0
(3) Choose an b and c in the matrix A 0 0 1 SO that the character-
1.

is tic polynomial of A is -A3 -|- 4A2 -|- 5A + 6.

Answer: Cl 7 b , and C =
1 0 -1
(4) Suppose that it is known that the matrix A \/§ a 17 has eigen-
2 0 b
values 2 and 3 and that the eigenvalue 2 has algebraic multiplicity 2.
Then Cl = and b = .
Cl 114 48
(5) The matrices A =
eigenvalues. Then a =
2 1]
d and B = -1
25

and d =
48
86
have the same

(6) Let A 10
n
1
n
2
n I
(a) The eigenvalues of A are , , and .
(b) The matrix A has three one-dirnensional eigenspaces. They are
spanned by the vectors l , , ), ( , , ),
and ( , , ), respectively.
 Eigenvalues and Eigenvectors 91

1 1 1 1
1 1 1 1
(7) Let A
1 1 1 1
1 1 1 1

(a) The eigenvalues of A - I are (which has algebraic multiplicity

) and (which has algebraic multiplicity l.
(b) The determinant of A - I is
(8) Let T be the operator on R3 whose matrix representation is

l 1 -1
0 0 1
0
0 Then the eigenvalues of the operator T5 - 3T4 -|- T 3 -
L Q 2
T2+T-3]
l
2
are , ,and .
(9) If A is a square matrix and A2 = I then its possible eigenvalues are
, and .

11.3 Problems

(1) Suppose that A and B are n X n matrices. Prove that c7(AB) = 07(BA).
Hint. Show that if A is an eigenvalue of AB, then it is also an eigenvalue
of BA. Deal with the cases A = 0 and A Quo 0 separately.
(2) Let c e R. Suppose that A is an n X n matrix and that the sum of the
entries in each column of A is C. Prove that c is an eigenvalue of A.
Hint. Consider the sum of the row vectors of the matrix A - el.
(3) Prove that every square matrix is the sum of two nonsingular matrices.
Hint. For the matrix A use A-cI and cl for some appropriate number e.
(4) This is a problem in cryptography. Read about Hill ciphers, then decode
the following Hill 3-cipher given that the first two words of the plaintext
are known to be "My candle". (See for example [11, Section 11.16.)
OGWGCGWGKK.EWVD.XZJOHZWLNYH USTFAIOS.A.KBN
JRCENYQZV,IE LTGCGWGKC YYBLSDWWODLBVFFOS.H

In many discussions of Hill ciphers letters of the alphabet are assigned

numbers from 0 to 25 and arithmetic is done modulo 26. The encoding
here is done slightly differently. Here each letter is assigned its numeri-
cal position in the alphabet (including Z that is assigned 26). Further-
more, a space between words is assigned 27, a comma is assigned 28,
92 Eazercises and Problems in Linear Algebra

and a period is assigned zero. Thus, for this code, all arithmetic should
be done modulo 29. (One reason for this is that some computer algebra
systems have problems calculating inverses mod 26.) Note: the cipher-
text contains exactly three spaces.

11.4 Answers to Odd-Numbered Exercises

(1) -2, -1, 1, -1, 3, 1

(3) 6, 6, 4
(5) 2, 6 (or 6, 2)
(7) (a) -1, 3, 3, 1
(b) -3
 Chapter 12

DIAGONALIZATION
OF MATRICES

12.1 Background

Topics: similarity of matrices, triangular and diagonal matrices, diago-

nalization, annihilating and minimal polynomials, algebraic and geometric
multiplicity of an eigenvalue, the Cayley-Hamilton theorem.

Definition 12.1.1. Two operators R and T on a vector space V are

SIMILAR if there exists an invertible operator S on V such that R = 5-1T5.

Proposition 12.1.2. If V is a vector space, then similarity is an equiva-

lence relation on £ ( V ) .

Definition 12.1.3. Let V be a finite dimensional vector space and B =

{e1, . . . ,e"} be a basis for V. An operator T on V is DIAGONAL if there
exist scalars a l , . . . , a'fl such that Ten' = arek for each k e Nn- Equivalently,
T is diagonal if its matrix representation [T] = [to] has the property that
iii = 0 whenever i go j .

Asking whether a particular operator on some finite dimensional vector

space is diagonal is, strictly speaking, nonsense. As defined the operator
property of being diagonal is definitely not a vector space concept. It makes
sense only for a vector space for which a basis has been specified. This
important, if obvious, fact seems to go unnoticed in many beginning linear
algebra texts, due, I suppose, to a rather obsessive fixation on R" in such
courses. Here is the relevant vector space property.

93
94 Eazercises and Problems in Linear Algebra

Definition 12.1.4. An operator T on a finite dimensional vector space

V is DIAGONALIZABLE if there exists a basis for V with respect to which T
is diagonal. Equivalently, an operator on a finite dimensional vector space
with basis is diagonalizable if it is similar to a diagonal operator. If a matrix
D is diagonalizable and A = S-1DS is diagonal, we say that the matrix S
DIAGONALIZES D.

Theorem 12.1.5. Let A be an n X n matrix with n linear independent

eigenvectors. If S is a matrix with these eigenvectors as columns, then S
diagonalizes A. The entries along the diagonal of the resulting diagonal
matrix are all eigenvalues of A.

Definition 12.1.6. A polynomial is MONIC if its leading coefficient is 1.

Thus a polynomial of degree n is manic if it takes the form 32" +an_1a37n*-1 -I-
. . . _|_ (1158 _|_ Go .

Definition 12.1.7. Let p be a polynomial of degree at least one and T

be an operator on some vector space. We say that p is an ANNIHILATING
POLYNOMIAL for T (or that p ANNIHILATES T) if p(T) = 0. For example, if
T3 - IT2 + T - 7] = 0, then the polynomial p defined by p(a:) = 1:3 - 41:2 -I-
:u - 7 is an annihilating polynomial for T.

Definition 12.1.8. Let T be an operator on a finite dimensional vector

space. The MINIMAL POLYNOMIAL of T is the unique manic polynomial of
smallest degree which annihilates T. (It is left as a problem to verify the
existence and the uniqueness of such a polynomial: see problem 8.)

Theorem 12.1.9 (Cayley-Hamilton Theorem). On a finite dimen-

sional vector space the characteristic polynomial of an operator T annihi-
lates T .

Paraphrase: Every matrix satisfies its characteristic equation.

Definition 12.1.10. A square matrix A = [Um] is UPPER TRIANGULAR if

a t J' = 0 whenever 11 > J'. A matrix is TRIANGULABLE or TRIANGULARIZ-
ABLE) if it is similar to an upper triangular matrix.

Theorem 12.1.11. Let T be an operator on a #nite dimensional vector

space and let {A1, , As1J} be its distinct eigenvalues. Then:

( 1 ) T is triangle/,lable if and only if its minimal polynomial can be factored

into a product of linear factors. That is, if and only if there are positive
 Diagonalization of Matrices 95

integers T o , . . . ,To such that

iT(a:l Z (cc - Aw - - (33 - A)"°

( 2 ) T is diagonalizable if and only if its minimal polynomial has the form

mT(a:) = (at - A l ) - - (33 - As).

Corollary 12.1.12. Every operator on a complex finite dimensional vector

space is triangulable.

Definition 12.1.13. An operator is NILPOTENT if some power of the oper-

atom is 0.

12.2 Exercises

(1) Let A = 11'1

1
1
1
1
1
1 . The characteristic polynomial of A is AP(A - 3)'I
1
-l

where p = and q = .
The minimal polynomial of A is A"(A - 3)"' where T' = and
S

(2) Let T be the operator on R4 whose matrix representation is

0 1 0 -1
-2 3 0 -1
-2 1 2 -1 .
2 -1 0 3
The characteristic polynomial of T is (A - 2)1' where p =
The minimal polynomial of T is (A - by where T =
(3) Let T be the operator on R3 whose matrix representation is
3 1 -1`l

(a) Find the characteristic polynomial of T.

Answer: CT(A) = (A - 1)1'(A - Zlq where p and q
(b) Find the minimal polynomial of T.
Answer: MT(A) = (A - 1)""(A - 21S where T' and S
96 Eazercises and Problems in Linear Algebra

(c) Find the eigenspaces M1 and M2 of T.

Answer: M1 = span{ } and M2 = span { •

(4) Let T be the operator on R5 whose matrix representation is

1 0 0 1 -1
0 1 -2 3 -3
0 0 -1 2 -2 .
1 -1 1 0 1
1 -1 1 -1 2

(a) Find the characteristic polynomial of T.

Answer: CT(A) = (A -|- 1)P(A - l)q where p and q
(b) Find the minimal polynomial of T.
Answer: MT(A) = (A+ 1)""(A - 1lS where T' and S

(c) Find the eigenspaces VI and VI of T.

Answer: VI = span { and
$2 = span { .

(5) Let T be an operator whose matrix representation is

0 0 0 0 0
0 0 0 0 0
3 1 0 0 0 .
0 0 0 1 2
0 0 0 -1 -1

(a) Regarding T as an operator on R5 find its characteristic polyno-

mial and minimal polynomial.
Answer: CT(A) = AP(A2 -|- l)'1 where p = and q =
and MT(A) = X'"(A2 -1- US where r = and S

(b) Regarded as an operator on R5 is T diagonalizable? Is it

triangulable? .
(c) Regarded as an operator on Q5 is T diagonalizable? Is it
triangulable? .
(6) Let T be the operator on R3 whose matrix representation is
r2 0 0
-1 3 2 .
 Diagonalization of Matrices 97

(a) Find the characteristic and minimal polynomials of T.

Answer: CT(A) = (A - l)p(A - 2)q where p = and q =
and MT(A) = (A - 1)""(A - 2)"' where r = and s = .
(b) What can be concluded from the form of the minimal polynomial?
Answer: .
(c) Find a matrix S (if one exists) that diagonalizes [T1. What is
the diagonal form A of [T] produced by this matrix? Answer:
Cl 1. Cl
CO

b 4 where Cl = b , and c 7 and

II

C 7

-b a b

L 1
A 0 0
<

0 ,LL 0 where A = and p

II

(7) Let T be the operator on R3 whose matrix representation is

r so -6 12
-18 11 18
-6 -3 26

(a) Find the characteristic and minimal polynomials of T.

Answer: CT(Al = ( A - 5)P(A- 20)q where p = and q =
and MT(A) = ()-5)"°(A-20)S* where T' = and s = .
(b) What can be concluded from the form of the minimal polynomial?
Answer: .
(c) Find a matrix S (if one exists) that diagonalizes [T1. What is the
diagonal form A of [T] produced by this matrix? Answer: S =
Cl b C
d a a where a = b 7 ,C and d 7 and 7

b c N
b
0 0

; 1
<

0 ,LL 0 where A = and p

II

(8) Let 7% be the space of polynomials of degree strictly less than n and
D be the differentiation operator on 7% Then
(a) the only eigenvalue of D is A = ,
(b) the corresponding eigenspace is the span of 7
98 Eazercises and Problems in Linear Algebra

(c) the algebraic multiplicity of A is ; and

(d) the geometric multiplicity of A is .
(9) Suppose that A is a 2 X 2 matrix of real numbers, that A2 I , and
that A is neither the identity matrix I nor -I. Then tr A = and
det A =
(10) Suppose that A is as in the preceding exercise and that the entries
in its first row are 3 and -1 (in that order). Then the entries in the
second row are and .

12.3 Problems

(1) Prove that the trace function is a similarity invariant on the family of
n X n matrices, that is, prove that if A and B are similar n X n matrices,
then t r A = tr B. Hint. Prove {'list that if M and N are n X n matrices,
then MN and NM have the same trace.
(2) Prove that the determinant function is a similarity invariant on the
family of n X n matrices, that is, prove that if A and B are similar
n X n matrices, then det A = det B.
(3) Prove that if two matrices are diagonalized by the same matrix, then
they commute.
(4) Prove that if a matrix A is diagonalizable, then so is every matrix
similar to A.
(5) Show that if A is a diagonalizable n X n matrix of real (or complex)
numbers, then tr A is the sum of the eigenvalues of A and det A is their
product.
(6) Suppose that T is an operator on a finite dimensional complex vector
space and that o(T) = {0}. Show that T is nilpotent.
(7) Let T be an operator on a finite dimensional vector space.
(a) Show that if p is an annihilating polynomial for T, then the minimal
polynomial WET divides p. Hint. Suppose that p annihilates T (so
that the degree of p is at least as large as the degree of M T ) . Divide
p by m f . Then there exist polynomials q (the quotient) and r (the
remainder) such that
p = m f q -l- 7° and degree of T < degree of WET .
Conclude that r = 0.
 Diagonalization of Matrices 99

(b) Show that the minimal polynomial mT and the characteristic poly-
nomial cT have exactly the same roots (although not necessarily
the same multiplicities). Hint. To show that every root of mT is
also a root of CT, it is enough to know that mf divides CT. Why is
that true?
To obtain the converse, suppose that A is a root of CT: that is,
suppose that A is an eigenvalue of T . Use the spectral mapping
theorem 11.1.2 to show that MT(A) = 0.

(8) Let T be an operator on a Finite dimensional vector space V. Show

that there is a unique manic polynomial of smallest degree which anni-
hilates T. Hint. This asks for a proof of the existence and the unique-
ness of the minimal polynomial for the operator T. The existence part is
easy: If there are any polynomials at all which annihilate T, surely there
is one of smallest degree. (How do we know that there is at least one
polynomial that annihilates T?) We want the annihilating polynomial
of smallest degree to be manic is this a problem?
To verify the uniqueness of the minimal polynomial, consider the
case of degree one separately. That is, suppose that p and q are manic
annihilating polynomials of degree one. How do we know that p = q?
Then consider polynomials of higher degree. Suppose that p and q are
manic annihilating polynomials of smallest degree k where k > 1. What
can you say about p - q?

12.4 Answers to Odd-Numbered Exercises

(1) 2, 1, 1, 1

(3) (a) 1, 2
(b) 1, 2
(c) (1,0,2), (1, 1,2)
(5) (a) 3, 1, 2, 1
(b) no, no
(c) no, yes

(7) (a) 1, 2, 1, 1
(b) T is diagonalizable
(c) 2, 1, 0, 3, 5, 20
This page intentionally left blank
 Chapter 13

SPECTRAL THEOREM
FOR VECTOR SPACES

13.1 Background

Topics: the spectral theorem for #note dimensional vector spaces (writing
a diagonalizable operator as a linear combination of projections).

The central fact asserted by the finite dimensional vector space version
of the spectral theorem is that every diagonalizable operator on such a space
can be written as a linear combination of projection operators where the
coefficients of the linear combination are the eigenvalues of the operator
and the ranges of the projections are the corresponding eigenspaces. Here
is a formal statement of the theorem.
Theorem 13.1.1 (Spectral theorem: vector space version). Let T
be a diagonalizable operator on a #nite dimensional vector space V, and
A l , . . . , As be the (distinct) eigenvalues of T . For each j let My be the
eigenspace associated with * j and Et be the projection of V onto My along
M1+~~~+mj_1+mi+1+~-~+M,.. Then
(1) T ; >1E1 -I- ~°+A1<=E1<v
®I=&+~ '-l-Ek, and
(iii) EEG = 0 when i go j.

For a proof of this result see, for example, [6], page 215, Theorem II.

101
102 Exercises and Problems in Linear Algebra

13.2 Exercises

(1) Let T be the operator on R2 whose matrix representation is

I -16 171
(a) Find the characteristic polynomial and minimal polynomial for T.
Answer: CT(A) = and MT(A)

(b) The eigenspace M1 associated with the smaller eigenvalue A1 is the

span of (1, l.
(c) The eigenspace M2 associated with the larger eigenvalue A2 is the
span of (1, l.
(d) We wish to write T as a linear combination of projection operators.
Find the (matrix representations of the) appropriate projections E1
and E2 onto the eigenspaces M1 and M2, respectively.
r 1-l

Answer: E1 7 where Cl and b 7 and E2

7 where C and d

(e) Compute the sum and product of E1 and E2 .

Answer: E1 -|- E2 7 and E1E2

(f) Write T as a linear combination of the projections found in (d) .

Answer: [T] = E1 -|- E2 .
(g) Find a matrix S that diagonalizes [T]. What is the associated
diagonal form A of [T]?

Answer: S
V ;l»
Cl
where a and b 7 and A

(2) Let T be the operator on R3 whose matrix representation is

-6 1 12
 Spectral Theo'/"em for Vector Spaces 103

(a) Find the characteristic polynomial and minimal polynomial for T.

Answer: CT(A) = and MT(A)

(b) The eigenspace M1 associated with the smallest eigenvalue A1 is

the span of (3, , l.
(c) The eigenspace M2 associated with the middle eigenvalue A2 is the
span of (1, , l.
(d) The eigenspace M3 associated with the largest eigenvalue A3 is the
span of (1, , l.
(e) We wish to write T as a linear combination of projection operators.
Find the (matrix representations of the) appropriate projections
El, EQ, and E3 onto the eigenspaces Mi, M2, and M3, respectively.

I
Cl C Cl -b d c
Answer: E1 2a C 2a E2 = -2a a C 7 and E3 =
-b C -b b -d c
c -I -a
c -b -2a , where a = 7 b ,C 7 and d
c d a
(f) Write T as a linear combination of the projections found in (e).
Answer: [T] = E1 -|- E2 -|- E3.
(g) Find a matrix S that diagonalizes 1T1. What is the associated
diagonal form A of [T]?

Answer: S l Cl

2a
b
Cl
b
C 1 where a b and
L l
7 7 7

-e -b -b

and A
U
II

(3) Find a matrix A whose eigenvalues are 1 and 4, and whose eigenvectors
are (3, 1) and (2, 1), respectively.

Answer: A
104 Exercises and Problems in Linear Algebra

(4) Let T be the operator on R3 whose matrix representation is

r O _o 17
-1 1 1 •

-1 2 0
(a) Write T as a linear combination of projections.
Answer: T = C1 E1 -|- C2 E2 + C3 E3 where C1 = ,C2 7

C3 = 7

I 1
a b -b b a b b -b a
$1 a b -b E2 I b CL b and E3 b b a
a -b b b a b b b a
where a = and b =
(b) Calculate the following:

l 1 . $1 Es
l 1
l l
E1E2 7 ; E2 Es

(c) E1 +E2 4-E3

(d) Write T3 as a linear combination of E17 EQ, and E3 .

Answer: T3 = E1 -|- E2 -|- E3.

(5) Let A 113

n
0
4
n
0

1 3 I
(a) Find the eigenvalues of A. Answer: The smaller eigenvalue is
A1 = , and the larger eigenvalue is A2 = .
 Spectral Theo'/"em for Vector Spaces 105

(b) The eigenspace associated with A1 is span{(1, 0, ) , (1, , 0)};

and the eigenspace associated with A2 is span{( , 1 , 0)}.
(c) Find a factorization of A in the form SAS-1 where A is a diagonal
matrix.
1 Cl b c 0 0
Answer: S = 0 -a a , A 0 c 0
a b b 0 0 d

r J J
L
j
L I 7 where a = 7 b ,c 7 d 7 J 7

and k = .
(d) Find a square root of A.
b b la
Answer: \/-A = 7 where a 7 b 7 and c

13.3 Problem

(1) Explain carefully how to use spectral theory (the theory of eigenvalues
V 10
and eigenvectors) to find a square root of the matrix A = 10] .
I

Illustrate your discussion by carrying out the computation.

13.4 Answers to Odd-Numbered Exercises

(l) (a) A2 10A+ 9, $2 - 1 0 ) + 9

(be 1
(c) 2

N
(dl 2, -1 2
1 0 0 0
(Q)
{0 1 0 0 ]
(f) 1, 9
(Q) 1 2, 0 9
7
N 1 0
106 Exercises and Problems in Linear Algebra

-5 18
(3) -3 10
(5) (a) 1, 4
(b) 1, _L 0
(c) 1, 0, 1, 4, 0, 1
(d) 1, 0, 2
 Chapter 14

SOME APPLICATIONS OF THE

SPECTRAL TI-IEOREM

14.1 Background

Topics: systems of linear differential equations, initial conditions, steady

state solutions, the functional calculus for operators on finite dimensional
vector spaces, Markov processes.

Theorem 14.1.1. Let

du i Au ( 14.1.1 )
dt
be a vector Di]§"erential equation (that is, a system of ordinary linear difer-
ential equations where A is an n X n matrix and suppose that U0 = u(0)
is an initial value of the system. If A is a diagonalizable matrix (so that
A = SAS for some diagonal matrix A and some inuertible matrix S ) ,
then the equation (14.1.1l has the solution
u(t) = l A i U0 = 5eAt5-1II() .
Definition 14.1.2. A MARKOV MATRIX is a square matrix with nonneg-
ative entries and with each column adding to 1.

Proposition 14.1.3 (Facts about Markov matrices.). L e t A be a

Markov matrix. Then

( i ) A1 = 1 is an eigenvalue.
(ii) The eigenvector 81 corresponding to A1 is nonnegative and it is a
steady state.

107
108 Exercises and Problems in Linear Algebra

(iii) The other eigenvalues satisfy IAkl < 1.

(iv If any power of A has all entries strictly positive, then IAkl < 1 for all
k go 1 and Akug -> u oo where the steady state u oo is a multiple of 81 .

14.2 Exercises

(1) Let A

(a) The eigenvalues of A are A1 = and A2 = .

(b) The corresponding eigenvectors are 81 = ( L a ) and 82 = (a, -a)
where a i

(c) Then
sAt _ C L l + e - b t
- 1- e bt
1-e-bt
1 -|- e
where a ]
and b

(d) Let u(t) = (;z:(t), y(t)). The general solution to the system of equa-
. du . ... ..
tlons = Au wlth initial conditions 51:0 = 3 and Yo = 1 is
:c(tl a + be-et and y(tl = a - be-ct w h e r e s = 7 b= , and
c
. • d
Au under the
(e) Find the steady state solution to the system

initial conditions given in (d). That is, find u OO where

:COO = limt_,oo ;v(tl and yOO = 1im1&_>00 y(t).

Answer: u OO

(2) Suppose that at time t the population y(t) of a predator and the
population a:(t) of its prey are governed by the equations

%= 456 - 29

do :u-l-y.

If at time t = 0 the populations are x = 300 and y = 200, then

the populations at all future times t are ;c(t) = asbt + 200eet and
y(t) = debt + bect where a = ,b= ,c= , and d = .
The long run ratio of populations of prey to predator approaches
to .
 Some Applications of the Spectral Theorem 109

(3) Use matrix methods to solve the initial value problem consisting of
the system of differential equations
d
= nu - 'U - 'w

do _
'u,+2v-'w
dt -
dw
= u - U +2w
dt
and the initial conditions
'u,(0l = 2 11(0) = -2 w(0) = 7.
a€bt 0€bt at.
Answer: u(t) eat; 'u(t) Co 7 and w(t) = bebt + de"
wheres= 7 b= ,C ,andd= .
(4) Consider the initial value problem: y" - y' - 2y = 0 with the initial
conditions Yo = 3 and 3/6 = 3.
du
(a) Express the differential equation in the form it = Au where l l l l l l l l l - - _

1
1_ `l
u (y, z) and Z = y'. Then A is the matrix where a 7

b , and C .
(be The smaller eigenvalue of A is and the larger is The
corresponding eigenvectors are ( 1, a) and (Lb) where Cl

and b =
a O
(c) The diagonal form of A is A = 0 b where a =
b i .
[ ] and

(d) Find the diagonalizing matrix S for A. That is, find S so that
A = S-1AS. Answer: S =
beet
[ ;]
1
_*_ edt
where a
_act
=
edt
and b =

(el The matrix

sAt is 1 -beet + bedt
-|-
ect -|- bedt
where a = 7

b = ,c = , and d = .
(f) The solution to the initial value problem is y(t) =

(5) Use the spectral theorem to solve the initial value problem
y/// 3y//_l_2y/
0
where y(0) = 2, 3/(0) = 0, and y"(0) = 3.
110 Exercises and Problems in Linear Algebra

Answer: y(t) Cl + bet + cent where a 7 b ,c 7

and d =
1
(6) Let G0 = 0 and G1 Q' For each k > 0 let Gk:+2 be the average of
Go and Gk_l_1.
(a) Find the transition matrix A that takes the vector (G1¢+1, Go) to
the vector (Gk-2, Gk:+1).

Answer: A

(b) Find a diagonal matrix A that is similar to A.

Answer: A

(c) Find a matrix S such that A = SAS-1.

Answer: S

(d) Determine the long run behavior of the numbers Go.

Answer: GOO := lirnk__,oo Go =

(7) Let T be the operator on R2 whose matrix representation is

V _ 16 17l' Use the spectral theorem to find \/T. (A square root of T
is an operator whose square is T.)
Answer: \/T = f-1 ,~.ll where Cl = 7 b , and C
L

(8) Let A i
1 2
approximation.)
{
4
Find
1
3
A100. (Write an exact answer not a decimal

1
Answer: A100 4
 Some Applications of the Spectral Theorem 111

(9) Let T be the operator on R3 whose matrix representation is

Fo
-1
-1
_o

1
2
1
0
I
11
•

(a) Write T as a linear combination of projections.

Answer: T = C1 E1 + c2 E2 -|- C3 E3 where 01 = ,C2 7

C3 7

1 1
Cl b -b b a b b -b a
$1 Cl b -b 7 E2 b a b 7 and E3 b b a
Cl -b b b a b b b a
where a and b

(b) Calculate the following: E1 E2 7 E1E3

7 E2E3

(c) E1'l'E2'l'E3

(d) Write T3 as a linear combination of E17 EQ, and E3 .

Answer: T3 = E1 -|- E2 -|- E3.

(10) Let A be the matrix whose eigenvalues are A1 17 A2 1I2,

H
1
and >3 1/3, and whose corresponding eigenvectors are 81 0 7
1

M U
1 0
82 -1 and 3 -1
0 e 0
112 Exercises and Problems in Linear Algebra

(a) Solve the difference equation Xk+1 Axk (where Xi

H
H
10
subject to the initial condition X0 20
30
Answer: Uk a(-blk cdr' , U p = cdr , and 'Wk = a(-b)k where
a = 7 b ,C 7 and d = .
(b) Each Xi can be written as a linear combination of the vectors
( , , ) and ( , , l.
(c) The value of X1000 is approximately ( 7 7

(11) Let A be as in the preceding exercise. Solve the differential equa-

H
10
. dx
son it Ax subject to the initial conditions X0 20 Answer:
30
x(t) = (be-t s e c t 7 sect ,be-t) where a = ,b= , and c .
(12) Suppose three cities A, B, and C are centers for trucks. Every month
half of the trucks in A and half of those in B go to C. The other half
of the trucks in A and the other half of the trucks in B stay where
they are. Every month half of the trucks in C go to A and the other
half go to B.
(a) What is the (Markov) transition matrix that acts on the vector

H
GO
60 (where 00 is the number of trucks initially in A, etc.)?
C0

Answer:

(b) If there are always 450 trucks, what is the long run distribution
of trucks? Answer: 0OO = , bOO = ,Coo .

14.3 Problems

(1) Initially a 2100 gallon tank M is full of water and an 1800 gallon tank
N is full of water in which 100 pounds of salt has been dissolved. Fresh
water is pumped into tank M at a rate of 420 gallons per minute and
 Some Applications of the Spectral Theorem 113

salt water is released from tank N at the same rate. Additionally, the
contents of M are pumped into N at a rate of 490 gallons per minute
and the contents of N are pumped into M at a rate sufficient to keep
both tanks full.
How long does it take (to the nearest second) for the concentration
of salt in tank M to reach a maximum? And how much salt is there
(to three significant figures) in M at that time?
(2) Show that if A is a diagonalizable n X n matrix, then

det(exp A) = exp(tr A).

Hint. What would be a reasonable definition of exp A if A were a diag-

onal matrix?
(3) Explain carefully how to use matrix methods to solve the initial value
problem
in _ y/
6y=0

under the initial conditions Yo 2 and 3/6 14. Carry out the com-
potations you describe.
(4) Explain carefully how to use matrix methods to solve the initial value
problem consisting of the system of differential equations
do
v+'w
dt and the initial conditions v(0)
dw w(0)
'U w
dt
Carry out the computation you describe.
(5) Show how to use the spectral theorem to solve the initial value problem
consisting of the system of differential equations
du
7u 5114-5w

do
2u-I-310-2w
5
dw
So $114-6w
dt
and the initial conditions

u(0) = 2 'u(0) 1 w(0) 1.

114 Exercises and Problems in Linear Algebra

(6) Explain carefully how to use the spectral theorem to find a square root
of the matrix A
2 1
2] • . .
. Illustrate your dlscusslon by carrying out
the computation.
0 1 0
(7) Let A 0 0 0
n n

(a) Does A have a cube root? Explain.

(b) Does A have a square root? Explain.

(8) Let A be a symmetric 2 X 2 matrix whose trace is 20 and whose deter-

minant is 64. Suppose that the eigenspace associated with the smaller
eigenvalue of A is the line as-y = 0. Find a matrix B such that 82 = A.

14.4 Answers to Odd-Numbered Exercises

(1) (a) 0, -2
(b) 5
(c) 2» 2
(d) 2, 1, 2

(e)
H2
2
(3) 3, 2, 5, 4
7 3
(5) 2 3, 2' 2
(Y) 2, -4, 5
1
(9) la) -1: 1 3, 0, 2 7

0 0 0 0 0 0 0 0 0
(be 0 0 0 7 0 0 0 7 0 0 0
0 0 0 0 0 0 0 0 0
1 0 0
(c) 0 1 0
0 0 1
(d) -1, 1, 2 7
1
(11) 30, 20, 2
 Chapter 15

EVERY OPERATOR IS
DIAGONALIZABLE PLUS
NILPOTENT

15.1 Background

Topics: generalized eigenspaces, nilpotent operators

Definition 15.1.1. An operator T on a vector space is NILPOTENT if

T" = 0 for some n e N. Similarly, a square matrix is NILPOTENT if some
power of it is the zero matrix.

Theorem 15.1.2. Let T be an operator on a finite dimensional vector

space V . Suppose that the minimal polynomial for T factors completely
into linear factors

mT(a:) = (cc - A • • • (33 - 1<:)'"

where A l , . . . As are the (clistinetl eigenvalues of T . For each j let WJ

ker(T - API)"°j and Et be the projection of V onto We along W1 -|- ' - ' +
Wj_1 + Wj+1 + ' ' ' + Wk. Then

V=W1€BW2€B°-° ® Wk,
each WJ is invariant under T, and I E1 + -|- Ek . Furthermore, the
operator

D A1E1 -I--°-I-)v¢E1'C>

115
116 Exercises and Problems in Linear Algebra

is diagonalizable, the operator

N T D

is nilpotent, and N eommates with D.

A proof of this theorem can be found in [6], pages 220-223.

Corollary 15.1.3. Every operator on a finite dimensional complex vector

space is the sum of a diagonal operator and a nilpotent one.

Definition 15.1.4. Since, in the preceding theorem, T = D + N where D

is diagonalizable and N is nilpotent, we say that D is the DIAGONALIZABLE
PART of T and N is the NILPOTENT PART of T. The subspace WJ = ker(T -
* j I)"°i is GENERALIZED EIGENSPACE associated with the eigenvalue As .

15.2 Exercises

(1) Let T be the operator on R2 whose matrix representation is

(a) Explain briefly why T is not diagonalizable.

u
Answer:
(b) Find the diagonalizable and nilpotent parts of T. Answer: D
Cl C

{ b]
b a
and N
[ 1
C
-C C
where a = ,b , and c

(2) Let T be the operator on R2 whose matrix representation is

(a) Explain briefly why T is not diagonalizable.

u
Answer: •

(b) Find the diagonalizable and nilpotent parts of T. Answer:

D =
{ 1 and N =
[ 1
1 1 0
(3) Let T be the operator on R3 whose matrix representation is 0 1 0
0 0 0
(a) The characteristic polynomial of T is (A)P(A 1)q where p
and q = .
 Every Operator is Diagonalizable Plus Nilpotent 117

(1b) The minimal polynomial of T is ( y n ( - 1 S

where r and
S

(c) Explain briefly how we know from (b) that T is not diagonalizable.
Answer: •

(d) The eigenspace M1 (corresponding to the smaller eigenvalue of T)

is span{( , , l)} and the eigenspace M2 (corresponding to
the larger eigenvalue) is span{(l, , )}.
(e) Explain briefly how we know from (d) that T is not diagonalizable.
Answer: •

(f) The generalized eigenspace W1 (corresponding to the smaller

eigenvalue) is W1 = span{( , , l)} and the general-
ized eigenspace W2 (corresponding to the larger eigenvalue) is
span{(l,a,a), (a, l,a)}, where a = .
(g) The (matrix representing the) projection E1 of R3 along W2 onto

1
a Cl a
W1 is a Cl a where Cl = and b
a Cl b
(h) The (matrix representing the) projection E2 of R3 along W1 onto

1
a b b
W2 is b a b where a = and b
b b b

1
a b b
(i) The diagonalizable part of T is D b a b and the nilpotent
part of T is b b b

1
b a b
N = b b b where a = and b
b b b

1
a b a
(j) A matrix S that diagonalizes D is a a b where a and
b a a
-D
II

1
a Cl a
(k) The diagonal form A of the diagonalizable part D of T is a b a
a Cl b
where a = and b = .
(1) Show that D commutes with N by computing DN and N D .
118 Exercises and Problems in Linear Algebra

Cl b Cl

Answer: DN = ND = Cl a Cl where a and b

_Cl a a_

1
1 1 0
(4) Let T be the operator on R3 whose matrix representation is 0 1 1
0 0 0

(a) The characteristic polynomial of T is (A)P(A - 1)q where p

and q = .
(be The minimal polynomial of T is ( y n ( - 11S where 7° = and
S

(c) Explain briefly how we know from (b) that T is not diagonalizable.
Answer: •

(d) The eigenspace M1 (corresponding to the smaller eigenvalue of T)

is span{(1, , )} and the eigenspace M2 (corresponding to
the larger eigenvalue) is span{(l, , )}
(e) Explain briefly how we know from (d) that T is not diagonalizable.
Answer: •

(f) The generalized eigenspace W1 (corresponding to the smaller

eigenvalue) is W1 = span{(l, , )} and the general-
ized eigenspace W2 (corresponding to the larger eigenvalue) is
span{(l,a,a), (a, l,a)}, where a = .
(g) The (matrix representing the) projection E1 of R3 along W2 onto
a Cl b
W1 is a Cl -b where a = and b
a Cl b

I
(h) The (matrix representing the) projection E2 of R3 along W1 onto
a b
W2 is b a
1
where a = and b
_b b b

(i) The diagonalizable part of T is D and the nilpotent

part of T is _b b b

1
b a Cl
N i b b b where a and b
b b b
 Every Operator is Diagonalizable Plus Nilpotent 119

(j) A matrix S that diagonalizes D is

-D
II -a b al wheres and

1
a Cl a
(k) The diagonal form A of the diagonalizable part D of T is a b a
a Cl b
where a = and b = .
(1) When comparing this exercise with the preceding one it may seem
that the correct answer to part (i) should be that the diagonal-

1
1 0 0
izable part of T is D 0 1 0 and the nilpotent part of [T]
0 1 0 0 0 0
is N 0 0 1 because D is diagonal, N is nilpotent, and
n

[T] = D + N. Explain briefly why this is not correct.

Answer:

(5) Let T be the operator on R3 whose matrix representation is

a 1 -1]

(a) The characteristic polynomial of T is (A - l)t'(A 2)q where p =

and q = .
(b) The minimal polynomial of T is (A - 1 lfrf( - QlS where 7° =
and S i .
(c) The eigenspace M1 (corresponding to the smaller eigenvalue of T)
is span{(l, , )} and the eigenspace M2 (corresponding to
the larger eigenvalue) is span{(l, , )}.
(d) The generalized eigenspace W1 (corresponding to the smaller eigen-
value) is span{(1, , )} and the generalized eigenspace W2
(corresponding to the larger eigenvalue) is span{(l, a, b), (0, b, a)},
where a = and b = .
120 Exercises and Problems in Linear Algebra

a b
(e) The diagonalizable part of T is D b and the nilpotent

part of T is
r n L

N where Cl 7 b , and C

lzc b

1
a a b
(f) A matrix S that diagonalizes D is b a b where a 7

c b a
b , and C

(g) The diagonal form A of the diagonalizable part D of T is

VX n no
,U 0 | where A = and ,LL

(h) Show that D commutes with N by computing DN and ND.

1
Cl IJ - C
Answer: DN = ND = Cl IJ -c where a = 7 b 7 and
2a b -a
U
II

(6) Let T be the operator on R4 whose matrix representation is

0 1 0 -1
-2 3 0 -1
-2 1 2 -1 .
2 -1 0 3

(a) The characteristic polynomial of T is (A - 2)1' where p

(b) The minimal polynomial of T is (A - by where T' =
a b b b
b a b b
(c) The diagonalizable part of T is D b b a b
where Cl

b b b a
and b
-a b c -b
.
(d) The nilpotent part of T 1S N =
• -CL
_CL
b c
c
-b
_b where Cl 7
b
b= , and C = . CL
b c b
 Every Operator is Diagonalizable Plus Nilpotent 121

(7) Let T be the operator on R5 whose matrix representation is

1 0 0 1 -1
0 1 -2 3 -3
0 0 -1 2 -2 .
1 -1 1 0 1
1 -1 1 -1 2

(a) Find the characteristic polynomial of T.

Answer: CT(A) = (A + 1)*°(A - 1)q where p = and q
(b) Find the minimal polynomial of T.
Answer: MT(A) = (A + 1)"°(A - 1)S where r = and S

(c) Find the eigenspaees M1 and M2 of T.

Answer: M1 = span{(a, 1, b, a, 0)} where a = and b 7 and
M2 = span{(l, a,b, b,b), (b,b, b, l,a)} where a and b
(d) Find the diagonalizable part of T.
a b b b b
b Cl -C C -C
Answer: D b b -a C -C where a 7 b 7 and
b b b Cl b
b b b b a
c

(e) Find the nilpotent part of T.

Cl a a b -b
Cl a a b -b
Answer: N Z Cl a a Cl a where a and b
b -b b -b b
b -b b -b b
(f) Find a matrix S that diagonalizes the diagonalizable part D of T.
What is the diagonal form A of D associated with this matrix?
Cl b Cl a Cl

b a b a a
Answer: S b a Cl b a whe r e a and b
Cl a Cl b b
Cl a Cl a b
122 Exercises and Problems in Linear Algebra

0 0 0 0
0 Cl 0 0 0
and A 0 0 a 0 0 wh e r e a
0 0 0 Cl 0
0 0 0 0 a

15.3 Problems

(1) Explain in detail how to find the diagonalizable and nilpotent parts
V-3 -4 57
of the matrix A |
-6 • Carry out the computations you
-2 1 4
describe.
2 0 2
(2) Consider the matrix A 0 0 2 In each part below explain
n n n
|-

carefully what you are doing.

(a) Find the characteristic polynomial for A.
(b) Find the minimal polynomial for A. What can you conclude from
the form of the minimal polynomial?
(c) Find the eigenspace associated with each eigenvalue of A. Do the
eigenvectors of A span R3? What can you conclude from this?
(d) Find the generalized eigenspaces W1 and W2 associated with the
eigenvalues of A.
(e) Find the projection operators E1 of R3 onto W1 along W2 and E2
of R3 onto W2 along W1.
(f) Find the diagonalizable part D of A. Express D both as a single
matrix and as a linear combination of projections.
(g) Find a matrix S that diagonalizes D. What is the resulting diagonal
form A of D?
(h) Find the nilpotent part N of A. What is the smallest power of N
that vanishes?

(3) Let T be the operator on R3 whose matrix representation is

VI -1 07

(a) Explain how to find the characteristic polynomial for T. Then carry
out the computation.
 Every Operator is Diagonalizable Plus Nilpotent 123

(b) What is the minimal polynomial for a matrix? Find the minimal
polynomial for T and explain how you know your answer is correct .
What can you conclude from the form of this polynomial?
(c) Find the eigenspaces associated with each eigenvalue of T. Do the
eigenvectors of T span R3? What can you conclude from this?
(d) Find the generalized eigenspaces W1 and W2 associated with the
eigenvalues of T.
(e) Find the projection operators E1 of R3 onto W1 along W2 and E2
of R3 onto W2 along W1.
(f) Find the diagonalizable part D of T. Express D both as a single
matrix and as a linear combination of projections.
(g) Find a matrix S that diagonalizes D. What is the resulting diagonal
form A of D?
(ii) Find the nilpotent part N of T. What is the smallest power of N
that vanishes?

15.4 Answers to Odd-Numbered Exercises

(1) (a) The single one-dimensional eigenspace does not span R2. (OR: the
minimal polynomial (A - 3>2
has a second degree factor see
theorem 12.1.11.1
(b) 3, 0, 1
(3) (21) 1, 2
(b) 1, 2
(c) The minimal polynomial has a second degree factor (see
Theorem 12111).
(d) 0, 0, 0, 0
(e) The eigenspaces do not span R3.
(f) 0, 0, 0
(s) 0, 1
(h) 1, 0
(1) 1, 0
(J) 0, 1
(k) 0, 1
(1) 0, 1
(5) (a) 1, 2
(b) 1, 2
(c) 0, 2, 1, 2
124 Exercises and Problems in Linear Algebra

(d) 0, 2, 1, 0
(e) 1, 0, 2
(f) 1, 0, 2
(Q) 1, 2
(1@) 4, 0, 2

(7) (21) 1, 4
(b) 1, 2
(c) 0, 1, 1, 0
(d) 1, 0, 2
(Q) 0,
(f) 0,
 Part 5

THE GEOMETRY OF
INNER PRODUCT SPACES
This page intentionally left blank
 Chapter 16

COMPLEX ARITHMETIC

16.1 Background

Topics: complex arithmetic, absolute value and argument of a complex

number, De Moivre 's theorem.

Notation 16.1.1. Let Cl and b be real numbers and Z = a -|- bi. Then
Red := a (the REAL PART of z) and I m p := b (the IMAGINARY PART of z).

16.2 Exercises

(1) Re I 2-I-3?\
3-411/
'2 -3¢\ _
(2) I m g 2
-l-311/

(3) * 21 + 7'£
1 Qi
|=av-bwherea and b

(4) Are(-Qf + 27) =

- (1-I-2vgli
(5) Let Z 2 ' . Then 210 = a(1 -bi) where a
1+2i
and b = 7 we = , and Are 210 =

(6) If z 1 (1-2), then

\/5 •
Z
365
a -|- bi where a and b

(7) The cube roots of -27 are a + bi, a - bi, and c -|- di where a 7

b= ,C= , and d = .

127
128 Exercises and Problems in Linear Algebra

(8) The complex number w = 1 + 21 has 13 thirteenth roots. The sum of

these roots is a + bi where a = and b = .
(9) Let Z1 = (1, to, l + in, Z2 I (12, 0, 1 i and Z3 (-1 , l + Qi, 7 + 3'£). 7

Show that the set { Z17 Z 2 7 Z 3 } is linearly dependent in (23 by finding

scalars a and 5 such that O5Z1 -1- Bz2 - z3 = 0.
Answer: a Z and B=
1 1 1 i
QQ 1 i
(10) Let A - 1 Then the rank of A is and the
0 1+ i 0 0
2 0 2 Qi
nullity of A is
-2a C 2b
1
(11) Let A Then A-1 a+ b a b c where
2
a-b a+b c
b
6
II

i 0 -z'
(12) If A -21 1 -1-4i 7 then
2 i i 3
a at IN-ai -a-l-bi
A 1
Cl do -c+ai -a-di where a b
7 7

-a + bi IN-ai -a-l-bi
C = , and d =
(13) Consider the initial value problem: y" + 49 = 0 with the initial condi-
sons Yo = 3 and 3/6 = 2.
du
(a) Express the differential equation in the form it = Au where llllllllllll-

r
u (y, 2) and Z = y' . Then A is the matrix where
a = ,b= , and C = .
(b) The eigenvalues of A are and , and the corresponding
eigenvectors are (1, al and (l, -a) where a = .
l'\ 'I

(c) The diagonal form of A is A = where a =

(d) Find the diagonalizing matrix S for A. That is, find S so that
r '|
A = S-1AS. Answer: S = 1 1 where a =
 Complex Arithmetic 129

(e) Find the matrix €A¢. Express your answer in terms of real trigono-
metric functions not complex exponentials. Answer: eA¢
f(tl
r' CL QU) where a = 7 b 7 f(tl Z 7
-696)
L f(tl
and g(t) = .
(f) The solution to the initial value problem is y(t) =

16.3 Problems

(1) Explain in detail how to find all the cube roots of ii.
(2) Show that three points 217 Z2, and Z3 in the complex plane form an
equilateral triangle if and only if

Z1
2
+ Z2 2 + Z3 2 = Z1Z2 + Z1Z3 -|- Z2Z3.

16.4 Answers to Odd-Numbered Exercises

<1 _Q
25

(3) 7, 2
(5) 512, \/5, 1024, -g
(7)
3 w
27 3 2 3 7 3, 0
(9) 2 - 2 , l + 32
(11) 1, 2, 0
(13) (a) 0, 1, 4
(b) 22, -22, 22
(c) 22
(dl 22
(e) 2, 2, cos 22, sin 215,
(f) 3cos2t-I-sin2t
This page intentionally left blank
 Chapter 17

REAL AND COMPLEX INNER

PRODUCT SPACES

17.1 Background

Topics: inner products in real and complex vector spaces, the Schwarz
inequality, the parallelogram law, the Pythagorean theorem, the norm
induced by an inner product, the metric induced by a norm, orthogonal
(or perpendicular) vectors, the angle between two vectors, rowspace and
columnspace of a matrix.

Definition 17.1.1. Let V be a (real or complex) vector space. A function

that associates to each pair of vectors X and y in V a (real or complex) num-
-
ber (x, y) (often written X yl is an INNER PRODUCT (or a DOT PRODUCT)
on V provided that the following four conditions are satisfied:

(al If x, y, Z Q V , then

( x + y , z = (x,z) + (mz)-
(b) I f x , y 6 V and a 6 (I (or R), then

(0(x,y) = 0¢xx,y)~

(cl If x, y Q V, then

(kJ) = (mx)-
(d) For every nonzero X in V we have (x, X) > 0.

131
132 Exercises and Problems in Linear Algebra

Conditions (a) and (b) show that an inner product is linear in its first
variable. It is easy to see that an inner product is conjugate linear in its
second variable. (A complex valued function f on V is CONJUGATE LINEAR
if f ( x + Y) = f(x) -|- f(y) and f(0x) = Ef(x) for all x, y e V and
a e C.) When a mapping is linear in one variable and conjugate linear
in the other, it is often called SESQUILINEAR (the prefix "sesqui-" means
"one and a half"). Taken together conditions (a)-(d) say that the inner
product is a positive definite conjugate symmetric sesquilinear form. When
a vector space has been equipped with an inner product we define the NORM
(or LENGTH) of a vector X by

IIxII :z V<X>»
(Notice that this is the same definition we used in 3.1.2 for vectors in RTL.)
Notation 17.1.2. There are many more or less standard notations for
the inner product of two vectors X and y. The two that we will use inter-
-
changeably in these exercises are X y and ( x , y) .

Example 17.1.3. For vectors as 3817 $ 2 , . . . , n) and y Z (y1,y2, • •

belonging to Rn define
'al

k=1

Then Rn is an inner product space.

Example 17.1.-4. For vectors as 3817 1-27 7 son) and y = (Y1,Y21 aYn
belonging to (In define
'al

(at, y) xkyk,
k 1

Then (Zn is an inner product space.

Example 17.1.5. Let Z2 be the set of all square summable sequences

of complex numbers. (A sequence as = l$Ukl1'Q:=1 is SQUARE SUMMABLE if
2 1 2 1 lxkl2 < oo.) (The vector space operations are defined pointwise.) For
vectors x = (131, 332, . . . ) and y = (y1, YE7 . . . ) belonging to Z2 define
OO

(at, y)
k=1

Then Z2 is an inner product space. (It is important to recognize that in

order for this definition to make sense, it must be verified that the infinite
series actually converges.)
 Real and Complex Inner Product Spaces 133

Example 1'7.1.6. For a < b let C([a, b]) be the family of all continuous
complex valued functions on the interval [a,b]. For every f , g e C([a, b1)
define
b
(f,9) f(11)9(tl ds.
a

Then C([a, b]) is an inner product space.

Definition 17.1.7. Let X and y be nonzero vectors in a real vector

space V. Then &(x, y ), the ANGLE between X and y , is defined by

<(x, y)
_ arccos (x, y) •

IIxII llyll
(Notice that this is the same definition as the one given in 3.1.3 for vectors
in R".I
Definition 17.1.8. Two vectors X and y in an inner product space V
are ORTHOGONAL (or PERPENDICULAR) if (x, y) = 0. In this case we write
X J_ y. Similarly if M and N are nonempty subsets of Vwe write M J_ N
if every vector in M is orthogonal to every vector in N. When one of the
sets consists of a single vector we write x J_ N instead of {a:} J_ N. When
M is a nonempty subset of V we denote by M-L the set of all vectors as such
that 33 J_ M. This is the ORTHOGONAL COMPLEMENT of M .

Definition 17.1.9. A real valued function f on an inner product space V

is UNIFORMLY CONTINUOUS if for every number 6 > 0 there exists a number
5 > 0 such that lf(a3) - f(y)l < 6 whenever llx - yll < 5 in V.

The following result is one we have already seen for R" (see 3.1.4).
Theorem 17.1.10 (Cauchy-Schwarz inequality). If X and y are vec-
tors in a (real 07" eomplexl inner product space, then

l(x,y)l _< IIxII Ilyll-

17.2 Exercises

(1 In (33 let X = (3 + Qi, l - am) and y = (1 - 21z, 1 - i,4 + pa). Then

IIxII Z ; Ilyll = ; and lx, y) =
(2) In (22 let X = (2 - 4i,4i) and y = (2 -|- 4i,4). Then llxll 7

Ilyll Z ; and (x, y) =

134 Exercises and Problems in Linear Algebra

(3) In (c2 let X = (3 - Qi, ii) and y = (1 + i, 1 - z). Then IIxII = 7

= , a n d (x, y ) = 1 - \ f + ( v ' a - b ) i wheres = and

b Z •

(4) Make the family of 2 X 2 matrices of real numbers into an inner

product space by defining (A, B) := tr(AtB) (see problem 8) for A,
1 4 Q2 a -1
B Q M aIR). L t U
e -3 5 an
d V
a + l
Z
-1
values of a such that U J_ V in the inner product space
[ 1 Z

{ Find all
M 2,2lR).
Answer: a = and
i i i 2'£ 'i 'i
(5) Let w Z 7 7 7 X 7 7 7 y Z
W W \/5 W W \/5
'i i 2'£ i i
- and Z 0, -
\/5 W W W W
7 7 7 7

(a) Which three of these vectors form an orthonormal basis for @3?
Answer: , , and .
(b) Write (1, 0, 0) as a linear combination of the three basis vectors
you chose in part (a). (Use 0 as the coefficient of the vector that
does not belong to the basis.)
Answer: (1,0,0) i w + 2i X +6y + C Z W h ere Cl 7
\/5 \ 2a
b= , and C

(6) Find all real numbers a such that the angle between the vectors
2i-|-2j-|- ( 0 - 2 ) k and 2i -|- ( a - Qlj -|- 2k is g. Answer: a
and .
(7) Let f (as) = x and g(a:) = 1:2 for 0 < x _< 1. Then the cosine of the angle
between f and g in the inner product space C([0, 11) of all continuous
real valued functions on [0, 1] is Z where a . Z

(8) Let f (as) = as and g (as) = cos try: for 0 < Hz: _< 1. In the inner product
space C(10, 1]) of all continuous real valued functions on [0, 1] the cosine
of the angle between f and s is -
of
02 where a 7 b 7

and C Z

(9) Let f(a:) = 332 and g(;z:) = 1-ew where 0 < at < 1 and c is a constant. If
C , then f J_ g in the inner product space C([0, l]) of continuous
real valued function on [0, l].
 Real and Complex Inner Product Spaces 135

(10) In R4 the subspace perpendicular to both (1,4,4, 1) and (2, 9, 8, 2) is

the span of the vectors (-4,a, b, a) and (-b, a,a, b) where a =
and b =
(11) A complex valued function f on the interval [-¢r,7r] is said to be
SQUARE INTEGRABLE if f
If (a:)l2 do: < oo. Let F([-7r,7r]) be the
family of all square integrable complex valued functions on [-7r,7r] .
This family is an inner product space under the usual pointwise oper-
ations of addition and scalar multiplication and the inner product
defined by
'ii
1
(f 7 Q)
2tr
f (so) g (so) do
'IT

for all f , g 6 .7E'(1-tr,7r]). Actually the preceding sentence is a lie:

to be correct we should identify any two square integrable functions
that differ only on a set of Lebesgue measure zero and work with the
resulting equivalence classes. Then we have an inner product space.
What we actually have here is a so-called semi-inner product. For the
purposes of the current exercise, however, this correction turns out to
be unimportant; ignore it.
For each integer n (positive, negative, or zero) define a function
eTL by

Ina:
eT(;c) e for tr<a3<7r.

(a) Then each en belongs to .7E'([-7r,¢r]) and Ilene fOr every

integer n.
(b) If m ;é n, then (e,,,,e,,) =
0 'f - _<a:< 0,
Now let f (513) 1 or
1 if 0 _<a:<7r.
(c) Then (f ,en} = .
<
(d) If n is odd, then f , e n) = .
(e) If n is even but not zero, then f , e n) = < .
(f) Write the sum of the middle eleven terms of the Fourier series for
f in simplified form. Hint: Use problem 9 in this chapter.
5
Answer: f , e7;)e7; a + bsina: -|- csin3x + dsin5a3 where
n 5
b ,C and d
6
II

Z
7 7
136 Exercises and Problems in Linear Algebra

17.3 Problems

(1) Prove that if X 6 V and (x, y) 0 for all y 6 V, then X = 0.

(2) Let S, T : V -> W be linear transformations between real inner prod-
uct spaces. Prove that if (Sv, W) = (Tv, w for all V e V and w e W, >
then S = T.
(3) Prove that if V is a complex inner product space and T 6 £(V)
satisfies (Tz, z) = 0 for all Z e V, then T = 0. Hint. In the hypothesis
replace Z first by X -|- y and then by X + iy.
(4) Show that the preceding result does not hold for real inner product
spaces.
(5) Let V be a complex inner product space and S , T e £ ( V ) . Prove that
if (Sx,x) = (Tx, x) for all X e V, then S = T.
(6) Prove the Schwarz inequality 17.1.10. Hint. Let of = (y, y) and B =
-(x, y) and expand flax + Byll2. This hint leads to a slick, easy, and
totally unenlightening proof. Perhaps you can find a more perspicuous
one .
(7) [The polarization identity] If .CU and y are vectors in a complex inner
product space, then
2 2 - - 2 - - 2
(buzz, y) i(ll¢v+yll lx all +zllcv+zyll -ZIIHI-Wll l-
What is the correct identity for a real inner product space?
(8) Let M2,2(R) be the vector space of all 2 X 2 matrices of real numbers.
Show that this space can be made into an inner product space by
defining (A, B) := tr(AtB) for all A, B e M 2,2lRl°
(9) Prove that for every real number 0
€i9
cos 0 -|- 'i sin H.

Derive from this that cos 0 = _1 € i 9 -|- 'é49

and sin H = 1 €i9 w
2 2i

(10) Let X and y be vectors in an inner product space. Prove that

2
IIX + y 112 + IIX yI 2IIXII2 + 2llyll 2
Give a geometric interpretation of this result.
(11) Show that the norm function I - I : V -> R on an inner product space
is uniformly continuous.
 Real and Complex Inner Product Spaces 137

17.4 Answers to Odd-Numbered Exercises

(1) 4, QI, 2 + 12£

(3) 4, 2, 3, 5

(5) (21) We iv, y

(be 3, 0, 0
(7) W
(9) §

(11) (a) 1
(b) 0
1
(c) 2
'i
(d) TI/IT
(G) 0
1 2 2 2
(f) 2 ' 7 r " 3 7 r ' 5 7 r
This page intentionally left blank
 Chapter 18

ORTI-IONORMAL SETS
OF VECTORS

18.1 Background

Topics: orthonormal sets of vectors, orthonormal bases, orthogonal com-

plements, orthogonal direct sums, Gram-Schmidt orthonormalization, the
QR-factorization of a matrix.

Definition 18.1.1. A set B of vectors in an inner product space is

ORTHONORMAL if X J_ y whenever X and y are distinct vectors in B and
llxll = 1 for every X e B. The set B is a MAXIMAL ORTHONORMAL SET pro-
vided that it is orthonormal and the only orthonormal set which contains
B is B itself.

Theorem 18.1.2. Let B = {e1, . . . , e " } be an orthonormal set in an inner

product space V . Then the following are equivalent.

(a) B is a maximal orthonormal set in V .

(b) If (x,ek} = 0 f o r = 1 , . . . , Na thenx= 0.
(c) The span o f B is all of V .
(d) IfX 6 V, then X = 2k=1<x, e":}ek. (the Fourier series for x.)
(e) (x,y) = Ek=1(x,e"'}(ek,y) for all x, y e V.
If) IIxII 2 = E k = 1 l ( x , e k ) l 2 for every X e V.
(go d i V = n.

139
140 Exercises and Problems in Linear Algebra

Definition 18.1.3. An orthonormal set in an inner product space V that

satisfies any (and hence all) of the conditions listed in the preceding theorem
is called an ORTHONORMAL BASIS for V. In this case the scalars (x, ek), that
appear in Theorem 18.1.2 are the FOURIER COEFFICIENTS of X with respect
to B.

Definition 18.1.4. An n X n matrix of real numbers is an ORTHOGONAL

MATRIX if its column vectors are an orthonormal basis for Rn. An n X n
matrix of complex numbers is a UNITARY MATRIX if its column vectors are
an orthonormal basis for CC".

Theorem 18.1.5 (QR-factorization). If A is an n X n matrix of real

numbers, then there exist an orthogonal matrix Q and an upper triangular
matrix R such that A = QR. If A is an n X n matrix of complex numbers,
then there exist a unitary matrix Q and an upper triangular matrix R such
that A = QR.

(For a proof of the preceding theorem see [8], pages 425-427.)

Definition 18.1.6. Let M and N be subspaces of an inner product

space V. We say that the space V is the ORTHOGONAL DIRECT SUM of
M and N if M -|- N = V and M J_ N. In this case we write

V M ® N.

Caution: Since the same notation is used for direct sums of vector spaces
and orthogonal direct sums of inner product spaces, close attention should
be paid to the context in which these concepts arise. For example, if M is
the x-axis and N is the line y = as in R2, is it true that R2 = M ® N? Yes,
if R2 is regarded as a vector space. No, if it is regarded as an inner product
space.

18.2 Exercises

(1) Use the Gram-Schmidt procedure to find an orthonormal basis for the
subspace of R4 spanned by W1 = (1, 0, 0, 0), W2 = (1, 1, 1, Of, and W3 =
(1,2,0, 1). The basis consists of the vectors e1 = (1, 0, 0, 0), 82 =
1 1
- (0, 1, 1, b); and e3 = (b, 1, -1, 1) where a = ,b= 7
a C
and C i .
 Orthonorfrrzal Sets of Vectors 141

(2) Let 774 = 774(10, II) be the vector space of polynomials of degree strictly
less than 4 with an inner product defined by

( p,q ) p (so) q (23) do

0

for all p, q 6 774. Let w0(a:) = 1, w1(a:) = buzz, w2(a3) = 562, and w3(a3) =
323 for 0 < as < 1. Use the Gram-Schmidt process to convert the ordered
basis {w0, w1, w2, w3} to an orthonormal basis {e0, e1, e2, e3} for 774.

Answer: e0 (at)
et) \/E(ba: - 1) where a = and b
82612) (ex2 - be: + 1) where a = and b
83(H2) v-a(ba:3 - c132 + do: - 1) where a 7

b i ,C 7 and d

3
(3) Find the QR factorization of A
[ Q]
4

Hg J
1 b C C
Answer: A QR = where a 7 7 and
Cl C b
C 1

l
0 0 1
(4) Let A 0 1 1 The Q R-factorization of A is A = QR
1

where Q and R

(5) Let A The Q R-factorization of A is A = QR

b -ab 3a 9a pa
where Q a Zab and R 0 2b b where Cl
-b -ab 0 0 ab
and b
142 Exercises and Problems in Linear Algebra

18.3 Problems

(1) Let {e1,e2, ,e"} be a Finite orthonormal subset of an inner product

space V and X 6 V. Show that

TL

§jl(x,ek=)l2 < IIxII 2

k 1

Hint. Multiply out (x - Ek=1<X7 e"')e'*, X - Ek=1<X7 ek)ek).

(2) Let M be a subspace of an inner product space V.
(a) Show that M C M .
(b) Prove that equality need not hold in (a) .
(c) Show that if V is Finite dimensional, then M = M .

(3) Let M and N be subspaces of an inner product space. Prove that

(m+n)_|_ = mJ_ onJ_.
(4) Let M be a subspace of a finite dimensional inner product space V.
Prove that V = M ® M .
(5) Give an example to show that the conclusion of the preceding problem
need not hold in an infinite dimensional space.
(6) Prove that if an inner product space V is the orthogonal direct sum
M ® N of two subspaces M and N, then N = M .
(7) Prove that if f is a linear functional on a finite dimensional inner prod-
uct space V, then there exists a unique vector a e V such that

f(xl (X, a)
for every X 6 V.

(8) In beginning
OO
calculus, you found (by making use of the p-test) that the
series Z p1 converges. But you were not given
. •
a means of discover-
k=1
ing what the series converges to. Now you have enough machinery to
accomplish this.
We denote by L210, 2¢r] the vector space of all complex valued func-
tions f defined on the interval [0, 2¢r] such that
27r

lf(tll2 dt < o o .
0
 Orthonorfrrzal Sets of Vectors 143

(As in Exercise 11 of Chapter 17 this isn't quite correct: the members

of L2 are technically equivalence classes of functions. For the purposes
of this problem, use the preceding not-quite-right definition.)
On the space L2[0, 2¢r] we define the following inner product
2'rr
1
(f 7 g ) 27T 0 f (t)g(tl dt.

For each integer n (positive, negative, or zero) define the function e"
on [0, Z7r] by
Ina:
@"(HE) e

for0<a:<27r.

(a) Show that {eTL: n is an integer} is an orthonormal set in L2[0, Zfr].

In part (b) you may use without proof the following fact: for every
function f in the inner product space $2 [0, Zfr]
OO
2 2
llfll l(f,e')l (*)
Z o

That is, in L2 [0, 2¢r] the square of the length of a vector is the sum of
the squares of its Fourier coefficients with respect to the orthonor-
mal family given in part (a). This is the infinite dimensional version
of Parseval 's formula.
OO
1
(b) Find the sum of the inf Mite series k2 Hint. Apply (*) to the
k=1
function f (iv) co.

18.4 Answers to Odd-Numbered Exercises

(1) \/5, 0, \/5

(3) 5, 3, 4
1 1
(5)
This page intentionally left blank
 Chapter 19

QUADRATIC FORMS

19.1 Background

Topics: quadratic forms, quadric surfaces, positive (and negative) definite,

positive (and negative) semidefinite, indefinite.

Definition 19.1.1. A symmetric matrix A on a real inner product space

V is POSITIVE DEFINITE if (Ax, x) > 0 for every X go 0 in V. It is NEGATIVE
DEFINITE if (Ax, x) < 0 for every X go 0 in V. It is POSITIVE SEMIDEFINITE
if (Ax, x) > _ 0 for every X ¢ 0 in V. It is NEGATIVE SEMIDEFINITE if
( A x , x) < 0 for every X 7é 0 in V. It is INDEFINITE if there are vectors X
and y in V such that (Ax, x) > 0 and (Ay, y) < 0. Of course an operator
on a finite dimensional vector space is positive definite, negative definite,
or indefinite if its matrix representation is positive definite, etc.

The following useful result (and its proof) can be found on page 250
of 191.
Theorem 19.1.2. Let A be a symmetric n X n matrix of real numbers.
Then the following conditions are equivalent:

( a ) A is positive definite;
lb) XtAX > 0 for every X 7é 0 in RTL;
( c ) every eigenvalue A of A is strictly positive;
( d l every leading principal svbmatrix As, (k = 1, . . . , n ) has strictly positive
determinant, and
(e) when A has been put in echelon form (without row exchanges) the pivots
are all strictly positive.

145
146 Exercises and Problems in Linear Algebra

In the preceding, the leading principal submatrifv As is the k X k matrix

that appears in the upper left corner of A.

19.2 Exercises

(1) Suppose that A is a 3 X 3 matrix such that (Ax, xl 131 2 + 5582 2

la b C

33332 + 61:15:32 - 45131133 + 81325133 for all X 6 R3. Then A b d e

L C e f
wheres= ,b= ,c= ,d= ,e = 7 and f =
(2) A curve C is given by the equation 25132 - 72a3y-|- 23y2 = 50. What kind
of curve is C?
Answer: It is a(n) .
(3) The equation 51:2 + 8a:y + 5y2 = 1 describes an ellipse. The principal
axes of the ellipse lie along the lines y = and y = .
(4) The graph of the equation 13132 - Say -|- 7y2 = 45 is an ellipse. The
length of its semimajor axis is and the length of its sernirninor
axis is .
(5) Consider the equation 21:2 -|- 2y2 - 22 - Zxy + 4332 -|- 4yz = 3.
(a) The graph of the equation is what type of quadric surface?
Answer:
(b) In standard form the equation for this surface is
u2 + ,U2 + ,w2 .
(c) Find three orthonormal vectors with the property that in the coor-
dinate system they generate, the equation of the surface is in stan-
dard form.
1 1 1
Answer: II ), ( 0), and \/5 (1,
W
•
7 7 7 7

(6) Determine for each of the following matrices whether it is positive def-
inite, positive semidefinite, negative definite, negative semidefinite, or
indefinite.
2 1
(a) The matrix -1 2 1 is
-1 -1 2
 Quadratic Forms 147

1 1
(b) The matrix 1 is
1

(7) Determine for each of the following matrices whether it is positive def-
inite, positive semidefinite, negative definite, negative semidefinite, or
indefinite.

(a) The matrix

\'12 3
2 5 4 is
Q A n

1 2 0
2 6 -2
(be The matrix Q
-2 5
0 0 -2
0 1 2 2

(cl The matrix 1 0 1 is

2 1 0

2 2 4
(8) Let B 2 b 8 For what range of values of b is B positive definite?
A O iv

Answer :

(9) Let A = 11
_1
a
1
1
Cl
_ For what range of values of a is A positive definite?

Answer :

19.3 Problem

(1) You are given a quadratic form q(w, y, z) = a;c2 + by2 -|- ez2 + Zdazy -I-
2ea:z-|- Zfyz. Explain in detail how to determine whether the associated
level surface q(a:, y, z) = e encloses a region of finite volume in R3 and, if
it does, how to find that volume. Justify carefully all claims you make.
Among other things, explain how to use the change of variables theorem
for multiple integrals to express the volume of an ellipsoid in terms of
the lengths of the principal axes of the ellipsoid.
148 Exercises and Problems in Linear Algebra

Apply the method you have developed to the equation

11362 + by2 -|- 1122 + 4133; - 10:z:z + 4yz = 8.

19.4 Answers to Odd-Numbered Exercises

(1) 1, 3, -2, 5, 4, -3
(3) -az, as
(5) (a) hyperboloid of one sheet
(b) -1, 1
(Q) 1, -2, 1, -1, 1, 1
(7) (al indefinite
(b) positive definite
(c) positive definite
(9) a > 1
 Chapter 20

OPTIMIZATION

20.1 Background

Topics: critical (stationary) points of a function of several variables; local

(relative) maxima and minima, global (absolute) maxima and minima.

Definition 20.1.1. Let f : IR" -> R be a smooth scalar Held (that is, a
real valued function on Rn with derivatives of all orders) and p G R " . The
HESS1AN MATRIX (or SECOND DERIVATIVE MATRIX) of f at p, denoted by
H f l p ) , is the symmetric n X n matrix
i n

Hf(p l <92 f ( p)
3$Ei 35I3j i=1j=1
Um(p )]-

Theorem 20.1.2 (Second Derivative Test). Let p be a critical point of

a smooth scalar field f (that is, a point where the gradient of f is zero). If
the Hessian matrix Hf is positive definite at p , then f has a local minimum
there. If Hf is negative definite at p , then f has a local maximum there. If
Hf is indefinite at p, then f has a saddle point there.

20.2 Exercises

(1) Notice that the function f deeMed by f(;z:,y) = (x2 - Zeal cosy has a
critical point (stationary point) at the point (l, w). The eigenvalues of
the Hessian matrix of f are and , so we conclude
that the point (1, 71) is a (local minimum, local
maximum, saddle point).

149
150 Exercises and Problems in Linear Algebra

(2) Use matrix methods to classify the critical point of the function
f ( : u , y ) = 21:2 -l-Zaiy-l-21:4-y4 - by3 -|- 7y 2 -ily-l-5

as a local maximum, local minimum, or saddle point.

(a) The only critical point is located at 7 •

(b) It is a

(3) Use matrix methods to classify the critical point of the function
1 2
f(x2 y, al I 5134 - my-I-3/2 .CUZ'*'Z -.CU +3
as a local maximum, local minimum, or saddle point.
(a) The only critical point is located at l 7 7 .
(b) It is a

(4) Notice that the function f deeMed by f ( x , y ) = -1 -|- 4(e"' - as) -

5:13 siny + $y2 has a critical point (stationary point) at the origin. Since
the eigenvalues of the Hessian matrix of f are
(both positive, both negative, of different signs) we conclude that the
origin is a (local minimum, local maximum,
saddle point).
(5) Use matrix methods to classify each critical point of the function

f(56» y) = y 3 - § a : 3 - 2 y 2 + 2 a : 2 + y - 7
as a local maximum, local minimum, or saddle point.
Answer: (0, §) is a
(0, ) is a
( , §) is a
( , ) is a .
(6) Use matrix methods to classify each critical point of the function

fw,y,2)==$2y 451:-ysinz for 0 < z < or

as a local maximum, local minimum, or saddle point.
Answer: The critical points are (-1, 7 7 which is
a ; and (1, , 7 which is a
(local minimum, local maximum, saddle
point) .
 Optimization 151

(7) The function f defined by f(a:,y) = $z:2y2 - 256 - 2y has a sta-

tionary point at l , l, At this stationary point f has a
(local minimum, local maximum, saddle
point) .

20.3 Problems

(1) Use matrix methods to classify each critical point of the function
f(vv,y>2) I $633/+ 22 - 3:6 - y + Hz + 5
as a local maximum, local minimum, or saddle point. Justify your con-
clusions carefully.
(2) Let f(a:,y, z) = ;z:2 y - yeZ -|- 21: -|- Z. The only critical point of f is
located at (-1, 1, 0). Use the second derivative test to classify this point
as a local maximum, local minimum, or saddle point. State the reasons
for your conclusion clearly.
(3) Notice that the function f defined by f(;z:,y,z) = 3329 -|- Qxy -|- y -
yez-1 -|- 21: -|- z -|- 7 has a critical point (stationary point) at (-2, 1, 1).
Use the second derivative test to classify this point as a local maximum,
local minimum, or saddle point. State the reasons for your conclusion
clearly.
(4) Explain in detail how to use matrix methods to classify each critical
point of the function
_ 1 2 1
f(;v» y) - -5a3y -|- -
.CU
- y

as a local maximum, local minimum, or saddle point. Carry out the

computations you describe.

20.4 Answers to Odd-Numbered Exercises

(1) -2, -1, local maximum

1
(3) ( El) 1 2 -
7 12
- 7

(b) local minimum

(5) saddle point, 1, local minimum, 1, local maximum, 1, 1, saddle point
(7) 1, 1, saddle point
This page intentionally left blank
 Part 6

ADJOINT OPERATORS
This page intentionally left blank
 Chapter 21

ADJOINTS AND TRANSPOSES

21.1 Background

Topics: adjoint of an operator, transpose of an operator, conjugate trans-

pose.

Definition 21.1.1. Let T: V -> W be a linear transformation between

real inner product spaces. If there exists a linear map T t : W -> V that
satisfies

(Tv, w ) ( v, T o w )
for all V 6 V and w 6 W, then Tt is the TRANSPOSE of T.

In connection with the definition above see Problem 1.

Theorem 21.1.2. Let T : R" -> RM be a linear transformation. Then the

transpose linear transformation T t exists. Furthermore, the matrix repre-
sentation [rt] of this transformation is the transpose of the matrix repre-
sentation of T.

Definition 21.1.3. Let V be a real inner product space and T be an

operator on V whose transpose exists. If T = Tt, then T is SYMMETRIC. If
T cornrnutes with its transpose (that is, if TTt = T t T ) it is NORMAL.

Definition 21.1.4. If [a] is an m x n matrix, its CONJUGATE TRANSPOSE

is the 77, X m matrix [a] .

155
156 Exercises and Problems in Linear Algebra

Definition 21.1.5. Let T: V -> W be a linear transformation between

complex inner product spaces. If there exists a linear map T* : W -> V that
satisfies

(Tv, w ) ( v, T*w
for all v Q V and w Q W, then T* is the ADJOINT (or CONJUGATE TRANS-
posE, or HERMITIAN CONJUGATE) of T. (In many places, T* is denoted by
TH or by To.)

Theorem 21.1.6. Let T : C" -> (lm be a linear transformation. Then the
adjoint linear transformation T* exists. Furthermore, the matrix represen-
tation [T*] of this transformation is the conjugate transpose of the matrix
representation of T.

Definition 21.1.7. Let V be a complex inner product space and T be an

operator on V whose adjoint exists. If T = T*, then T is SELF-ADJOINT
(or HERMITIAN). If T commutes with its adjoint (that is, if TT* = T * T )
it is NORMAL. A matrix is NORMAL if it is the representation of a normal
operator.

Definition 21.1.8. Let V and W be inner product spaces. We make the

vector space V ® W into an inner product space as follows. For V I 7 V2 e V
and w1,w2 e W let

(lV1»W1l» (V2»W2l) = (V1»V2) + (W1»W2)-

(It is an easy exercise to verify that this is indeed an inner product
on V ® W.)

21.2 Exercises

(1) Let C([0, l], (I) be the family of all continuous complex valued functions
on the interval [0, 11. The usual inner product on this space is given by
1

(f g )
7
f g
0

Let (IS be a fixed continuous complex valued function on [0, l]. Define
the operator Met on the complex inner product space C([0, 1],©l by
MM f ) (bf. Then
m
 Adjoint's and Transposes 157

3- i 2 + Qi
(2) Let A 1
. Find Hermitian (that is, self-adjoint1 matri-

ces B and C such that A B -|- ic. Hint. Consider A :|: A*.
Answer: B =
a= 7 b =
1 40
a b co
,C
b-l-c7
d
and C
and d
1
7
-a
a IN-ci b 40 1-
CZ

1 7 where

(3) Let 773 be the space of polynomial functions of degree strictly less than 3
defined on the interval [0, l]. Define the inner product of two polynomi-
als p, q e 773 by (p, q) = f01 p(t)q(t) dt. Then the matrix representation
of the transpose of the differentiation operator D on the space 773 (with

respect to its usual basis {1,t, t2}) is . Hint.

The answer is not the transpose of the matrix representation of D.

(4) Let V be a complex inner product space. Define an operator T : VQBV ->
V ® V by

T(x»y) Z (y,-x)~

Then T*(u, v) 7 I.

21.3 Problems

(1) Let T : V -> W be a linear map between real inner product spaces. If
S : W -> V is a function that satisfies

(Tv, w (v, Sw)

for all V 6 V and all w 6 W, then S is linear (and is therefore the

transpose of T).
(2) Prove Theorem 21.1.2. Show that, in fact, every linear map between
finite dimensional real inner product spaces has a transpose.
(3) Let T be a self-adjoint operator on a complex inner product space V.
Prove that ( T x , x) is real for every X e V.
(4) Let T be an operator on a complex inner product space whose adjoint
T* exists. Prove that T*T = 0 if and only if T = 0.
158 Exercises and Problems in Linear Algebra

(5) Let V be a complex inner product space and let Q25 be defined on the
set Ql(V) of operators on V whose adjoint exists by

W) T*.

Show that if S, T 6 Ql(V) and a 6 (C, then (S -|- T)* = S* -|- T* and
(0T)* = E T * . Hint. Use Problem 5 in Chapter 17.
Note: Similarly, if V is a real inner product space, Ql(V) is the set of
operators whose transpose exists, and q25(T) := T*, then (15 is linear.
(6) Let T be a linear operator on a complex inner product space V. Show
if T has an adjoint, then so does T* and T** = T . Hint: Use Problem 5
in Chapter 17. ( Here T** means (T*l*.)
Note: The real inner product space version of this result says that if
T is an operator on a real inner product space whose transpose exists,
then the transpose of T* exists and T** = T.
(7) Let S and T be operators on a complex inner product space V. Show
that if S and T have adjoints, then so does ST and (ST)* = T*S*.
Hint. Use Problem 5 in Chapter 17.
Note: The real inner product space version of this says that if S and
T are operators on a real inner product space and if S and T both
have transposes, then so does ST and (ST)* = T*S*.
(8) Let A : V -> V be an operator on a real inner product space. Suppose
that At exists and that it commutes with A (that is, suppose AAt =
AtA). Show that kerA = her At.
(9) Let A and B be Hermitian operators on a complex inner product
space. Prove that AB is Hermitian if and only if AB = BA.
(10) Show that if T: V -> W is an invertible linear map between complex
inner product spaces and both T and T-1 have adjoints, then T* is
invertible and (T*)-1 = (T-1)*.
Note: The real inner product space version of this says that if
T : V -> W is an invertible linear map between real inner product
spaces and both T and T-1 have transposes, then Tt is invertible and
(Ttl-1 Z (T-l)t.

(11) Every eigenvalue of a self-adjoint operator on a complex inner prod-

uct space is real. Hint. Let x be an eigenvector associated with an
eigenvalue A of an operator A. Consider Allxll2.
 Adjoint's and Transposes 159

(12) Let A be a self-adjoint operator on a complex inner product space.

Prove that eigenvectors associated with distinct eigenvalues of A are
orthogonal. Hint. Use problem II. Let x and y be eigenvectors asso-
ciated with distinct eigenvalues A and ,LL of A. Start your proof by
showing that Mac, y) = ,u.(a:,y).

21.4 Answers to Odd-Numbered Exercises

(l) <l>

(3)
I 12
0
-24
30
-261
30
This page intentionally left blank
 Chapter 22

THE FOUR FUNDAMENTAL

SUBSPACES

22.1 Background

Topics: column space; row space, nullspace, left nullspace, lead variables
and free variables in a matrix, rank, row rank and column rank of a matrix.
Definition 22.1.1. A linear transformation T : R" -> lR"" (and its asso-
ciated standard matrix [TD have four fundamental subspaces: the kernels
and ranges of T and Tt. Over the years a rather elaborate terminology has
grown up around these basic notions.
The NULLSPACE of the matrix [T] is the kernel of the linear map T.
The LEFT NULLSPACE of the matrix [T] is the kernel of the linear map Tt.
The COLUMN SPACE of [T] is the subspace of RM spanned by the column
vectors of the matrix [T]. This is just the range of the linear map T.
And finally, the ROW SPACE of [T] is the subspace of R" spanned by the
row vectors of the matrix [T]. This is just the range of the linear map Tt.
For a linear transformation T : CC" -> CM the terminology is the same.
EXCEPT: in the preceding five paragraphs each appearance of ccTt77 must
be replaced by a "T*" (and, of course, IR" by QC" and RI" by CM).
Definition 22.1.2. The ROW RANK of a matrix is the dimension of its row
space and the COLUMN RANK of a matrix is the dimension of its column
space.

Proposition 22.1.3. The rank of a matrix A is the dimension of the

largest square submatrix of A with nonzero determinant.

161
162 Exercises and Problems in Linear Algebra

Two useful facts that you may wish to keep in mind are:

(i) row equivalent matrices have the same row space (for a proof see [6] ,
page 56); and
(ii) the row rank of a matrix is the same as its column rank (for a proof
see [6], page 72).

Note that according to the second assertion the rank of a linear map T, the
row rank of its matrix representation [T], and the column rank of [T] are
all equal.

Theorem 22.1.4 (Fundamental Theorem of Linear Algebra). If T

is an operator on a #note dimensional complex inner product space, then

kerT = (ranT*)J_.

Corollary 22.1.5. If T is an operator on a #note dimensional complex

inner product space, then

kerT* = ( r a n T ) .

Corollary 22.1.6. If T is an operator on a #note dimensional complex

inner product space, then

ranT = ( k e r T * ) .

Corollary 22.1.7. If T is an operator on a #note dimensional complex

inner product space, then

ranT* = ( k e r T ) .

Note: With the obvious substitutions of Tt for T*, the preceding theorem
and its three corollaries remain true for finite dimensional real inner product
spaces.

22.2 Exercises

(1) Let T : (23 -> (23 be the operator whose matrix representation is

[T] 1 -I-i
_1 Qi -6 + 5'£ 1
 The Four Fundamental Subspaces 163

(a) The kernel of T (the nullspace of [T]) is the span of

{( 7 7 10)}.

(b) The range of T (the column space of [T1) is the span of

{(1, 0, ), (0, 1, )}-

(c) The kernel of T* (the left nullspace of T*) is the span of

{(3, 7 )}-
(d) The range of T* (the row space of [T1) is the span of

{(10, 0, ), (0, 10, )}~

(2) Find a basis for each of the four fundamental subspaces associated with
the matrix

1
1 2 0 1
A 0 1 1 0
1 2 0 1

(a) The column space of A is the plane in R3 whose equation is

. I t is the span of {(1,0, ), (0, 1, )}.
(b) The nullspace of A is the span of {( , -1, ,0),( ,0, 0, 1)}.
(c) The row space of A is the span of {(1,0, , 1), (0, ,1,0)}.
(d) The left nullspace of A is the line in R3 whose equations are
= = 0. It is the span of {( , , 1)}.

'1 2 0 2 -1 1
(3) Let A = |»,
r:
6 1 1 -2 1 .
10 1 5 -4 3
(a) Find a basis for the column space of A.
Answer:

{(1» 0, )» (0, 7

(b) The column space of A is a plane in R3 . What is its equation?

Answer:
(c) The dimension of the row space of A is
164 Exercises and Problems in Linear Algebra

(d) Fill in the missing coordinates of the following vector so that it lies
in the row space of A.

(4, 7 6, 7 7

(e) The dimension of the nullspace of A is .

(f) Fill in the missing coordinates of the following vector so that it lies
in the nullspace of A.

7 1, ,1,1,1

(4) Let T : R5 -> R3 be defined by

T(v,w,a2,y,z)=(v Hz: -|- z,'u + w-y,w+:u-y-z).

(a) Find the matrix representation of T .

(b) The kernel of T (the nullspace of [T]) is the span of

{( 7 , L 0, 0), (0, 1, 0, 1, 0), (-1, 1, 0, 7 1)}-

(c) The range of T (the column space of [T]) is the span of

{(1, 7 -1), (0, 1, )}~

Geometrically this is a .
(d) The range of Tt (the row space of [T]) is the span of

{(1, 0, , 0 , 1l>( 7 7 7 - 1 7 -1)}-

(e) The kernel of Tt (the left nullspace of [TD is the span of

{( 1, 7 )}-
Geometrically this is a

1 0 2 0 -1
1 2 4 -2 -1
(5) Let A be the matrix 0 Find the following sub-
1 1 -1 0
2 3 7 -3 -2
spaces associated with A.
 The Four Fundamental Subspaces 165

(a) The column space of A is the span of

{(1, 0, -1/2, I, (0, 1, 7 )}~

(b) The row space of A is the span of

{(1, 0, 7 0, ), (0, 1, 1, 7 of.

(c) The nullspace of A is the span of
{( -1: 1: 0: O): (0, 1, 0,
7 7 0),
(1, 0, 0, 7 )}.

(d) The nullspace of At is the span of

{( 7 , 1 , 0), (-1/2, 7 0, 1>}.

1 -2 -1 3 2
2 4 2 -6 -4
5 -10 -1 15 0
(6) Let A
3 -6 1 9 -4 .
3 -6 -1 9 1
0 0 2 0 -5
(a) The nullspace of A is the span of
{(2, 1, 0, 0, 0), ( 7 0, 0, 1, 0),
( 7 0, 7 0, 1>}.

(b) The row space of A is the span of

{(1, -2, 0, 7 -1/2), (0, 0, 1, 7 )}-
(c) The column space of A is the span of

{(1» -2: 0: 7 1/2, iv (0, 0, 1, 7 7 1I2)}~

(d) The left nullspace of A is the span of

{(2, 0, -1, 1, 0, 0 ) , ( ,1, 0, 0, 0, 0), (-1/2, 0, 7 0, 1, 0),
( 7 0, -1/2, 0, 0, 1)}.

2 1
(7) Let A |2 4 3
166 Exercises and Problems in Linear Algebra

(a) Fill in coordinates of the following vector X so that it is perpendic-

ular to the rowspace of A. Answer: X = (10, , l.
(b) Fill in coordinates of the following vector y so that it is perpendic-
ular to the columnspace of A. Answer: y = (3, 7 ).
(8) In this exercise we prove a slightly different version of the fundamental
theorem of linear algebra than the one given in Theorem 22.1.4. Here
we work with real inner product spaces and the scope of the result is
not restricted to finite dimensional spaces, but we must assume that
the linear map with which we are dealing has a transpose.

Theorem 22.2.1 (Fundamental theorem of linear algebra). Suppose

that V and W are arbitrary real inner product spaces and that the linear
transformation T: V -> W has a transpose. Then
kerT = ( r a n T t ) .
We prove the preceding theorem. For each step in the proof give the appro-
priate reason. Choose from the following list.
DK Definition of "Kernel"
DO Definition of "Orthogonal"
DOC Definition of "Orthogonal Complement"
DR Definition of "Range"
DT Definition of "Transpose"
H Hypothesis
PIP Elementary Property of Inner Products

Proof. We must show two things: (i) kerT C (ran Tt)J- and (ii)
(ranTt) C kerT.
To prove (i) we suppose that as e kerT and prove that x e (ran Tt)J_.
Let 'U be a vector in ran Tt. Then there exists a vector w in W such that
'U = Ttw (reason: l. We compute the inner product of as and 'U.
(buzz, U) = (at, T%w)
= (Taz, w) (reason:
= (0, w) (reason: and
0 (reason:

From this we infer that as J_ 'U (reason: and consequently that

x e (ran Ttl-L (reason: l.
 The Four Fundamental Subspaces 167

To prove the converse we suppose that Hz: 6 (ran Ttl-L and show that
x 6 her T. We know that an J_ ran Tt (reason: and l. If w e W
then the vector Ttw belongs to ran Tt (reason: ), so J: J_ Ttw for all
w e W. Thus for all w e W

0 (SU, Ttfw) (reason:

(Tai, w) (reason:

It follows from this that Tab 0 (reason: ). That is, as Q kerT

(reason: l.

(9) The matrix

33
3/
Z
I has rank one if and only if the point (33, y, z)
lies on the parametrized curve r(t) ( , t, ) in R3. Hint.
Use Proposition 22.1.3.
(10) Let A be the 3 X 4 matrix whose nullspace is the subspace of R4
spanned by the vectors (l,0, l,0) and (0, 1, l,0). Then the vectors
( , , , go and (0, , , ) form an orthonor-
mal basis for the row space of A.
( I I ) Let T : R3 -> R2 be the linear transformation whose matrix represen-
t t' . 1 0 2
[
a 1on is
1 1 4 1
and let X (5, 4, -9).

(a) Find u 6 kerT and V 6 ran Tt such that X u -|- V.

Answer: u ( , , ) and V =( , 7

(b) Find y 6 ranT and Z 6 her Tt such that Tx = y -|- z.

Answer: y l 7 l and Z l 7 l.

22.3 Problems

(1) Let T: R5 -> R4 be a linear transformation whose matrix representa-

tion is

1 2 0 3
-2 -4 3 1 0
[T] -1 -2 3 -4 3
1 2 3 -14 9
168 Exercises and Problems in Linear Algebra

Gauss-Jordan reduction applied to [T] yields the matrix

1 2 0 -5 3
0 0 1 -3 2
on

and applied to the transpose of [T] yields

II

0 0 0 0 0
0 0 0 0 0
1 0 1
0 1 1 31
1
C: 0 0 0 0
0 0 0 0
0 0 0 0

(a) From the matrices above we can read off the dimension of the range
of T and write down a basis for it. Explain carefully.
(b) From the matrices above we can read off the dimension of the
range of the transpose of T and write down a basis for it. Explain
carefully.
(c) From the matrices above we can write down two equations that
a vector (u, w, co, y, z) must satisfy to be in the kernel of T.
Explain carefully. What are the equations? Also explain carefully
how we obtain from these equations the dimension of the ker-
nel of T and find a basis for it. Carry out the calculation you
describe.
(d) From the matrices above we can write down two equations that a
vector (w, x, y, z) must satisfy to be in the kernel of the transpose
of T. Explain carefully. What are the equations? Also explain care-
fully how we obtain from these equations the dimension of the ker-
nel of Tt and find a basis for it. Carry out the calculation you
describe.

(2) Let T: R6 -> R3 be a linear transformation whose matrix representa-

tion is

"1 2 0 2 1 1
[T] r 3 6
10
1
1
1
5
2
4
1
3
7
 The Four Fundamental Subspaces 169

Gauss-Jordan reduction applied to [T] yields the matrix B =

F1 2 0 2 -1 11
0 1 -5 1 |
-2 , and applied to the transpose of [T] yields
0 0 0 0 0 1

-1 0 2-
0 1 1
0 0 0
C ' 0 0 0 '

0 0 0
0 0 0

(a) From the matrices above we can read off the dimension of the range
of T and write down a basis for it. Explain carefully.
(b) From the matrices above we can read off the dimension of the
range of the transpose of T and write down a basis for it. Explain
carefully.
(c) From the matrices above we can write down two equations that a
vector ('u.,v, w , ac, y, z) in R6 must satisfy to be in the kernel of T.
Explain carefully. What are the equations? Also explain carefully
how we obtain from these equations the dimension of the kernel of
T and find a basis for it. Carry out the calculation you describe.
(d) From the matrices above we can write down two equations that a
vector (as, y, z) in R3 must satisfy to be in the kernel of the trans-
pose of T. Explain carefully. What are the equations? Also explain
carefully how we obtain from these equations the dimension of the
kernel of Tt and find a basis for it. Carry out the calculation you
describe.

(3) Let T: R6 -> R5 be the linear transformation whose matrix represen-

tation is

1 2 -1 -2 3 0
0 0 0 1 1 2
A [T] 2 4 -2 -4 7 4
0 0 0 -1 1 2
3 6 -3 -6 7 8
170 Exercises and Problems in Linear Algebra

You may use the following fact: the reduced row echelon forms of the
augmented matrix [A | b ] and of At are

1 2 1 0 0 8 361 -|- 262 - 63

0 0 0 1 0 -261 -1- 62 -1- bg
B 0 0 0 0 1 -261 + 63 and
0 0 0 0 0 0 62 -|- 64
0 0 0 0 0 0 -761 -|- 263 -|- 65

` 1 0 0 0 7
0 1 0 1 0
0 0 1 0 _Q .
7 respectlvely.
0 0 0 0 0
0 0 0 0 0
_0 0 0 0 0

t
u
Suppose that X 'u
w
y
Z

(a) What are the free variables of the system Ax = 0 and which are
the lead variables? How do you know?
(be What is the rank of A? Why?
(c) Write a general solution to the homogeneous equation Ax = 0 as
a linear combination of vectors in R6 using the free variables as
coefficients. Explain.
(d) Explain how to find the dimension of and a basis for the kernel of
T. Do so.
(e) Explain how to find the dimension of and a basis for the range of
T. Do so.
61
62
(f) What conditions must the vector b 63 satisfy in order that the
64
65
nonhomogeneous equation Ax = b have solutions?
 The Four Fundamental Subspaces 171

(g) Find, if possible, the general solution to the nonhomogeneous equa-

1
-3 1
son Ax 2 . (Write your answer as a general solution to the
3
3
homogeneous equation plus a particular solution.)
(ii) Explain how to find the dimension of and a basis for the range of
Tt. Do so.
(i) Explain how to find the dimension of and a basis for the kernel of
Tt. Do so.

(4) Prove the three corollaries to the fundamental theorem of linear

algebra 22.1.4 for complex inner product spaces.

22.4 Answers to Odd-Numbered Exercises

(1) (a) 9-31, 3 - t z

(be 3, -2
(c) -2, -1
(d) -9-31, - 3 - i

(3) (a) 2, 1
(b) Za: -|- y - z = 0
(c) 2
(dl 8, -22, 2, -8
(G) 4
(f) -4, 6
1 1
(5) la) §= §= §
(be 2, -1, -1
IcI -2, 1, 0, 1
1 l 3
(dl §' -§' -5
(7) (21) -5: 0
(b) 3, -3
(9) 62, t3
(11) (21) 6, 6, -3, -1, -2, -6
(b) -13, -27, 0, 0
This page intentionally left blank
 Chapter 23

ORTI-IOGONAL PROJECTIONS

23.1 Background

Topics: orthogonal and unitary operators, orthogonal projections

Definition 23.1.1. A linear operator T : V -> V on a real inner product

space is ORTHOGONAL if it is invertible and Tt = T-1. A matrix is ORTHOG-
ONAL if it is the representation of an orthogonal operator. An operator T on
a complex inner product space V is UNITARY if it is invertible and T* = T-1.
A matrix is UNITARY if it is the representation of a unitary operator.

The definitions for orthogonal and unitary matrices given above differ
from the ones offered in 18.1.4. In Problem 8 you are asked to show that in
both cases the definitions are equivalent.

Definition 23.1.2. An operator T on an inner product space V is an

ISOMETRY if it preserves the distance between vectors. Equivalently, T is
an isometric if lITxll = llxll for every x e V.

Definition 23.1.3. Let V be an inner product space and suppose that it is

the vector space direct sum of M and N. Then the projection EmnZ V -> V
is an ORTHOGONAL PROJECTION if M J_ N (that is if V is the orthogonal
direct sum of M and N ) .

Proposition 23.1.4. A projection E on a complex inner product space V

is on orthogonal projection if and only if E is self-adjoint. On a real inner
product space a projection is orthogonal if and only if it is symmetric.

For a proof of (the real case of) this result see exercise 4.

173
174 Exercises and Problems in Linear Algebra

Definition 23.1.5. We say that a linear operator T on an inner product

space V is POSITIVE (and write T > 0) if (Tx, X) > 0 for all x e V. If
S and T are two linear operators on V, we say that Q DOMINATES (or
MAJORIZES) P if Q - P > 0. In this case we write P g Q.

23.2 Exercises

(I) The matrix representation of the orthogonal projection operator tak-

ing R3 onto the plane .CU + y -|- Z = 0 is

(2) Find a vector u (u1,u2,u3) in (C3 such that the matrix

_1
m
-1
m U1
'i -¢
m m U2 is unitary.
1-i 1-'é
m m U3

1
Answer: u (2-l-a1, 3-bi, c4-di) where a = 7 b ,c 7

d= 7 m , and n = .
(3) The orthogonal projection of the vector (2,0, -1,3) on the plane
spanned by ( - l , l , 0 , l ) and (0,l,1, 1 ) in R4 is ( l , a , b , a l where
a = and b = . The matrix that implements this orthogo-
C -d e -d
1 -d e d e
al projection is 5 where C 7 d 7 and
e d C d
-d e d e
e

(4) Let E be a projection operator on a real inner product space. Below

we prove (the real case of) Proposition 23.1.41 that E is an orthogonal
projection if and only if E = Et. Fill in the missing reasons and steps.
Choose reasons from the following list.

(DK) Definition of "kernel" .

(DL) Definition of "linear" .
(DO) Definition of "orthogonal" .
(DOP) Definition of "orthogonal projection" .
(DT) Definition of "transpose" .
 Orthogonal Projections 175

(GPa) Problem 5 in chapter 5.

(GPb) Problem 1 in chapter 10.
(GPc) Problem 5 in chapter 17.
(GPd) Problem 6 in chapter 21.
(H1) Hypothesis that M J_ N.
(H2) Hypothesis that E = Et.
(PIP) Elementary property of Inner Products.
(Ti) Theorem 10.1.2, part (i).
(Tiii) Theorem 10.1.2, part (iii).
(Tiv) Theorem 10.1.2, part (iv).
(VA) Vector space arithmetic (consequences of vector
space axioms, etc.)
Let E = EMN be a projection operator on a real inner product
space V = M ® N. Suppose first that E is an orthogonal projec-
tion. Then M J_ N (reason: ). If x and y are elements in V,
then there exist unique vectors 111, p 6 M and n , q e N such that
x = m + n and y = p - l - q (reason: )~ Then
(Ex, y = <E(m + n), p -I- q )
= (Em + En, p + q ) (reason: and
= (0 ' E I L P + q) (reason: and
= (E n , p + q ) (reason:
(n,p-l-q ) (reason: and
( u p ) + ( n ,o ) (reason:
0 + (n, <1) (reason: and
(Ill,q ) + ( n 7q ) (reason: and
(x, q) (reason: )
(x,E q ) (reason: and
(x, 0 Eq )
-|- (reason:
(x,E p 4-E q > (reason: and
(x,E( p -|- q )) (reason: and
(x, Etty) (reason: )
(Etx, y (reason: ).
From this we conclude that E = Et (reason:
176 Exercises and Problems in Linear Algebra

Conversely, suppose that E = Et. To show that M J_ N it is enough

to show that III J_ n for arbitrary elements III e M and n e N.
(n m ) 7
(En, III (reason: and )
In, Etm) (reason:
In Em
7 (reason: )
In 07 (reason: and
0 (reason:
Thus III J_ n (reason: l.
Note: Of course, the complex inner product space version of the pre-
ceding result says that if E is a projection operator on a complex inner
product space, then E is an orthogonal projection if and only if it is
self-adjoint.
(5) Let P be the orthogonal projection of R3 onto the subspace spanned
To A I" '|
1
by the vectors ( l , 0 , l ) and ( l , l , - 1 ) . Then [P] = 6

where a = ,b= , and C .

(6) Find the image of the vector b = (1, 2, 7) under the orthogonal pro-
r 1 1 7

jection of R3 onto the column space of the matrix A

Answer: l , , l.
(7) Let u = (3,-1,1,4,2) and V (1,2,-1,0,1). Then the orthogonal
projection of u onto V is ( , , , , ).
8'
(8) Let u = (8,v§, W,-1, 1) and = (1, -1, 0, 2, \/§). Then the orthog-
oral projection of u onto V is v where a = and b =

(9) Let u = (5,4,3, 5) and V = (1, 2, 0, -2). Then the orthogonal projec-
son of u onto V is v where a = and b =

(10) Find the point q in R3 on the ray connecting the origin to the point
(2,4, 8) that is closest to the point (1, 1, 1).
Answer: q 5l i
7 7

(11) Let e1 = kg, -,

2
3
-8
and 62 = (--,
1 2
-,
3 3
be vectors in R3. Notice that
{et, e2} is an orthonormal set.
 Orthogonal Projections 177

(a) Find a vector 83 whose first coordinate is positive such that

B = { € 1 v € 2 7 € 3 } is an orthonormal basis for R3. Answer:
1
3( 7 7 m
(b) Suppose that X is a vector in R3 whose Fourier coefficients with
respect to the basis B are: (x,e1) = -2, {x,e2) = -1; and
(x, e l = 3. Then x = l , , ).
(c) Let y be a vector in R3 whose Fourier coefficients with respect to
B are

(y, 61) \ / 8 - ;

(y, 62 ) 5- JE ; and

(y, 63 ) 3+m+?.
Then the length of the vector y is .
(d) The orthogonal projection of the vector b = (0, 3, 0) onto the plane
spanned by 81 and 62 is 3 l , , l.
(e) The orthogonal projection of the vector b = (0, 3, 0) onto the line
spanned by e3 is 3 ( , , l.
(f) What vector do you get when you add the results of the projections
you found in parts (d) and (elf Answer: l , , ).

23.3 Problems

(1) Prove that an operator T : V -> V on a finite dimensional real inner

product space V is orthogonal if and only if it is an isometric. Similarly,
on a finite dimensional complex inner product space an operator is
unitary if and only if it is an isometric.
(2) Prove that an operator T : V -> V on a finite dimensional real inner
product space V is orthogonal if and only if TtT = I. What is the
corresponding necessary and sufficient condition on a finite dimensional
complex inner product space for an operator to be unitary?
(3) Show that if an operator U on a complex inner product space is both
Hermitian and unitary, then o(U) C {-1, l}.
178 Exercises and Problems in Linear Algebra

(4) Let P and Q be orthogonal projections on a real inner product space.

Show that their sum P -|- Q is an orthogonal projection if and only if
PQ = QP = 0. Hint. Use Proposition 23.1.4.
(5) Explain in detail how to find the matrix that represents the orthog-
onal projection of R3 onto the plane Hz: -|- y - Zz = 0. Carry out the
computation you describe.
(6) Let P and Q be orthogonal projection operators on a real inner product
space V.
(a) Show that the operator PQ is an orthogonal projection if and only
if P commutes with Q.
(b) Show that if P commutes with Q, then

ran(PQ) = ran P VW ran Q.

Hint. To show that ramP O rank C ran(PQ) start with a vector y

in ran P O ran Q and examine PQy.

(7) Let P and Q be orthogonal projections on an inner product space V.

Prove that the following are equivalent:
(a) P S Q;
(be lIPxll < lIQxll for all X e V;
(c) ramP C ranQ;
(dl QP = P; and
(e) PQ = P.
Hint. First show that (d) and (e) are equivalent. Then show that
(a) => (b) => (c) => (d) => (a). To prove that (b) => (c) take an
arbitrary element as in the range of P; show that lIQxll = llxll and
that consequently ll(I - Q)xll = 0. To prove that (d) => (a) show that
(I - P)Q is an orthogonal projection; then consider ll(I - P)QII2.
(8) In 18.1.4 and 23.1.1 the definitions for unitary matrices differ. Show
that they are, in fact, equivalent. Argue that the same is true for the
definitions given for orthogonal matrices.

23.4 Answers to Odd-Numbered Exercises

1
(1) 3
2
 Orthogonal Projections 179

(3) 3, 4, 3, 1, 2
(5) 5, 2, 1
(7) _2 _ 4_ 2
_ ( ) _
2
7'7' 7' '7
(9) 4, 3
(11) (a) 2, -1, 2
(b) 1, -3, 2
(c) 4
(d) 1, 4, 1
(Q) -2, 1, -2
(f) 0, 3, 0
This page intentionally left blank
 Chapter 24

LEAST SQUARES
APPROXIMATION

24.1 Background

Topic: least squares approximation.

24.2 Exercises

(1) Let A

(a) Find an orthonormal basis {e1,e2,e3} for R3 such that {e1,e2}

spans the column space of A.

el = l(a, b, -6)

e2 = i(b, Q, b)

e3 = i(b,

where a ,b= , and n = .

(b) To which of the four fundamental subspaces of A does 83 belong?
Answer: 83 belongs to the of A.
(c) What is the least squares solution to Ax = b when b (1, 2,7)?
Answer: SE = 7 .

181
182 Exercises and Problems in Linear Algebra

(2) Find the best least squares fit by a straight line to the following data:
33 lwhent= 1., a : = 3 w h e n t 0; Hz: 2whent=l;and:c 3
when t = 2.
Answer: as + t.

(3) At times t = -2, -1, 0, 1, and 2 the data y = 4, 2, -1, 0, and 0,

respectively, are observed. Find the best line to fit this data. Answer:
y=Ct+DwhereC= a d D = .
(4) The best (least squares) line lit to the data: y = 2 at t = -1; y = 0
a b
att=0,y=-3att=1;y=-5att=2isy = - - - - t where
10 5
a and b = .
(5) Consider the following data: y = 20 when t = -2; y = 6 when t = -L
y=2whent=0;y=8whent=l,y=24whent= 2. Find the
parabola that best fits the data in the least squares sense.
Answer: y = C -|- Dt -|- E152 where C = , D = 7 and
E i .
(6) Consider the following data: y = 2 when t = -1, y = 0 when t = 0;
y = -3 when t = 1; y = -5 when t = 2. Find the parabola that best
fits the data in the least squares sense.
Answer: y = C -|- Dt -|- E152 where C = 7 D 7 and
E i

(7) Find the plane 50z = a -|- bu -|- ev that is the best least squares fit to
the following data: Z = 3 when u = l a n d ' u = 1 , z = 6 w h e n u Oand
v = 3 , z = 5 w h e n ' u , = 2 a n d v = 1 ° , z = 0 w h e n 'LL = 0 and 0. i

Answer: a = , b=
(8) Consider the following data: y = 4 at t = -1; y = 5 at t = 0; y = 9 at
t 1.
i

(a) Then the best (least squares) line that fits the data is y = c -|- dt
where c = and d = .
(b) The orthogonal projection of b (4, 5, 9) onto the column space
I'1 _11

of A =
i 1
1
0
1 i is ( 7 7 .
 Least Squares Approximation 183

(9) The best least squares solution to the following (inconsistent) system
u= 1
of equations 'U = 1 is 'LL and 'U

u -|- U 0

24.3 Problems

(1) Explain in detail how to use matrix methods to find the best (least
squares) solution to the following (inconsistent) system of equations
'U Z 1
'U Z 1 Carry out the computation you describe.
u -|- 'U 0
(2) The following data y are observed at times t: y = 4 when t = -2, y = 3
whent=-Ly:lwhent=0;andy=0whent=2.
(a) Explain how to use matrix methods to find the best (least squares)
straight line approximation to the data. Carry out the computation
you describe.
(b) Find the orthogonal projection of y = (4,3, 1,0) on the column
space of the matrix
1 -2
1 -1
A i .
1 0
1 2
(c) Explain carefully what your answer in (b) has to do with part (a) .
(d) At what time does the largest error occur? That is, when does the
observed data differ most from the values your line predicts?

24.4 Answers to Odd-Numbered Exercises

(l) (a) 1, 2, 3
(b) left nullspace
(c) 1, 2
(3) -1, 1
(5) 2, 1, 5
(7) -6, 73, 101
1 1
9 -, -
( ) 3 3
This page intentionally left blank
 Part 7

SPECTRAL THEORY OF INNER

PRODUCT SPACES
This page intentionally left blank
 Chapter 25

SPECTRAL TI-IE()REl\/I
FOR REAL INNER
PRODUCT SPACES

25.1 Background

Topics: the spectral theorem for finite dimensional real inner product
spaces.

Definition 25.1.1. An operator T on a finite dimensional real inner prod-

uct space with an orthonormal basis is ORTHOGONALLY DIAGONALIZABLE
if there exists an orthogonal matrix that diagonalizes T.

The following theorem (together with its analog for complex spaces) is
the fundamental structure theorem for inner product spaces. It says that
any symmetric operator on a finite dimensional real inner product space
can be written as a linear combination of orthogonal projections. The coef-
ficients are the eigenvalues of the operator and the ranges of the orthogonal
projections are the eigenspaces of the operator.
Theorem 25.1.2 (Spectral Theorem for Finite Dimensional Real
Inner Product Spaces). Let T be a symmetric operator on a #nite
dimensional real inner product space V, and A1, . . . ,As be the (distinct)
eigenvalues of T . For each j let My be the eigenspace associated with As and
-
Et be the projection of V onto My along M1+- ° + M j _ 1 +Mj+1-I-~ ' -+Mk.

187
188 Exercises and Problems in Linear Algebra

Then T is orthogonally dtagonallzable, the elgenspaces of T are mutually

orthogonal, each EJ is an orthogonal projection, and the following hold:

(i) T ; A1E1 -|- - + AkEk,

(ii) I = E 1 - l ' ° ' ° - l - E k , and
(iii) EEG = 0 when i ,e j .

25.2 Exercises

(1) Let T be the operator on R3 whose matrix representation is

1-41CO

olly

Cell£IO [\l©
I

I
I

let I~I
I

Le)l©
OIICO
I

I
I
too OIICO

l\l©

Low:
011
I

I
I

(a) Find the characteristic polynomial and minimal polynomial for T.

Answer: e;r(A) =
MTIAI I .
(b) The eigenspace M1 associated with the smallest eigenvalue A1 is
the span of (1, , l.
(c) The eigenspace M2 associated with the middle eigenvalue A2 is the
span of ( , , -1).
(d) The eigenspace M3 associated with the largest eigenvalue A3 is the
span of ( , 1, l.
(e) Find the (matrix representations of the) orthogonal projections E1 ,
EQ, and E3 onto the eigenspaces Mi, MQ, and M3, respectively.

1 1
1 Cl a Cl b -C -C
1
Answer: E1 = a a a E2 c a a 7 E3
m n
Cl a Cl c a a

1
d d d
1
2 d Cl -CL where a b , c d 7 7 7

d -a Cl
m , and n = .
(f) Write T as a linear combination of the projections found in (el-
Answer: [T] = E1 -I- E2 -|- E3.
 Spectral Theorem for Real Inner Product Spaces 189

(g) Find an orthogonal matrix Q (that is, a matrix such that Qt =

Q-1) that diagonalizes T. What is the associated diagonal form A
of T?
C
0
Answer: Q

b
7
7
,C A
\/5
and A

and u
,U
n um where
6
II

7 7 7 ,U 7

(2) Let T be the operator on R3 whose matrix representation is

2 2 1

_1 IJ
(a) The eigenspace M1 associated with the smallest eigenvalue A1 is
the span of (1, , l.
(b) The eigenspace M2 associated with the middle eigenvalue A2 is the
span of (1, , l.
(c) The eigenspace M3 associated with the largest eigenvalue A3 is the
span of (1, , l.
(d) Find the (matrix representations of the) orthogonal projections E1 ,
EQ, and E3 onto the eigenspaces Mi, MQ, and M3, respectively.
a -CL -b a -a Cl
1 1
Answer: E1 Cl a b E2 a Cl -a
mn m
b b c a -a Cl

1
d a a
1
E3= a a d wheres= 7 b , C 7 d 7
n
d d d
m , and n .
(e) Write T as a linear combination of the projections found in (d).
Answer: [T] = E1 -|- E2 -|- E3.
(f) Find an orthogonal matrix Q (that is, a matrix such that Qt = Q-1)
that diagonalizes T. What is the associated diagonal form A of T?
a
We
of
a
Answer: Q We - 7 \/5
and A ,U where
C n
0
We 7
b ,C A and u
6
II

i
7 7 7 ,U 7
190 Exercises and Problems in Linear Algebra

25.3 Problem

(1) Let A

(a) Does A satisfy the hypotheses of the spectral theorem 25.1.2 for
symmetric operators on a Iinite dimensional real inner product
space? Explain.
(b) Explain how to find an orthogonal matrix that diagonalizes the
matrix A. Carry out the computation you describe.
(c) Explain in careful detail how to write the matrix A in part (be
as a linear combination of orthogonal projections. Carry out the
computations you describe.

25.4 Answers to the Odd-Numbered Exercises

(1) (a) A3 - 2A2 A + 2 (or (A + 1>(A 1) O - 2 ) ) ; A3 -QA2 - A + 2 (or

(A + l)(A 1)(,\-2))
(b) 1, 1
(c) 2, -1
(dl 0, -1
(Q) 1, 4, 2, 0, 3, 6
(f) -1, 1, 2
(g) 1, 3, 2, -1, 1, 2
 Chapter 26

SPECTRAL THEOREM FOR

COMPLEX INNER PRODUCT
SPACES

26.1 Background

Topics: the spectral theorem for Ignite dimensional complex inner product
spaces.

Definition 26.1.1. An operator T on a finite dimensional complex inner

product space with an orthonormal basis is UNITARILY DIAGONALIZABLE if
there exists a unitary matrix that diagonalizes T.

Theorem 26.1.2 (Spectral Theorem for Finite Dimensional Com-

pl ex Inner Product Spaces). Let T be a normal operator on a finite
dimensional complex inner product space V , and A1, . . . , As be the (distinct)
eigenvalues of T . For each j let My be the eigenspace associated with As and
-
Et be the projection of V onto My along M1+- ° + M j _ 1 +Mj+1 -I-~ ' - + M k .
Then T is unitarily diagonalizable, the eigenspaces of T are mutually
orthogonal, each EJ is an orthogonal projection, and the following hold:
-
(i) T A1E1 -|- ' • -|- As,Ek,
;

(ii) I I E 1 -l-'°'-l-Ek, and

(iii) E,~Ej = 0 when i go j .

Theorem 26.1.3. Let T be an operator on a finite dimensional complex

inner product space V . Then the following are equivalent:

191
192 Exercises and Problems in Linear Algebra

( 1 ) T is normal;
(2) T is unitarily diagonalizable; and
( 3 ) V has an orthonormal basis consisting of eigenvectors of T.

For a discussion and proofs of the preceding theorems see, fOr example,
[2], page 2501§'. or [3], page 2841

26.2 Exercises

2 H .
(1) Let A

(a) Use the spectral theorem 26.1.2 to write A as a linear combination

of orthogonal projections.
Answer: A = 0E1 -|- BE2 where a = 5 E1
ii
7 7

1 2 -1 - 1 F 1 1 -I-'i
and E2 .
3
(b) Find a square root of A.

Answer: \ A=
1
g [ 4 1 -I-'i
.

(2) Let T be the operator on (22 whose matrix representation is

u
(a) The eigenspace VI associated with the eigenvalue -i is the span of
(1, l.
(b) The eigenspace VI associated with the eigenvalue i is the span of
(1, l.
(c) The (matrix representations of the) orthogonal projections E1 and
r n AS
E2 onto the eigenspaces VI and VI, respectively, are E1 7

and E2 where Cl and b

(d) Write T as a linear combination of the projections found in (c).

Answer: [T] = E1 + EQ.
 Spectral Theorem for Complex Inner Product Spaces 193

(e) A unitary matrix U that diagonalizes [T] is where

and b = . The associated diagonal form A = U* [T]U
6
II

of [T] is
1
1-i
(3)L@tn=3
1 l44-2£
1 'i 44-2£ 1 i .
- i 44-2£ I

J
a b b
(a) The matrix N is normal because NN* = N * N b a b where
A A

a= and b = .
(b) Thus according to the spectral theorem 26.1.2 N can be written as a
linear combination of orthogonal projections. Written in this form
N A1E1 -|- A2 E2 where A1 I 7

1 1
Cl a a b -a Cl

$2 , E 1 = a a a andE2= a b 7 Cl

Cl a a a Cl b
where Cl and b

(c) A unitary matrix U that diagonalizes N is a .

a -b - C
where
1
d n
201
7 b ,C and d The asso-
6
II

c i t e d diagonal form A = U*NU of N is

(4) Let T be an operator whose matrix representation is

(a) Regarded as an operator on R2 is T triangulable?

u . A s an
operator on R2 is it diagonalizable? .
(b) Show that T regarded as an operator on Q2 is diagonalizable by
finding numbers C and d such that the matrix S = f-f) -OT is

invertible and s-1T5 is diagonal.

Answer: C = and d =
194 Exercises and Problems in Linear Algebra

(c) Show that despite being diagonalizable (as an operator on (C2) T

is not normal.

Answer: TT* T*T.

(d) Explain briefly why the result of part (c) does not contradict
Theorem 26.13.

(5) Let T be the operator on (23 whose matrix representation is

8-1 5 21 2 +4i
-54-21 8 i -44-2i
6 .
-2-41, -44-2i 14-1-21

(a) Find the characteristic polynomial and minimal polynomial for T.

Answer: CT(Al = .
WLTIAI I .
(b) The eigenspace M1 associated with the real eigenvalue A1 is the
span of (1, , l.
(c) The eigenspace M2 associated with the complex eigenvalue A2 with
negative imaginary part is the span of (1, , l.
(d) The eigenspace M3 associated with the remaining eigenvalue A3 is
the span of (1, , l.
(e) Find the (matrix representations of the) orthogonal projections E1 ,
EQ, and E3 onto the eigenspaces Mi, MQ, and M3, respectively.

I 1
1 -b be 1 b
1 1
Answer: E1 b Cl -c E2 -b a
n
-be - C d

1
l 1 -b -b
b a a where a = 7 ,c 7 ,e 7

P b Cl a
m 7n , and p = .
(f) Write T as a linear combination of the projections found in (e).
Answer: [T] = E1 + E2 -|- E3.
(g) Find a unitary matrix U which diagonalizes T. What is the asso-
ciated diagonal form A of T?
 Spectral Theorem for Complex Inner Product Spaces 195

|g.§l
a

|(» l.l@

P
\be
d
Answer: U \be
and A ,U where

I"
_ bd n
\be
a = ,b= ,C A

on
II

II
,U

*-.
7 7 7

and I/ i .
(h) The operator T is normal because TT* T*T
a -be 260
be Cl -2b where Cl 7 b , C 7 and
6
-Zinc -2b d
d

(6) Let T be the operator on (23 whose matrix representation is

I
1 _
§
5 -|- Zi 2 - i
.
O - f; 7

5 - 'L 2 4-Z i
1
2 2 -|- Zi 5 i

(a) Find the characteristic polynomial and minimal polynomial for T.

Answer: c;r(A) = .
MTIAI I .
(b) The eigenspace M1 associated with the real eigenvalue A1 is the
span of (1, , l.
(c) The eigenspace M2 associated with the complex eigenvalue A2 with
negative imaginary part is the span of l , , -1).
(d) The eigenspace M3 associated with the remaining eigenvalue A3 is
the span of ( , -1, l.
(e) Find the (matrix representations of the) orthogonal projections E1 ,
EQ, and E3 onto the eigenspaces Mi, MQ, and M3, respectively.

1 1
a Cl a b b b
1 1
Answer: E1 Cl Cl CL E2 I b C C , E3 I
m n
a Cl a b C c

1
where a 7 b , C 7 d 7
6
e 7 m , and n = .
(f) Write T as a linear combination of the projections found in (e).
Answer: [T] = E1 -|- E2 -|- E3.
196 Exercises and Problems in Linear Algebra

(g) Find an orthogonal matrix Q (that is, a matrix such that Qt =

Q-1) that diagonalizes T. What is the associated diagonal form A
of T?
C
0
Answer: Q

b
7
7
,C
JE
and A

and
,U
n um where

A
6

M
II

7 7 7 7 1/ Z

26.3 Problems

(1) Let N be a normal operator on a finite dimensional complex inner

product space V. Show that lINxll = jIN*xll for all x e V.
(2) Let N be a normal operator on a complex finite dimensional inner
product space V. Show that if Al, . . . , As are the eigenvalues of N, then
X, . . . , are the eigenvalues of N*.
(3) Let T be as in exercise 4. Show by direct computation that there is no
D. AS
invertible 2 X 2 matrix S = of real numbers such that S-ITS
is upper triangular.

26.4 Answers to Odd-Numbered Exercises

(1 ) (a) 1> 4 , - 1 + £ > 1 7 1 - 2

(b) 1 -12,5
8 2
(3) (a) 3 ' 3
1 2
(b) 2, l + i, 3 7 _
3
1 1 1
0, 0 1 -I-i
0 0 1 + 'L 'I
(5) (a) XI - 5A2 + SA - 6 (or (AS - QA + 2>(A 3)); >3 - 5A2 + SA - 6 (or
(AS - QA + 2)(A - 3>)
(b) i, -Qi
(c) -QL, 0
 Spectral Theorem for Complex Inner Product Spaces 197

(d) i, ¢
(Q) 1, to, 2, 4, 0, 6, 2, 3
(f) 3, 1 - ¢ , 1+'£
(g) 1 7 2 7 3 7 ' 7 0 7 3 7 1 - 1+
(h) 19, 7, i, 40
This page intentionally left blank
 BIBLIOGRAPHY

1. Howard Anton and Chris Rorres, Elementary Linear Algebra: Applications

Version, eighth ed., John Wiley and Sons, New York, 2000. vii, 45, 81
2. William C. Brown, A second Coarse in Linear Algebra, John Wiley, New York,
1988. viii, 175
3. Charles W. Curtis, Linear Algebra: An Introductory Approach, Springer,
New York, 1984. 175
4. Paul R. Halmos, Finite-Dimensional Vector Spaces, D. Van Nostrand,
Princeton, 1958. vii, 58
Jim Hefferon, Linear Algebra, 2006, http://joshua.smcvt.edu/linearalgebra. 45
5.6. Kenneth Hoffman and Ray Kunze, Linear Algebra, second ed., Prentice Hall,
Englewood Cliffs, N.J., 1971. vii, 89, 103, 147
7. Nathan Jacobson, Lectures in Abstract Algebra, vol. II, Linear Algebra,
Springer-verlag, New York, 1984. 49
8. Steven Roman, Advanced Linear Algebra, second ed., Springer-Verlag,
New York, 2005. viii, 125
9. Gilbert Strang, Linear Algebra and its Applications, second ed., Academic
Press, New York, 1980. vii, 131

199
This page intentionally left blank
 INDEX

,_{(x,y) (angle between X and y), 25, additive

133 inverses, 33
M ® N (direct sum of subspaces), 40 adjoint
V ® W (external direct sum of vector of a linear map, 156
spaces), 40 angle, 25, 133
V/M (quotient of V by M), 41 annihilating polynomial, 94
X - y (inner product of X and y), 25, associative, 33
131, 132
M J_ N (two sets are orthogonal), 133 basis, 55
X J_ y (two vectors are orthogonal) , orthonormal, 140
133 standard, 55
M < V (M is a subspace of V), 39 bijective, 61
=~
V W (V and W are isomorphic) , bounded function, 40
63
(x,y) (inner product), 131 C"
[T] (matrix representation of T), 76 as an inner product space, 132
[A, B] (commutator of A and B), 3 C([a, bl)
[ac] (equivalence class containing x), as an inner product space, 133
41 Cauchy-Schwarz inequality, 25, 133
TS (notation for composition of Cayley-Hamilton theorem, 94
linear maps), 62 characteristic polynomial, 89
(x, y) (inner product of X and y), 25, cofactor, 13
132 column
II:z:ll (norm of a vector x), 132 rank, 161
At (transpose of A), 4 column index, 3
ML (the orthogonal complement column space, 161
of m ) , 133 combination
linear, 49
action commutative, 33
of a matrix, 11 diagram, 71

201
202 Exercises and Problems in Linear Algebra

commutator, 3 EMN (projection along M onto N),

commute, 3, 71 83
complement eigenspace, 89
orthogonal, 133 generalized, 116
complementary subspaces, 40 eigenvalue, 89
lm Z (imaginary part of a complex eigenvector, 89
number), 127 electrical networks
conjugate problem on, 54
Hermitian, 156 equivalent
linear, 132 row, 12
even function, 43
transpose, 156
exact sequence, 69
of a linear map, 156
expansion
of a matrix, 155
Laplace, 13
continuous
external direct sum, 40
uniformly, 133
continuously differentiable, 64 factorization
cross product, 25 QR-, 140
finite dimension, 56
definite first isomorphism theorem, 72
negative, 145 Fourier
positive, 145 coefficients, 140
dependence series, 139
linear, 50 function
determinant, 12 even, 43
det A (determinant of A), 12 odd,43
diagonal matrix, 93 square integrable, 135
diagonalizable, 94 uniformly continuous, 133
conditions to be, 95 functional
orthogonally, 187 linear, 61
part of an operator, 116 fundamental theorem of linear
unitarily, 191 algebra, 162, 166
diagonalizing matrix, 94
generalized eigenspace, 116
diagram, 71
commutative, 71
Hermitian
differentiable
conjugate, 156
continuously, 64 operator, 156
differential equations, 107 Hessian matrix, 149
dimension, 56
dim V (dimension of V), 56 Iv (identity operator on V), 62
direct sum identity
external, 40 additive, 33
internal, 40 matrix, 4
orthogonal, 140 In (n X n identity matrix), 4
dominate, 174 identity map, 62
 Index 203

indefinite, 145 combination, 49

independence trivial, 49
linear, 50 conjugate, 132
index dependence, 50
column, 3 functional, 61
row, 3 independence, 50
inequality map, 61
(Cauchy-lSchwarz, 133 adjoint of a, 156
infinite dimensional, 56 conjugate transpose of a, 156
injective, 61 Hermitian conjugate of a, 156
inner product, 25, 131 transpose of a, 155
space operator, 62
C" as a, 132 sesqui-, 132
R" as a, 132 transformation, 62
C([0,bi) as a, 133
Z2 as a, 132 majorize, 174
integrable, 40 map
internal direct sum, 40 linear, 61
inverse Markov matrix, 107
additive, 33
matrix
left, 63
conjugate transpose of a, 155
right, 63
diagonal, 93
invertible, 63
diagonalizing, 94
isometric, 173
Hessian, 149
isomorphic, 63
Markov, 107
isomorphism, 63
nilpotent, 115
normal, 156
kerT (the kernel of T), 62
kernel, 62 notation for, 3
orthogonal, 140, 173
£(V) representation of an linear map,
linear operators on V, 62 76
QW, W) second derivative, 149
linear maps between vector standard, 76
spaces, 62 symmetric, 4
Z2 transpose of a, 4
as an inner product space, 132 unitary, 140, 173
square summable sequences, 132 upper triangular, 4
Laplace expansion, 13 maximal
leading principal submatrix, 146 linearly independent set, 57
left orthonormal set, 139
inverse, 7, 63 minimal polynomial, 94
nullspace, 161 existence and uniqueness of, 99
length, 25 minimal spanning set, 57
Lie bracket, 3 minor, 13
linear, 61 manic polynomial, 94
204 Exercises and Problems in Linear Algebra

negative point spectrum, 89

definite, 145 polarization identity, 136
semidefinite, 145 polynomial
networks annihilating, 94
problem on, 54 characteristic, 89
nilpotent, 95, 115 minimal, 94
part of an operator, 116 existence and uniqueness of,
nonsingular, 13 99
norm, 25, 132 rnonic, 94
normal positive
matrix, 156 definite, 145
operator, 155, 156 operator, 174
nullity, 62 semidefinite, 145
nullspace, 62, 161 principal subrnatrix, 146
left, 161 product
cross, 25
odd function, 43 inner, 25, 131
one-to-one, 61 of vector spaces, 40
correspondence, 61
projection
onto, 61
along one subspace onto
operator
another, 83
adjoint of an, 156
orthogonal, 173
diagonalizable part of an, 116
Hermitian, 156
Q R-factorization, 140
linear, 62
quotient
nilpotent, 115
map
nilpotent part of an, 116
for vector spaces, 72
normal, 156
orthogonal, 173 theorem
positive, 174 for vector spaces, 72
self-adjoint, 156 vector space, 41
symmetric, 155 Rn
transpose of an, 155
unitary, 173 as an inner product space, 132
orthogonal, 133 ranT (the range of T), 62
complement, 133 range,62
diagonalization, 187 rank
direct sum, 140 column, 161
matrix, 140, 173 of a linear map, 62
operator, 173 of a matrix, 76
projection, 173 row, 161
orthonormal, 139 Re Z (real part of a complex
basis, 140 nurnnber), 127
representation
perpendicular, 133 matrix, 76
pivoting, 11 Riemann integrable function, 40
 Index 205

right subspace, 39
inverse, 7, 63 complementary, 40
TOW sum
equivalent matrices, 12 external direct, 40
operations, 11 internal direct, 40
rank, 161 summable
row index, 3 square, 132
row space, 161 surjective, 61
swapping, 11
scaling, 11 symmetric
Schwarz inequality, 25, 133 matrix, 4
second derivative operator, 155
matrix, 149 system
test, 149 of differential equations, 107
self-adjoint
operator, 156 trace, 4
semidefinite t r A (trace of A), 4
negative, 145 transformation
positive, 145 linear, 62
sequence transpose
exact, 69 conjugate, 156
sesquilinear, 132 of a matrix, 4
short exact sequence, 69 of an operator, 155
o ( T ) (spectrum of T), 89 triangulable, 94
similar, 93 conditions to be, 94
singular, 13 triangular
space upper, 4
vector, 33 trivial
span, 49 linear combination, 49
spanA (the span of the set A), 50
spectral mapping theorem, 89 uniformly continuous, 133
spectral theorem unitary
for finite dimensional complex diagonalization, 191
inner product spaces, 191 matrix, 140, 173
for finite dimensional real inner operator, 173
product spaces, 187 upper
for finite dimensional vector triangular, 4
spaces, 101 upper triangular, 94
spectrum, 89
square integrable function, 135 vector
square summable, 132 space, 33
standard complex, 33
basis for R", 55 real, 33
matrix for a linear map, 76
submatrix
leading principal, 146