Comprehensive  Introduction

             to  Linear  Algebra

PART  III  -­  OPERATORS  AND  TENSORS

Joel  G.  Broida

S.  Gill  Williamson
Comprehensive Introduction to Linear Algebra
PART III - OPERATORS AND TENSORS

Joel G. Broida
S. Gill Williamson

Creative Commons CC0 1.0, 2012

 Preface (Part II)

This book, Part 3 - Operators and Tensors, covers Chapters 9 through 12 of the book A Com-
prehensive Introduction to Linear Algebra (Addison-Wesley, 1986), by Joel G. Broida and S. Gill
Williamson. Selections from Chapters 9 and 10 are covered in most upper division courses in
linear algebra. Chapters 11 and 12 introduce multilinear algebra and Hilbert space. The orig-
inal Preface, Contents and Index are included. Three appendices from the original manuscript
are included as well as the original Bibliography. The latter is now (2012) mostly out of date.
Wikipedia articles on selected subjects are generally very informative.

v
Preface (Parts I, II, III)

As a text, this book is intended for upper division undergraduate and begin-
ning graduate students in mathematics, applied mathematics, and fields of
science and engineering that rely heavily on mathematical methods. However,
it has been organized with particular concern for workers in these diverse
fields who want to review the subject of linear algebra. In other words, we
have written a book which we hope will still be referred to long after any final
exam is over. As a result, we have included far more material than can possi-
bly be covered in a single semester or quarter. This accomplishes at least two
things. First, it provides the basis for a wide range of possible courses that can
be tailored to the needs of the student or the desire of the instructor. And
second, it becomes much easier for the student to later learn the basics of
several more advanced topics such as tensors and infinite-dimensional vector
spaces from a point of view coherent with elementary linear algebra. Indeed,
we hope that this text will be quite useful for self-study. Because of this, our
proofs are extremely detailed and should allow the instructor extra time to
work out exercises and provide additional examples if desired.

vii
viii PREFACE

A major concern in writing this book has been to develop a text that
addresses the exceptional diversity of the audience that needs to know some-
thing about the subject of linear algebra. Although seldom explicitly
acknowledged, one of the central difficulties in teaching a linear algebra
course to advanced students is that they have been exposed to the basic back-
ground material from many different sources and points of view. An experi-
enced mathematician will see the essential equivalence of these points of
view, but these same differences seem large and very formidable to the
students. An engineering student for example, can waste an inordinate amount
of time because of some trivial mathematical concept missing from their
background. A mathematics student might have had a concept from a different
point of view and not realize the equivalence of that point of view to the one
currently required. Although such problems can arise in any advanced mathe-
matics course, they seem to be particularly acute in linear algebra.
To address this problem of student diversity, we have written a very self-
contained text by including a large amount of background material necessary
for a more advanced understanding of linear algebra. The most elementary of
this material constitutes Chapter 0, and some basic analysis is presented in
three appendices. In addition, we present a thorough introduction to those
aspects of abstract algebra, including groups, rings, fields and polynomials
over fields, that relate directly to linear algebra. This material includes both
points that may seem “trivial” as well as more advanced background material.
While trivial points can be quickly skipped by the reader who knows them
already, they can cause discouraging delays for some students if omitted. It is
for this reason that we have tried to err on the side of over-explaining
concepts, especially when these concepts appear in slightly altered forms. The
more advanced reader can gloss over these details, but they are there for those
who need them. We hope that more experienced mathematicians will forgive
our repetitive justification of numerous facts throughout the text.
A glance at the Contents shows that we have covered those topics nor-
mally included in any linear algebra text although, as explained above, to a
greater level of detail than other books. Where we differ significantly in con-
tent from most linear algebra texts however, is in our treatment of canonical
forms (Chapter 8), tensors (Chapter 11), and infinite-dimensional vector
spaces (Chapter 12). In particular, our treatment of the Jordan and rational
canonical forms in Chapter 8 is based entirely on invariant factors and the
 PREFACE ix

Smith normal form of a matrix. We feel this approach is well worth the effort
required to learn it since the result is, at least conceptually, a constructive
algorithm for computing the Jordan and rational forms of a matrix. However,
later sections of the chapter tie together this approach with the more standard
treatment in terms of cyclic subspaces. Chapter 11 presents the basic formal-
ism of tensors as they are most commonly used by applied mathematicians,
physicists and engineers. While most students first learn this material in a
course on differential geometry, it is clear that virtually all the theory can be
easily presented at this level, and the extension to differentiable manifolds
then becomes only a technical exercise. Since this approach is all that most
scientists ever need, we leave more general treatments to advanced courses on
abstract algebra. Finally, Chapter 12 serves as an introduction to the theory of
infinite-dimensional vector spaces. We felt it is desirable to give the student
some idea of the problems associated with infinite-dimensional spaces and
how they are to be handled. And in addition, physics students and others
studying quantum mechanics should have some understanding of how linear
operators and their adjoints are properly defined in a Hilbert space.
One major topic we have not treated at all is that of numerical methods.
The main reason for this (other than that the book would have become too
unwieldy) is that we feel at this level, the student who needs to know such
techniques usually takes a separate course devoted entirely to the subject of
numerical analysis. However, as a natural supplement to the present text, we
suggest the very readable “Numerical Analysis” by I. Jacques and C. Judd
(Chapman and Hall, 1987).
The problems in this text have been accumulated over 25 years of teaching
the subject of linear algebra. The more of these problems that the students
work the better. Be particularly wary of the attitude that assumes that some of
these problems are “obvious” and need not be written out or precisely articu-
lated. There are many surprises in the problems that will be missed from this
approach! While these exercises are of varying degrees of difficulty, we have
not distinguished any as being particularly difficult. However, the level of dif-
ficulty ranges from routine calculations that everyone reading this book
should be able to complete, to some that will require a fair amount of thought
from most students.
Because of the wide range of backgrounds, interests and goals of both
students and instructors, there is little point in our recommending a particular
x PREFACE

course outline based on this book. We prefer instead to leave it up to each

teacher individually to decide exactly what material should be covered to meet
the needs of the students. While at least portions of the first seven chapters
should be read in order, the remaining chapters are essentially independent of
each other. Those sections that are essentially applications of previous
concepts, or else are not necessary for the rest of the book are denoted by an
asterisk (*).
Now for one last comment on our notation. We use the symbol ˙ to denote
the end of a proof, and ∆ to denote the end of an example. Sections are labeled
in the format “Chapter.Section,” and exercises are labeled in the format
“Chapter.Section.Exercise.” For example, Exercise 2.3.4 refers to Exercise 4
of Section 2.3, i.e., Section 3 of Chapter 2. Books listed in the bibliography
are referred to by author and copyright date.
Contents (Part I, II, III ) #

0 Foundations 1
0.1 Sets 2
0.2 Mappings 4
0.3 Orderings and Equivalence Relations 7
0.4 Cardinality and the Real Number System 11
0.5 Induction 17
0.6 Complex Numbers 19
0.7 Additional Properties of the Integers 25

1 An Introduction to Groups 30
1.1 Definitions 30
1.2 Permutation Groups 35
1.3 Homomorphisms of Groups 49
1.4 Rings and Fields 53
1.5 More on Groups and Rings 56

xi
xii CONTENTS

2 Vector Spaces 68
2.1 Definitions 68
2.2 Linear Independence and Bases 75
2.3 Direct Sums 85
2.4 Inner Product Spaces 94
2.5 Orthogonal Sets 104

3 Linear Equations and Matrices 115

3.1 Systems of Linear Equations 115
3.2 Elementary Row Operations 121
3.3 Row and Column Spaces 127
3.4 The Rank of a Matrix 135
3.5 Solutions to Systems of Linear Equations 138
3.6 Matrix Algebra 147
3.7 Invertible Matrices 157
3.8 Elementary Matrices 163

4 Determinants 170
4.1 Definitions and Elementary Properties 171
4.2 Additional Properties of Determinants 176
4.3 Expansion by Minors 186
4.4 Determinants and Linear Equations 199
4.5 Block Matrices 204
4.6 The Cauchy-Binet Theorem 208

5 Linear Transformations and Matrices 215

5.1 Linear Transformations 215
5.2 Further Properties of Linear Transformations 224
5.3 Matrix Representations 233
5.4 Change of Basis 243

6 Polynomials 252
6.1 Definitions 252
6.2 Factorization of Polynomials 261
6.3 Polynomial Ideals 269
6.4 Polynomials Over Algebraically Closed Fields 276
6.5 The Field of Quotients 281
6.6 Polynomials Over Finite Fields * 285

7 Linear Transformations and Polynomials 296

7.1 Minimal Polynomials 297
7.2 Eigenvalues and Eigenvectors 302
7.3 Characteristic Polynomials 308
CONTENTS xiii

7.4 Annihilators 322

7.5 Invariant Subspaces 329
7.6 The Primary Decomposition Theorem 337
7.7 More on Diagonalization 344
7.8 Projections 352
7.9 Quotient Spaces 360
7.10 The Triangular Form Theorem * 365
7.11 Nilpotent Transformations * 369
7.12 The Triangular Form Theorem Again * 376

8 Canonical Forms 382

8.1 Elementary Canonical Forms 382
8.2 Matrices Over the Ring of Polynomials 389
8.3 The Smith Canonical Form 397
8.4 Similarity Invariants 406
8.5 The Rational Canonical Form 412
8.6 The Jordan Canonical Form 420
8.7 Cyclic Subspaces * 432
8.8 The Elementary Divisor Theorem * 439

9 # Linear Forms 446

9.1 Bilinear Functionals 446
9.2 Double Duals and Annihilators 450
9.3 The Transpose of a Linear Transformation 458
9.4 Bilinear Forms 461
9.5 Symmetric and Antisymmetric Bilinear Forms 467
9.6 Diagonalization of Symmetric Bilinear Forms 474
9.7 Hermitian Forms 481
9.8 Simultaneous Diagonalization * 485

10 # Linear Operators 490

10.1 Linear Functionals and Adjoints 490
10.2 Isometric and Unitary Operators 498
10.3 Normal Operators 508
10.4 Diagonalization of Normal Operators 514
10.5 The Spectral Theorem 523
10.6 The Matrix Exponential Series 528
10.7 Positive Operators 537

11 # Multilinear Mappings and Tensors 543

11.1 Definitions 544
11.2 Special Types of Tensors 553
11.3 The Exterior Product 563
xiv CONTENTS

11.4 Tensor Algebras 574

11.5 The Tensor Product of Vector Spaces 577
11.6 Volumes in ®3 584
11.7 Volumes in ®n 589
11.8 Linear Transformations and Volumes 593
11.9 Orientations and Volumes 603
11.10 The Metric Tensor and Volume Forms 609

12 # Hilbert Spaces 619

12.1 Mathematical Preliminaries 619
12.2 Operator Norms 635
12.3 Hilbert Spaces 640
12.4 Closed Subspaces 649
12.5 Hilbert Bases 655
12.6 Bounded Operators on a Hilbert Space 665
12.7 Hermitian, Normal and Unitary Operators 676

Appendices 680
A Metric Spaces 680
B Sequences and Series 696
C Path Connectedness 720

Bibliography 723

Index 727
 CHAPTER 9

Linear Forms

We are now ready to elaborate on the material of Sections 2.4, 2.5 and 5.1.
Throughout this chapter, the field F will be assumed to be either the real or
complex number system unless otherwise noted.

9.1 BILINEAR FUNCTIONALS

Recall from Section 5.1 that the vector space V* = L(V, F): V ‘ F is defined
to be the space of linear functionals on V. In other words, if ƒ ∞ V*, then for
every u, v ∞ V and a, b ∞ F we have

ƒ(au + bv) = aƒ(u) + bƒ(v) ∞ F .

The space V* is called the dual space of V. If V is finite-dimensional, then

viewing F as a one-dimensional vector space (over F), it follows from
Theorem 5.4 that dim V* = dim V. In particular, given a basis {eá} for V, the
proof of Theorem 5.4 showed that a unique basis {øi} for V* is defined by the
requirement that
øi(eé) = ∂ié

where we now again use superscripts to denote basis vectors in the dual space.
We refer to the basis {øi} for V* as the basis dual to the basis {eá} for V.

446
 9.1 BILINEAR FUNCTIONALS 447

Elements of V* are usually referred to as 1-forms, and are commonly denoted

by Greek letters such as ß, ƒ, œ and so forth. Similarly, we often refer to the øi
as basis 1-forms.
Since applying Theorem 5.4 to the special case of V* directly may be
somewhat confusing, let us briefly go through a slightly different approach to
defining a basis for V*.
Suppose we are given a basis {eè, . . . , eñ} for a finite-dimensional vector
space V. Given any set of n scalars ƒá, we define the linear functionals ƒ ∞ V*
= L(V, F) by ƒ(eá) = ƒá. According to Theorem 5.1, this mapping is unique. In
particular, we define n linear functionals øi by øi(eé) = ∂ié. Conversely, given
any linear functional ƒ ∞ V*, we define the n scalars ƒá by ƒá = ƒ(eá). Then,
given any ƒ ∞ V* and any v = Ívjeé ∞ V, we have on the one hand

ƒ(v) = ƒ(Ívieá) = Íviƒ(eá) = ̓ávi

while on the other hand

øi(v) = øi(Íévj eé) = Íévjøi (eé) = Íévj∂ié = vi .

Therefore ƒ(v) = Íáƒáøi(v) for any v ∞ V, and we conclude that ƒ = Íáƒáøi.

This shows that the øi span V*, and we claim that they are in fact a basis for
V*.
To show that the øi are linearly independent, suppose Íáaáøi = 0. We must
show that every aá = 0. But for any j = 1, . . . , n we have

0 = Íáaáøi(eé) = Íáaá∂ié = aé

which verifies our claim. This completes the proof that {øi} forms a basis for
V*.
There is another common way of denoting the action of V* on V that is
quite similar to the notation used for an inner product. In this approach, the
action of the dual basis {øi} for V* on the basis {eá} for V is denoted by
writing øi(eé) as
Óøi, eé Ô = ∂ié .

However, it should be carefully noted that this is not an inner product. In par-
ticular, the entry on the left inside the bracket is an element of V*, while the
entry on the right is an element of V. Furthermore, from the definition of V*
as a linear vector space, it follows that Ó , Ô is linear in both entries. In other
words, if ƒ, œ ∞ V*, and if u, v ∞ V and a, b ∞ F, we have
448 LINEAR FORMS

a! + b" ,!u = a ! ,!u + b " ,!u

! ,!au + bv = a ! ,!u + b ! ,!v !!.

These relations define what we shall call a bilinear functional Ó , Ô: V* ª

V ‘ F on V* and V (compare this with definition IP1 of an inner product
given in Section 2.4).
We summarize these results as a theorem.

Theorem 9.1 Let {eè, . . . , eñ} be a basis for V, and let {ø1, . . . , øn} be the
corresponding dual basis for V* defined by øi(eé) = ∂ié. Then any v ∞ V can
be written in the forms
n n n
v = ! vi ei = !" i (v)ei = ! " i ,!v ei
i=1 i=1 i=1
and any ƒ ∞ V* can be written as
n n n
! = # !i" = # ! (ei )" = # ! ,!ei " i !!.
i i

i=1 i=1 i=1

This theorem provides us with a simple interpretation of the dual basis. In

particular, since we already know that any v ∞ V has the expansion v = Ívieá
in terms of a basis {eá}, we see that øi(v) = Óøi, vÔ = vi is just the ith coord-
inate of v. In other words, øi is just the ith coordinate function on V (relative
to the basis {eá}).
Let us make another observation. If we write v = Ívieá and recall that
ƒ(eá) = ƒá, then (as we saw above) the linearity of ƒ results in

Óƒ, vÔ = ƒ(v) = ƒ(Ívieá) = Íviƒ(eá) = ̓ávi

which looks very much like the standard inner product on ®n. In fact, if V is
an inner product space, we shall see that the components of an element ƒ ∞
V* may be related in a direct way to the components of some vector in V (see
Section 11.10).
It is also useful to note that given any nonzero v ∞ V, there exists ƒ ∞ V*
with the property that ƒ(v) ≠ 0. To see this, we use Theorem 2.10 to first
extend v to a basis {v, vì, . . . , vñ} for V. Then, according to Theorem 5.1,
there exists a unique linear transformation ƒ: V ‘ F such that ƒ(v) = 1 and
ƒ(vá) = 0 for i = 2, . . . , n. This ƒ so defined clearly has the desired property.
An important consequence of this comes from noting that if vè, vì ∞ V with
vè ≠ vì, then vè - vì ≠ 0, and thus there exists ƒ ∞ V* such that
 9.1 BILINEAR FUNCTIONALS 449

0 ≠ ƒ(vè - vì) = ƒ(vè) - ƒ(vì) .

This proves our next result.

Theorem 9.2 If V is finite-dimensional and vè, vì ∞ V with vè ≠ vì, then

there exists ƒ ∞ V* with the property that ƒ(vè) ≠ ƒ(vì).

Example 9.1 Consider the space V = ®2 consisting of all column vectors of

the form
! v1 $
v = ## &&!!.
2
"v %

Relative to the standard basis we have

!1$ !0$
v = v1 # & + v 2 # & = v1e1 + v 2 e2 !!.
"0% "1%

If ƒ ∞ V*, then ƒ(v) = ̓ávi, and we may represent ƒ by the row vector ƒ =
(ƒ1, ƒ2). In particular, if we write the dual basis as øi = (aá, bá), then we have

"1%
1 = !1 (e1 ) = (a1,!b1 ) $ ' = a1
#0&

"0%
0 = !1 (e2 ) = (a1,!b1 ) $ ' = b1
#1&

"1%
0 = ! 2 (e1 ) = (a2 ,!b2 ) $ ' = a2
#0&

"0%
1 = ! 2 (e2 ) = (a2 ,!b2 ) $ ' = b2
#1&

so that ø1 = (1, 0) and ø2 = (0, 1). Note that, for example,

" v1 %
!1 (v) = (1,!0) $$ '' = v1
2
#v &
as it should. ∆
450 LINEAR FORMS

Exercises

1. Find the basis dual to the given basis for each of the following:
(a) ®2 with basis e1 = (2, 1), e2 = (3, 1).
(b) ®3 with basis e1 = (1, -1, 3), e2 = (0, 1, -1), e3 = (0, 3, -2).

2. Let V be the space of all real polynomials of degree ¯ 1. Define ø1, ø2 ∞

V* by
1 2
!1 ( f ) = " 0 f (x)dx !!!!!!and!!!!!!! 2 ( f ) = " 0 f (x)dx !!.
Find a basis {e1, e2} for V that is dual to {ø1, ø2}.

3. Let V be the vector space of all polynomials of degree ¯ 2. Define the

linear functionals ø1, ø2, ø3 ∞ V* by
1
!1 ( f ) = " 0 f (x)dx,!!!!!!!!!!! 2 ( f ) = f #(1),!!!!!!!!!!! 3 ( f ) = f (0)
where fæ(x) is the usual derivative of f(x). Find the basis {eá} for V which
is dual to {øi}.

4. (a) Let u, v ∞ V and suppose that ƒ(u) = 0 implies ƒ(v) = 0 for all ƒ ∞ V*.
Show that v = ku for some scalar k.
(b) Let ƒ, ß ∞ V* and suppose that ƒ(v) = 0 implies ß(v) = 0 for all v ∞
V. Show that ß = kƒ for some scalar k.

5. Let V = F[x], and for a ∞ F, define ƒa: V ‘ F by ƒa(f ) = f(a). Show that:
(a) ƒa is linear, i.e., that ƒa ∞ V*.
(b) If a ≠ b, then ƒa ≠ ƒb.

6. Let V be finite-dimensional and W a subspace of V. If ƒ ∞ W*, prove that

ƒ can be extended to a linear functional Ï ∞ V*, i.e., Ï(w) = ƒ(w) for all
w ∞ W.

9.2 DOUBLE DUALS AND ANNIHILATORS

We now discuss the similarity between the dual space and inner products. To
elaborate on this relationship, let V be finite-dimensional over the real field ®
with an inner product Ó , Ô: V ª V ‘ F defined on it. (There should be no
confusion between the inner product on V and the action of a bilinear func-
tional on V* ª V because both entries in the inner product expressions are
elements of V.) In fact, throughout this section we may relax our definition of
inner product somewhat as follows. Referring to our definition in Section 2.4,
 9.2 DOUBLE DUALS AND ANNIHILATORS 451

we keep properties (IP1) and (IP2), but instead of (IP3) we require that if u ∞
V and Óu, vÔ = 0 for all v ∞ V, then u = 0. Such an inner product is said to be
nondegenerate. The reader should be able to see easily that (IP3) implies
nondegeneracy, and hence all inner products we have used so far in this book
have been nondegenerate. (In Section 11.10 we will see an example of an
inner product space with the property that Óu, uÔ = 0 for some u ≠ 0.)
If we leave out the second vector entry in the inner product Óu, Ô, then
what we have left is essentially a linear functional on V. In other words, given
any u ∞ V, we define a linear functional Lu ∞ V* by

Lu(v) = Óu, vÔ

for all v ∞ V. From the definition of a (real) inner product, it is easy to see that
this functional is indeed linear. Furthermore, it also has the property that

Lau+ bv = aLu + bLv

for all u, v ∞ V and a, b ∞ F. What we have therefore done is define a linear

mapping L: V ‘ V* by L(u) = Lu for all u ∞ V. Since the inner product is
nondegenerate, we see that if u ≠ 0 then Lu(v) = Óu, vÔ can not vanish for all
v ∞ V, and hence Lu ≠ 0. This means that Ker L = {0}, and hence the
mapping must be one-to-one (Theorem 5.5). But both V and V* are of
dimension n, and therefore this mapping is actually an isomorphism of V onto
V*. This proves our next theorem.

Theorem 9.3 Let V be finite-dimensional over ®, and assume that V has a

nondegenerate inner product defined on it. Then the mapping u ’ Lu is an
isomorphism of V onto V*.

Looking at this isomorphism as a mapping from V* onto V, we can

reword this theorem as follows.

Corollary Let V be as in Theorem 9.3. Then, given any linear functional

L ∞ V*, there exists a unique u ∞ V such that L(v) = Óu, vÔ = Lu(v) for all v ∞
V. In other words, given any L ∞ V*, there exists a unique u ∞ V such that Lu
= L.

Note that if V is a vector space over ç with the more general Hermitian
inner product defined on it, then the definition Lu(v) = Óu, vÔ shows that Lau =
a*Lu , and the mapping u ’ Lu is no longer an isomorphism of V onto V*.
Such a mapping is not even linear, and is in fact called antilinear (or conju-
gate linear). We will return to this more general case later.
452 LINEAR FORMS

Let us now consider vector spaces V and V* over an arbitrary (i.e., possi-
bly complex) field F. Since V* is a vector space, we can equally well define
the space of linear functionals on V*. By a procedure similar to that followed
above, the expression Ó , uÔ for a fixed u ∞ V defines a linear functional on V*
(note that here Ó , Ô is a bilinear functional and not an inner product). In other
words, we define the function fu: V* ‘ F by

fu(ƒ) = Óƒ, uÔ = ƒ(u)

for all ƒ ∞ V*. It follows that for all a, b ∞ F and ƒ, ø ∞ V* we have

fu(aƒ + bø) = Óaƒ + bø, uÔ = aÓƒ, uÔ + bÓø, uÔ = afu(ƒ) + bfu(ø)

and hence fu is a linear functional from V* to F. In other words, fu is in the

dual space of V*. This space is called the double dual (or second dual) of V,
and is denoted by V**.
Note that Theorem 9.3 shows us that V* is isomorphic to V for any finite-
dimensional V, and hence V* is also finite-dimensional. But then applying
Theorem 9.3 again, we see that V** is isomorphic to V*, and therefore V is
isomorphic to V**. Our next theorem verifies this fact by explicit construction
of an isomorphism from V onto V**.

Theorem 9.4 Let V be finite-dimensional over F, and for each u ∞ V define

the function fu: V* ‘ F by fu(ƒ) = ƒ(u) for all ƒ ∞ V*. Then the mapping f:
u ’ fu is an isomorphism of V onto V**.

Proof We first show that the mapping f: u ’ fu defined above is linear. For
any u, v ∞ V and a, b ∞ F we see that

fau+bv (! ) = ! ,!au + bv
= a ! ,!u + b ! ,!v
= afu (! ) + bfv (! )
= (afu + bfv )(! )!!.

Since this holds for all ƒ ∞ V*, it follows that fau+ bv = afu + bfv, and hence
the mapping f is indeed linear (so it defines a vector space homomorphism).
Now let u ∞ V be an arbitrary nonzero vector. By Theorem 9.2 (with vè =
u and vì = 0) there exists a ƒ ∞ V* such that fu(ƒ) = Óƒ, uÔ ≠ 0, and hence
clearly fu ≠ 0. Since it is obviously true that f0 = 0, it follows that Ker f = {0},
and thus we have a one-to-one mapping from V into V** (Theorem 5.5).
 9.2 DOUBLE DUALS AND ANNIHILATORS 453

Finally, since V is finite-dimensional, we see that dim V = dim V* =

dim V**, and hence the mapping f must be onto (since it is one-to-one). ˙

The isomorphism f: u ’ fu defined in Theorem 9.4 is called the natural

(or evaluation) mapping of V into V**. (We remark without proof that even
if V is infinite-dimensional this mapping is linear and injective, but is not
surjective.) Because of this isomorphism, we will make the identification V =
V** from now on, and hence also view V as the space of linear functionals on
V*. Furthermore, if {øi} is a basis for V*, then the dual basis {eá} for V will
be taken to be the basis for V**. In other words, we may write

øi(eé) = eé(øi) = ∂ié

so that
ƒ(v) = v(ƒ) = ̓ávi .

Now let S be an arbitrary subset of a vector space V. We call the set of

elements ƒ ∞ V* with the property that ƒ(v) = 0 for all v ∞ S the annihilator
of S, and we denote it by S0. In other words,

S0 = {ƒ ∞ V*: ƒ(v) = 0 for all v ∞ S} .

It is easy to see that S0 is a subspace of V*. Indeed, suppose that ƒ, ø ∞ S0, let
a, b ∞ F and let v ∞ S be arbitrary. Then

(aƒ + bø)(v) = aƒ(v) + bø(v) = 0 + 0 = 0

so that aƒ + bø ∞ S0. Note also that we clearly have 0 ∞ S0, and if T ™ S, then
S0 ™ T0.
If we let S be the linear span of a subset S ™ V, then it is easy to see that
S0 = S0. Indeed, if u ∞ S is arbitrary, then there exist scalars aè, . . . , ar such
that u = Íaávi for some set of vectors {v1, . . . , vr} ∞ S. But then for any ƒ ∞
S0 we have
ƒ(u) = ƒ(Íaávi) = Íaáƒ(vi) = 0

and hence ƒ ∞ S0. Conversely, if ƒ ∞ S0 then ƒ annihilates every v ∞ S and

hence ƒ ∞ S0. The main conclusion to deduce from this observation is that to
find the annihilator of a subspace W of V, it suffices to find the linear func-
tionals that annihilate any basis for W (see Example 9.2 below).
454 LINEAR FORMS

Just as we talked about the second dual of a vector space, we may define
the space S00 in the obvious manner by

S00 = (S0)0 = {v ∞ V: ƒ(v) = 0 for all ƒ ∞ S0} .

This is allowed because of our identification of V and V** under the isomor-
phism u ’ fu . To be precise, note that if v ∞ S ™ V is arbitrary, then for any
ƒ ∞ S0 we have fv(ƒ) = ƒ(v) = 0, and hence fv ∞ (S0)0 = S0 0. But by our
identification of v and fv (i.e., the identification of V and V**) it follows that
v ∞ S00, and thus S ™ S00. If S happens to be subspace of V, then we can in
fact say more than this.

Theorem 9.5 Let V be finite-dimensional and W a subspace of V. Then

(a) dim W0 = dim V - dim W.
(b) W00 = W.

Proof (a) Assume that dim V = n and dim W = m ¯ n. If we choose a basis

{wè, . . . , wm} for W, then we may extend this to a basis

{wè, . . . , wm , vè, . . . , vn-m}

for V (Theorem 2.10). Corresponding to this basis for V, we define the dual
basis
{ƒ1, . . . , ƒm, œ1, . . . , œn-m}

for V*. By definition of dual basis we then have œi(vé) = ∂ié and œi(wé) = 0 for
all wé. This shows that œi ∞ W0 for each i = 1, . . . , n - m. We claim that {œi}
forms a basis for W0.
Since each œi is an element of a basis for V*, the set {œi} must be linearly
independent. Now let ß ∞ W0 be arbitrary. Applying Theorem 9.1 (and
remembering that wá ∞ W) we have
m n$m n$m
i j
! = # ! ,!wi " + # ! ,!v j % = # ! ,!v j % j !!.
i=1 j=1 j=1

This shows that the œi also span W0, and hence they form a basis for W0.
Therefore dim W0 = n - m = dim V - dim W.
(b) Recall that the discussion preceding this theorem showed that W ™
W00. To show that W = W00, we need only show that dim W = dim W00.
However, since W0 is a subspace of V* and dim V* = dim V, we may apply
part (a) to obtain
 9.2 DOUBLE DUALS AND ANNIHILATORS 455

dimW 00 = dimV* ! dimW 0

= dimV* ! (dimV ! dimW )
= dimW !!.!!˙

Example 9.2 Let W ™ ®4 be the two-dimensional subspace spanned by the

(column) vectors wè = (1, 2, -3, 4) and wì = (0, 1, 4, -1). To find a basis for
W0, we seek dim W0 = 4 - 2 = 2 independent linear functionals ƒ of the form
ƒ(x, y, z, t) = ax + by + cz + dt such that ƒ(wè) = ƒ(wì) = 0. (This is just
ƒ(w) = ̓áwi where w = (x, y, z, t) and ƒ = (a, b, c, d).) This means that we
must solve the set of linear equations

! (1,!2,!"3,!4) = a + 2b " 3c + 4t = 0
! (0,!1,!4,!"1) =!!!!!!!!b + 4c "!! t!= 0

which are already in row-echelon form with c and t as free variables (see
Section 3.5). We are therefore free to choose any two distinct sets of values
we like for c and t in order to obtain independent solutions.
If we let c = 1 and t = 0, then we obtain a = 11 and b = -4 which yields the
linear functional ƒ1(x, y, z, t) = 11x - 4y + z. If we let c = 0 and t = 1, then we
obtain a = -6 and b = 1 so that ƒ2(x, y, z, t) = -6x + y + t. Therefore a basis
for W0 is given by the pair {ƒ1, ƒ2}. In component form, these basis (row)
vectors are simply
!1 = (11,!"4,!1,!0)
! 2 = ("6,!1,!0,!1)!!.!!!

This example suggests a general approach to finding the annihilator of a

subspace W of Fn. To see this, first suppose that we have m ¯ n linear equa-
tions in n unknowns:
n
! aij x j = 0
j=1

for each i = 1, . . . , m. If we define the m linear functionals ƒi by

n
! i (x1,!…!,!xn ) = " aij x j
j=1
then we see that the solution space of our system of equations is nothing more
than the subspace of Fn that is annihilated by {ƒi}. Recalling the material of
Section 3.5, we know that the solution space to this system is found by row-
reducing the matrix A = (aáé). Note also that the row vectors Aá are just the
456 LINEAR FORMS

coordinates of the linear functional ƒi relative to the basis of Fn* that is dual
to the standard basis for Fn.
Now suppose that for each i = 1, . . . , m we are given the vector n-tuple
vá = (aáè, . . . , aáñ) ∞ Fn. What we would like to do is find the annihilator of
the subspace W ™ Fn that is spanned by the vectors vá. From the previous
section (and the above example) we know that any linear functional ƒ on Fn
must have the form ƒ(xè, . . . , xñ) = Íi ˆ= 1cáxá, and hence the annihilator we
seek satisfies the condition
n
! (vi ) = ! (ai1,!…!,!ain ) = " aij c j = 0
j=1
for each i = 1, . . . , m. In other words, the annihilator (cè, . . . , cñ) is a solution
of the homogeneous system
n
! aij c j = 0!!.
j=1

Example 9.3 Let W ™ ®5 be spanned by the four vectors

v1 = (2,!!2,!3,!4,!!1) v2 = (!1,!1,!2,!5,!2)
v3 = (0,!0,!!1,!!2,!3) v4 = (1,!!1,!2,!3,!0)!!.

Then W0 is found by row-reducing the matrix A whose rows are the basis
vectors of W:
" !!2 !2 !!3 !!4 !1 %
$ !1 !!1 !!2 !!5 !!2 '
A=$ '!!.
$ !!0 !!0 !1 !2 !!3'
# !!1 !1 !!2 !!3 !!0 &

Using standard techniques, the reduced matrix is easily found to be

"1 !1 !0 !1 !0 %
$0 !!0 !1 !!2 !0 '
$ '!!.
$0 !!0 !0 !!0 !1 '
#0 !!0 !0 !!0 !0 &

This is equivalent to the equations

c1 ! c2 !!!!!!!!!c4 !!!!!!= 0
c3 +!2c4 !!!!!!= 0
c5 = 0
 9.2 DOUBLE DUALS AND ANNIHILATORS 457

and hence the free variables are cì and c4. Note that the row-reduced form of
A shows that dim W = 3, and hence dim W0 = 5 - 3 = 2. Choosing cì = 1 and
c4 = 0 yields cè = 1 and c3 = 0, and hence one of the basis vectors for W0 is
given by ƒ1 = (1, 1, 0, 0, 0). Similarly, choosing cì = 0 and c4 = 1 results in the
other basis vector ƒ2 = (1, 0, -2, 1, 0). ∆

Exercises

1. Let U and W be subspaces of V (which may be infinite-dimensional).

Prove that:
(a) (U + W)0 = U0 ⁄ W0.
(b) (U ⁄ W)0 = U0 + W0.
Compare with Exercise 2.5.2.

2. Let V be finite-dimensional and W a subspace of V. Prove that W* is iso-

morphic to V*/W0 and (independently of Theorem 9.5) also that

dim W0 = dim V - dim W .

[Hint: Consider the mapping T: V* ‘ W* defined by Tƒ = ƒw where ƒw

is the restriction of ƒ ∞ V* to W. Show that T is a surjective linear
transformation and that Ker T = W0. Now apply Exercise 1.5.11 and
Theorems 5.4 and 7.34.]

3. Let V be an n-dimensional vector space. An (n - 1)-dimensional subspace

of V is said to be a hyperspace (or hyperplane). If W is an m-
dimensional subspace of V, show that W is the intersection of n - m
hyperspaces in V.

4. Let U and W be subspaces of a finite-dimensional vectors space V. Prove

that U = W if and only if U0 = W0.

5. Let {e1, . . . , e5} be the standard basis for ®5, and let W ™ ®5 be spanned
by the three vectors
w1 = e1 + 2e2 +!! e3
w2 =!!!!!!!!! e2 + 3e3 + 3e4 + e5
w3 = e1 + 4e2 + 6e3 + 4e4 + e5 !!.

Find a basis for W0.

458 LINEAR FORMS

9.3 THE TRANSPOSE OF A LINEAR TRANSFORMATION

Suppose U and V are vector spaces over a field F, and let U* and V* be the
corresponding dual spaces. We will show that any T ∞ L(U, V) induces a
linear transformation T* ∞ L(V*, U*) in a natural way. We begin by recalling
our discussion in Section 5.4 on the relationship between two bases for a
vector space. In particular, if a space V has two bases {eá} and {eõá}, we seek
the relationship between the corresponding dual bases {øi} and {øùi} for V*.
This is given by the following theorem.

Theorem 9.6 Let {eá} and {eõá} be two bases for a finite-dimensional vector
space V, and let {øi} and {øùi} be the corresponding dual bases for V*. If P is
the transition matrix from the basis {eá} to the basis {eõá}, then (Pî)T is the
transition matrix from the {øi} basis to the {øùi} basis.

Proof Let dim V = n. By definition of P = (páé) we have

n
ei = ! e j p ji
j=1
for each i = 1, . . . , n. Similarly, let us define the (transition) matrix Q = (qáé)
by the requirement that
n
!i = " ! j q ji !!.
j=1

We must show that Q = (Pî)T. To see this, first note that the ith column of Q
is Qi = (qèá, . . . , qñá) and the jth row of PT is PTé = (pTéè, . . . , pTéñ). From the
definition of dual bases, we then see that

! i j = "i ,!e j = #k" k qki ,! #r er prj = #k, r qki prj " k ,!er
= #k, r qki prj! k r = #k qki pkj = #k pT jk qki
= (PT Q) ji !!.

In other words, PTQ = I. Since P is a transition matrix it is nonsingular, and

hence this shows that Q = (PT)î = (Pî)T (Theorem 3.21, Corollary 4). ˙

Now suppose that T ∞ L(V, U). We define a mapping T*: U* ‘ V* by

the rule
T*ƒ = ƒ ı T

for all ƒ ∞ U*. (The mapping T* is frequently written Tt.) In other words, for
any v ∞ V we have
 9.3 THE TRANSPOSE OF A LINEAR TRANSFORMATION 459

(T*ƒ)(v) = (ƒ ı T)(v) = ƒ(T(v)) ∞ F .

To show that T*ƒ is indeed an element of V*, we simply note that for vè, vì ∞
V and a, b ∞ F we have (using the linearity of T and ƒ)

(T *! )(av1 + bv2 ) = ! (T (av1 + bv2 ))

= ! (aT (v1 ) + bT (v2 ))
= a! (T (v1 )) + b! (T (v2 ))
= a(T *! )(v1 ) + b(T *! )(v2 )

(this also follows directly from Theorem 5.2). Furthermore, it is easy to see
that the mapping T* is linear since for any ƒ, œ ∞ U* and a, b ∞ F we have

T*(aƒ + bœ) = (aƒ + bœ) ı T = a(ƒ ı T) + b(œ ı T) = a(T*ƒ) + b(T*œ) .

Hence we have proven the next result.

Theorem 9.7 Suppose T ∞ L(V, U), and define the mapping T*: U* ‘ V*
by T*ƒ = ƒ ı T for all ƒ ∞ U*. Then T* ∞ L(U*, V*).

The linear mapping T* defined in this theorem is called the transpose of

the linear transformation T. The reason for the name transpose is shown in the
next theorem. Note that we make a slight change in our notation for elements
of the dual space in order to keep everything as simple as possible.

Theorem 9.8 Let T ∞ L(V, U) have matrix representation A = (aáé) with

respect to the bases {vè, . . . , vm} for V and {uè, . . . , uñ} for U . Let the dual
spaces V* and U* have the corresponding dual bases {või} and {uõi}. Then the
matrix representation of T* ∞ L(U*, V*) with respect to these bases for U*
and V* is given by AT.

Proof By definition of A = (aáé) we have

n
Tvi = ! u j a ji
j=1
for each i = 1, . . . , m. Define the matrix representation B = (báé) of T* by
m
T *u = ! v j b ji
i

j=1
for each i = 1, . . . , n. Applying the left side of this equation to an arbitrary
basis vector vÉ, we find
460 LINEAR FORMS

(T*uõi)vÉ = uõi(TvÉ) = uõi(ÍéuéaéÉ) = Íéuõi(ué)aéÉ = Íé∂iéaéÉ = aáÉ

while the right side yields

Íébéávõj(vÉ) = Íébéá∂jÉ = bÉá .

Therefore bÉá = aáÉ = aTÉá , and thus B = AT. ˙

Example 9.4 If T ∞ L(V, U), let us show that Ker T* = (Im T)0. (Remember
that T*: U* ‘ V*.) Let ƒ ∞ Ker T* be arbitrary, so that 0 = T*ƒ = ƒ ıT. If
u ∞ U is any element in Im T, then there exists v ∞ V such that u = Tv. Hence

ƒ(u) = ƒ(Tv) = (T*ƒ)v = 0

and thus ƒ ∞ (Im T)0. This shows that Ker T* ™ (Im T)0.
Now suppose œ ∞ (Im T)0 so that œ(u) = 0 for all u ∞ Im T. Then for any
v ∞ V we have
(T*œ)v = œ(Tv) ∞ œ(Im T) = 0

and hence T*œ = 0. This shows that œ ∞ Ker T* and therefore (Im T)0 ™
Ker T*. Combined with the previous result, we see that Ker T* = (Im T)0. ∆

Example 9.5 Suppose T ∞ L(V, U) and recall that r(T) is defined to be the
number dim(Im T). We will show that r(T) = r(T*). From Theorem 9.5 we
have
dim(Im T)0 = dim U - dim(Im T) = dim U - r(T)

and from the previous example it follows that

nul T* = dim(Ker T*) = dim(Im T)0 .

Therefore (using Theorem 5.6) we see that

r(T *) = dimU* ! nulT * = dimU ! nulT * = dimU ! dim(ImT )0

= r(T )!!.
 9.3 THE TRANSPOSE OF A LINEAR TRANSFORMATION 461

Exercises

1. Suppose A ∞ Mmxn(F). Use Example 9.5 to give a simple proof that

rr(A) = cr(A).

2. Let V = ®2 and define ƒ ∞ V* by ƒ(x, y) = 3x - 2y. For each of the fol-

lowing linear transformations T ∞ L(®3, ®2), find (T*ƒ)(x, y, z):
(a) T(x, y, z) = (x + y, y + z).
(b) T(x, y, z) = (x + y + z, 2x - y).

3. If S ∞ L(U, V) and T ∞ L(V, W), prove that (T ı S)* = S* ı T*.

4. Let V be finite-dimensional, and suppose that T ∞ L(V). Show that the

mapping T ’ T* defines an isomorphism of L(V) onto L(V*).

5. Let V = ®[x], suppose a, b ∞ ® are fixed, and define ƒ ∞ V* by

b
!( f ) = " a f (x) dx !!.
If D is the usual differentiation operator on V, find D*ƒ.

6. Let V = Mn(F), let B ∞ V be fixed, and define T ∞ L(V) by

T(A) = AB - BA .

If ƒ ∞ V* is defined by ƒ(A) = Tr A, find T*ƒ.

9.4 BILINEAR FORMS

In order to facilitate our treatment of operators (as well as our later discussion
of the tensor product), it is worth generalizing slightly some of what we have
done so far in this chapter. Let U and V be vector spaces over F. We say that a
mapping f: U ª V ‘ F is bilinear if it has the following properties for all uè,
uì ∞ U, for all vè, vì ∞ V and all a, b ∞ F:

(1) f(auè + buì, vè) = af(uè, vè) + bf(uì, vè).

(2) f(uè, avè + bvì) = af(uè, vè) + bf(uè, vì).

In other words, f is bilinear if for each v ∞ V the mapping u ’ f(u, v) is

linear, and if for each u ∞ U the mapping v ’ f(u, v) is linear. In the
particular case that V = U, then the bilinear map f: V ª V ‘ F is called a
bilinear form on V. (Note that a bilinear form is defined on V ª V, while a
bilinear functional was defined on V* ª V.) Rather than write expressions like
462 LINEAR FORMS

f(u, v), we will sometimes write the bilinear map as Óu, vÔ if there is no need to
refer to the mapping f explicitly. While this notation is used to denote several
different operations, the context generally makes it clear exactly what is
meant.
We say that the bilinear map f: U ª V ‘ F is nondegenerate if f(u, v) = 0
for all v ∞ V implies that u = 0, and f(u, v) = 0 for all u ∞ U implies that v = 0.

Example 9.6 Suppose A = (aáé) ∞ Mn(F). Then we may interpret A as a

bilinear form on Fn as follows. In terms of the standard basis {eá} for Fn, any
X ∞ Fn may be written as X = Í xieá , and hence for all X, Y ∞ Fn we define
the bilinear form fA by

fA(X, Y) = Íi, jaáéxi y j = XTAY .

Here the row vector XT is the transpose of the column vector X, and the
expression XTAY is just the usual matrix product. It should be easy for the
reader to verify that fA is actually a bilinear form on Fn. ∆

Example 9.7 Suppose å, ∫ ∞ V*. Since å and ∫ are linear, we may define a
bilinear form f: V ª V ‘F by

f(u, v) = å(u)∫(v)

for all u, v ∞ V. This form is usually denoted by å · ∫ and is called the

tensor product of å and ∫. In other words, the tensor product of two elements
å, ∫ ∞ V* is defined for all u, v ∞ V by

(å · ∫)(u, v) = å(u)∫(v) .

We may also define the bilinear form g: V ª V ‘F by

g(u, v) = å(u)∫(v) - å(v)∫(u) .

We leave it to the reader to show that this is indeed a bilinear form. The map-
ping g is usually denoted by å°∫, and is called the wedge product or the
antisymmetric tensor product of å and ∫. In other words

(å°∫)(u, v) = å(u)∫(v) - å(v)∫(u) .

Note that å°∫ is just å · ∫ - ∫ · å. ∆

 9.4 BILINEAR FORMS 463

Generalizing Example 9.6 leads to the following theorem.

Theorem 9.9 Given a bilinear map f: Fm ª Fn ‘ F, there exists a unique

matrix A ∞ Mmxn(F) such that f = fA. In other words, there exists a unique
matrix A such that f(X, Y) = XTAY for all X ∞ Fm and Y ∞ Fn.

Proof In terms of the standard bases for Fm and Fn, we have the column
vectors X = Í i ˜= 1 xi eá ∞ Fm and Y = Íj ˆ= 1 yj eé ∞ Fn. Using the bilinearity of
f we then have

f(X, Y) = f(Íáxi eá , Íéyj eé) = Íi, j xi yjf(eá, eé) .

If we define aáé = f(eá, eé), then we see that our expression becomes

f(X, Y) = Íi, j xi aáéyj = XTAY .

To prove the uniqueness of the matrix A, suppose there exists a matrix Aæ

such that f = fAæ. Then for all X ∞ Fm and Y ∞ Fn we have

f(X, Y) = XTAY = XTAæY

and hence XT(A - Aæ)Y = 0. Now let C = A - Aæ so that

XTCY = Íi, j cáéxi yj = 0

for all X ∞ Fm and Y ∞ Fn. In particular, choosing X = eá and Y = eé, we find

that cáé = 0 for every i and j. Thus C = 0 so that A = Aæ. ˙

The matrix A defined in this theorem is said to represent the bilinear map
f relative to the standard bases for Fm and Fn. It thus appears that f is repre-
sented by the mn elements aáé = f(eá, eé). It is extremely important to realize
that the elements aáé are defined by the expression f(eá, eé) and, conversely,
given a matrix A = (aáé), we define the expression f(eá, eé) by requiring that
f(eá, eé) = aáé . In other words, to say that we are given a bilinear map f: Fm ª
Fn ‘ F means that we are given values of f(eá, eé) for each i and j. Then,
given these values, we can evaluate expressions of the form f(X, Y) =
Íi, j xiyj f(eá, eé). Conversely, if we are given each of the f(eá, eé), then we have
defined a bilinear map on Fm ª Fn.
464 LINEAR FORMS

We denote the set of all bilinear maps on U and V by B(U ª V, F), and the
set of all bilinear forms as simply B(V) = B(V ª V, F). It is easy to make
B(U ª V, F) into a vector space over F. To do so, we simply define

(af + bg)(u, v) = af(u, v) + bg(u, v)

for any f, g ∞ B(U ª V, F) and a, b ∞ F. The reader should have no trouble

showing that af + bg is itself a bilinear mapping.
It is left to the reader (see Exercise 9.4.1) to show that the association
A ’ fA defined in Theorem 9.9 is actually an isomorphism between
Mmxn (F) and B(Fm ª Fn, F). More generally, it should be clear that Theo-
rem 9.9 applies equally well to any pair of finite-dimensional vector spaces U
and V, and from now on we shall treat it as such.

Theorem 9.10 Let V be finite-dimensional over F, and let V* have basis

{øi}. Define the elements fij ∞ B(V) by

fij(u, v) = øi(u)øj(v)

for all u, v ∞ V. Then {fij} forms a basis for B(V) which thus has dimension
(dim V)2.

Proof Let {eá} be the basis for V dual to the {øi} basis for V*, and define
aáé = f(eá, eé). Given any f ∞ B(V), we claim that f = Íi, j aáéfij. To prove this, it
suffices to show that f(er, es) = (Íi, j aáéfij)(er, es) for all r and s. We first note
that
( !i, j aij f ij )(er ,!es ) = !i, j aij" i (er )" j (es ) = !i, j aij# i r# j s = ars
= f (er ,!es )!!.

Since f is bilinear, it follows from this that f(u, v) = (Íi, j aáéf ij)(u, v) for all u,
v ∞ V so that f = Íi, j aáéfij. Hence {fij} spans B(V).
Now suppose that Íi, j aáéfij = 0 (note that this 0 is actually an element of
B(V)). Applying this to (er, es) and using the above result, we see that

0 = (Íi, j aáéfij)(er , es) = ars .

Therefore {fij} is linearly independent and hence forms a basis for B(V). ˙
 9.4 BILINEAR FORMS 465

It should be mentioned in passing that the functions fij defined in Theorem

9.10 can be written as the tensor product øi · øj: V ª V ‘ F (see Example
9.7). Thus the set of bilinear forms øi · øj forms a basis for the space V* ·
V* which is called the tensor product of the two spaces V*. This remark is
not meant to be a complete treatment by any means, and we will return to
these ideas in Chapter 11.
We also note that if {eá} is a basis for V and dim V = n, then the matrix A
of any f ∞ B(V) has elements aáé = f(eá, eé), and hence A = (aáé) has n2 inde-
pendent elements. Thus, dim B(V) = n2 as we saw above.

Theorem 9.11 Let P be the transition matrix from a basis {eá} for V to a new
basis {eæá}. If A is the matrix of f ∞ B(V) relative to {eá}, then Aæ = PTAP is
the matrix of f relative to the basis {eæá}.

Proof Let X, Y ∞ V be arbitrary. In Section 5.4 we showed that the transition

matrix P = (páé) defined by eæá = P(eá) = Íéeépéá also transforms the components
of X = Íáxieá = Íéxæj eæé as xi = Íépáéxæj. In matrix notation, this may be written
as [X]e = P[X]eæ (see Theorem 5.17), and hence [X]eT = [X]eæT PT. From
Theorem 9.9 we then have

f(X, Y) = [X]eTA[Y]e = [X]eæT[P]TA[P][Y]eæ = [X]eæTAæ[Y]eæ .

Since X and Y are arbitrary, this shows that Aæ = PTAP is the unique repre-
sentation of f in the new basis {eæá}. ˙

Just as the transition matrix led to the definition of a similarity transforma-

tion, we now say that a matrix B is congruent to a matrix A if there exists a
nonsingular matrix P such that B = PTAP. It was shown in Exercise 5.2.12
that if P is nonsingular, then r(AP) = r(PA) = r(A). Since P is nonsingular, r(P)
= r(PT), and hence r(B) = r(PTAP) = r(AP) = r(A). In other words, congruent
matrices have the same rank. We are therefore justified in defining the rank
r(f ) of a bilinear form f on V to be the rank of any matrix representation of f.
We leave it to the reader to show that f is nondegenerate if and only if r(f ) =
dim V (see Exercise 9.4.3).

Exercises

1. Show that the association A ’ fA defined in Theorem 9.9 is an isomor-

phism between Mm x m(F) and B(Fm ª Fn, F).
466 LINEAR FORMS

2. Let V = Mmxn(F) and suppose A ∞ Mm(F) is fixed. Then for any X, Y ∞

V we define the mapping fA: V ª V ‘F by fA(X, Y) = Tr(XTAY). Show
that this defines a bilinear form on V.

3. Prove that a bilinear form f on V is nondegenerate if and only if r(f ) =

dim V.

4. (a) Let V = ®3 and define f ∞ B(V) by

f(X, Y) = 3x1y1 - 2x1y2 + 5x2y1 + 7x2y2 - 8x2y3 + 4x3y2 - x3y3 .

Write out f(X, Y) as a matrix product XTAY.

(b) Suppose A ∞ Mn(F) and let f(X, Y) = XTAY for X, Y ∞ Fn. Show
that f ∞ B(Fn).

5. Let V = ®2 and define f ∞ B(V) by

f(X, Y) = 2x1y1 - 3x1y2 + x2y2 .

(a) Find the matrix representation A of f relative to the basis v1 = (1, 0),
v2 = (1, 1).
(b) Find the matrix representation B of f relative to the basis võ1 = (2, 1),
võ2 = (1, -1).
(c) Find the transition matrix P from the basis {vá} to the basis {võá} and
verify that B = PTAP.

6. Let V = Mn(ç), and for all A, B ∞ V define

f(A, B) = n Tr(AB) - (Tr A)(Tr B) .

(a) Show that this defines a bilinear form on V.

(b) Let U ™ V be the subspace of traceless matrices. Show that f is degen-
erate, but that fU = f|U is nondegenerate.
(c) Let W ™ V be the subspace of all traceless skew-Hermitian matrices A
(i.e., Tr A = 0 and A¿ = A*T = -A). Show that fW = f|W is negative defi-
nite, i.e., that fW(A, A) < 0 for all nonzero A ∞ W.
(d) Let Vÿ ™ V be the set of all matrices A ∞ V with the property that
f(A, B) = 0 for all B ∞ V. Show that Vÿ is a subspace of V. Give an explicit
description of Vÿ and find its dimension.
 9.4 BILINEAR FORMS 467

9.5 SYMMETRIC AND ANTISYMMETRIC BILINEAR FORMS

An extremely important type of bilinear form is one for which f(u, u) = 0 for
all u ∞ V. Such forms are said to be alternating. If f is alternating, then for
every u, v ∞ V we have

0 = f (u + v,!u + v)
= f (u,!u) + f (u,!v) + f (v,!u) + f (v,!v)
= f (u,!v) + f (v,!u)
and hence
f (u,!v) = ! f (v,!u)!!.

A bilinear form that satisfies this condition is called antisymmetric (or skew-
symmetric). If we let v = u, then this becomes f(u, u) + f(u, u) = 0. As long as
F is not of characteristic 2 (see the discussion following Theorem 4.3; this is
equivalent to the statement that 1 + 1 ≠ 0 in F), we can conclude that f(u, u) =
0. Thus, as long as the base field F is not of characteristic 2, alternating and
antisymmetric forms are equivalent. We will always assume that 1 + 1 ≠ 0 in
F unless otherwise noted, and hence we always assume the equivalence of
alternating and antisymmetric forms.
It is also worth pointing out the simple fact that the diagonal matrix ele-
ments of any representation of an alternating (or antisymmetric) bilinear form
will necessarily be zero. This is because the diagonal elements are given by
aáá = f(eá, eá) = 0.

Theorem 9.12 Let f ∞ B(V) be alternating. Then there exists a basis for V in
which the matrix A of f takes the block diagonal form

A = M•~~~•M•0•~~~•0

where 0 is the 1 x 1 matrix (0), and

" !!0 1 %
M =$ '!!.
# !1 0 &

Moreover, the number of blocks consisting of the matrix M is just (1/2)r(f).

Proof We first note that the theorem is clearly true if f = 0. Next we note that
if dim V = 1, then any vector vá ∞ V is of the form vá = aáu for some basis
vector u and scalar aá. Therefore, for any vè, vì ∞ V we have
468 LINEAR FORMS

f(vè, vì) = f(aèu, aìu) = aèaìf(u, u) = 0

so that again f = 0. We now assume that f ≠ 0 and that dim V > 1, and proceed
by induction on dim V. In other words, we assume the theorem is true for
dim V < n, and proceed to show that it is also true for dim V = n.
Since dim V > 1 and f ≠ 0, there exist nonzero vectors uè, uì ∞ V such that
f(uè, uì) ≠ 0. Moreover, we can always multiply uè by the appropriate scalar so
that
f(uè, uì) = 1 = -f(uì, uè) .

It is also true that uè and uì must be linearly independent because if uì = kuè,

then f(uè, uì) = f(uè, kuè) = kf(uè, uè) = 0. We can now define the two-
dimensional subspace U ™ V spanned by the vectors {uè, uì}. By definition,
the matrix (aáé) ∞ Mì(F) of f restricted to U is given by aáé = f(uá, ué), and
hence it is easy to see that (aáé) is given by the matrix M defined in the
statement of the theorem.
Since any u ∞ U is of the form u = auè + buì, we see that

f(u, uè) = af(uè, uè) + bf(uì, uè) = -b

and
f(u, uì) = af(uè, uì) + bf(uì, uì) = a .

Now define the set

W = {w ∞ V: f(w, u) = 0 for every u ∞ U} .

We claim that V = U • W (compare this with Theorem 2.22). To show that

U ⁄ W = {0}, we assume that v ∞ U ⁄ W. Then v ∞ U has the form v = åuè +
∫uì for some scalars å and ∫. But v ∞ W so that 0 = f(v, uè) = -∫ and 0 =
f(v, uì) = å, and hence v = 0.
We now show that V = U + W. Let v ∞ V be arbitrary, and define the
vectors
u = f (v,!u2 )u1 ! f (v,!u1 )u2 " U
w = v ! u!!.

If we can show that w ∞ W, then we will have shown that v = u + w ∞ U + W

as desired. But this is easy to do since we have

f (u,!u1 ) = f (v,!u2 ) f (u1,!u1 ) ! f (v,!u1 ) f (u2 ,!u1 ) = f (v,!u1 )

f (u,!u2 ) = f (v,!u2 ) f (u1,!u2 ) ! f (v,!u1 ) f (u2 ,!u2 ) = f (v,!u2 )

and therefore we find that

 9.5 SYMMETRIC AND ANTISYMMETRIC BILINEAR FORMS 469

f (w,!u1 ) = f (v ! u,!u1 ) = f (v,!u1 ) ! f (u,!u1 ) = 0

f (w,!u2 ) = f (v ! u,!u2 ) = f (v,!u2 ) ! f (u,!u2 ) = 0!!.

These equations show that f(w, u) = 0 for every u ∞ U, and thus w ∞ W. This
completes the proof that V = U • W, and hence it follows that dim W =
dim V - dim U = n - 2 < n.
Next we note that the restriction of f to W is just an alternating bilinear
form on W, and therefore, by our induction hypothesis, there exists a basis
{u3, . . . , un} for W such that the matrix of f restricted to W has the desired
form. But the matrix of V is the direct sum of the matrices of U and W, where
the matrix of U was shown above to be M. Therefore {uè, uì, . . . , uñ} is a
basis for V in which the matrix of f has the desired form.
Finally, it should be clear that the rows of the matrix of f that are made up
of the portion M • ~ ~ ~ • M are necessarily linearly independent (by defini-
tion of direct sum and the fact that the rows of M are independent). Since each
M contains two rows, we see that r(f) = rr(f) is precisely twice the number of
M matrices in the direct sum. ˙

Corollary 1 Any nonzero alternating bilinear form must have even rank.

Proof Since the number of M blocks in the matrix of f is (1/2)r(f), it follows

that r(f) must be an even number. ˙

Corollary 2 If there exists a nondegenerate, alternating form on V, then

dim V is even.

Proof This is Exercise 9.5.7. ˙

If f ∞ B(V) is alternating, then the matrix elements aáé representing f rela-

tive to any basis {eá} for V are given by

aáé = f(eá, eé) = -f(eé, eá) = -aéá .

Any matrix A = (aáé) ∞ Mn(F) with the property that aáé = -aéá (i.e., A = -AT)
is said to be antisymmetric. If we are given any element aáé of an anti-
symmetric matrix, then we automatically know aéá. Because of this, we say
that aáé and aéá are not independent. Since the diagonal elements of any such
antisymmetric matrix must be zero, this means that the maximum number of
independent elements in A is given by (n2 - n)/2. Therefore, the subspace of
B(V) consisting of nondegenerate alternating bilinear forms is of dimension
n(n - 1)/2.
470 LINEAR FORMS

Another extremely important class of bilinear forms on V is that for which

f(u, v) = f(v, u) for all u, v ∞ V. In this case we say that f is symmetric, and
we have the matrix representation

aáé = f(eá, eé) = f(eé, eá) = aéá .

As expected, any matrix A = (aáé) with the property that aáé = aéá (i.e., A = AT)
is said to be symmetric. In this case, the number of independent elements of
A is [(n2 - n)/2] + n = (n2 + n)/2, and hence the subspace of B(V) consisting
of symmetric bilinear forms has dimension n(n + 1)/2.
It is also easy to prove generally that a matrix A ∞ Mn(F) represents a
symmetric bilinear form on V if and only if A is a symmetric matrix. Indeed,
if f is a symmetric bilinear form, then for all X, Y ∞ V we have

XTAY = f(X, Y) = f(Y, X) = YTAX .

But XTAY is just a 1 x 1 matrix, and hence (XTAY)T = XTAY. Therefore

(using Theorem 3.18) we have

YTAX = XTAY = (XTAY)T = YTATXT T = YTATX .

Since X and Y are arbitrary, this implies that A = AT. Conversely, suppose
that A is a symmetric matrix. Then for all X, Y ∞ V we have

XTAY = (XTAY)T = YTATXT T = YTAX

so that A represents a symmetric bilinear form. The analogous result holds for
antisymmetric bilinear forms as well (see Exercise 9.5.2).
Note that adding the dimensions of the symmetric and antisymmetric sub-
spaces of B(V) we find

n(n - 1)/2 + n(n + 1)/2 = n2 = dim B(V) .

This should not be surprising since, for an arbitrary bilinear form f ∞ B(V)
and any X, Y ∞ V, we can always write

f(X, Y) = (1/2)[f(X, Y) + f(Y, X)] + (1/2)[f(X, Y) - f(Y, X)] .

In other words, any bilinear form can always be written as the sum of a sym-
metric and an antisymmetric bilinear form.
 9.5 SYMMETRIC AND ANTISYMMETRIC BILINEAR FORMS 471

There is another particular type of form that is worth distinguishing. In

particular, let V be finite-dimensional over F, and let f = Ó , Ô be a symmetric
bilinear form on V. We define the mapping q: V ‘ F by

q(X) = f(X, X) = ÓX, XÔ

for every X ∞ V. The mapping q is called the quadratic form associated

with the symmetric bilinear form f. It is clear that (by definition) q is
represented by a symmetric matrix A, and hence it may be written in the
alternative forms

q(X) = XTAX = Íi, jaáéxixj = Íáaáá(xi)2 + 2Íi< jaáéxixj .

This expression for q in terms of the variables xi is called the quadratic poly-
nomial corresponding to the symmetric matrix A. In the case where A hap-
pens to be a diagonal matrix, then aáé = 0 for i ≠ j and we are left with the
simple form q(X) = aèè(x1)2 + ~ ~ ~ + aññ(xn)2. In other words, the quadratic
polynomial corresponding to a diagonal matrix contains no “cross product”
terms.
While we will show below that every quadratic form has a diagonal repre-
sentation, let us first look at a special case.

Example 9.8 Consider the real quadratic polynomial on Fn defined by

q(Y) = Íi, jbáéyiyj

(where báé = béá as usual for a quadratic form). If it happens that bèè = 0 but, for
example, that bèì ≠ 0, then we make the substitutions

y1 = x1 + x 2
y 2 = x1 ! x 2
yi = x i for i = 3,!…!,!n!!.

A little algebra (which you should check) then shows that q(Y) takes the form

q(Y) = Íi, j cáéxixj

where now cèè ≠ 0. This means that we can focus our attention on the case
q(X) = Íi, jaáéxixj where it is assumed that aèè ≠ 0.
472 LINEAR FORMS

Thus, given the real quadratic form q(X) = Íi, jaáéxixj where a11 ≠ 0, let us
make the substitutions

x1 = y1 ! (1/a11 )[a12 y 2 +!+ a1n y n ]

x i = yi for each i = 2,!…!,!n!!.

Some more algebra shows that q(X) now takes the form

q(x1, . . . , xn) = a11(y1)2 + qæ(y2, . . . , yn)

where qæ is a new quadratic polynomial. Continuing this process, we eventu-

ally arrive at a new set of variables in which q has a diagonal representation.
This is called completing the square. ∆

Given any quadratic form q, it is possible to fully recover the values of f

from those of q. To show this, let u, v ∞ V be arbitrary. Then

q(u + v) = u + v,!u + v
= u,!u + u,!v + v,!u + v,!v
= q(u) + 2 f (u,!v) + q(v)

and therefore
f (u,!v) = (1/2)[q(u + v) ! q(u) ! q(v)]!!.

This equation is called the polar form of f.

Theorem 9.13 Let f be a symmetric bilinear form on a finite-dimensional

space V. Then there exists a basis {eá} for V in which f is represented by a
diagonal matrix. Alternatively, if f is represented by a (symmetric) matrix A in
one basis, then there exists a nonsingular transition matrix P to the basis {eá}
such that PTAP is diagonal.

Proof Since the theorem clearly holds if either f = 0 or dim V = 1, we assume

that f ≠ 0 and dim V = n > 1, and proceed by induction on dim V. If q(u) =
f(u, u) = 0 for all u ∞ V, then the polar form of f shows that f = 0, a contradic-
tion. Therefore, there must exist a vector vè ∞ V such that f(vè, vè) ≠ 0. Now
let U be the (one-dimensional) subspace of V spanned by vè, and define the
subspace W = {u ∞ V: f(u, vè) = 0}. We claim that V = U • W.
Suppose v ∞ U ⁄ W. Then v ∞ U implies that v = kvè for some scalar k,
and hence v ∞ W implies 0 = f(v, vè) = k f(vè, vè). But since f(vè, vè) ≠ 0 we
must have k = 0, and thus v = kvè = 0. This shows that U ⁄ W = {0}.
 9.5 SYMMETRIC AND ANTISYMMETRIC BILINEAR FORMS 473

Now let v ∞ V be arbitrary, and define

w = v - [f(v, vè)/f(vè, vè)]v1 .

Then
f(w, vè) = f(v, vè) - [f(v, vè)/f(vè, vè)]f(vè, vè) = 0

and hence w ∞ W. Since the definition of w shows that any v ∞ V is the sum
of w ∞ W and an element of U, we have shown that V = U + W, and hence V
= U • W.
We now consider the restriction of f to W, which is just a symmetric
bilinear form on W. Since dim W = dim V - dim U = n - 1, our induction
hypothesis shows there exists a basis {eì, . . . , eñ} for W such that f(eá, eé) = 0
for all i ≠ j where i, j = 2, . . . , n. But the definition of W shows that f(eá, vè) =
0 for each i = 2, . . . , n, and thus if we define eè = vè, the basis {eè, . . . , eñ} for
V has the property that f(eá, eé) = 0 for all i ≠ j where now i , j = 1, . . . , n. This
shows that the matrix of f in the basis {eá} is diagonal. The alternate statement
in the theorem follows from Theorem 9.11. ˙

In the next section, we shall show explicitly how this diagonalization can
be carried out.

Exercises

1. (a) Show that if f is a nondegenerate, antisymmetric bilinear form on V,

then n = dim V is even.
(b) Show that there exists a basis for V in which the matrix of f takes the
block matrix form
" !!0 !D %
$ '
#!D !0 &

where D is the (n/2) x (n/2) matrix

!0 ! 0 1$
# &
# 0 ! 1 0 &!!.
#" " "&
# &
"1 ! 0 0%

2. Show that a matrix A ∞ Mñ(F) represents an antisymmetric bilinear form

on V if and only if A is antisymmetric.
474 LINEAR FORMS

3. Reduce each of the following quadratic forms to diagonal form:

(a) q(x, y, z) = 2x2 - 8xy + y2 - 16xz + 14yz + 5z2.
(b) q(x, y, z) = x2 - xz + y2.
(c) q(x, y, z) = xy + y2 + 4xz + z2.
(d) q(x, y, z) = xy + yz.

4. (a) Find all antisymmetric bilinear forms on ®3.

(b) Find a basis for the space of all antisymmetric bilinear forms on ®n.

5. Let V be finite-dimensional over ç. Prove:

(a) The equation

(Ef)(u, v) = (1/2)[f(u, v) - f(v, u)]

for every f ∞ B(V) defines a linear operator E on B(V).

(b) E is a projection, i.e., E2 = E.
(c) If T ∞ L(V), the equation

(T¿f)(u, v) = f(Tu, Tv)

defines a linear operator T¿ on B(V).

(d) E T¿ = T¿ E for all T ∞ B(V).

6. Let V be finite-dimensional over ç, and suppose f, g ∞ B(V) are antisym-

metric. Show there exists an invertible T ∞ L(V) such that f(Tu, Tv) =
g(u, v) for all u, v ∞ V if and only if f and g have the same rank.

7. Prove Corollary 2 of Theorem 9.12.

9.6 DIAGONALIZATION OF SYMMETRIC BILINEAR FORMS

Now that we know any symmetric bilinear form f can be diagonalized, let us
look at how this can actually be carried out. After this discussion, we will give
an example that should clarify everything. (The algorithm that we are about to
describe may be taken as an independent proof of Theorem 9.13.) Let the
(symmetric) matrix representation of f be A = (aáé) ∞ Mn(F), and first assume
that aèè ≠ 0. For each i = 2, . . . , n we multiply the ith row of A by aèè, and then
add -aáè times the first row to this new ith row. In other words, this
combination of two elementary row operations results in Aá ‘ aèèAá - aáèAè.
Following this procedure for each i = 2, . . . , n yields the first column of A in
 9.6 DIAGONALIZATION OF SYMMETRIC BILINEAR FORMS 475

the form A1 = (aèè, 0, . . . , 0) (remember that this is a column vector, not a row
vector). We now want to put the first row of A into the same form. However,
this is easy because A is symmetric. We thus perform exactly the same opera-
tions (in the same sequence), but on columns instead of rows, resulting in
Ai ‘ aèèAi - aáèA1. Therefore the first row is also transformed into the form
Aè = (aèè, 0, . . . , 0). In other words, this sequence of operations results in the
transformed A having the block matrix form

! a11 0$
# &
"0 B%

where B is a matrix of size less than that of A. We can also write this in the
form (aèè) • B.
Now look carefully at what we did for the case of i = 2. Let us denote the
multiplication operation by the elementary matrix Em, and the addition opera-
tion by Ea (see Section 3.8). Then what was done in performing the row oper-
ations was simply to carry out the multiplication (EaEm)A. Next, because A is
symmetric, we carried out exactly the same operations but applied to the
columns instead of the rows. As we saw at the end of Section 3.8, this is
equivalent to the multiplication A(EmTEaT). In other words, for i = 2 we
effectively carried out the multiplication

EaEmAEmTEaT .

For each succeeding value of i we then carried out this same procedure, and
the final net effect on A was simply a multiplication of the form

Es ~ ~ ~ EèAEèT ~ ~ ~ EsT

which resulted in the block matrix (aèè) • B shown above. Furthermore, note
that if we let S = EèT ~ ~ ~ EsT = (Es ~ ~ ~ Eè)T, then (aèè) • B = STAS must be
symmetric since (STAS)T = STATS = STAS. This means that in fact the
matrix B must also be symmetric.
We can now repeat this procedure on the matrix B and, by induction, we
eventually arrive at a diagonal representation of A given by

D = Er ~ ~ ~ EèAEèT ~ ~ ~ ErT

for some set of elementary row transformations Eá. But from Theorems 9.11
and 9.13, we know that D = PTAP, and therefore PT is given by the product
476 LINEAR FORMS

er ~ ~ ~ eè(I) = Er ~ ~ ~ Eè of elementary row operations applied to the identity

matrix exactly as they were applied to A. It should be emphasized that we
were able to arrive at this conclusion only because A is symmetric, thereby
allowing each column operation to be the transpose of the corresponding row
operation. Note however, that while the order of the row and column opera-
tions performed is important within their own group, the associativity of the
matrix product allows the column operations (as a group) to be performed
independently of the row operations.
We still must take into account the case where aèè = 0. If aèè = 0 but aáá ≠ 0
for some i > 1, then we can bring aáá into the first diagonal position by inter-
changing the ith row and column with the first row and column respectively.
We then follow the procedure given above. If aáá = 0 for every i = 1, . . . , n,
then we can pick any aáé ≠ 0 and apply the operations Aá ‘ Aá + Aé and Ai ‘
Ai + Aj. This puts 2aáé ≠ 0 into the ith diagonal position, and allows us to pro-
ceed as in the previous case (which then goes into the first case treated). (Note
also that this last procedure requires that our field is not of characteristic 2
because we assumed that aáé + aáé = 2aáé ≠ 0.)

Example 9.9 Let us find the transition matrix P such that D = PTAP is diag-
onal, with A given by
" !!1 !3 !!2 %
$ '
!$ !3 !!7 !5 '!!.
$ '
# !!2 !5 !!8 &

We begin by forming the matrix (A\I):

" !!1 !3 !!!2 !1 0 0 %

$ '
$ !3 !!7 !!5 !!0 1 0 '!!.
$ !!2 !5 !!!8 !!0 0 1 '&
#

Now carry out the following sequence of elementary row operations to both A
and I, and identical column operations to A only:

# 1 "3 !2 1 0 0&
% (
A2 + 3A3 ! % 0 "2 !1 !!3 1 0 (
A3 " 2A1 ! %$ 0 !1 !4 "2 0 1 ('
 9.6 DIAGONALIZATION OF SYMMETRIC BILINEAR FORMS 477

" 1 !!0 !0 1 0 0%
$ '
$ 0 !2 !1 !!3 1 0 '!!!!
$ 0 !!1 !4 !2 0 1 '&
#
º º
A2 + 3A1 A3 - 2A1

# 1 !!0 !0 ! 1 0 0&
% (
% 0 "2 !1 !! 3 1 0 (
% "1 1 2 ('
2A 3 + A 2 ! $ 0 !!0 !9

" 1 !!0 !!0 !!1 0 0 %

$ '
$ 0 !2 !!0 !!3 1 0 '!!.
$ 0 !!0 18 !1 1 2 '
# &
º
2A3 + A2

We have thus diagonalized A, and the final form of the matrix (A\I) is just
(D\PT). ∆

Since Theorem 9.13 tells us that every symmetric bilinear form has a diag-
onal representation, it follows that the associated quadratic form q(X) has the
diagonal representation

q(X) = XTAX = aèè(x1)2 + ~ ~ ~ + aññ(xn)2

where A is the diagonal matrix representing the (symmetric) bilinear form.

Let us now specialize this discussion somewhat and consider only real
symmetric bilinear forms. We begin by noting that in general, the diagonal
representation of a symmetric bilinear form f has positive, negative, and zero
entries. We can always renumber the basis vectors so that the positive entries
appear first, followed by the negative entries and then the zero entries. It is in
fact true, as we now show, that any other diagonal representation of f has the
same number of positive and negative entries. If there are P positive entries
and N negative entries, then the difference S = P - N is called the signature
of f.

Theorem 9.14 Let f ∞ B(V) be a real symmetric bilinear form. Then every
diagonal representation of f has the same number of positive and negative
entries.
478 LINEAR FORMS

Proof Let {eè, . . . , eñ} be the basis for V in which the matrix of f is diagonal
(see Theorem 9.13). By suitably numbering the eá, we may assume that the
first P entries are positive and the next N entries are negative (also note that
there could be n - P - N zero entries). Now let {eæè, . . . , eæñ} be another basis
for V in which the matrix of f is also diagonal. Again, assume that the first Pæ
entries are positive and the next Næ entries are negative. Since the rank of f is
just the rank of any matrix representation of f, and since the rank of a matrix is
just the dimension of its row (or column) space, it is clear that r(f) = P + N =
Pæ + Næ. Because of this, we need only show that P = Pæ.
Let U be the linear span of the P vectors {eè, . . . , eP}, let W be the linear
span of {eæPæ+1, . . . , eæñ}, and note that dim U = P and dim W = n - Pæ. Then
for all nonzero vectors u ∞ U and w ∞ W, we have f(u, u) > 0 and f(w, w) ¯ 0
(this inequality is ¯ and not < because if Pæ + Næ ≠ n, then the last of the basis
vectors that span W will define a diagonal element in the matrix of f that is 0).
Hence it follows that U ⁄ W = {0}, and therefore (by Theorem 2.11)

dim(U + W ) = dimU + dimW ! dim(U " W ) = P + n ! P# ! 0

= P ! P# + n!!.

Since U and W are subspaces of V, it follows that dim(U + W) ¯ dim V = n,

and therefore P - Pæ + n ¯ n. This shows that P ¯ Pæ. Had we let U be the span
of {eæè, . . . , eæPæ} and W be the span of {eP+1 , . . . , eñ}, we would have found
that Pæ ¯ P. Therefore P = Pæ as claimed. ˙

While Theorem 9.13 showed that any quadratic form has a diagonal repre-
sentation, the important special case of a real quadratic form allows an even
simpler representation. This corollary is known as Sylvester’s theorem or the
law of inertia.

Corollary Let f be a real symmetric bilinear form. Then f has a unique diag-
onal representation of the form

" Ir %
$ '
!$ !I s '
$ '
# 0t &

where Ir and Is are the r x r and s x s unit matrices, and 0t is the t x t zero
matrix. In particular, the associated quadratic form q has a representation of
the form

q(xè, . . . , xñ) = (x1)2 + ~ ~ ~ + (xr)2 - (xr+1)2 - ~ ~ ~ - (xr+s)2 .

 9.6 DIAGONALIZATION OF SYMMETRIC BILINEAR FORMS 479

Proof Let f be represented by a (real) symmetric n x n matrix A. By Theorem

9.14, there exists a nonsingular matrix Pè such that D = PèTAPè = (dáé) is a
diagonal representation of f with a unique number r of positive entries fol-
lowed by a unique number s of negative entries. We let t = n - r - s be the
unique number of zero entries in D. Now let Pì be the diagonal matrix with
diagonal entries
"1 d
ii !!!!for i = 1,!…,!r
$$
(P2 )ii = #1 !dii for i = r +1,!…,!r + s !!.
$ 1 for i = r + s +1,!…,!n
$%

Since Pì is diagonal, it is obvious that (Pì)T = Pì. We leave it to the reader to

multiply out the matrices and show that

PìT DPì = PìT PèTAPèPì = (PèPì)TA(PèPì)

is a congruence transformation that takes A into the desired form. ˙

We say that a real symmetric bilinear form f ∞ B(V) is nonnegative (or

positive semidefinite) if q(X) = XTAX = Íi, j aáéxixj = f(X, X) ˘ 0 for all X ∞
V, and we say that f is positive definite if q(X) > 0 for all nonzero X ∞ V. In
particular, from Theorem 9.14 we see that f is nonnegative semidefinite if and
only if the signature S = r(f ) ¯ dim V, and f will be positive definite if and
only if S = dim V.

Example 9.10 The quadratic form (x1)2 - 4x1x2 + 5(x2)2 is positive definite
because it can be written in the form

(x1 - 2x2)2 + (x2)2

which is nonnegative for all real values of x1 and x2, and is zero only if x1 =
x2 = 0.
The quadratic form (x1)2 + (x2)2 + 2(x3)2 - 2x1x3 - 2x2x3 can be written in
the form
(x1 - x3)2 + (x2 - x3)2 .

Since this is nonnegative for all real values of x1, x2 and x3 but is zero for
nonzero values (e.g., x1 = x2 = x3 ≠ 0), this quadratic form is nonnegative but
not positive definite. ∆
480 LINEAR FORMS

Exercises

1. Determine the rank and signature of the following real quadratic forms:
(a) x2 + 2xy + y2.
(b) x2 + xy + 2xz + 2y2 + 4yz + 2z2.

2. Find the transition matrix P such that PTAP is diagonal where A is given
by:

" !!1 !!2 !3 % " 0 !!1 !!1 %

$ ' $ '
(a)!!!$ !!2 !!5 !4 ' (b)!!!$ 1 !2 !!2 '
$ ' $ '
# !3 !4 !!8 & # 1 !!2 !1 &

" !!1 !!1 !2 !3 %

$ '
$ !!1 !!2 !5 !1 '
(c)!!
$!2 !5 !6 !9 '
$ '
# !3 !1 !9 11 &

3. Let f be the symmetric bilinear form associated with the real quadratic
form q(x, y) = ax2 + bxy + cy2. Show that:
(a) f is nondegenerate if and only if b2 - 4ac ≠ 0.
(b) f is positive definite if and only if a > 0 and b2 - 4ac < 0.

4. If A is a real, symmetric, positive definite matrix, show there exists a non-

singular matrix P such that A = PT P.

The remaining exercises are all related.

5. Let V be finite-dimensional over ç, let S be the subspace of all symmetric

bilinear forms on V, and let Q be the set of all quadratic forms on V.
(a) Show that Q is a subspace of all functions from V to ç.
(b) Suppose T ∞ L(V) and q ∞ Q. Show that the equation (T¿q)(v) =
q(Tv) defines a quadratic form T¿q on V.
(c) Show that the function T¿ is a linear operator on Q, and show that T¿
is invertible if and only if T is invertible.

6. (a) Let q be the quadratic form on ®2 defined by q(x, y) = ax2 + 2bxy +

cy2 (where a ≠ 0). Find an invertible T ∞ L(®2) such that

(T¿q)(x, y) = ax2 + (c - b2/a)y2 .

 9.6 DIAGONALIZATION OF SYMMETRIC BILINEAR FORMS 481

[Hint: Complete the square to find Tî (and hence T).]

(b) Let q be the quadratic form on ®2 defined by q(x, y) = 2bxy. Find an
invertible T ∞ L(®2) such that

(T¿q)(x, y) = 2bx2 - 2by2 .

(c) Let q be the quadratic form on ®3 defined by q(x, y, z) = xy + 2xz +

z2. Find an invertible T ∞ L(®3) such that

(T¿q)(x, y, z) = x2 - y2 + z2 .

7. Suppose A ∞ Mn(®) is symmetric, and define a quadratic form q on ®n by

n
q(X) = ! aij xi x j !!.
i, j=1

Show there exists T ∞ L(®n) such that

n
(T †q)(X) = ! ci (xi )2
i=1

where each cá is either 0 or ±1.

9.7 HERMITIAN FORMS

Let us now briefly consider how some of the results of the previous sections
carry over to the case of bilinear forms over the complex number field. Much
of this material will be elaborated on in the next chapter.
We say that a mapping f: V ª V ‘ ç is a Hermitian form on V if for all
uè, uì, v ∞ V and a, b ∞ ç we have

(1) f(auè + buì, v) = a*f(uè, v) + b*f(uì, v).

(2) f(uè, v) = f(v, uè)*.

(We should point out that many authors define a Hermitian form by requiring
that the scalars a and b on the right hand side of property (1) not be the com-
plex conjugates as we have defined it. In this case, the scalars on the right
hand side of property (3) below will be the complex conjugates of what we
482 LINEAR FORMS

have shown.) As was the case for the Hermitian inner product (see Section
2.4), we see that

f (u,!av1 + bv2 ) = f (av1 + bv2 ,!u)* = [a*f (v1,!u) + b*f (v2 ,!u)]*
= af (v1,!u)* + bf (v2 ,!u)* = af (u,!v1 ) + bf (u,!v2 )

which we state as

(3) f(u, avè + bvì) = af(u, vè) + bf(u, vì).

Since f(u, u) = f(u, u)* it follows that f(u, u) ∞ ® for all u ∞ V.

Along with a Hermitian form f is the associated Hermitian quadratic
form q: V ‘ ® defined by q(u) = f(u, u) for all u ∞ V. A little algebra
(Exercise 9.7.1) shows that f may be obtained from q by the polar form
expression of f which is

f(u, v) = (1/4)[q(u + v) - q(u - v)] - (i/4)[q(u + iv) - q(u - iv)] .

We also say that f is nonnegative semidefinite if q(u) = f(u, u) ˘ 0 for all u ∞

V, and positive definite if q(u) = f(u, u) > 0 for all nonzero u ∞ V. For
example, the usual Hermitian inner product on çn is a positive definite form
since for every nonzero X = (x1, . . . , xn) ∞ çn we have
n n
q(X) = f (X,! X) = X,! X = ! (x i )*x i = ! |x i |2 > 0!!.
i=1 i=1
As we defined in Section 8.1, we say that a matrix H = (háé) ∞ Mn(ç) is
Hermitian if háé = héá*. In other words, H is Hermitian if H = H*T. We denote
the operation of taking the transpose along with taking the complex conjugate
of a matrix A by A¿ (read “A dagger”). In other words, A¿ = A*T. For reasons
that will become clear in the next chapter, we frequently call A¿ the
(Hermitian) adjoint of A. Thus H is Hermitian if H¿ = H.
Note also that for any scalar k we have k¿ = k*. Furthermore, using
Theorem 3.18(d), we see that

(AB)¿ = (AB)*T = (A*B*)T = B¿A¿ .

By induction, this obviously extends to any finite product of matrices. It is

also clear that
A¿¿ = A .
 9.7 HERMITIAN FORMS 483

Example 9.11 Let H be a Hermitian matrix. We show that f(X, Y) = X¿HY

defines a Hermitian form on çn. Let Xè, Xì, Y ∞ çn be arbitrary, and let a,
b ∞ ç. Then (using Theorem 3.18(a))

f (aX1 + bX2 ,!Y ) = (aX1 + bX2 )† HY

= (a*X1† + b*X2 † )HY
= a*X1† HY + b*X2 † HY
= a*f (X1,!Y ) + b*f (X2 ,!Y )

which shows that f(X, Y) satisfies property (1) of a Hermitian form. Now,
since X¿HY is a (complex) scalar we have (X¿HY)T = X¿HY, and therefore

f(X, Y)* = (X¿HY)* = (X¿HY)¿ = Y¿HX = f(Y, X)

where we used the fact that H¿ = H. Thus f(X, Y) satisfies property (2), and
hence defines a Hermitian form on çn.
It is probably worth pointing out that X¿HY will not be a Hermitian form
if the alternative definition mentioned above is used. In this case, one must
use f(X, Y) = XTHY* (see Exercise 9.7.2). ∆

Now let V have basis {eá}, and let f be a Hermitian form on V. Then for
any X = Í xieá and Y = Í yieá in V, we see that

f(X, Y) = f(Íáxieá, Íéyjeé) = Íi, j xi*yj f(eá, eé) .

Just as we did in Theorem 9.9, we define the matrix elements háé representing
a Hermitian form f by háé = f(eá, eé). Note that since f(eá, eé) = f(eé, eá)*, we see
that the diagonal elements of H = (háé) must be real. Using this definition for
the matrix elements of f, we then have

f(X, Y) = Íi, jxi*háéyj = X¿HY .

Following the proof of Theorem 9.9, this shows that any Hermitian form f has
a unique representation in terms of the Hermitian matrix H.
If we want to make explicit the basis referred to in this expression, we
write f(X, Y) = [X]e¿H[Y]e where it is understood that the elements háé are
defined with respect to the basis {eá}. Finally, let us prove the complex ana-
logues of Theorems 9.11 and 9.14.
484 LINEAR FORMS

Theorem 9.15 Let f be a Hermitian form on V, and let P be the transition

matrix from a basis {eá} for V to a new basis {eæá}. If H is the matrix of f with
respect to the basis {eá} for V, then Hæ = P¿HP is the matrix of f relative to the
new basis {eæá}.

Proof We saw in the proof of Theorem 9.11 that for any X ∞ V we have
[X]e = P[X]e æ, and hence [X]e¿ = [X]eæ¿P¿. Therefore, for any X, Y ∞ V we
see that
f(X, Y) = [X]e¿H[Y]e = [X]eæ¿P¿HP[Y]e æ = [X]eæ¿Hæ[Y]eæ

where Hæ = P¿HP is the (unique) matrix of f relative to the basis {eæá}. ˙

Theorem 9.16 Let f be a Hermitian form on V. Then there exists a basis for
V in which the matrix of f is diagonal, and every other diagonal representation
of f has the same number of positive and negative entries.

Proof Using the fact that f(u, u) is real for all u ∞ V along with the appropri-
ate polar form of f, it should be easy for the reader to follow the proofs of
Theorems 9.13 and 9.14 and complete the proof of this theorem (see Exercise
9.7.3). ˙

We note that because of this result, our earlier definition for the signature
of a bilinear form applies equally well to Hermitian forms.

Exercises

1. Let f be a Hermitian form on V and q the associated quadratic form.

Verify the polar form

f(u, v) = (1/4)[q(u + v) - q(u - v)] - (i/4)[q(u + iv) - q(u - iv)] .

2. Verify the statement made at the end of Example 9.11.

3. Prove Theorem 9.16.

4. Show that the algorithm described in Section 9.6 applies to Hermitian

matrices if we allow multiplication by complex numbers and, instead of
multiplying by ET on the right, we multiply by E*T.

5. For each of the following Hermitian matrices H, use the results of the pre-
vious exercise to find a nonsingular matrix P such that PTHP is diagonal:
 9.7 HERMITIAN FORMS 485

" !!1 i % " 1 2 + 3i %

(a)!!$ ' (b)!!$ '
# !i 2 & # 2 ! 3i !1 &

" 1 i 2 + i% " 1 1+ i 2i %
$ ' $ '
(c)!!!$ !i 2 1! i ' (d)!!!$1 ! i 4 2 ! 3i '
$ ' $ '
# 2 ! i 1+ i 2 & # !2i 2 + 3i 7 &

9.8 SIMULTANEOUS DIAGONALIZATION *

We now apply the results of Sections 8.1, 9.5 and 9.6 to the problem of simul-
taneously diagonalizing two real quadratic forms. After the proof we shall
give an example of how this result applies to classical mechanics.

Theorem 9.17 Let XTAX and XTBX be two real quadratic forms on an n-
dimensional Euclidean space V, and assume that XTAX is positive definite.
Then there exists a nonsingular matrix P such that the transformation X = PY
reduces XTAX to the form

XTAX = YTY = (y1)2 + ~ ~ ~ + (yn)2

and XTBX to the form

XTBX = YTDY = (y1)2 + ~ ~ ~ + –(yn)2

where , . . . , – are roots of the equation

det(B - ¬A) = 0 .

Moreover, the ‡ are real and positive if and only if XTBX is positive definite.

Proof Since A is symmetric, Theorem 9.13 tells us there exists a basis for V
that diagonalizes A. Furthermore, the corollary to Theorem 9.14 and the dis-
cussion following it shows that the fact A is positive definite means that the
corresponding nonsingular transition matrix R may be chosen so that the
transformation X = RY yields

XTAX = YTY = (y1)2 + ~ ~ ~ + (yn)2 .

Note that YTY = XTAX = YTRTARY implies that

486 LINEAR FORMS

RTAR = I .

We also emphasize that R will not be orthogonal in general.

Now observe that RTBR is a real symmetric matrix since B is, and hence
(by the corollary to Theorem 8.2) there exists an orthogonal matrix Q such
that
QT RT BRQ = (RQ)T B(RQ) = diag(, . . . , –) = D

where the ‡ are the eigenvalues of RTBR. If we define the nonsingular (and
not generally orthogonal) matrix P = RQ, then

PTBP = D
and
PTAP = QTRTARQ = QTIQ = I .

Under the transformation X = PY, we are then left with

XTAX = YTPTAPY = YTY

as before, while

XT BX = YTPT BPY = YTDY = (y1)2 + ~ ~ ~ + –(yn)2

as desired.
Now note that by definition, the ‡ are roots of the equation

det(RTBR - ¬I) = 0 .

Using RTAR = I this may be written as

det[RT(B - ¬A)R] = 0 .

Since det R = det RT ≠ 0, we find that (using Theorem 4.8)

det(B - ¬A) = 0 .

Finally, since B is a real symmetric matrix, there exists an orthogonal

matrix S that brings it into the form

STBS = diag(µè, . . . , µ ñ) = Dÿ

where the µá are the eigenvalues of B. Writing X = SY, we see that

 9.8 SIMULTANEOUS DIAGONALIZATION 487

XTBX = YTSTBSY = YT DÿY = µè(y1)2 + ~ ~ ~ + µ ñ(yn)2

and thus XTBX is positive definite if and only if YTDÿY is positive definite,
i.e., if and only if every µá > 0. Since we saw above that

PTBP = diag(, . . . , –) = D

it follows from Theorem 9.14 that the number of positive µá must equal the
number of positive ‡. Therefore XTBX is positive definite if and only if every
¬á > 0. ˙

Example 9.12 Let us show how Theorem 9.17 can be of help in classical
mechanics. This example requires a knowledge of both the Lagrange equa-
tions of motion and Taylor series expansions. The details of the physics are
given in, e.g., the classic text by Goldstein (1980). Our purpose is simply to
demonstrate the usefulness of this theorem.
Consider the small oscillations of a conservative system of N particles
about a point of stable equilibrium. We assume that the position rá of the ith
particle is a function of n generalized coordinates qá, and not explicitly on the
time t. Thus we write rá = rá(qè, . . . , qñ), and
n
dri !r
= r!i = " i q! j
dt j=1
!q j
where we denote the derivative with respect to time by a dot.
Since the velocity vá of the ith particle is given by rãá , the kinetic energy T
of the ith particle is (1/2)má(vá)2 = (1/2)márãáÂrãá, and hence the kinetic energy
of the system of N particles is given by
N n
1
T = ! mi r!i i!ri = ! M jk q! j q! k
i=1
2 j, k=1
where
N
1 !r !r
M jk = " mi i i i = M kj !!.
i=1
2 !q j !qk
Thus the kinetic energy is a quadratic form in the generalized velocities qãá. We
also assume that the equilibrium position of each qá is at qá = 0. Let the poten-
tial energy of the system be V = V(qè, . . . , qñ). Expanding V in a Taylor
series expansion about the equilibrium point, we have (using an obvious nota-
tion for evaluating functions at equilibrium)

n "
!2V %
n "
!V % 1
V (q1,!…,!qn ) = V (0) + ( $ ' qi + ( $$ '' qi q j +!!!!.
i=1 # !q &
i 0 2 !q
i, j=1 # i
!q j&
0
488 LINEAR FORMS

At equilibrium, the force on any particle vanishes, and hence we must have
($V/$qá)à = 0 for every i. Furthermore, we may shift the zero of potential and
assume that V(0) = 0 because this has no effect on the force on each particle.
We may therefore write the potential as the quadratic form
n
V= ! bij qi q j
i, j=1
where the báé are constants, and báé = béá. Returning to the kinetic energy, we
expand Máé about the equilibrium position to obtain

n "
!M %
M ij (q1,!…,!qn ) = M ij (0) + ( $ ij ' qk +!!!!.
k=1 #
!qk &0

To a first approximation, we may keep only the first (constant) term in this
expansion. Then denoting Máé(0) by aáé = aéá we have
n
T= ! aij q!i q! j !!.
i, j=1
so that T is also a quadratic form.
The Lagrange equations of motion are

d " !L % !L
$ '=
dt # !q!i & !qi

where L = T - V is called the Lagrangian. Since T is a function of the qãá and

V is a function of the qá , the equations of motion take the form

d " !T % !V
$ '=( !!. (*)
dt # !q!i & !qi

Now, the physical nature of the kinetic energy tells us that T must be a posi-
tive definite quadratic form, and hence we seek to diagonalize T as follows.
Define new coordinates qæè, . . . , qæñ by qá = Íépáéqæé where P = (páé) is a
nonsingular constant matrix. Then differentiating with respect to time yields
qãá = Íépáéqãæé so that the qãá are transformed in the same manner as the qá. By
Theorem 9.17, the transformation P may be chosen so that T and V take the
forms
T = (qãæè)2 + ~ ~ ~ + (qãæñ)2
and
V = ¬è(qæè)2 + ~ ~ ~ + ¬ñ(qæñ)2 .
 9.8 SIMULTANEOUS DIAGONALIZATION 489

Since V = 0 at qè = ~ ~ ~ = qñ = 0, the fact that P is nonsingular tells us that V =

0 at qæè = ~ ~ ~ = qæñ = 0 as well. Thus we see that V is also positive definite,
and hence each ¬á > 0. This means that we may write ¬á = øá2 where each øá is
real and positive.
Since P is a constant matrix, the equations of motion (*) are just as valid
for T and V expressed in terms of qæá and qãæá. Therefore, substituting these
expressions for T and V into (*), we obtain the equations of motion

d 2 qi!
2
= "#i 2 qi! !!.
dt

For each i = 1, . . . , n the solution to this equation is

qæá = åá cos(øát + ∫á)

where åá and ∫á are constants to be determined from the initial conditions of

the problem.
The coordinates qæá are called the normal coordinates for the system of
particles, and the form of the solution shows that the particles move according
to simple harmonic motion. ∆

For additional applications related to this example, we refer the reader to

any advanced text on classical mechanics, such as those listed in the bibliog-
raphy. (See, eg., Marion, chapter 13.6.)
 CHAPTER 10

Linear Operators

Recall that a linear transformation T ∞ L(V) of a vector space into itself is

called a (linear) operator. In this chapter we shall elaborate somewhat on the
theory of operators. In so doing, we will define several important types of
operators, and we will also prove some important diagonalization theorems.
Much of this material is directly useful in physics and engineering as well as
in mathematics. While some of this chapter overlaps with Chapter 8, we
assume that the reader has studied at least Section 8.1.

10.1 LINEAR FUNCTIONALS AND ADJOINTS

Recall that in Theorem 9.3 we showed that for a finite-dimensional real inner
product space V, the mapping u ’ Lu = Óu, Ô was an isomorphism of V onto
V*. This mapping had the property that Lauv = Óau, vÔ = aÓu, vÔ = aLuv, and
hence Lau = aLu for all u ∞ V and a ∞ ®. However, if V is a complex space
with a Hermitian inner product, then Lauv = Óau, vÔ = a*Óu, vÔ = a*Luv, and
hence Lau = a*Lu which is not even linear (this was the definition of an anti-
linear (or conjugate linear) transformation given in Section 9.2). Fortunately,
there is a closely related result that holds even for complex vector spaces.
Let V be finite-dimensional over ç, and assume that V has an inner prod-
uct Ó , Ô defined on it (this is just a positive definite Hermitian form on V).
Thus for any X, Y ∞ V we have ÓX, YÔ ∞ ç. For example, with respect to the

490
 10.1 LINEAR FUNCTIONALS AND ADJOINTS 491

standard basis {eá} for çn (which is the same as the standard basis for ®n), we
have X = Íxieá and hence (see Example 2.13)

X,!Y = !i x i ei ,! ! j y j e j = !i, j x i *y j ei ,!e j = !i, j x i *y j"ij

= !i x i *yi = X*T Y !!.

Note that we are temporarily writing X*T rather than X¿. We will shortly
explain the reason for this (see Theorem 10.2 below). In particular, for any
T ∞ L(V) and X ∞ V we have the vector TX ∞ V, and hence it is meaningful
to write expressions of the form ÓTX, YÔ and ÓX, TYÔ.
Since we are dealing with finite-dimensional vector spaces, the Gram-
Schmidt process (Theorem 2.21) guarantees that we can always work with an
orthonormal basis. Hence, let us consider a complex inner product space V
with basis {eá} such that Óeá, eéÔ = ∂áé. Then, just as we saw in the proof of
Theorem 9.1, we now see that for any u = Íuj ej ∞ V we have

Óeá, uÔ = Óeá, ÍéujeéÔ = ÍéujÓeá, eéÔ = Íéuj∂áé = ui

and thus
u = ÍáÓeá, uÔeá .

Now consider the vector Teé. Applying the result of the previous para-
graph we have
Teé = ÍáÓeá, TeéÔeá .

But this is precisely the definition of the matrix A = (aáé) that represents T rel-
ative to the basis {eá}. In other words, this extremely important result shows
that the matrix elements aáé of the operator T ∞ L(V) are given by

aáé = Óeá, TeéÔ .

It is important to note however, that this definition depended on the use of an

orthonormal basis for V. To see the self-consistency of this definition, we go
back to our original definition of (aáé) as Teé = ÍÉekakj . Taking the scalar
product of both sides of this equation with eá yields (using the orthonormality
of the eá)

Óeá, Teé Ô = Óeá, ÍÉeÉaÉé Ô = ÍÉaÉéÓeá, eÉÔ = ÍÉaÉé∂áÉ = aáé .

We now prove the complex analogue of Theorem 9.3.

492 LINEAR OPERATORS

Theorem 10.1 Let V be a finite-dimensional inner product space over ç.

Then, given any linear functional L on V, there exists a unique u ∞ V such
that Lv = Óu, vÔ for all v ∞ V.

Proof Let {eá} be an orthonormal basis for V and define u = Íá(Leá)*eá . Now
define the linear functional Lu on V by Luv = Óu, vÔ for every v ∞ V. Then, in
particular, we have

Lu eá = Óu, eáÔ = ÓÍé(Leé)*eé, eáÔ = ÍéLeéÓeé, eáÔ = ÍéLeé∂éá = Leá .

Since L and Lu agree on a basis for V, they must agree on any v ∞ V, and
hence L = Lu = Óu, Ô.
As to the uniqueness of the vector u, suppose uæ ∞ V has the property that
Lv = Óuæ, vÔ for every v ∞ V. Then Lv = Óu, vÔ = Óuæ, vÔ so that Óu - uæ, vÔ = 0.
Since v was arbitrary we may choose v = u - uæ. Then Óu - uæ, u - uæÔ = 0
which implies that (since the inner product is just a positive definite Hermitian
form) u - uæ = 0 or u = uæ. ˙

The importance of finite-dimensionality in this theorem is shown by the

following example.

Example 10.1 Let V = ®[x] be the (infinite-dimensional) space of all poly-

nomials over ®, and define an inner product on V by

1
Óf ,!gÔ = ! 0 f (x)g(x) dx
for every f, g ∞ V. We will give an example of a linear functional L on V for
which there does not exist a polynomial h ∞ V with the property that Lf =
Óh, f Ô for all f ∞ V.
To show this, define the nonzero linear functional L by

Lf = f(0) .

(L is nonzero since, e.g., L(a + x) = a.) Now suppose there exists a polynomial
h ∞ V such that Lf = f(0) = Óh, fÔ for every f ∞ V. Then, in particular, we have

L(xf) = 0f(0) = 0 = Óh, xfÔ

for every f ∞ V. Choosing f = xh we see that

1
0 = Óh,!x 2 hÔ = ! 0 x 2 h2 dx !!.
 10.1 LINEAR FUNCTIONALS AND ADJOINTS 493

Since the integrand is strictly positive, this forces h to be the zero polynomial.
Thus we are left with Lf = Óh, fÔ = Ó0, fÔ = 0 for every f ∞ V, and hence L = 0.
But this contradicts the fact that L ≠ 0, and hence no such polynomial h can
exist.
Note the fact that V is infinite-dimensional is required when we choose f =
xh. The reason for this is that if V consisted of all polynomials of degree ¯
some positive integer N, then f = xh could have degree > N. ∆

Now consider an operator T ∞ L(V), and let u be an arbitrary element of

V. Then the mapping Lu: V ‘ ç defined by Luv = Óu, TvÔ for every v ∞ V is
a linear functional on V. Applying Theorem 10.1, we see that there exists a
unique uæ ∞ V such that Óu, TvÔ = Luv = Óuæ, vÔ for every v ∞ V. We now
define the mapping T¿: V ‘ V by T¿u = uæ. In other words, we define the
adjoint T¿ of an operator T ∞ L(V) by

Ó T¿u, vÔ = Óu, TvÔ

for all u, v ∞ V. The mapping T¿ is unique because uæ is unique for a given u.

Thus, if Tÿ¿u = uæ = T¿u, then (Tÿ¿ - T¿)u = 0 for every u ∞ V, and hence Tÿ¿ -
T¿ = 0 or Tÿ¿ = T¿.
Note further that

ÓTu, vÔ = Óv, TuÔ* = ÓT¿v, uÔ* = Óu, T¿vÔ .

However, it follows from the definition that Óu, T¿vÔ = ÓT¿¿u, vÔ. Therefore
the uniqueness of the adjoint implies that T¿¿ = T.
Let us show that the map T¿ is linear. For all uè, uì, v ∞ V and a, b ∞ ç we
have
ÓT † (au1 + bu2 ),!vÔ = Óau1 + bu2 ,!TvÔ
= a*Óu1,!TvÔ + b*Óu2 ,!TvÔ
= a*ÓT †u1,!vÔ + b*ÓT †u2 ,!vÔ
= ÓaT †u1,!vÔ + ÓbT †u2 ,!vÔ
= ÓaT †u1 + bT †u2 ,!vÔ!! .

Since this is true for every v ∞ V, we must have

T¿(auè + buì) = aT¿uè + bT¿uì .

Thus T¿ is linear and T¿ ∞ L(V).

494 LINEAR OPERATORS

If {eá} is an orthonormal basis for V, then the matrix elements of T are

given by aáé = Óeá, TeéÔ. Similarly, the matrix elements báé of T¿ are related to
those of T because

báé = Óeá, T¿eéÔ = ÓTeá, eéÔ = Óeé, TeáÔ* = aéá* .

In other words, if A is the matrix representation of T relative to the orthonor-

mal basis {eá}, then A*T is the matrix representation of T¿. This explains the
symbol and terminology for the Hermitian adjoint used in the last chapter.
Note that if V is a real vector space, then the matrix representation of T¿ is
simply AT, and we may denote the corresponding operator by TT.
We summarize this discussion in the following theorem, which is valid
only in finite-dimensional vector spaces. (It is also worth pointing out that T¿
depends on the particular inner product defined on V.)

Theorem 10.2 Let T be a linear operator on a finite-dimensional complex

inner product space V. Then there exists a unique linear operator T¿ on V
defined by ÓT¿u, vÔ = Óu, TvÔ for all u, v ∞ V. Furthermore, if A is the matrix
representation of T relative to an orthonormal basis {eá}, then A¿ = A*T is the
matrix representation of T¿ relative to this same basis. If V is a real space,
then the matrix representation of T¿ is simply AT.

Example 10.2 Let us give an example that shows the importance of finite-
dimensionality in defining an adjoint operator. Consider the space V = ®[x] of
all polynomials over ®, and let the inner product be as in Example 10.1.
Define the differentiation operator D ∞ L(V) by Df = df/dx. We show that
there exists no adjoint operator D¿ that satisfies ÓDf, gÔ = Óf, D¿gÔ.
Using ÓDf, gÔ = Óf, D¿gÔ, we integrate by parts to obtain

1 1
Óf ,!D †gÔ = ÓDf ,!gÔ = ! 0 (Df )g dx = ! 0 [D( fg) " fDg] dx
= ( fg)(1) " ( fg)(0) " Óf ,!DgÔ! .

Rearranging, this general result may be written as

Óf, (D + D¿)gÔ = (fg)(1) - (fg)(0) .

We now let f = x2(1 - x)2p for any p ∞ V. Then f(1) = f(0) = 0 so that we are
left with
 10.1 LINEAR FUNCTIONALS AND ADJOINTS 495

1
0 = Óf ,!(D + D † )gÔ = " 0 x 2 (1 ! x)2 p(D + D† )g dx
= Óx 2 (1 ! x)2 (D + D † )g,! pÔ .

Since this is true for every p ∞ V, it follows that x2(1 - x)2(D + D¿)g = 0. But
x2(1 - x)2 > 0 except at the endpoints, and hence we must have (D + D¿)g = 0
for all g ∞ V, and thus D + D¿ = 0. However, the above general result then
yields
0 = Óf, (D + D¿)gÔ = (fg)(1) - (fg)(0)

which is certainly not true for every f, g ∞ V. Hence D¿ must not exist.
We leave it to the reader to find where the infinite-dimensionality of V =
®[x] enters into this example. ∆

While this example shows that not every operator on an infinite-dimen-

sional space has an adjoint, there are in fact some operators on some infinite-
dimensional spaces that do indeed have an adjoint. A particular example of
this is given in Exercise 10.1.3. In fact, the famous Riesz representation theo-
rem asserts that any continuous linear functional on a Hilbert space does
indeed have an adjoint. While this fact should be well known to anyone who
has studied quantum mechanics, we defer further discussion until Chapter 12
(see Theorem 12.26).
As defined previously, an operator T ∞ L(V) is Hermitian (or self-
adjoint) if T¿ = T. The elementary properties of the adjoint operator T¿ are
given in the following theorem. Note that if V is a real vector space, then the
properties of the matrix representing an adjoint operator simply reduce to
those of the transpose. Hence, a real Hermitian operator is represented by a
(real) symmetric matrix.

Theorem 10.3 Suppose S, T ∞ L(V) and c ∞ ç. Then

(a) (S + T)¿ = S¿ + T¿.
(b) (cT)¿ = c*T¿.
(c) (ST)¿ = T¿S¿.
(d) T¿¿ = (T¿)¿ = T.
(e) I¿ = I and 0¿ = 0.
(f) (T ¿)î = (Tî)¿.

Proof Let u, v ∞ V be arbitrary. Then, from the definitions, we have

(a)!!Ó(S + T )† u,!vÔ = Óu,!(S + T )vÔ = Óu,!Sv + TvÔ = Óu,!SvÔ + Óu,!TvÔ
= ÓS †u,!vÔ + ÓT †u,!vÔ = Ó(S † + T † )u,!vÔ.
(b)!!Ó(cT )† u,!vÔ = Óu,!cTvÔ = cÓu,!TvÔ = cÓT †u,!vÔ = Óc*T †u,!vÔ.
496 LINEAR OPERATORS

(c)!!Ó(ST )† u,!vÔ = Óu,!(ST )vÔ = Óu,!S(Tv)Ô = ÓS †u,!TvÔ

= ÓT † (S †u),!vÔ = Ó(T †S † )u,!vÔ.
(d) This was shown in the discussion preceding Theorem 10.2.
(e) ÓIu, vÔ = Óu, vÔ = Óu, IvÔ = ÓI¿u, vÔ.
Ó0u, vÔ = Ó0, vÔ = 0 = Óu, 0vÔ = Ó0¿u, vÔ.
(f ) I = I¿ = (T Tî)¿ = (Tî)¿T¿ so that (Tî)¿ = (T¿)î.

The proof is completed by noting that the adjoint and inverse operators are
unique. ˙

Corollary If T ∞ L(V) is nonsingular, then so is T¿.

Proof This follows from Theorems 10.3(f ) and 5.10. ˙

We now group together several other useful properties of operators for

easy reference.

Theorem 10.4 (a) Let V be an inner product space over either ® or ç, let
T ∞ L(V), and suppose that Óu, TvÔ = 0 for all u, v ∞ V. Then T = 0.
(b) Let V be an inner product space over ç, let T ∞ L(V), and suppose
that Óu, TuÔ = 0 for all u ∞ V. Then T = 0.
(c) Let V be a real inner product space, let T ∞ L(V) be Hermitian, and
suppose that Óu, TuÔ = 0 for all u ∞ V. Then T = 0.

Proof (a) Let u = Tv. Then, by definition of the inner product, we see that
ÓTv, TvÔ = 0 implies Tv = 0 for all v ∞ V which implies that T = 0.
(b) For any u, v ∞ V we have (by hypothesis)

0 = Óu + v,!T (u + v)Ô
= Óu,!TuÔ + Óu,!TvÔ + Óv,!TuÔ + Óv,!TvÔ
(*)
= 0 + Óu,!TvÔ + Óv,!TuÔ + 0
= Óu,!TvÔ + Óv,!TuÔ

Since v is arbitrary, we may replace it with iv to obtain

0 = iÓu, TvÔ - iÓv, TuÔ .

Dividing this by i and adding to (*) results in 0 = Óu, TvÔ for any u, v ∞ V. By
(a), this implies that T = 0.
(c) For any u, v ∞ V we have Óu + v, T(u + v)Ô = 0 which also yields (*).
Therefore, using (*), the fact that T¿ = T, and the fact that V is real, we obtain
 10.1 LINEAR FUNCTIONALS AND ADJOINTS 497

0 = ÓT †u,!vÔ + Óv,!TuÔ = ÓTu,!vÔ + Óv,!TuÔ = Óv,!TuÔ + Óv,!TuÔ

= 2Óv,!TuÔ.

Since this holds for any u, v ∞ V we have T = 0 by (a). (Note that in this par-
ticular case, T¿ = TT.) ˙

Exercises

1. Suppose S, T ∞ L(V).
(a) If S and T are Hermitian, show that ST and TS are Hermitian if and
only if [S, T] = ST - TS = 0.
(b) If T is Hermitian, show that S¿TS is Hermitian for all S.
(c) If S is nonsingular and S¿TS is Hermitian, show that T is Hermitian.

2. Consider V = Mñ(ç) with the inner product ÓA, BÔ = Tr(B¿A). For each
M ∞ V, define the operator TM ∞ L(V) by TM(A) = MA. Show that
(TM)¿ = TM¿ .

3. Consider the space V = ç[x]. If f = Íaáxi ∞ V, we define the complex

conjugate of f to be the polynomial f* = Íaá*xi ∞ V. In other words, if
t ∞ ®, then f*(t) = (f(t))*. We define an inner product on V by
1
Óf ,!gÔ = ! 0 f *(t)g(t) dt !!.
For each f ∞ V, define the operator Tf ∞ L(V) by Tf (g) = fg. Show that
(Tf)¿ = Tf*.

4. Let V be the space of all real polynomials of degree ¯ 3, and define an

inner product on V by
1
Óf ,!gÔ = ! 0 f (x)g(x) dx !!.
For any t ∞ ®, find a polynomial ht ∞ V such that Óht, f Ô = f(t) for all f ∞
V.

5. If V is as in the previous exercise and D is the usual differentiation oper-

ator on V, find D¿.

6. Let V = ç2 with the standard inner product.

(a) Define T ∞ L(V) by Teè = (1, -2), Teì = (i, -1). If v = (zè zì), find
T¿v.
498 LINEAR OPERATORS

(b) Define T ∞ L(V) by Teè = (1 + i, 2), Teì = (i, i). Find the matrix rep-
resentation of T¿ relative to the usual basis for V. Is it true that [T, T¿] =
0?

7. Let V be a finite-dimensional inner product space and suppose T ∞ L(V).

Show that Im T¿ = (Ker T)Ê.

8. Let V be a finite-dimensional inner product space, and suppose E ∞ L(V)

is idempotent, i.e., E2 = E. Prove that E¿ = E if and only if [E, E¿] = 0.

9. For each of the following inner product spaces V and L ∞ V*, find a
vector u ∞ V such that Lv = Óu, vÔ for all v ∞ V:
(a) V = ®3 and L(x, y, z) = x - 2y + 4z.
(b) V = ç2 and L(zè, zì) = zè - zì.
(c) V is the space of all real polynomials of degree ¯ 2 with inner product
as in Exercise 4, and Lf = f(0) + Df(1). (Here D is the usual differentia-
tion operator.)

10. (a) Let V = ®2, and define T ∞ L(V) by T(x, y) = (2x + y, x - 3y). Find
T¿(3, 5).
(b) Let V = ç2, and define T ∞ L(V) by T(zè, zì) = (2zè + izì, (1 - i)zè).
Find T¿(3 - i, 1 + i2).
(c) Let V be as in Exercise 9(c), and define T ∞ L(V) by Tf = 3f + Df.
Find T¿f where f = 3x2 - x + 4.

10.2 ISOMETRIC AND UNITARY OPERATORS

Let V be a complex inner product space with the induced norm. Another
important class of operators U ∞ L(V) is that for which ˜ Uv ˜ = ˜ v ˜ for all v ∞
V. Such operators are called isometric because they preserve the length of the
vector v. Furthermore, for any v, w ∞ V we see that

˜ Uv - Uw ˜ = ˜ U(v - w) ˜ = ˜ v - w ˜

so that U preserves distances as well. This is sometimes described by saying

that U is an isometry.
If we write out the norm as an inner product and assume that the adjoint
operator exists, we see that an isometric operator satisfies

Óv, vÔ = ÓUv, UvÔ = Óv, (U¿U)vÔ

 10.2 ISOMETRIC AND UNITARY OPERATORS 499

and hence Óv, (U¿U - 1)vÔ = 0 for any v ∞ V. But then from Theorem 10.4(b))
it follows that
U¿U = 1 .

In fact, this is sometimes taken as the definition of an isometric operator. Note

that this applies equally well to an infinite-dimensional space.
If V is finite-dimensional, then (Theorems 3.21 and 5.13) it follows that
U¿ = Uî, and hence
U¿U = UU¿ = 1 .

Any operator that satisfies either U¿U = UU¿ = 1 or U¿ = Uî is said to be

unitary. It is clear that a unitary operator is necessarily isometric. If V is
simply a real space, then unitary operators are called orthogonal.
Because of the importance of isometric and unitary operators in both
mathematics and physics, it is worth arriving at both of these definitions from
a slightly different viewpoint that also aids in our understanding of these
operators. Let V be a complex vector space with an inner product defined on
it. We say that an operator U is unitary if ˜ Uv ˜ = ˜ v ˜ for all v ∞ V, and in
addition, it has the property that it is a mapping of V onto itself. Since ˜ Uv ˜ =
˜ v ˜ , we see that Uv = 0 if and only if v = 0, and hence Ker U = {0}. Therefore
U is one-to-one and Uî exists (Theorem 5.5). Since U is surjective, the
inverse is defined on all of V also. Note that there has been no mention of
finite-dimensionality. This was avoided by requiring that the mapping be sur-
jective.
Starting from ˜ Uv ˜ = ˜ v ˜ , we may write Óv, (U¿U)vÔ = Óv, vÔ. As we did in
the proof of Theorem 10.4, if we first substitute v = vè + vì and then v = vè +
ivì, divide the second of these equations by i and then add to the first, we find
that Óvè, (U¿U)vìÔ = Óvè, vìÔ. Since this holds for all vè, vì ∞ V, it follows that
U¿U = 1. If we now multiply this equation from the left by U we have UU¿U
= U, and hence (UU¿)(Uv) = Uv for all v ∞ V. But as v varies over all of V,
so does Uv since U is surjective. We then define væ = Uv so that (UU¿)væ = væ
for all væ ∞ V. This shows that U¿U = 1 implies UU¿ = 1. What we have just
done then, is show that a surjective norm-preserving operator U has the
property that U¿U = UU¿ = 1. It is important to emphasize that this approach
is equally valid in infinite-dimensional spaces.
We now define an isometric operator Ø to be an operator defined on all of
V with the property that ˜ Øv ˜ = ˜ v ˜ for all v ∞ V. This differs from a unitary
operator in that we do not require that Ø also be surjective. Again, the
requirement that Ø preserve the norm tells us that Ø has an inverse (since it
must be one-to-one), but this inverse is not necessarily defined on the whole
of V. For example, let {eá} be an orthonormal basis for V, and define the
“shift operator” Ø by
Ø(eá) = ei+1 .
500 LINEAR OPERATORS

This Ø is clearly defined on all of V, but the image of Ø is not all of V since it
does not include the vector eè. Thus, Øî is not defined on eè.
Exactly as we did for unitary operators, we can show that Ø¿Ø = 1 for an
isometric operator Ø. If V happens to be finite-dimensional, then obviously
ØØ¿ = 1. Thus, on a finite-dimensional space, an isometric operator is also
unitary.
Finally, let us show an interesting relationship between the inverse Øî of
an isometric operator and its adjoint Ø¿. From Ø¿Ø = 1, we may write
Ø¿(Ø v) = v for every v ∞ V. If we define Øv = væ, then for every væ ∞ Im Ø
we have v = Øîvæ, and hence

Ø¿væ = Øîvæ for væ ∞ Im Ø .

On the other hand, if wæ ∞ (Im Ø)Ê, then automatically Ówæ, ØvÔ = 0 for every
v ∞ V. Therefore this may be written as ÓØ¿wæ, vÔ = 0 for every v ∞ V, and
hence (choose v = Ø¿wæ)

Ø¿wæ = 0 for wæ ∞ (Im Ø)Ê .

In other words, we have

$&!"1 on Im!!
†
! =% !!.
'& 0 on (Im !)#

For instance, using our earlier example of the shift operator, we see that
Óeè, eáÔ = 0 for i ≠ 1, and hence eè ∞ (Im Ø)Ê. Therefore Ø¿(eè) = 0, so that we
clearly can not have ØØ¿ = 1.
Our next theorem summarizes some of this discussion.

Theorem 10.5 Let V be a complex finite-dimensional inner product space.

Then the following conditions on an operator U ∞ L(V) are equivalent:
(a) U¿ = Uî.
(b) ÓUv, UwÔ = Óv, wÔ for all v, w ∞ V.
(c) ˜ Uv ˜ = ˜ v ˜ .

Proof (a) ¶ (b): ÓUv, UwÔ = Óv, (U¿U)wÔ = Óv, IwÔ = Óv, wÔ.
(b) ¶ (c): ˜ Uv ˜ = ÓUv, UvÔ1/2 = Óv, vÔ1/2 = ˜ v ˜ .
(c) ¶ (a): Óv, (U¿U)vÔ = ÓUv, UvÔ = Óv, vÔ = Óv, IvÔ, and therefore
Óv, (U¿U - I)vÔ = 0. Hence (by Theorem 10.4(b)) we must have U¿U = I, and
thus U¿ = Uî (since V is finite-dimensional). ˙
 10.2 ISOMETRIC AND UNITARY OPERATORS 501

From part (c) of this theorem we see that U preserves the length of any
vector. In particular, U preserves the length of a unit vector, hence the desig-
nation “unitary.” Note also that if v and w are orthogonal, then Óv, wÔ = 0 and
hence ÓUv, UwÔ = Óv, wÔ = 0. Thus U maintains orthogonality as well.
Condition (b) of this theorem is sometimes described by saying that a
unitary transformation preserves inner products. In general, we say that a
linear transformation (i.e., a vector space homomorphism) T of an inner
product space V onto an inner product space W (over the same field) is an
inner product space isomorphism of V onto W if it also preserves inner
products. Therefore, one may define a unitary operator as an inner product
space isomorphism.
It is also worth commenting on the case of unitary operators defined on a
real vector space. Since in this case the adjoint reduces to the transpose, we
have U¿ = UT = Uî. If V is a real vector space, then an operator T = L(V) that
satisfies TT = Tî is said to be an orthogonal transformation. It should be
clear that Theorem 10.5 also applies to real vector spaces if we replace the
adjoint by the transpose. We will have more to say about orthogonal transfor-
mations below.

Theorem 10.6 Let V be finite-dimensional over ç (resp. ®). A linear trans-

formation U ∞ L(V) is unitary (resp. orthogonal) if and only if it takes an
orthonormal basis for V into an orthonormal basis for V.

Proof We consider the case where V is complex, leaving the real case to the
reader. Let {eá} be an orthonormal basis for V, and assume that U is unitary.
Then from Theorem 10.5(b) we have

ÓUeá, UeéÔ = Óeá, eéÔ = ∂áé

so that {Ueá} is also an orthonormal set. But any orthonormal set is linearly
independent (Theorem 2.19), and hence {Ueá} forms a basis for V (since there
are as many of the Ueá as there are eá).
Conversely, suppose that both {eá} and {Ueá} are orthonormal bases for V
and let v, w ∞ V be arbitrary. Then

Óv,!wÔ = Ó!i vi ei ,! ! j w j e jÔ = !i, j vi *w jÓei ,!e jÔ = !i, j vi *w j"ij

= !i vi *wi !!.

However, we also have

502 LINEAR OPERATORS

ÓUv,!UwÔ = ÓU( !i vi ei ),!U( ! j w j e j )Ô = !i, j vi *w j ÓUei ,!Ue j Ô

= !i, j vi *w j"ij = !i vi *wi = Óv,!wÔ .

This shows that U is unitary (Theorem 10.5). ˙

Corollary Let V and W be finite-dimensional inner product spaces over ç.

Then there exists an inner product space isomorphism of V onto W if and only
if dim V = dim W.

Proof Clearly dim V = dim W if V and W are isomorphic. On the other hand,
let {eè, . . . , eñ} be an orthonormal basis for V, and let {eõè, . . . , eõñ} be an
orthonormal basis for W. (These bases exist by Theorem 2.21.) We define the
(surjective) linear transformation U by the requirement Ueá = eõá. U is unique
by Theorem 5.1. Since ÓUeá, UeéÔ = Óeõá, eõéÔ = ∂áé = Óeá, eéÔ, the proof of Theorem
10.6 shows that U preserves inner products. In particular, we see that ˜ Uv ˜ =
˜ v ˜ for every v ∞ V, and hence Ker U = {0} (by property (N1) of Theorem
2.17). Thus U is also one-to-one (Theorem 5.5). ˙

From Theorem 10.2 we see that a complex matrix A represents a unitary

operator relative to an orthonormal basis if and only if A¿ = Aî. We therefore
say that a complex matrix A is a unitary matrix if A¿ = Aî. In the special
case that A is a real matrix with the property that AT = Aî, then we say that A
is an orthogonal matrix. (These classes of matrices were also discussed in
Section 8.1.) The reason for this designation is shown in the next example,
which is really nothing more than another way of looking at what we have
done so far.

Example 10.3 Suppose V = ®n and X ∞ V. In terms of an orthonormal basis

{eá} for V we may write X = Íáxieá. Now suppose we are given another
orthonormal basis {eõá} related to the first basis by eõá = A(eá) = Íéeéaéá for some
real matrix (aáé). Relative to this new basis we have A(X) = Xä = Íáxõieõá where
xi = Íéaáéxõj (see Section 5.4). Then

2
X = Ó!i x i ei ,! ! j x j e jÔ = !i, j x i x jÓei ,!e jÔ = !i, j x i x j"ij
= !i (x i )2 = !i, j, k aij aik x j x k = !i, j, k aT ji aik x j x k
= ! j, k (AT A) jk x j x k !!.

If A is orthogonal, then AT = Aî so that (ATA)éÉ = ∂éÉ and we are left with

 10.2 ISOMETRIC AND UNITARY OPERATORS 503

˜ X ˜ 2 = Íá(xi)2 = Íé(xõj)2 = ˜ Xä ˜ 2

so that the length of X is unchanged under an orthogonal transformation. An

equivalent way to see this is to assume that A simply represents a rotation so
that the length of a vector remains unchanged by definition. This then forces
A to be an orthogonal transformation (see Exercise 10.2.2).
Another way to think of orthogonal transformations is the following. We
saw in Section 2.4 that the angle œ between two vectors X, Y ∞ ®n is defined
by
ÓX,!YÔ
cos! = !!.
˜X˜ ˜Y ˜

Under the orthogonal transformation A, we then have Xä = A(X) and also

ÓX,!Y Ô
cos! = !!.
˜X˜ ˜Y ˜

But ˜ Xä ˜ = ˜ X ˜ and ˜ Yä ˜ = ˜ Y ˜ , and in addition,

ÓX,!YÔ = Ó!i x i ei ,! ! j y j e j Ô = !i x i yi = !i, j, k aij x j aik y k

= ! j, k" jk x j y k = ! j x j y j = ÓX,!Y Ô

so that œ = œ ä (this also follows from the real vector space version of Theorem
10.5). Therefore an orthogonal transformation also preserves the angle
between two vectors, and hence is nothing more than a rotation in ®n. ∆

Theorem 10.7 The following conditions on a matrix A are equivalent:

(a) A is unitary.
(b) The rows Aá of A form an orthonormal set.
(c) The columns Ai of A form an orthonormal set.

Proof We begin by by noting that, using the usual inner product on çn, we
have
(AA¿)áé = ÍÉaáÉa¿Éé = ÍÉaáÉa*éÉ = ÍÉa*éÉaáÉ = ÓAé, AáÔ
and
(A¿A)áé = ÍÉa¿áÉaÉé = ÍÉa*ÉáaÉé = ÓAi, AjÔ .

Now, if A is unitary, then AA¿ = I implies (AA¿)áé = ∂áé which then implies
that ÓAé, AáÔ = ∂áé so that (a) is equivalent to (b). Similarly, we must have
504 LINEAR OPERATORS

(A¿A)áé = ∂áé = ÓAi, AjÔ so that (a) is also equivalent to (c). Therefore (b) must
also be equivalent to (c). ˙

Note that the equivalence of (b) and (c) in this theorem means that the
rows of A form an orthonormal set if and only if the columns of A form an
orthonormal set. But the rows of A are just the columns of AT, and hence A is
unitary if and only if AT is unitary.
It should be obvious that this theorem applies just as well to orthogonal
matrices. Looking at this in the other direction, we see that in this case AT =
Aî so that ATA = AAT = I, and therefore

(AT A)ij = !k aT ik akj = !k aki akj = "ij

(AAT )ij = !k aik aT kj = !k aik a jk = "ij !!.

Viewing the standard (orthonormal) basis {eá} for ®n as row vectors, we have
Aá = Íéaáéeé, and hence

ÓAi ,!A jÔ = Ó!k aik ek ,! !r a jr erÔ = !k, r aik a jrÓek ,!erÔ

= !k, r aik a jr"kr = !k aik a jk = "ij !!.

Furthermore, it is easy to see that a similar result holds for the columns of A.
Our next theorem details several useful properties of orthogonal and uni-
tary matrices.

Theorem 10.8 (a) If A is an orthogonal matrix, then det A = ±1.

(b) If U is a unitary matrix, then \det U\ = 1. Alternatively, det U = ei! for
some real number ƒ.

Proof (a) We have AAT = I, and hence (from Theorems 4.8 and 4.1)

1 = det I = det(AAT) = (det A)(det AT) = (det A)2

so that det A = ±1.

(b) If UU¿ = I then, as above, we have

1 = det I = det(UU † ) = (detU )(detU † ) = (detU )(detU T )*

2
= (detU )(detU )* = detU !!.
 10.2 ISOMETRIC AND UNITARY OPERATORS 505

Since the absolute value is defined to be positive, this shows that \det U\ = 1
and hence det U = ei! for some real ƒ. ˙

Example 10.4 Let us take a look at rotations in ®2 as shown, for example, in

the figure below. Recall from Example 10.3 that if we have two bases {eá} and
{eõá}, then they are related by a transition matrix A = (aáé) defined by eõá =
Íéeéaéá. In addition, if X = Íxieá = Íxõieõá, then xi = Íéaáéxõj. If both {eá} and {eõá}
are orthonormal bases, then

Óeá, eõéÔ = Óeá, ÍÉeÉaÉéÔ = ÍÉaÉéÓeá, eÉÔ = ÍÉaÉé∂áÉ = aáé .

Using the usual dot product on ®2 as our inner product (see Section 2.4,
Lemma 2.3) and referring to the figure below, we see that the elements aáé are
given by (also see Section 0.6 for the trigonometric identities)i

a11 = e1 ie1 = e1 e1 cos! = cos!

a12 = e1 ie2 = e1 e2 cos(" /2 + ! ) = #sin !
a21 = e2 ie1 = e2 e1 cos(" /2 # ! ) = sin !
a22 = e2 ie2 = e2 e2 cos! = cos!

xì
xõì
X

θ
xõè

θ xè

Thus the matrix A is given by

# cos! "sin ! &

(aij ) = % (!!.
$ sin ! cos! '

We leave it to the reader to compute directly that ATA = AAT = I and det A =
+1. ∆

Example 10.5 Referring to the previous example, we can show that any
(real) 2 x 2 orthogonal matrix with det A = +1 has the form
506 LINEAR OPERATORS

# cos! "sin ! &

(aij ) = % (
$ sin ! cos! '

for some œ ∞ ®. To see this, suppose A has the form

!a b $
# &
"c d%

where a, b, c, d ∞ ®. Since A is orthogonal, its rows form an orthonormal set,

and hence we have

a2 + b2 = 1, c2 + d2 = 1, ac + bd = 0, ad - bc = 1

where the last equation follows from det A = 1.

If a = 0, then the first of these equations yields b = ±1, the third then yields
d = 0, and the last yields -c = 1/b = ±1 which is equivalent to c = -b. In other
words, if a = 0, then A has either of the following forms:

" !!0 1 % " 0 !1 %

$ ' or $ '!!.
# !1 0 & # 1 !!0 &

The first of these is of the required form if we choose œ = -90o = -π/2, and
the second is of the required form if we choose œ = +90o = +π/2.
Now suppose that a ≠ 0. From the third equation we have c = -bd/a, and
substituting this into the second equation, we find (a2 + b2)d2 = a2. Using the
first equation, this becomes a2 = d2 or a = ±d. If a = -d, then the third equation
yields b = c, and hence the last equation yields -a2 - b2 = 1 which is im-
possible. Therefore a = d, the third equation then yields c = -b, and we are left
with
" a !c %
$ '!!.
# c !a &

Since det A = a 2 + c2 = 1, there exists a real number œ such that a = cos œ and
c = sin œ which gives us the desired form for A. ∆

Exercises

1. Let GL(n, ç) denote the subset of Mñ(ç) consisting of all nonsingular

matrices, U(n) the subset of all unitary matrices, and L(n) the set of all
 10.2 ISOMETRIC AND UNITARY OPERATORS 507

nonsingular lower-triangular matrices.

(a) Show that each of these three sets forms a group.
(b) Show that any nonsingular n x n complex matrix can be written as a
product of a nonsingular upper-triangular matrix and a unitary matrix.
[Hint: Use Exercises 5.4.14 and 3.7.7.]

2. Let V = ®n with the standard inner product, and suppose the length of any
X ∞ V remains unchanged under A ∞ L(V). Show that A must be an
orthogonal transformation.

3. Let V be the space of all continuous complex-valued functions defined on

[0, 2π], and define an inner product on V by
1 2!
Óf ,!gÔ = " f *(x)g(x)dx !!.
2! 0
Suppose there exists h ∞ V such that \h(x)\ = 1 for all x ∞ [0, 2π], and
define Th ∞ L(V) by Thf = hf. Prove that T is unitary.

4. Let W be a finite-dimensional subspace of an inner product space V, and

recall that V = W • WÊ (see Exercise 2.5.11). Define U ∞ L(V) by

U(wè + wì) = wè - wì

where wè ∞ W and wì ∞ WÊ.

(a) Prove that U is a Hermitian operator.
(b) Let V = ®3 have the standard inner product, and let W ™ V be spanned
by the vector (1, 0, 1). Find the matrix of U relative to the standard basis
for V.

5. Let V be a finite-dimensional inner product space. An operator Ø ∞ L(V)

is said to be a partial isometry if there exists a subspace W of V such that
˜ Øw ˜ = ˜ w ˜ for all w ∞ W, and ˜ Øw ˜ = 0 for all w ∞ WÊ. Let Ø be a partial
isometry and suppose {w1, . . . , wk} is an orthonormal basis for W.
(a) Show that ÓØu, ØvÔ = Óu, vÔ for all u, v ∞ W. [Hint: Use Exercise
2.4.7.]
(b) Show that {Øw1, . . . , Øwk} is an orthonormal basis for Im Ø.
(c) Show there exists an orthonormal basis {vá} for V such that the first k
columns of [Ø]v form an orthonormal set, and the remaining columns are
zero.
(d) Let {u1, . . . , ur} be an orthonormal basis for (Im Ø)Ê. Show that
{Ø w1, . . . , Ø wk, u1, . . . , ur} is an orthonormal basis for V.
(e) Suppose T ∞ L(V) satisfies T(Ø wá) = wá (for 1 ¯ i ¯ k) and Tuá = 0
(for 1 ¯ i ¯ r). Show that T is well-defined, and that T = Ø¿.
508 LINEAR OPERATORS

(f ) Show that Ø¿ is a partial isometry.

6. Let V be a complex inner product space, and suppose H ∞ L(V) is

Hermitian. Show that:
(a) ˜ v + iHv ˜ = ˜ v - iHv ˜ for all v ∞ V.
(b) u + iHu = v + iHv if and only if u = v.
(c) 1 + iH and 1 - iH are nonsingular.
(d) If V is finite-dimensional, then U = (1 - iH)(1 + iH)î is a unitary
operator. (U is called the Cayley transform of H. This result is also true
in an infinite-dimensional Hilbert space but the proof is considerably more
difficult.)

10.3 NORMAL OPERATORS

We now turn our attention to characterizing the type of operator on V for

which there exists an orthonormal basis of eigenvectors for V. We begin by
taking a look at some rather simple properties of the eigenvalues and eigen-
vectors of the operators we have been discussing.
To simplify our terminology, we remark that a complex inner product
space is also called a unitary space, while a real inner product space is some-
times called a Euclidean space. If H is an operator such that H¿ = -H, then H
is said to be anti-Hermitian (or skew-Hermitian). Furthermore, if P is an
operator such that P = S¿S for some operator S, then we say that P is positive
(or positive semidefinite or nonnegative). If S also happens to be nonsingular
(and hence P is also nonsingular), then we say that P is positive definite. Note
that a positive operator is necessarily Hermitian since (S¿S)¿ = S¿S. The rea-
son that P is called positive is shown in part (d) of the following theorem.

Theorem 10.9 (a) The eigenvalues of a Hermitian operator are real.

(b) The eigenvalues of an isometry (and hence also of a unitary transfor-
mation) have absolute value one.
(c) The eigenvalues of an anti-Hermitian operator are pure imaginary.
(d) A positive (positive definite) operator has eigenvalues that are real and
nonnegative (positive).

Proof (a) If H is Hermitian, v ≠ 0, and Hv = ¬v, we have

!Óv,!vÔ = Óv,! ! vÔ = Óv,!HvÔ = ÓH †v,!vÔ = ÓHv,!vÔ

= Ó! v,!vÔ = !*Óv,!vÔ!!.

But Óv, vÔ ≠ 0, and hence ¬ = ¬*.

 10.3 NORMAL OPERATORS 509

(b) If Ø is an isometry, v ≠ 0, and Øv = ¬v, then we have (using Theorem

2.17)
˜ v ˜ = ˜ Øv ˜ = ˜ ¬v ˜ = \¬\ ˜ v ˜ .

But ˜ v ˜ ≠ 0, and hence \¬\ = 1.

(c) If H¿ = -H, v ≠ 0, and Hv = ¬v, then

!Óv,!vÔ = Óv,! ! vÔ = Óv,!HvÔ = ÓH †v,!vÔ = Ó " Hv,!vÔ = Ó " ! v,!vÔ

= "!*Óv,!vÔ .

But Óv, vÔ ≠ 0, and hence ¬ = -¬*. This shows that ¬ is pure imaginary.
(d) Let P = S¿S be a positive definite operator. If v ≠ 0, then the fact that
S is nonsingular means that Sv ≠ 0, and hence ÓSv, SvÔ = ˜ Sv ˜ 2 > 0. Then, for
Pv = (S¿S)v = ¬v, we see that

¬Óv, vÔ = Óv, ¬vÔ = Óv, PvÔ = Óv, (S¿S)vÔ = ÓSv, SvÔ .

But Óv, vÔ = ˜ v ˜ 2 > 0 also, and therefore we must have ¬ > 0.

If P is positive, then S is singular and the only difference is that now for
v ≠ 0 we have ÓSv, SvÔ = ˜ Sv ˜ 2 ˘ 0 which implies that ¬ ˘ 0. ˙

We say that an operator N is normal if N¿N = NN¿. Note this implies that
for any v ∞ V we have

˜Nv˜2 = ÓNv,!NvÔ = Ó(N † N )v,!vÔ = Ó(NN † )v,!vÔ = ÓN †v,!N †vÔ

=!˜N †v˜2 !!.

Now let ¬ be a complex number. It is easy to see that if N is normal then so is

N - ¬1 since (from Theorem 10.3)

(N ! "1)† (N ! "1) = (N † ! "*1)(N ! "1) = N † N ! " N † ! "*N + " * "1

= (N ! "1)(N † ! "*1) = (N ! "1)(N ! "1)† !!.

Using N - ¬1 instead of N in the previous result we obtain

˜ Nv - ¬v ˜ 2 = ˜ N¿v - ¬*v ˜ 2 .

Since the norm is positive definite, this equation proves the next theorem.
510 LINEAR OPERATORS

Theorem 10.10 Let N be a normal operator and let ¬ be an eigenvalue of N.

Then Nv = ¬v if and only if N¿v = ¬*v.

In words, if v is an eigenvector of a normal operator N with eigenvalue ¬,

then v is also an eigenvector of N¿ with eigenvalue ¬*. (Note it is always true
that if ¬ is an eigenvalue of an operator T, then ¬* will be an eigenvalue of T¿.
See Exercise 10.3.6.)

Corollary If N is normal and Nv = 0 for some v ∞ V, then N¿v = 0.

Proof This follows from Theorem 10.10 by taking ¬ = ¬* = 0. Alternatively,

using N¿N = NN¿ along with the fact that Nv = 0, we see that

ÓN¿v, N¿vÔ = Óv, (NN¿)vÔ = Óv, (N¿N)vÔ = 0 .

Since the inner product is positive definite, this requires that N¿v = 0. ˙

Theorem 10.11 (a) Eigenvectors belonging to distinct eigenvalues of a

Hermitian operator are orthogonal.
(b) Eigenvectors belonging to distinct eigenvalues of an isometric opera-
tor are orthogonal. Hence the eigenvectors of a unitary operator are orthogo-
nal.
(c) Eigenvectors belonging to distinct eigenvalues of a normal operator
are orthogonal.

Proof As we note after the proof, Hermitian and unitary operators are special
cases of normal operators, and hence parts (a) and (b) follow from part (c).
However, it is instructive to give independent proofs of parts (a) and (b).
Assume that T is an operator on a unitary space, and Tvá = ¬ává for i = 1, 2
with ¬è ≠ ¬ì. We may then also assume without loss of generality that ¬è ≠ 0.
(a) If T = T¿, then (using Theorem 10.9(a))

!2Óv1,!v2Ô = Óv1,! !2 v2Ô = Óv1,!Tv2Ô = ÓT †v1,!v2Ô = ÓTv1,!v2Ô

= Ó!1v1,!v2Ô = !1*Óv1,!v2Ô = !1Óv1,!v2Ô .

But ¬è ≠ ¬ì, and hence Óvè, vìÔ = 0.

(b) If T is isometric, then T¿T = 1 and we have

Óvè, vìÔ = Óvè, (T¿T)vìÔ = ÓTvè, TvìÔ = ¬è*¬ìÓvè, vìÔ .

But by Theorem 10.9(b) we have \\2 = * = 1, and thus * = 1/.
Therefore, multiplying the above equation by ¬è, we see that ¬èÓvè, vìÔ =
 10.3 NORMAL OPERATORS 511

¬ìÓvè, vìÔ and hence, since ¬è ≠ ¬ì, this shows that Óvè, vìÔ = 0.
(c) If T is normal, then

Óvè, TvìÔ = ¬ìÓvè, vìÔ

while on the other hand, using Theorem 10.10 we have

Óvè, TvìÔ = ÓT¿vè, vìÔ = Ó¬è*vè, vìÔ = ¬èÓvè, vìÔ .

Thus Óvè, vìÔ = 0 since ¬è ≠ ¬ì. ˙

We note that if H¿ = H, then H¿H = HH = HH¿ so that any Hermitian

operator is normal. Furthermore, if U is unitary, then U¿U = UU¿ ( = 1) so
that U is also normal.
A Hermitian operator T defined on a real inner product space is said to be
symmetric. This is equivalent to requiring that with respect to an orthonormal
basis, the matrix elements aáé of T are given by

aáé = Óeá, TeéÔ = ÓTeá, eéÔ = Óeé, TeáÔ = aéá .

Therefore, a symmetric operator is represented by a real symmetric matrix. It

is also true that antisymmetric operators (i.e., TT = -T) and anti-Hermitian
operators (H¿ = -H) are normal. Therefore, part (a) and the unitary case in
part (b) in the above theorem are really special cases of part (c).

Theorem 10.12 (a) Let T be an operator on a unitary space V, and let W be

a T-invariant subspace of V. Then WÊ is invariant under T¿.
(b) Let U be a unitary operator on a unitary space V, and let W be a U-
invariant subspace of V. Then WÊ is also invariant under U.

Proof (a) For any v ∞ W we have Tv ∞ W since W is T-invariant. Let w ∞

WÊ be arbitrary. We must show that T ¿w ∞ WÊ. But this is easy because

ÓT¿w, vÔ = Ów, TvÔ = 0

by definition of WÊ. Thus T¿w ∞ WÊ so that WÊ is invariant under T¿.

(b) The fact that U is unitary means Uî = U¿ exists, and hence U is non-
singular. In other words, for any væ ∞ W there exists v ∞ W such that Uv = væ.
Now let w ∞ WÊ be arbitrary. Then

ÓUw, væÔ = ÓUw, UvÔ = Ów, (U¿U)vÔ = Ów, vÔ = 0

by definition of WÊ. Thus Uw ∞ WÊ so that WÊ is invariant under U. ˙

512 LINEAR OPERATORS

Recall from the discussion in Section 7.7 that the algebraic multiplicity of
a given eigenvalue is the number of times the eigenvalue is repeated as a root
of the characteristic polynomial. We also defined the geometric multiplicity as
the number of linearly independent eigenvectors corresponding to this eigen-
value (i.e., the dimension of its eigenspace).

Theorem 10.13 Let H be a Hermitian operator on a finite-dimensional

unitary space V. Then the algebraic multiplicity of any eigenvalue ¬ of H is
equal to its geometric multiplicity.

Proof Let V¬ = {v ∞ V: Hv = ¬v} be the eigenspace corresponding to the

eigenvalue ¬. Furthermore, V¬ is obviously invariant under H since Hv =
¬v ∞ V¬ for every v ∞ V¬. By Theorem 10.12(a), we then have that V¬Ê is
also invariant under H¿ = H. Furthermore, by Theorem 2.22 we see that V =
V¬ • V¬Ê. Applying Theorem 7.20, we may write H = Hè • Hì where Hè =
H\V¬ and Hì = H\V¬Ê.
Let A be the matrix representation of H, and let Aá be the matrix represen-
tation of Há (i = 1, 2). By Theorem 7.20, we also have A = Aè • Aì. Using
Theorem 4.14, it then follows that the characteristic polynomial of A is given
by
det(xI - A) = det(xI - Aè) det(xI - Aì) .

Now, Hè is a Hermitian operator on the finite-dimensional space V¬ with only

the single eigenvalue ¬. Therefore ¬ is the only root of det(xI - Aè) = 0, and
hence it must occur with an algebraic multiplicity equal to the dimension of
V¬ (since this is just the size of the matrix Aè). In other words, if dim V¬ = m,
then det(xI - Aè) = (x - ¬m). On the other hand, ¬ is not an eigenvalue of Aì
by definition, and hence det(xI - Aì) ≠ 0. This means that det(xI - A) contains
(x - ¬) as a factor exactly m times. ˙

Corollary Any Hermitian operator H on a finite-dimensional unitary space

V is diagonalizable.

Proof Since V is a unitary space, the characteristic polynomial of H will

factor into (not necessarily distinct) linear terms. The conclusion then follows
from Theorems 10.13 and 7.26. ˙

In fact, from Theorem 8.2 we know that any normal matrix is unitarily
similar to a diagonal matrix. This means that given any normal operator T ∞
L(V), there is an orthonormal basis for V that consists of eigenvectors of T.
 10.3 NORMAL OPERATORS 513

We develop this result from an entirely different point of view in the next
section.

Exercises

1. Let V be a unitary space and suppose T ∞ L(V). Define T+ = (1/2)(T + T¿)

and T- = (1/2i)(T - T¿).
(a) Show that T+ and T- are Hermitian, and that T = T+ + iT-.
(b) If T\æ+ and Tæ- are Hermitian operators such that T = Tæ+ + iTæ-, show
that Tæ+ = T+ and Tæ- = T-.
(c) Prove that T is normal if and only if [T+, T-] = 0.

2. Let N be a normal operator on a finite-dimensional inner product space V.

Prove Ker N = Ker N¿ and Im N = Im N¿. [Hint: Prove that (Im N¿)Ê =
Ker N, and hence that Im N¿ = (Ker N)Ê.]

3. Let V be a finite-dimensional inner product space, and suppose T ∞ L(V)

is both positive and unitary. Prove that T = 1.

4. Let H ∞ Mn(ç) be Hermitian. Then for any nonzero x ∞ çn we define the

Rayleigh quotient to be the number

Óx,!HxÔ
R(x) = !!.
˜x˜2

Prove that max{R(x): x ≠ 0} is the largest eigenvalue of H, and that

min{R(x): x ≠ 0} is the smallest eigenvalue of H.

5. Let V be a finite-dimensional unitary space, and suppose E ∞ L(V) is such

that E2 = E = E¿. Prove that V = Im E • (Im E)Ê.

6. If V is finite-dimensional and T ∞ L(V) has eigenvalue ¬, show that T¿

has eigenvalue ¬*.

10.4 DIAGONALIZATION OF NORMAL OPERATORS

We now turn to the problem of diagonalizing operators. We will discuss sev-

eral of the many ways to approach this problem. Because most commonly
used operators are normal, we first treat this general case in detail, leaving
unitary and Hermitian operators as obvious special cases. Next, we go back
514 LINEAR OPERATORS

and consider the real and complex cases separately. In so doing, we will gain
much insight into the structure of orthogonal and unitary transformations.
While this problem was treated concisely in Chapter 8, we present an entirely
different viewpoint in this section to acquaint the reader with other approaches
found in the literature. If the reader has studied Chapter 8, he or she should
keep in mind the rational and Jordan forms while reading this section, as many
of our results (such as Theorem 10.16) follow almost trivially from our earlier
work. We begin with some more elementary facts about normal transforma-
tions.

Theorem 10.14 Let V be a unitary space.

(a) If T ∞ L(V) and (T¿T)v = 0 for some v ∞ V, then Tv = 0.
(b) If H is Hermitian and Hkv = 0 for k ˘ 1, then Hv = 0.
(c) If N is normal and Nkv = 0 for k ˘ 1, then Nv = 0.
(d) If N is normal, and if (N - ¬1)kv = 0 where k ˘ 1 and ¬ ∞ ç, then
Nv = ¬v.

Proof (a) Since (T¿T)v = 0, we have 0 = Óv, (T¿T)vÔ = ÓTv, TvÔ which
implies that Tv = 0 because the inner product is positive definite.
(b) We first show that if H2mv = 0 for some positive integer m, then Hv =
0. To see this, let T = H2m-1 and note that T¿ = T because H is Hermitian (by
induction from Theorem 10.3(c)). Then T¿T = TT = H2m, and hence

0 = ÓH2mv, vÔ = Ó(T¿T)v, vÔ = ÓTv, TvÔ

which implies that 0 = Tv = H2m-1v. Repeating this process, we must eventu-

ally obtain Hv = 0.
Now, if Hkv = 0, then H2mv = 0 for any 2m ˘ k, and therefore applying the
above argument, we see that Hv = 0.
(c) Define the Hermitian operator H = N¿N. Since N is normal, we see
that
(N¿N)2 = N¿NN¿N = N¿2N2

and by induction,
(N¿N)k = N¿kNk .

By hypothesis, we then find that

Hkv = (N¿N)k v = (N¿kNk)v = 0

 10.4 DIAGONALIZATION OF NORMAL OPERATORS 515

and hence (N¿N)v = Hv = 0 by part (b). But then Nv = 0 by part (a).

(d) Since N is normal, it follows that N - ¬1 is normal, and therefore by
part (c) we have (N - ¬1)v = 0. ˙

Just as we did for operators, we say that a matrix N is normal if N¿N =

NN¿. We now wish to show that any normal matrix can be diagonalized by a
unitary similarity transformation. Another way to phrase this is as follows. We
say that two matrices A, B ∞ Mn(ç) are unitarily similar (or equivalent) if
there exists a unitary matrix U ∞ Mn(ç) such that A = U¿BU = UîBU. Thus,
we wish to show that any normal matrix is unitarily similar to a diagonal
matrix. This extremely important result is quite easy to prove with what has
already been shown. Let us first prove this in the case of normal operators
over the complex field. (See Theorem 8.2 for another approach.)

Theorem 10.15 Let N be a normal operator on a finite-dimensional unitary

space V. Then there exists an orthonormal basis for V consisting of eigenvec-
tors of N in which the matrix of N is diagonal.

Proof Let ¬è, . . . , ¬r be the distinct eigenvalues of the normal operator N.

(These all exist in ç by Theorems 6.12 and 6.13.) Then (by Theorem 7.13) the
minimal polynomial m(x) for N must be of the form

m(x) = (x - ¬è)nè ~ ~ ~ (x - ¬r)ë

where each ná ˘ 1. By the primary decomposition theorem (Theorem 7.23), we

can write V = Wè • ~ ~ ~ • Wr where Wá = Ker(N - ¬á1)ná. In other words,

(N - ¬á1)ná vá = 0

for every vá ∞ Wá. By Theorem 10.14(d), we then have Nvá = ¬ává so that
every vá ∞ Wá is an eigenvector of N with eigenvalue ¬á.
Now, the inner product on V induces an inner product on each subspace
Wá in the usual and obvious way, and thus by the Gram-Schmidt process
(Theorem 2.21), each Wá has an orthonormal basis relative to this induced
inner product. Note that by the last result of the previous paragraph, this basis
must consist of eigenvectors of N.
By Theorem 10.11(c), vectors in distinct Wá are orthogonal to each other.
Therefore, according to Theorem 2.15, the union of the bases of the Wá forms
a basis for V, which thus consists entirely of eigenvectors of N. By Theorem
7.14 then, the matrix of N is diagonal in this basis. (Alternatively, we see that
the matrix elements náé of N relative to the eigenvector basis {eá} are given by
náé = Óeá, NeéÔ = Óeá, ¬éeéÔ = ¬é∂áé.) ˙
516 LINEAR OPERATORS

Corollary 1 Let N be a normal matrix over ç. Then there exists a unitary

matrix U such that UîNU = U¿NU is diagonal. Moreover, the columns of U
are just the eigenvectors of N, and the diagonal elements of U¿NU are the
eigenvalues of N.

Proof The normal matrix N defines an operator on a finite-dimensional

unitary space V with the standard orthonormal basis, and therefore by
Theorem 10.15, V has an orthonormal basis of eigenvectors in which the
matrix N is diagonal. By Theorem 10.6, any such change of basis in V is
accomplished by a unitary transformation U, and by Theorem 5.18, the matrix
of the operator relative to this new basis is related to the matrix N in the old
basis by the similarity transformation UîNU (= U¿NU).
Now note that the columns of U are precisely the eigenvectors of N (see
the discussion preceding Example 7.4). We also recall that Theorem 7.14 tells
us that the diagonal elements of the diagonal form of N are exactly the eigen-
values of N. ˙

Corollary 2 A real symmetric matrix can be diagonalized by an orthogonal

matrix.

Proof Note that a real symmetric matrix A may be considered as an operator

on a finite-dimensional real inner product space V. If we think of A as a com-
plex matrix that happens to have all real elements, then A is Hermitian and
hence has all real eigenvalues. This means that all the roots of the minimal
polynomial for A lie in ®. If ¬è, . . . , ¬r are the distinct eigenvalues of A, then
we may proceed exactly as in the proof of Theorem 10.15 and Corollary 1 to
conclude that there exists a unitary matrix U that diagonalizes A. In this case,
since Wá = Ker(A - ¬á I)ná and A - ¬á I is real, it follows that the eigenvectors
of A are real and hence U is actually an orthogonal matrix. ˙

Corollary 2 is also proved from an entirely different point of view in

Exercise 10.4.9. This alternative approach has the advantage of presenting a
very useful geometric picture of the diagonalization process.

Example 10.6 Let us diagonalize the real symmetric matrix

" !!2 !2 %
A=$ '!!.
# !2 !!5 &

The characteristic polynomial of A is

 10.4 DIAGONALIZATION OF NORMAL OPERATORS 517

ÎA(x) = det(xI - A) = (x - 2)(x - 5) - 4 = (x - 1)(x - 6)

and therefore the eigenvalues of A are 1 and 6. To find the eigenvectors of A,

we must solve the matrix equation (¬áI - A)vá = 0 for the vector vá. For ¬è = 1
we have vè = (xè, yè), and hence we find the homogeneous system of equations

!x1 + 2y1 = 0
2x1 ! 4y1 = 0!!.

These imply that xè = 2yè, and hence a nonzero solution is vè = (2, 1). For ¬ì =
6 we have the equations

4x2 + 2y2 = 0
2x2 +!! y2 = 0

which yields vì = (1, -2).

Note that Óvè, vìÔ = 0 as it should according to Theorem 10.11, and that
˜ vè ˜ = “5” = ˜ vì ˜ . We then take the normalized basis vectors to be eá = vá/“5”
which are also eigenvectors of A. Finally, A is diagonalized by the orthogonal
matrix P whose columns are just the eá:

"2 5 !!1 5%
P = $$ '!!.
'
#1 5 !2 5&

We leave it to the reader to show that

!1 0$
PT AP = # &!!.
"0 6%

Another important point to notice is that Theorem 10.15 tells us that even
though an eigenvalue ¬ of a normal operator N may be degenerate (i.e., have
algebraic multiplicity k > 1), it is always possible to find k linearly indepen-
dent eigenvectors belonging to ¬. The easiest way to see this is to note that
from Theorem 10.8 we have \det U\ = 1 ≠ 0 for any unitary matrix U. This
means that the columns of the diagonalizing matrix U (which are just the
eigenvectors of N) must be linearly independent. This is in fact another proof
that the algebraic and geometric multiplicities of a normal (and hence
Hermitian) operator must be the same.
We now consider the case of real orthogonal transformations as indepen-
dent operators, not as a special case of normal operators. First we need a gen-
518 LINEAR OPERATORS

eral definition. Let V be an arbitrary finite-dimensional vector space over any

field F, and suppose T ∞ L(V). A nonzero T-invariant subspace W ™ V is said
to be irreducible if the only T-invariant subspaces contained in W are {0}
and W.

Theorem 10.16 (a) Let V be a finite-dimensional vector space over an

algebraically closed field F, and suppose T ∞ L(V). Then every irreducible T-
invariant subspace W of V is of dimension 1.
(b) Let V be a finite-dimensional vector space over ®, and suppose T ∞
L(V). Then every irreducible T-invariant subspace W of V is of dimension
either 1 or 2.

Proof (a) Let W be an irreducible T-invariant subspace of V. Then the

restriction TW of T to W is just a linear transformation on W, where TW(w) =
Tw ∞ W for every w ∞ W. Since F is algebraically closed, the characteristic
polynomial of TW has at least one root (i.e., eigenvalue ¬) in F. Therefore T
has at least one (nonzero) eigenvector v ∞ W such that Tv = ¬v ∞ W. If we
define S(v) to be the linear span of {v}, then S(v) is also a T-invariant sub-
space of W, and hence S(v) = W because W is irreducible. Therefore W is
spanned by the single vector v, and hence dim W = 1.
(b) Let W be an irreducible T-invariant subspace of V, and let m(x) be the
minimal polynomial for TW. Therefore, the fact that W is irreducible (so that
W is not a direct sum of T-invariant subspaces) along with the primary
decomposition theorem (Theorem 7.23) tells us that we must have m(x) =
f(x)n where f(x) ∞ ®[x] is a prime polynomial. Furthermore, if n were greater
than 1, then we claim that Ker f(T)n-1 would be a T-invariant subspace of W
(Theorem 7.18) that is different from {0} and W.
To see this, first suppose that Ker f(T)n-1 = {0}. Then the linear transfor-
mation f(T)n-1 is one-to-one, and hence f(T)n-1(W) = W. But then

0 = f(T)n(W) = f(T)f(T)n-1(W) = f(T)(W) .

However, f(T)W ≠ 0 by definition of m(x), and hence this contradiction shows

that we can not have Ker f(T)n-1 = {0}. Next, if we had Ker f(T)n-1 = W, this
would imply that f(T)n-1(W) = 0 which contradicts the definition of minimal
polynomial. Therefore we must have n = 1 and m(x) = f(x).
Since m(x) = f(x) is prime, it follows from the corollary to Theorem 6.15
that we must have either m(x) = x - a or m(x) = x2 + ax + b with a2 - 4ab < 0.
If m(x) = x - a, then there exists an eigenvector v ∞ W with Tv = av ∞ W, and
hence S(v) = W as in part (a). If m(x) = x2 + ax + b, then for any nonzero w ∞
W we have
 10.4 DIAGONALIZATION OF NORMAL OPERATORS 519

0 = m(T)w = T 2w + aTw + bw
and hence
T2w = T(Tw) = -aTw - bw ∞ W .

Thus S(w, Tw) is a T-invariant subspace of W with dimension either 1 or 2.

However W is irreducible, and therefore we must have W = S(w, Tw). ˙

Theorem 10.17 Let V be a finite-dimensional Euclidean space, let T ∞ L(V)

be an orthogonal transformation, and let W be an irreducible T-invariant sub-
space of V. Then one of the following two conditions holds:
(a) dim W = 1, and for any nonzero w ∞ W we have Tw = ±w.
(b) dim W = 2, and there exists an orthonormal basis {eè, eì} for W such
that the matrix representation of TW relative to this basis has the form

# cos! "sin ! &

% (!!.
$ sin ! !!cos! '

Proof That dim W equals 1 or 2 follows from Theorem 10.16(b). If dim W =

1, then (since W is T-invariant) there exists ¬ ∞ ® such that Tw = ¬w for any
(fixed) w ∞ W. But T is orthogonal so that

˜ w ˜ = ˜ Tw ˜ = ˜ ¬w ˜ = \¬\ ˜ w ˜

and hence \¬\ = 1. This shows that Tw = ¬w = ±w.

If dim W = 2, then the desired form of the matrix of TW follows essen-
tially from Example 10.5. Alternatively, we know that W has an orthonormal
basis {eè, eì} by the Gram-Schmidt process. If we write Teè = aeè + beì, then
˜ Teè ˜ = ˜ eè ˜ = 1 implies that a2 + b2 = 1. If we also write Teì = ceè + deì, then
similarly c2 + d2 = 1. Using ÓTeè, TeìÔ = Óeè, eìÔ = 0 we find ac + bd = 0, and
hence c = -bd/a. But then 1 = d2(1 + b2/a2) = d2/a2 so that a2 = d2 and c2 = b2.
This means that Teì = ±(-beè + aeì). If Teì = -beè + aeì, then the matrix of T
is of the form
" a !b %
$ '
# b !a &

and we may choose œ ∞ ® such that a = cos œ and b = sin œ (since det T = a2 +
b2 = 1). However, if Teì = beè - aeì, then the matrix of T is

" a !!b %
$ '
# b !a &
520 LINEAR OPERATORS

which satisfies the equation x 2 - 1 = (x - 1)(x + 1) = 0 (and has det T = -1).

But if T satisfied this equation, then (by the primary decomposition theorem
(Theorem 7.23)) W would be a direct sum of subspaces, in contradiction to
the assumed irreducibility of W. Therefore only the first case can occur. ˙

This theorem becomes quite useful when combined with the next result.

Theorem 10.18 Let T be an orthogonal operator on a finite-dimensional

Euclidean space V. Then V = Wè • ~ ~ ~ • Wr where each Wá is an irreducible
T-invariant subspace of V such that vectors belonging to distinct subspaces
Wá and Wé are orthogonal.

Proof If dim V = 1 there is nothing to prove, so we assume dim V > 1 and

that the theorem is true for all spaces of dimension less than dim V. Let Wè be
a nonzero T-invariant subspace of least dimension. Then Wè is necessarily
irreducible. By Theorem 2.22 we know that V = Wè • WèÊ where dim WèÊ <
dim V, and hence we need only show that WèÊ is also T-invariant. But this
follows from Theorem 10.12(b) applied to real unitary transformations (i.e.,
orthogonal transformations). This also means that T(WèÊ) ™ WèÊ, and hence
T is an orthogonal transformation on WèÊ (since it takes vectors in WèÊ to
vectors in WèÊ). By induction, WèÊ is a direct sum of pairwise orthogonal
irreducible T-invariant subspaces, and therefore so is V = Wè • WèÊ. ˙

From Theorem 10.18, we see that if we are given an orthogonal transfor-

mation T on a finite-dimensional Euclidean space V, then V = Wè • ~ ~ ~ • Wr
is the direct sum of pairwise orthogonal irreducible T-invariant subspaces Wá.
But from Theorem 10.17, we see that any such subspace Wá is of dimension
either 1 or 2. Moreover, Theorem 10.17 also showed that if dim Wá = 1, then
the matrix of T\Wá is either (1) or (-1), and if dim Wá = 2, then the matrix of
T\Wá is just the rotation matrix Rá given by

# cos!i "sin !i &

Ri = % (!!.
$ sin !i !!cos!i '

Since each Wá has an orthonormal basis and the bases of distinct Wá are
orthogonal, it follows that we can find an orthonormal basis for V in which
the matrix of T takes the block diagonal form (see Theorem 7.20)

(1) • ~ ~ ~ • (1) • (-1) • ~ ~ ~ • (-1) • Rè • ~ ~ ~ • Rm .

 10.4 DIAGONALIZATION OF NORMAL OPERATORS 521

These observations prove the next theorem.

Theorem 10.19 Let T be an orthogonal transformation on a finite-dimen-

sional Euclidean space V. Then there exists an orthonormal basis for V in
which the matrix representation of T takes the block diagonal form

Mè • ~ ~ ~ • Mr

where each Má is one of the following: (+1), (-1), or Rá .

Exercises

1. Prove that any nilpotent normal operator is necessarily the zero operator.

2. Let A and B be normal operators on a finite-dimensional unitary space V.

For notational simplicity, let va denote an eigenvector of A corresponding
to the eigenvalue a, let vb be an eigenvector of B corresponding to the
eigenvalue b, and let vab denote a simultaneous eigenvector of A and B,
i.e., Avab = avab and Bvab = bvab.
(a) If there exists a basis for V consisting of simultaneous eigenvectors of
A and B, show that the commutator [A, B] = AB - BA = 0.
(b) If [A, B] = 0, show that there exists a basis for V consisting entirely of
simultaneous eigenvectors of A and B. In other words, if [A, B] = 0, then
A and B can be simultaneously diagonalized. [Hint: There are several
ways to approach this problem. One way follows easily from Exercise
8.1.3. Another intuitive method is as follows. First assume that at least one
of the operators, say A, is nondegenerate. Show that Bva is an eigenvector
of A, and that Bva = bva for some scalar b. Next assume that both A and B
are degenerate. Then Ava,i = ava,i where the va,i (i = 1, . . . , ma) are
linearly independent eigenvectors corresponding to the eigenvalue a of
multiplicity ma. What does the matrix representation of A look like in the
{va,i} basis? Again consider Bva,i. What does the matrix representation of
B look like? Now what happens if you diagonalize B?]

3. If Nè and Nì are commuting normal operators, show that the product NèNì
is normal.

4. Let V be a finite-dimensional complex (real) inner product space, and

suppose T ∞ L(V). Prove that V has an orthonormal basis of eigenvectors
of T with corresponding eigenvalues of absolute value 1 if and only if T is
unitary (Hermitian and orthogonal).
522 LINEAR OPERATORS

5. For each of the following matrices A, find an orthogonal or unitary matrix

P and a diagonal matrix D such that P¿AP = D:

!1 2$ " 0 !1 % " 2 3 ! i3%

(a)!!# & (b)!!$ ' (c)!!$ '
"2 1% # 1 !!0 & # 3 + i3 5 &

!0 2 2$ !2 1 1$
# & # &
(d)!!!# 2 0 2 & (e)!!!# 1 2 1 &
# & # &
"2 2 0% "1 1 2%

6. Let A, B and C be normal operators on a finite-dimensional unitary space,

and assume that [A, B] = 0 but [B, C] ≠ 0. If all of these operators are
nondegenerate (i.e., all eigenvalues have multiplicity equal to 1), is it true
that [A, C] ≠ 0? Explain. What if any of these are degenerate?

7. Let V be a finite-dimensional unitary space and suppose A ∞ L(V).

(a) Prove that Tr(AA¿) = 0 if and only if A = 0.
(b) Suppose N ∞ L(V) is normal and AN = NA. Prove that AN¿ = N¿A.

8. Let A be a positive definite real symmetric matrix on an n-dimensional

Euclidean space V. Using the single variable formula (where a > 0)

"
# !" exp(!ax 2 2)dx = (2 $ a)1 2
show that
# ! !
$ !# exp[(!1 2)Óx,!AxÔ]d n x = (2" )n 2 (det A)!1 2
where dn x = dx1 ~ ~ ~ dxn. [Hint: First consider the case where A is
diagonal.]

9. (This is an independent proof of Corollary 2 of Theorem 10.15.) Let A =

(aáé) ∞ M3(®) be a real symmetric matrix. Thus A: ®3 ‘ ®3 is a Hermitian
linear operator with respect to the inner product Ó , Ô. Prove there exists an
orthonormal basis of eigenvectors of A using the following approach. (It
should be clear after you have done this that the same proof will work in
®n just as well.)
(a) Let S2 be the unit sphere in ®3, and define f: S2 ‘ ® by

f(x) = ÓAx, xÔ .
 10.4 DIAGONALIZATION OF NORMAL OPERATORS 523

Let M = sup f(x) and m = inf f(x) where the sup and inf are taken over S2.
Show that there exist points x1, x1æ ∞ S2 such that f(x1) = M and f(x1æ) = m.
[Hint: Use Theorem A15.]
(b) Let C = x(t) be any curve on S2 such that x(0) = x1, and let a dot
denote differentiation with respect to the parameter t. Note that xã(t) is
tangent to C, and hence also to S2. Show that ÓAx1, xã(0)Ô = 0, and thus
deduce that Ax1 is normal to the tangent plane at x1. [Hint: Consider
df(x(t))/dt\t=0 and note that C is arbitrary.]
(c) Show that Óxã(t), x(t)Ô = 0, and hence conclude that Ax1 = ¬1x1. [Hint:
Recall that S2 is the unit sphere.]
(d) Argue that Ax1æ = ¬1æx1æ, and in general, that any critical point of f(x) =
ÓAx, xÔ on the unit sphere will be an eigenvector of A with critical value
(i.e., eigenvalue) ¬i = ÓAxi, xiÔ. (A critical point of f(x) is a point x0 where
df/dx = 0, and the critical value of f is just f(x0).)
(e) Let [x1] be the 1-dimensional subspace of ®3 spanned by x1. Show that
[x1] and [x1]Ê are both A-invariant subspaces of ®3, and hence that A is
Hermitian on [x1]Ê ™ ®2. Note that [x1]Ê is a plane through the origin of
S2.
(f ) Show that f now must achieve its maximum at a point x2 on the unit
circle S1 ™ [x1]Ê, and that Ax2 = ¬2x2 with ¬2 ¯ ¬1.
(g) Repeat this process again by considering the space [x2]Ê ™ [x1]Ê, and
show there exists a vector x3 ∞ [x2]Ê with Ax3 = ¬3x3 and ¬3 ¯ ¬2 ¯ ¬1.

10.5 THE SPECTRAL THEOREM

We now turn to another major topic of this chapter, the so-called spectral
theorem. This important result is actually nothing more than another way of
looking at Theorems 8.2 and 10.15. We begin with a simple version that is
easy to understand and visualize if the reader will refer back to the discussion
prior to Theorem 7.29.

Theorem 10.20 Suppose A ∞ Mñ(ç) is a diagonalizable matrix with distinct

eigenvalues ¬è, . . . , ¬r. Then A can be written in the form

A = ¬èEè + ~ ~ ~ + ¬rEr

where the Eá are n x n matrices with the following properties:

(a) Each Eá is idempotent (i.e., Eá2 = Eá).
(b) EáEé = 0 for i ≠ j.
524 LINEAR OPERATORS

(c) Eè + ~ ~ ~ + Er = I.
(d) AEá = EáA for every Eá.

Proof Since A is diagonalizable by assumption, let D = PîAP be the diago-

nal form of A for some nonsingular matrix P (whose columns are just the
eigenvectors of A). Remember that the diagonal elements of D are just the
eigenvalues ¬á of A. Let Pá be the n x n diagonal matrix with diagonal element
1 wherever a ¬á occurs in D, and 0’s everywhere else. It should be clear that
the collection {Pá} obeys properties (a) - (c), and that

PîAP = D = ¬èPè + ~ ~ ~ + ¬rPr .

If we now define Eá = PPáPî, then we have

A = PDPî = ¬èEè + ~ ~ ~ + ¬rEr

where the Eá also obey properties (a) - (c) by virtue of the fact that the Pá do.
Using (a) and (b) in this last equation we find

AEá = (¬èEè + ~ ~ ~ + ¬rEr)Eá = ¬áEá

and similarly it follows that EáA = ¬áEá so that each Eá commutes with A, i.e.,
EáA = AEá. ˙

By way of terminology, the collection of eigenvalues ¬1, . . . , ¬r is called

the spectrum of A, the sum E1 + ~ ~ ~ + Er = I is called the resolution of the
identity induced by A, and the expression A = ¬èEè + ~ ~ ~ + ¬rEr is called the
spectral decomposition of A. These definitions also apply to arbitrary normal
operators as in Theorem 10.22 below.

Corollary Let A be diagonalizable with spectral decomposition as in

Theorem 10.20. If f(x) ∞ ç[x] is any polynomial, then

f(A) = f(¬è)Eè + ~ ~ ~ + f(¬r)Er .

Proof Using properties (a) - (c) in Theorem 10.20, it is easy to see that for
any m > 0 we have
Am = ¬èm Eè + ~ ~ ~ + ¬rm Er .

The result for arbitrary polynomials now follows easily from this result. ˙
 10.5 THE SPECTRAL THEOREM 525

Before turning to our proof of the spectral theorem, we first prove a simple
but useful characterization of orthogonal projections.

Theorem 10.21 Let V be an inner product space and suppose E ∞ L(V).

Then E is an orthogonal projection if and only if E2 = E = E¿.

Proof We first assume that E is an orthogonal projection. By definition this

means that E2 = E, and hence we must show that E¿ = E. From Theorem 7.27
we know that V = Im E • Ker E = Im E • (Im E)Ê. Suppose v, w ∞ V are
arbitrary. Then we may write v = vè + vì and w = wè + wì where vè, wè ∞ Im E
and vì, wì ∞ (Im E)Ê. Therefore

Óv, EwÔ = Óvè + vì, wèÔ = Óvè, wèÔ + Óvì, wèÔ = Óvè, wèÔ
and

Óv, E¿wÔ = ÓEv, wÔ = Óvè, wè + wìÔ = Óvè, wèÔ + Óvè, wìÔ = Óvè, wèÔ .

In other words, Óv, (E - E¿)wÔ = 0 for all v, w ∞ V, and hence E = E¿ (by

Theorem 10.4(a)).
On the other hand, if E2 = E = E¿, then we know from Theorem 7.27 that
E is a projection of V on Im E in the direction of Ker E, i.e., V = Im E •
Ker E. Therefore, we need only show that Im E and Ker E are orthogonal
subspaces. To show this, let w ∞ Im E and wæ ∞ Ker E be arbitrary. Then
Ew = w and Ewæ = 0 so that

Ówæ, wÔ = Ówæ, EwÔ = ÓE¿wæ, wÔ = ÓEwæ, wÔ = 0 .

(This was also proved independently in Exercise 10.3.5.) ˙

We are now in a position to prove the spectral theorem for normal opera-
tors. In order to distinguish projection operators from their matrix representa-
tions in this theorem, we denote the operators by πá and the corresponding
matrices by Eá.

Theorem 10.22 (Spectral Theorem for Normal Operators) Let V be a

finite-dimensional unitary space, and let N be a normal operator on V with
distinct eigenvalues ¬è, . . . , ¬r. Then
(a) N = ¬è πè + ~ ~ ~ + ¬r πr where each πá is the orthogonal projection of V
onto a subspace Wá = Im πá.
(b) πáπé = 0 for i ≠ j.
526 LINEAR OPERATORS

(c) πè + ~ ~ ~ + πr = 1.
(d) V = Wè • ~ ~ ~ • Wr where the subspaces Wá are mutually orthogonal.
(e) Wé = Im πé = Ker(N - ¬é1) is the eigenspace corresponding to ¬é.

Proof Choose any orthonormal basis {eá} for V, and let A be the matrix rep-
resentation of N relative to this basis. As discussed following Theorem 7.6,
the normal matrix A has the same eigenvalues as the normal operator N. By
Corollary 1 of Theorem 10.15 we know that A is diagonalizable, and hence
applying Theorem 10.20 we may write

A = ¬èEè + ~ ~ ~ + ¬rEr

where Eá2 = Eá, EáEé = 0 if i ≠ j, and Eè + ~ ~ ~ + Er = I. Furthermore, A is

diagonalized by a unitary matrix P, and as we saw in the proof of Theorem
10.20, Eá = PPáP¿ where each Pá is a real diagonal matrix. Since each Pá is
clearly Hermitian, this implies that Eá¿ = Eá, and hence each Eá is an orthogo-
nal projection (Theorem 10.21).
Now define πá ∞ L(V) as that operator whose matrix representation
relative to the basis {eá} is just Eá. From the isomorphism between linear
transformations and their representations (Theorem 5.13), it should be clear
that
N = !1"1 +!+ !r " r
" i† = " i
"i2 = "i
" i " j = 0 for i # j
"1 +!+ " r = 1!!.

Since πá2 = πá = πá¿, Theorem 10.21 tells us that each πá is an orthogonal

projection of V on the subspace Wá = Im πá. Since πè + ~ ~ ~ + πr = 1, we see
that for any v ∞ V we have v = πèv + ~ ~ ~ + πrv so that V = Wè + ~ ~ ~ + Wr . To
show that this sum is direct suppose, for example, that

wè ∞ Wè ⁄ (Wì + ~ ~ ~ + Wr) .

This means that wè = wì + ~ ~ ~ + wr where wá ∞ Wá for each i = 1, . . . , r. Since

wá ∞ Wá = Im πá, it follows that there exists vá ∞ V such that πává = wá for each
i. Then
wá = πává = πá2 vá = πáwá

and if i ≠ j , then πáπé = 0 implies

 10.5 THE SPECTRAL THEOREM 527

πáwé = (πáπé)vé = 0 .

Applying πè to wè = wì + ~ ~ ~ + wr, we obtain wè = πèwè = 0. Hence we have

shown that Wè ⁄ (Wì + ~ ~ ~ + Wr) = {0}. Since this argument can clearly be
applied to any of the Wá , we have proved that V = Wè • ~ ~ ~ • Wr.
Next we note that for each i, πá is the orthogonal projection of V on Wá =
Im πá in the direction of WáÊ = Ker πá, so that V = Wá • WáÊ. Therefore, since
V = Wè • ~ ~ ~ • Wr, it follows that for each j ≠ i we must have Wé ™ WáÊ, and
hence the subspaces Wá must be mutually orthogonal. Finally, the fact that
Wé = Ker(N - ¬é1) was proved in Theorem 7.29. ˙

The observant reader will have noticed the striking similarity between the
spectral theorem and Theorem 7.29. In fact, part of Theorem 10.22 is essen-
tially a corollary of Theorem 7.29. This is because a normal operator is diag-
onalizable, and hence satisfies the hypotheses of Theorem 7.29. However,
note that in the present case we have used the existence of an inner product in
our proof, whereas in Chapter 7, no such structure was assumed to exist. We
leave it to the reader to use Theorems 10.15 and 7.28 to construct a simple
proof of the spectral theorem that makes no reference to any matrix represen-
tation of the normal operator (see Exercise 10.5.1).

Theorem 10.23 Let ÍjÂ=1¬éEé be the spectral decomposition of a normal oper-

ator N on a finite-dimensional unitary space. Then for each i = 1, . . . , r there
exists a polynomial fá(x) ∞ ç[x] such that fá(¬é) = ∂áé and fá(N) = Eá.

Proof For each i = 1, . . . , r we must find a polynomial fá(x) ∞ ç[x] with the
property that fá(¬é) = ∂áé. It should be obvious that the polynomials fá(x)
defined by
x ! "j
fi (x) = $
" ! "j
j#i i

have this property. From the corollary to Theorem 10.20 we have p(N) =
Íép(¬é)Eé for any p(x) ∞ ç[x], and hence

fá(N) = Íéfá(¬é)Eé = Íé∂áéEé = Eá

as required. ˙
528 LINEAR OPERATORS

Exercises

1. Use Theorems 10.15 and 7.28 to construct a proof of Theorem 10.22 that
makes no reference to any matrix representations.

2. Let N be an operator on a finite-dimensional unitary space. Prove that N is

normal if and only if N¿ = g(N) for some polynomial g. [Hint: If N is
normal with eigenvalues ¬è, . . . , ¬r, use Exercise 6.4.2 to show the exis-
tence of a polynomial g such that g(‡) = ‡* for each i.]

3. Let T be an operator on a finite-dimensional unitary space. Prove that T is

unitary if and only if T is normal and \¬\ = 1 for every eigenvalue ¬ of T.

4. Let H be a normal operator on a finite-dimensional unitary space. Prove

that H is Hermitian if and only if every eigenvalue of H is real.

10.6 THE MATRIX EXPONENTIAL SERIES

We now use Theorem 10.20 to prove a very useful result, namely, that any
unitary matrix U can be written in the form e^H for some Hermitian matrix H.
Before proving this however, we must first discuss some of the theory of
sequences and series of matrices. In particular, we must define just what is
meant by expressions of the form e^H. If the reader already knows something
about sequences and series of numbers, then the rest of this section should
present no difficulty. However, for those readers who may need some review,
we have provided all of the necessary material in Appendix B.
Let {Sr} be a sequence of complex matrices where each Sr ∞ Mñ(ç) has
entries s(r)áé. We say that {Sr} converges to the limit S = (sáé) ∞ Mñ(ç) if each
of the n2 sequences {s(r)áé} converges to a limit sáé. We then write Sr ‘ S or
limr ‘Ÿ Sr = S (or even simply lim Sr = S). In other words, a sequence {Sr} of
matrices converges if and only if every entry of Sr forms a convergent
sequence.
Similarly, an infinite series of matrices
!
" Ar
r=1
where Ar = (a(r)áé) is said to be convergent to the sum S = (sáé) if the sequence
of partial sums
m
Sm = ! Ar
r=1
 10.6 THE MATRIX EXPONENTIAL SERIES 529

converges to S. Another way to say this is that the series ÍAr converges to S
if and only if each of the n2 series Ía(r)áé converges to sáé for each i, j = 1, . . . ,
n. We adhere to the convention of leaving off the limits in a series if they are
infinite.
Our next theorem proves several intuitively obvious properties of
sequences and series of matrices.

Theorem 10.24 (a) Let {Sr} be a convergent sequence of n x n matrices

with limit S, and let P be any n x n matrix. Then PSr ‘ PS and SrP ‘ SP.
(b) If Sr ‘ S and P is nonsingular, then PîSrP ‘ PîSP.
(c) If ÍAr converges to A and P is nonsingular, then ÍPîArP converges
to PîAP.

Proof (a) Since Sr ‘ S, we have lim s(r)áé = sáé for all i, j = 1, . . . , n.

Therefore

lim(PSr)áé = lim(ÍÉpáÉs(r)Éé) = ÍÉpáÉlim s(r)Éé = ÍÉpáÉsÉé = (PS)áé .

Since this holds for all i, j = 1, . . . , n we must have PSr ‘ PS. It should be
obvious that we also have Sr P ‘ SP.
(b) As in part (a), we have

lim(P !1Sr P)ij = lim( "k, m p!1ik s (r) km pmj )

= "k, m p!1ik pmj lim s (r) km
= "k, m p!1ik pmj skm
= (P !1SP)ij !!.

Note that we may use part (a) to formally write this as

lim(PîSrP) = Pîlim(SrP) = PîSP .

(c) If we write the mth partial sum as

m #m &
!1 !1
Sm = " P Ar P = P %%" Ar (( P
r=1 $ r=1 '

then we have
530 LINEAR OPERATORS

+- %m ( /-
#1 (r)
lim m!" (Sm )ij = $ lim , p ik '$ a kl ** plj 0
'
k, l -. & r=1 ) -1
m
= $ p#1ik plj lim $ a(r) kl
k, l r=1

=$ p#1ik plj akl

k, l

= P #1 AP!!.!!˙

Theorem 10.25 For any A = (aáé) ∞ Mñ(ç) the following series converges:
!
A A2 Ar
" r!r = I + A + 2! +!+ r! +!!.
r=0

Proof Choose a positive real number M > max{n, \aáé\} where the max is
taken over all i, j = 1, . . . , n. Then \aáé\ < M and n < M < M2. Now consider
the term A2 = (báé) = (ÍÉaáÉaÉé). We have (by Theorem 2.17, property (N3))
n n
|bij | ! " |aik |!|akj | < " M 2 = nM 2 < M 4 !!.
k=1 k=1
Proceeding by induction, suppose that for Ar = (cáé), it has been shown that
\cáé\ < M2r. Then Ar+1 = (dáé) where
n
|dij | ! " |aik ||ckj | < nMM 2r = nM 2r+1 < M 2(r+1) !!.
k=1
This proves that Ar = (a(r)áé) has the property that \a(r)áé\ < M2r for every r ˘ 1.
Now, for each of the n2 terms i, j = 1, . . . , n we have
!
|a(r)ij | ! M 2r
" r! < " r! = exp(M 2 )
r=0 r=0
so that each of these n2 series (i.e., for each i, j = 1, . . . , n) must converge
(Theorem B26(a)). Hence the series I + A + A2/2! + ~ ~ ~ must converge
(Theorem B20). ˙

We call the series in Theorem 10.25 the matrix exponential series, and
denote its sum by eA = exp A. In general, the series for eA is extremely diffi-
cult, if not impossible, to evaluate. However, there are important exceptions.
 10.6 THE MATRIX EXPONENTIAL SERIES 531

Example 10.7 Let A be the diagonal matrix

" !1 0 ! 0 %
$ '
$ 0 !2 ! 0 '
A= !!.
$" " " '
$ '
# 0 0 ! !n &

Then it is easy to see that

"! r 0 ! 0 %
$ 1 '
$ 0 !2 r ! 0 '
Ar = $ '
$ " " " '
$ '
# 0 0 ! !n r &

and hence
" e!1 0 ! 0 %
$ '
A2 $ 0 e !2 ! 0 '
exp A = I + A + +!!= $ '!!.!!!
2! $ " " " '
$ '
# 0 0 ! e !n &

Example 10.8 Consider the 2 x 2 matrix

" 0 !1 %
J =$ '
# 1 !!0 &

and let
# 0 "! &
A = !J = % (
$! !!0 '

where œ ∞ ®. Then noting that J2 = -I, we see that A2 = -œ2I, A3 = -œ3J, A4 =

œ4I, A5 = œ5J, A6 = -œ6I, and so forth. From elementary calculus we know
that
sin œ = œ - œ3/3! + œ5/5! - ~ ~ ~
and
cos œ = 1 - œ2/2! + œ4/4! - ~ ~ ~

and hence
532 LINEAR OPERATORS

e A = I + A + A 2 2!+!
= I + ! J " ! 2 I 2!" ! 3 J 3!+ ! 4 I 4!+ ! 5 J 5!" ! 6 I 6!+!
= I(1 " ! 2 2!+ ! 4 4!"!) + J(! " ! 3 3!+ ! 5 5!"!)
= (cos! )I + (sin ! )J !!.

In other words, using the explicit forms of I and J we see that

# cos! "sin ! &

eA = % (
$ sin ! cos! '

so that eœJ represents a rotation in ®2 by an angle œ. ∆

Theorem 10.26 Let A ∞ Mñ(ç) be diagonalizable, and let ¬è, . . . , ¬r be the

distinct eigenvalues of A. Then the matrix power series
!
" as A s
s=0
converges if and only if the series
"
# as !i s
s=0
converges for each i = 1, . . . , r.

Proof Since A is diagonalizable, choose a nonsingular matrix P such that D =

PîAP is diagonal. It is then easy to see that for every s ˘ 1 we have

asDs = asPîAsP = PîasAsP

where the n diagonal entries of Ds are just the numbers ‡s. By Theorem
10.24(c), we know that ÍasAs converges if and only if ÍasDs converges. But
by definition of series convergence, ÍasDs converges if and only if Ías¬ás
converges for every i = 1, . . . , r. ˙

Theorem 10.27 Let f(x) = aà + aèx + aìx2 + ~ ~ ~ be any power series with
coefficients in ç, and let A ∞ Mñ(ç) be diagonalizable with spectral decom-
position A = ¬èEè + ~ ~ ~ + ¬rEr. Then, if the series

f(A) = aàI + aèA + aìA2 + ~ ~ ~

converges, its sum is

 10.6 THE MATRIX EXPONENTIAL SERIES 533

f(A) = f(¬è)Eè + ~ ~ ~ + f(¬r)Er .

Proof As in the proof of Theorem 10.20, let the diagonal form of A be

D = PîAP = ¬èPè + ~ ~ ~ + ¬rPr

so that Eá = PPáPî. Now note that

P !1 f (A)P = a0 P !1P + a1P !1 AP + a2 P !1 APP !1 AP +!

= f (P !1 AP)
= a0 I + a1 D + a2 D 2 +!
= f (D)!!.

Using properties (a) - (c) of Theorem 10.20 applied to the Pá, it is easy to see
that Dk = ¬èk Pè + ~ ~ ~ + ¬rk Pr and hence

f(D) = f(¬è)Pè + ~ ~ ~ + f(¬r)Pr .

Then if f(A) = ÍAr converges, so does ÍPîArP = Pîf(A)P = f(D) (Theorem

10.24(c)), and we have

f(A) = f(PDPî) = Pf(D)Pî = f(¬è)Eè + ~ ~ ~ + f(¬r)Er . ˙

Example 10.9 Consider the exponential series eA where A is diagonalizable.

Then, if ¬è, . . . , ¬k are the distinct eigenvalues of A, we have the spectral
decomposition A = ¬èEè + ~ ~ ~ + ¬ÉEÉ. Using f(A) = eA, Theorem 10.27 yields

eA = e¬èEè + ~ ~ ~ + e¬ÉEÉ

in agreement with Example 10.7. ∆

We can now prove our earlier assertion that a unitary matrix U can be
written in the form e^H for some Hermitian matrix H.

Theorem 10.28 Every unitary matrix U can be written in the form e^H for
some Hermitian matrix H. Conversely, if H is Hermitian, then e^H is unitary.

Proof By Theorem 10.9(b), the distinct eigenvalues of U may be written in

the form e^, . . . , e^ƒ where each ‡ is real. Since U is also normal, it fol-
534 LINEAR OPERATORS

lows from Corollary 1 of Theorem 10.15 that there exists a unitary matrix P
such that P¿UP = PîUP is diagonal. In fact

PîUP = e^¬èPè + ~ ~ ~ + e^¬ÉPÉ

where the Pá are the idempotent matrices used in the proof of Theorem 10.20.
From Example 10.7 we see that the matrix e^¬èPè + ~ ~ ~ + e^¬ÉPÉ is just e^D
where
D = ¬èPè + ~ ~ ~ + ¬ÉPÉ

is a diagonal matrix with the ‡ as diagonal entries. Therefore, using Theorem

10.24(c) we see that

U = Pe^ D Pî = e^ P D Pî = e^ H

where H = PDPî. Since D is a real diagonal matrix it is clearly Hermitian, and

since P is unitary (so that Pî = P¿), it follows that H¿ = (PDP¿)¿ = PDP¿ = H
so that H is Hermitian also.
Conversely, suppose H is Hermitian with distinct real eigenvalues , . . . ,
ƒ. Since H is also normal, there exists a unitary matrix P that diagonalizes H.
Then as above, we may write this diagonal matrix as

PîHP = ¬èPè + ~ ~ ~ + ¬ÉPÉ

so that (from Example 10.7 again)

Pîe^HP = e^PîHP = e^¬èPè + ~ ~ ~ + e^¬ÉPÉ .

Using the properties of the Pá, it is easy to see that the right hand side of this
equation is diagonal and unitary since using

(e^¬èPè + ~ ~ ~ + e^¬ÉPÉ)¿ = e-^¬èPè + ~ ~ ~ + e-^¬ÉPÉ

we have
(e^¬èPè + ~ ~ ~ + e^¬ÉPÉ)¿(e^¬èPè + ~ ~ ~ + e^¬ÉPÉ) = I

and
(e^¬èPè + ~ ~ ~ + e^¬ÉPÉ)(e^¬èPè + ~ ~ ~ + e^¬ÉPÉ)¿ = I .

Therefore the left hand side must also be unitary, and hence (using Pî = P¿)
 10.6 THE MATRIX EXPONENTIAL SERIES 535

I = (P !1eiH P)† (P !1eiH P)

= P † (eiH )† PP !1eiH P
= P † (eiH )† eiH P

so that PPî = I = (e^H)¿e^H. Similarly we see that e^H(e^H)¿ = I, and thus e^H is
unitary. ˙

While this theorem is also true in infinite dimensions (i.e., in a Hilbert

space), its proof is considerably more difficult. The reader is referred to the
books listed in the bibliography for this generalization.
Given a constant matrix A, we now wish to show that

detA
= AetA !!. (1)
dt

To see this, we first define the derivative of a matrix M = M(t) to be that

matrix whose elements are just the derivatives of the corresponding elements
of M. In other words, if M(t) = (máé(t)), then (dM/dt)áé = dmáé/dt. Now note that
(with M(t) = tA)

etA = I + tA + (tA)2/2! + (tA)3/3! + ~ ~ ~

and hence (since the aáé are constant) taking the derivative with respect to t
yields the desired result:

detA /dt = 0 + A + tA 2 + (tA)2 A/2! +!

= A{I + tA + (tA)2 /2! +!}
= AetA !!.

Next, given two matrices A and B (of compatible sizes), we recall that
their commutator is the matrix [A, B] = AB - BA = -[B, A]. If [A, B] = 0,
then AB = BA and we say that A and B commute. Now consider the function
f(x) = exABe-xA. Leaving it to the reader to verify that the product rule for
derivatives also holds for matrices, we obtain (note that AexA = exAA)

df /dx = Ae xA Be!xA ! e xA Be!xA A = Af ! fA = [A,! f ]

d 2 f /dx 2 = [A,!df /dx] = [A,![A,! f ]]
!
536 LINEAR OPERATORS

Expanding f(x) in a Taylor series about x = 0, we find (using f(0) = B)

f (x) = f (0) + (df /dx)0 x + (d 2 f /dx 2 )0 x 2 /2! +!

= B + [A,!B]x + [A,![A,!B]]x 2 /2! +!!.

Setting x = 1, we finally obtain

e A Be!A = B + [A,!B] + [A,![A,!B]]/2! + [A,![A,![A,!B]]]/3! +! (2)

Note that setting B = I shows that eAe-A = I as we would hope.

In the particular case that both A and B commute with their commutator
[A, B], then we find from (2) that eABe-A = B + [A, B] and hence eAB =
BeA + [A, B]eA or
[e A ,!B] = [A,!B]e A !!. (3)

Example 10.10 We now show that if A and B are two matrices that both
commute with their commutator [A, B], then

e A e B = exp{A + B + [A,!B]/2}!!. (4)

(This is sometimes referred to as Weyl’s formula.) To prove this, we start

with the function f(x) = exAexBe-x(A+B). Then

df /dx = e xA Ae xB e!x(A+B) + e xA e xB Be!x(A+B) ! e xA e xB (A + B)e!x(A+B)

= e xA Ae xB e!x(A+B) ! e xA e xB Ae!x(A+B) (5)
= e xA [A,!e xB ]e!x(A+B)

As a special case, note [A, B] = 0 implies df/dx = 0 so that f is independent of

x. Since f(0) = I, it follows that we may choose x = 1 to obtain eAeBe-(A+B) =
I or eAeB = eA+B (as long as [A, B] = 0).
From (3) we have (replacing A by xB and B by A) [A, exB] = x[A, B]exB.
Using this along with the fact that A commutes with the commutator [A, B]
(so that exA[A, B] = [A, B]exA), we have

df/dx = xexA[A, B]exBe-x(A+B) = x[A, B]f .

Since A and B are independent of x, we may formally integrate this from 0 to

x to obtain
ln f(x)/f(0) = [A, B]x2/2 .
 10.6 THE MATRIX EXPONENTIAL SERIES 537

Using f(0) = 1, this is f(x) = exp{[A, B]x2/2} so that setting x = 1 we find

eAeBe-(A+B) = exp{[A, B]/2} .

Finally, multiplying this equation from the right by eA+B and using the fact
that [[A, B]/2, A + B] = 0 yields (4). ∆

Exercises

1. (a) Let N be a normal operator on a finite-dimensional unitary space.

Prove that
det eN = eTr N .

(b) Prove this holds for any N ∞ Mn(ç). [Hint: Use either Theorem 8.1 or
the fact (essentially proved at the end of Section 8.6) that the diagonaliz-
able matrices are dense in Mn(ç).]

2. If the limit of a sequence of unitary operators exists, is it also unitary?

Why?

3. Let T be a unitary operator. Show that the sequence {Tn: n = 0, 1, 2, . . . }

contains a subsequence {TnÉ: k = 0, 1, 2, . . . } that converges to a unitary
operator. [Hint: You will need the fact that the unit disk in ç2 is compact
(see Appendix A).]

10.7 POSITIVE OPERATORS

Before proving the main result of this section (the polar decomposition
theorem), let us briefly discuss functions of a linear transformation. We have
already seen two examples of such a function. First, the exponential series e A
(which may be defined for operators exactly as for matrices) and second, if A
is a normal operator with spectral decomposition A = ͬáEá, then we saw that
the linear transformation p(A) was given by p(A) = Íp(¬á)Eá where p(x) is any
polynomial in ç[x] (Corollary to Theorem 10.20).
In order to generalize this notion, let N be a normal operator on a unitary
space, and hence N has spectral decomposition ͬáEá. If f is an arbitrary
complex-valued function (defined at least at each of the ‡), we define a linear
transformation f(N) by
f(N) = Íf(¬á)Eá .
538 LINEAR OPERATORS

What we are particularly interested in is the function f(x) = “x” defined for all
real x ˘ 0 as the positive square root of x.
Recall (see Section 10.3) that we defined a positive operator P by the
requirement that P = S¿S for some operator S. It is then clear that P¿ = P, and
hence P is normal. From Theorem 10.9(d), the eigenvalues of P = ͬéEé are
real and non-negative, and we can define “P” by

“P” = Í铬”é” Eé
where each ¬é ˘ 0.
Using the properties of the Eé, it is easy to see that (“P”)2 = P. Furthermore,
since Eé is an orthogonal projection, it follows that Eé¿ = Eé (Theorem 10.21),
and therefore (“P”)¿ = “P” so that “P” is Hermitian. Note that since P = S¿S we
have
ÓPv, vÔ = Ó(S¿S)v, vÔ = ÓSv, SvÔ = ˜ Sv ˜ 2 ˘ 0 .

Just as we did in the proof of Theorem 10.23, let us write v = ÍEév = Ívé
where the nonzero vé are mutually orthogonal. Then

“P”(v) = Í“¬”é” Eév = Í“¬”é” vé

and hence we also have (using Óvé, vÉÔ = 0 if j ≠ k)

Ó“P” (v),!vÔ = Ó! j “¬”é!v j ,! !k vkÔ = ! j, k “¬”é!Óv j ,!vkÔ = ! j “¬”é Óv j ,!v jÔ

= ! j “¬”é ˜v j ˜2 " 0!!.

In summary, we have shown that “P” satisfies

(a) (“P”)2 = P
(b) (“P”)¿ = “P”
(c) Ó“P”(v), vÔ ˘ 0

and it is natural to ask about the uniqueness of any operator satisfying these
three properties. For example, if we let T = Í ±“¬”é” Eé, then we still have T 2 =
Í ¬éEé = P regardless of the sign chosen for each term. Let us denote the fact
that “P” satisfies properties (b) and (c) above by the expression “P” ˘ 0. In
other words, by the statement A ˘ 0 we mean that A¿ = A and ÓAv, vÔ ˘ 0 for
every v ∞ V (i.e., A is a positive Hermitian operator).
We now claim that if P = T2 and T ˘ 0, then T = “P”. To prove this, we
first note that T ˘ 0 implies T¿ = T (property (b)), and hence T must also be
normal. Now let ͵áFá be the spectral decomposition of T. Then
 10.7 POSITIVE OPERATORS 539

Í(µá)2Fá = T2 = P = ͬéEé .

If vá ≠ 0 is an eigenvector of T corresponding to µá, then property (c) tells us

that (using the fact that each µá is real since T is Hermitian)

0 ¯ ÓTvá, váÔ = Óµává, váÔ = µá ˜ vá ˜ 2 .

But ˜ vá ˜ > 0, and hence µá ˘ 0. In other words, any operator T ˘ 0 has non-
negative eigenvalues. Since each µá is distinct and nonnegative, so is each µá2,
and hence each µá2 must be equal to some ¬é . Therefore the corresponding Fá
and Eé must be equal (by Theorem 10.22(e)). By suitably numbering the
eigenvalues, we may write µá2 = ¬á, and thus µá = “¬”á” . This shows that

T = ͵áFá = Í“¬”á” Eá = “P”

as claimed.
We summarize this discussion in the next result which gives us three
equivalent definitions of a positive transformation.

Theorem 10.29 Let P be an operator on a unitary space V. Then the follow-

ing conditions are equivalent:
(a) P = T2 for some unique Hermitian operator T ˘ 0.
(b) P = S¿S for some operator S.
(c) P¿ = P and ÓPv, vÔ ˘ 0 for every v ∞ V.

Proof (a) ¶ (b): If P = T2 and T¿ = T, then P = TT = T¿T.

(b) ¶ (c): If P = S¿S, then P¿ = P and ÓPv, vÔ = ÓS¿Sv, vÔ = ÓSv, SvÔ =
˜ Sv ˜ 2 ˘ 0.
(c) ¶ (a): Note that property (c) is just our statement that P ˘ 0. Since
P¿ = P, we see that P is normal, and hence we may write P = ͬéEé. Defining
T = Í“¬”é” Eé , we have T¿ = T (since every Eé is Hermitian), and the preceding
discussion shows that T ˘ 0 is the unique operator with the property that P =
T2. ˙

We remark that in the particular case that P is positive definite, then P =

S¿S where S is nonsingular. This means that P is also nonsingular.
Finally, we are in a position to prove the last result of this section, the so-
called polar decomposition (or factorization) of an operator. While we state
and prove this theorem in terms of matrices, it should be obvious by now that
it applies just as well to operators.
540 LINEAR OPERATORS

Theorem 10.30 (Polar Decomposition) If A ∞ Mñ(ç), then there exist

unique positive Hermitian matrices Hè, Hì ∞ Mñ(ç) and (not necessarily
unique) unitary matrices Uè, Uì ∞ Mñ(ç) such that A = UèHè = HìUì. More-
over, Hè = (A¿A)1/2 and Hì = (AA¿)1/2 . In addition, the matrices Uè and Uì
are uniquely determined if and only if A is nonsingular.

Proof Let 2, . . . , –2 be the eigenvalues of the positive Hermitian matrix
A¿A, and assume the ¬á are numbered so that ¬á > 0 for i = 1, . . . , k and ¬á = 0
for i = k + 1, . . . , n (see Theorem 10.9(d)). (Note that if A is nonsingular, then
A¿A is positive definite and hence k = n.) Applying Corollary 1 of Theorem
10.15, we let {vè, . . . , vñ} be the corresponding orthonormal eigenvectors of
A¿A. For each i = 1, . . . , k we define the vectors wá = Avá/¬á. Then

Ówi ,!w j Ô = ÓAvi /!i ,!Av j /! j Ô = Óvi ,!A† Av j Ô/!i ! j

= Óvi ,!v j Ô! j 2 /!i ! j = "ij ! j 2 /!i ! j

so that wè, . . . , wÉ are also orthonormal. We now extend these to an ortho-

normal basis {wè, . . . , wñ} for çn. If we define the columns of the matrices
V, W ∞ Mñ(ç) by Vi = vá and Wi = wá, then V and W will be unitary by
Theorem 10.7.
Defining the Hermitian matrix D ∞ Mñ(ç) by

D = diag(, . . . , –)

it is easy to see that the equations Avá = ¬áwá may be written in matrix form as
AV = WD. Using the fact that V and W are unitary, we define Uè = WV¿ and
Hè = VDV¿ to obtain

A = WDV¿ = (WV¿)(VDV¿) = UèHè .

Since det(¬I - VDV¿) = det(¬I - D), we see that Hè and D have the same
nonnegative eigenvalues, and hence Hè is a positive Hermitian matrix. We can
now apply this result to the matrix A¿ to write A¿ = UÿèHÿè or A = Hÿè¿Uÿè¿ =
HÿèUÿè¿. If we define Hì = Hÿè and Uì = Uÿè¿, then we obtain A = HìUì as
desired.
We now observe that using A = UèHè we may write

A¿A = HèUè¿UèHè = (Hè)2

and similarly
AA¿ = HìUìUì¿Hì = (Hì)2
 10.7 POSITIVE OPERATORS 541

so that Hè and Hì are unique even if A is singular. Since Uè and Uì are

unitary, they are necessarily nonsingular, and hence Hè and Hì are nonsingular
if A = UèHè = HìUì is nonsingular. In this case, Uè = AHèî and Uì = HìîA
will also be unique. On the other hand, suppose A is singular. Then k ≠ n and
wÉ, . . . , wñ are not unique. This means that Uè = WV¿ (and similarly Uì) is
not unique. In other words, if Uè and Uì are unique, then A must be non-
singular. ˙

Exercises

1. Let V be a unitary space and let E ∞ L(V) be an orthogonal projection.

(a) Show directly that E is a positive transformation.
(b) Show that ˜ Ev ˜ ¯ ˜ v ˜ for all v ∞ V.

2. Prove that if A and B are commuting positive transformations, then AB is

also positive.

3. This exercise is related to Exercise 7.5.5. Prove that any representation of

a finite group is equivalent to a unitary representation as follows:
(a) Consider the matrix X = Ía´G D¿(a)D(a). Show that X is Hermitian
and positive definite, and hence that X = S2 for some Hermitian S.
(b) Show that D(a)¿XD(a) = X.
(c) Show that U(a) = SD(a)Sî is a unitary representation.

Supplementary Exercises for Chapter 10

1. Let T be a linear transformation on a space V with basis {eè, . . . , en}. If

T(eá) = Íj ˘iajiej for all i = 1, . . . , n and T(eè) ≠ ceè for any scalar c, show
that T is not normal.

2. Let A be a fixed n x n matrix, and let B be any n x n matrix such that A =

B2. Assume that B is similar to a diagonal matrix and has nonnegative
eigenvalues ¬1, . . . , ¬n. Let p(x) be a polynomial such that p(¬á2) = ¬á for
each i = 1, . . . , n. Show that p(A) = B and hence B is unique. How does
this relate to our discussion of “P” for a positive operator P?

3. Describe all operators that are both unitary and positive.

4. Is it true that for any A ∞ Mn(ç), AA¿ and A¿A are unitarily similar?
Explain.
542 LINEAR OPERATORS

5. In each case, indicate whether or not the statement is true or false and
give your reason.
(a) For any A ∞ Mn(ç), AA¿ has all real eigenvalues.
(b) For any A ∞ Mn(ç), the eigenvalues of AA¿ are of the form \¬\2
where ¬ is an eigenvalue of A.
(c) For any A ∞ Mn(ç), the eigenvalues of AA¿ are nonnegative real
numbers.
(d) For any A ∞ Mn(ç), AA¿ has the same eigenvalues as A¿A if A is
nonsingular.
(e) For any A ∞ Mn(ç), Tr(AA¿) = \Tr A\2.
(f ) For any A ∞ Mn(ç), AA¿ is unitarily similar to a diagonal matrix.
(g) For any A ∞ Mn(ç), AA¿ has n linearly independent eigenvectors.
(h) For any A ∞ Mn(ç), the eigenvalues of AA¿ are the same as the
eigenvalues of A¿A.
(i) For any A ∞ Mn(ç), the Jordan form of AA¿ is the same as the Jordan
form of A¿A.
(j) For any A ∞ Mn(ç), the null space of A¿A is the same as the null
space of A.

6. Let S and T be normal operators on V. Show that there are bases {uá} and
{vá} for V such that [S]u = [T]v if and only if there are orthonormal bases
{uæá} and {væá} such that [S]uæ = [T]væ.

7. Let T be normal and let k > 0 be an integer. Show that there is a normal S
such that Sk = T.

8. Let N be normal and let p(x) be a polynomial over ç. Show that p(N) is
also normal.

9. Let N be a normal operator on a unitary space V, let W = Ker N, and let

Nÿ be the transformation induced by N on V/W. Show that Nÿ is normal.
Show that Nÿî is also normal.

10. Discuss the following assertion: For any linear transformation T on a

unitary space V, TT¿ and T¿T have a common basis of eigenvectors.

11. Show that if A and B are real symmetric matrices and A is positive defi-
nite, then p(x) = det(B - xA) has all real roots.
 CHAPTER 11

Multilinear Mappings and

Tensors

In this chapter we generalize our earlier discussion of bilinear forms, which

leads in a natural manner to the concepts of tensors and tensor products. While
we are aware that our approach is not the most general possible (which is to
say it is not the most abstract), we feel that it is more intuitive, and hence the
best way to approach the subject for the first time. In fact, our treatment is
essentially all that is ever needed by physicists, engineers and applied
mathematicians. More general treatments are discussed in advanced courses
on abstract algebra.
The basic idea is as follows. Given a vector space V with basis {eá}, we
defined the dual space V* (with basis {øi}) as the space of linear functionals
on V. In other words, if ƒ = Íáƒáøi ∞ V* and v = Íévjeé ∞ V, then

! (v) = Ó! ,!vÔ = Ó"i!i# i ,! " j v j e jÔ = "i, j!i v j$ i j

= "i!i vi !!.

Next we defined the space B(V) of all bilinear forms on V (i.e., bilinear map-
pings on V ª V), and we showed (Theorem 9.10) that B(V) has a basis given
by {fij = øi · øj} where

fij(u, v) = øi · øj(u, v) = øi(u)øj(v) = uivj .

543
544 MULTILINEAR MAPPINGS AND TENSORS

It is this definition of the fij that we will now generalize to include linear func-
tionals on spaces such as, for example, V* ª V* ª V* ª V ª V.

11.1 DEFINITIONS

Let V be a finite-dimensional vector space over F, and let Vr denote the r-fold
Cartesian product V ª V ª ~ ~ ~ ª V. In other words, an element of Vr is an r-
tuple (vè, . . . , vr) where each vá ∞ V. If W is another vector space over F,
then a mapping T: Vr ‘ W is said to be multilinear if T(vè, . . . , vr) is linear
in each variable. That is, T is multilinear if for each i = 1, . . . , r we have

T(vè, . . . , avá + bvæá , . . . , vr)

= aT(vè, . . . , vá, . . . , vr) + bT(vè, . . . , væá, . . . , vr)

for all vá, væá ∞ V and a, b ∞ F. In the particular case that W = F, the mapping
T is variously called an r-linear form on V, or a multilinear form of degree
r on V, or an r-tensor on V. The set of all r-tensors on V will be denoted by
Tr (V). (It is also possible to discuss multilinear mappings that take their
values in W rather than in F. See Section 11.5.)
As might be expected, we define addition and scalar multiplication on
Tr (V) by

(S + T )(v1,!!…!,!vr ) = S(v1,!!…!,!vr ) + T (v1,!!…!,!vr )

(aT )(v1,!!…!,!vr ) = aT (v1,!!…!,!vr )

for all S, T ∞ Tr (V) and a ∞ F. It should be clear that S + T and aT are both r-
tensors. With these operations, Tr (V) becomes a vector space over F. Note
that the particular case of r = 1 yields T1 (V) = V*, i.e., the dual space of V,
and if r = 2, then we obtain a bilinear form on V.
Although this definition takes care of most of what we will need in this
chapter, it is worth going through a more general (but not really more
difficult) definition as follows. The basic idea is that a tensor is a scalar-
valued multilinear function with variables in both V and V*. Note also that by
Theorem 9.4, the space of linear functions on V* is V** which we view as
simply V. For example, a tensor could be a function on the space V* ª V ª V.
By convention, we will always write all V* variables before all V variables,
so that, for example, a tensor on V ª V* ª V will be replaced by a tensor on
V* ª V ª V. (However, not all authors adhere to this convention, so the reader
should be very careful when reading the literature.)
 11.1 DEFINITIONS 545

Without further ado, we define a tensor T on V to be a multilinear map on

V*s ª Vr:
T :!V !s " V r = V
"$
!
#$V
"!" !
% "V
"$"!"
#$ %V #F
s copies r copies

where r is called the covariant order and s is called the contravariant order
of T. We shall say that a tensor of covariant order r and contravariant order s
is of type (or rank) (rÍ). If we denote the set of all tensors of type (rÍ) by Tr Í(V),
then defining addition and scalar multiplication exactly as above, we see that
Tr Í(V) forms a vector space over F. A tensor of type (0º) is defined to be a
scalar, and hence T0 º(V) = F. A tensor of type (0¡) is called a contravariant
vector, and a tensor of type (1º) is called a covariant vector (or simply a
covector). In order to distinguish between these types of vectors, we denote
the basis vectors for V by a subscript (e.g., eá), and the basis vectors for V* by
a superscript (e.g., øj). Furthermore, we will generally leave off the V and
simply write Tr or Tr Í.
At this point we are virtually forced to introduce the so-called Einstein
summation convention. This convention says that we are to sum over
repeated indices in any vector or tensor expression where one index is a
superscript and one is a subscript. Because of this, we write the vector com-
ponents with indices in the opposite position from that of the basis vectors.
This is why we have been writing v = Íávieá ∞ V and ƒ = Íéƒéøj ∞ V*. Thus
we now simply write v = vieá and ƒ = ƒéøj where the summation is to be
understood. Generally the limits of the sum will be clear. However, we will
revert to the more complete notation if there is any possibility of ambiguity.
It is also worth emphasizing the trivial fact that the indices summed over
are just “dummy indices.” In other words, we have viei = vjej and so on.
Throughout this chapter we will be relabelling indices in this manner without
further notice, and we will assume that the reader understands what we are
doing.
Suppose T ∞ Tr, and let {eè, . . . , eñ} be a basis for V. For each i = 1, . . . ,
r we define a vector vá = eéaji where, as usual, aji ∞ F is just the jth component
of the vector vá. (Note that here the subscript i is not a tensor index.) Using the
multilinearity of T we see that

T(vè, . . . , vr) = T(ejèajè1, . . . , ej‹aj‹r) = ajè1 ~ ~ ~ aj‹r T(ejè , . . . , ej‹) .

The nr scalars T(ejè , . . . , ej‹) are called the components of T relative to the
basis {eá}, and are denoted by Tjè ~ ~ ~ j‹. This terminology implies that there
exists a basis for Tr such that the Tjè ~ ~ ~ j‹ are just the components of T with
546 MULTILINEAR MAPPINGS AND TENSORS

respect to this basis. We now construct this basis, which will prove that Tr is
of dimension nr.
(We will show formally in Section 11.10 that the Kronecker symbols ∂ij
are in fact the components of a tensor, and that these components are the same
in any coordinate system. However, for all practical purposes we continue to
use the ∂ij simply as a notational device, and hence we place no importance on
the position of the indices, i.e., ∂ij = ∂ji etc.)
For each collection {iè, . . . , ir} (where 1 ¯ iÉ ¯ n), we define the tensor
Øiè ~ ~ ~ i‹ (not simply the components of a tensor Ø) to be that element of Tr
whose values on the basis {eá} for V are given by

Øiè ~ ~ ~ i‹(ejè , . . . , ej‹) = ∂ièjè ~ ~ ~ ∂i‹j‹

and whose values on an arbitrary collection {vè, . . . , vr} of vectors are given
by multilinearity as

!i1!ir (v1,!!…!,!vr ) = !i1 !…!ir (e j1 a j1 1,!!…!,!e jr a jr r )

= a j1 1 !a jr r !i1!ir (e j1 ,!!…!,!e jr )
= a j1 1 !a jr r " i1 j1 !" ir jr
= ai1 1 !air r !!.

That this does indeed define a tensor is guaranteed by this last equation which
shows that each Øiè ~ ~ ~ i‹ is in fact linear in each variable (since vè + væè =
(ajè1 + aæjè1)ejè etc.). To prove that the nr tensors Øiè ~ ~ ~ i‹ form a basis for Tr,
we must show that they linearly independent and span Tr.
Suppose that åiè ~ ~ ~ i‹ Øiè ~ ~ ~ i‹ = 0 where each åiè ~ ~ ~ i‹ ∞ F. From the
definition of Øiè ~ ~ ~ i‹, we see that applying this to any r-tuple (ejè , . . . , ej‹) of
basis vectors yields åjè ~ ~ ~ j‹ = 0. Since this is true for every such r-tuple, it
follows that åiè ~ ~ ~ i‹ = 0 for every r-tuple of indices (iè, . . . , ir), and hence the
Øiè ~ ~ ~ i‹ are linearly independent.
Now let Tiè ~ ~ ~ i‹ = T(eiè , . . . , ei‹) and consider the tensor

Tiè ~ ~ ~ i‹ Øiè ~ ~ ~ i‹

in Tr. Using the definition of Øiè ~ ~ ~ i‹, we see that both Tiè ~ ~ ~ i‹Øiè ~ ~ ~ i‹ and T
yield the same result when applied to any r-tuple (ejè , . . . , ej‹) of basis
vectors, and hence they must be equal as multilinear functions on Vr. This
shows that {Øiè ~ ~ ~ i‹} spans Tr.
 11.1 DEFINITIONS 547

While we have treated only the space Tr, it is not any more difficult to
treat the general space Tr Í. Thus, if {eá} is a basis for V, {øj} is a basis for V*
and T ∞ Tr Í, we define the components of T (relative to the given bases) by

Tiè ~ ~ ~ i› jè ~ ~ ~ j‹ = T(øiè , . . . , øi›, ejè , . . . , ej‹ ) .

Defining the nr+s analogous tensors Øi!Ô⁄`Ÿ`Ÿ`Ÿi$Ô‹, it is easy to mimic the above
procedure and hence prove the following result.

Theorem 11.1 The set Tr Í of all tensors of type (rÍ) on V forms a vector space
of dimension nr+s.

Proof This is Exercise 11.1.1. ˙

Since a tensor T ∞ Tr Í is a function on V*s ª Vr, it would be nice if we

could write a basis (e.g., Øi!Ô⁄`Ÿ`Ÿ`Ÿi$Ô‹) for Tr Í in terms of the bases {eá} for V and
{øj} for V*. We now show that this is easy to accomplish by defining a
product on Tr Í, called the tensor product. The reader is cautioned not to be
intimidated by the notational complexities, since the concepts involved are
really quite simple.
Suppose that S ! T r1 s1 and T ! T r2 s2 . Let uè, . . . , urè, vè, . . . , vrì be
vectors in V, and å1, . . . , åsè, ∫1, . . . , ∫sì be covectors in V*. Note that the
product
S(å1, . . . , åsè, uè, . . . , urè) T(∫1, . . . , ∫sì, vè, . . . , vrì)

is linear in each of its rè + sè + rì + sì variables. Hence we define the tensor

product S · T ∞ T rs1+r+s2 (read “S tensor T”) by
1 2

(S · T)(å1, . . . , åsè, ∫1, . . . , ∫sì, uè, . . . , urè, vè, . . . , vrì) =

S(å1, . . . , åsè, uè, . . . , urè ) T(∫1, . . . , ∫sì, vè, . . . , vrì ) .

It is easily shown that the tensor product is both associative and distribu-
tive (i.e., bilinear in both factors). In other words, for any scalar a ∞ F and
tensors R, S and T such that the following formulas make sense, we have

(R ! S) ! T = R ! (S ! T )
R ! (S + T ) = R ! S + R ! T
(R + S) ! T = R ! T + S ! T
(aS) ! T = S ! (aT ) = a(S ! T )
548 MULTILINEAR MAPPINGS AND TENSORS

(see Exercise 11.1.2). Because of the associativity property (which is a conse-

quence of associativity in F), we will drop the parentheses in expressions such
as the top equation and simply write R · S · T. This clearly extends to any
finite product of tensors. It is important to note, however, that the tensor
product is most certainly not commutative.
Now let {eè, . . . , eñ} be a basis for V, and let {øj} be its dual basis. We
claim that the set {øjè · ~ ~ ~ · øj‹} of tensor products where 1 ¯ jÉ ¯ n forms
a basis for the space Tr of covariant tensors. To see this, we note that from the
definitions of tensor product and dual space, we have

øjè · ~ ~ ~ · øj‹(eiè , . . . , ei‹) = øjè(eiè) ~ ~ ~ øj‹(ei‹) = ∂jèiè ~ ~ ~ ∂j‹i‹

so that øjè · ~ ~ ~ · øj‹ and Øjè ~ ~ ~ j‹ take the same values on the r-tuples
(eiè, . . . , ei‹), and hence they must be equal as multilinear functions on Vr.
Since we showed above that {Øjè ~ ~ ~ j‹} forms a basis for Tr , we have proved
that {øjè · ~ ~ ~ · øj‹} also forms a basis for Tr .
The method of the previous paragraph is readily extended to the space Tr Í.
We must recall however, that we are treating V** and V as the same space. If
{eá} is a basis for V, then the dual basis {øj} for V* was defined by øj(eá) =
Óøj, eá Ô = ∂já. Similarly, given a basis {øj} for V*, we define the basis {eá} for
V** = V by eá(øj) = øj(eá) = ∂já. In fact, using tensor products, it is now easy
to repeat Theorem 11.1 in its most useful form. Note also that the next
theorem shows that a tensor is determined by its values on the bases {eá} and
{øj}.

Theorem 11.2 Let V have basis {eè, . . . , eñ}, and let V* have the corre-
sponding dual basis {ø1, . . . , øn}. Then a basis for Tr Í is given by the collec-
tion
{eiè · ~ ~ ~ · ei› · øjè · ~ ~ ~ · øj‹}

where 1 ¯ jè, . . . , jr, iè, . . . , is ¯ n, and hence dim Tr Í = nr+s.

Proof In view of Theorem 11.1, all that is needed is to show that

eiè · ~ ~ ~ · ei› · øjè · ~ ~ ~ · øj‹ = Øi!Ô⁄`Ÿ`Ÿ`Ÿi$Ô‹ .

The details are left to the reader (see Exercise 11.1.1). ˙

Since the components of a tensor T are defined with respect to a particular

basis (and dual basis), we might ask about the relationship between the com-
 11.1 DEFINITIONS 549

ponents of T relative to two different bases. Using the multilinearity of

tensors, this is a simple problem to solve.
First, let {eá} be a basis for V and let {øj} be its dual basis. If {eõá} is
another basis for V, then there exists a nonsingular transition matrix A = (aji)
such that
eõi = ejaji . (1)

(We emphasize that ajá is only a matrix, not a tensor. Note also that our defini-
tion of the matrix of a linear transformation given in Section 5.3 shows that ajá
is the element of A in the jth row and ith column.) Using Óøi, eéÔ = ∂ié, we have

Óøi, eõÉÔ = Óøi, eéajÉÔ = ajÉÓøi, eéÔ = ajÉ∂ié = aiÉ .

Let us denote the inverse of the matrix A = (aié) by Aî = B = (bié). In other

words, aiébjÉ = ∂iÉ and biéajÉ = ∂iÉ. Multiplying Óøi, eõÉÔ = aiÉ by bjá and
summing on i yields
Óbjáøi, eõÉÔ = bjáaiÉ = ∂jÉ .

But the basis {øùi} dual to {eõá} also must satisfy Óøùj, eõkÔ = ∂jÉ, and hence
comparing this with the previous equation shows that the dual basis vectors
transform as
øùj = bjiøi (2)

The reader should compare this carefully with (1). We say that the dual
basis vectors transform oppositely (i.e., use the inverse transformation matrix)
to the basis vectors. It is also worth emphasizing that if the nonsingular transi-
tion matrix from the basis {eá} to the basis {eõá} is given by A, then (according
to the same convention given in Section 5.4) the corresponding nonsingular
transition matrix from the basis {øi} to the basis {øùi} is given by BT =
(Aî)T. We leave it to the reader to write out equations (1) and (2) in matrix
notation to show that this is true (see Exercise 11.1.3).
We now return to the question of the relationship between the components
of a tensor in two different bases. For definiteness, we will consider a tensor
T ∞ T1 ™. The analogous result for an arbitrary tensor in Tr Í will be quite
obvious. Let {eá} and {øj} be a basis and dual basis for V and V* respective-
ly. Now consider another pair of bases {eõá} and {øùj} where eõá = eéajá and øùi =
biéøj. Then we have Tij k = T(øi, øj, ek) as well as Täpq r = T(øùp, øùq, eõr), and
therefore
550 MULTILINEAR MAPPINGS AND TENSORS

Täp q r = T(øùp, øùq, eõr) = bpi bqj akr T(øi, øj, ek) = bpi bqj akr Tijk .

This is the classical law of transformation of the components of a tensor of

type (1™). It should be kept in mind that (aié) and (bié) are inverse matrices to
each other. (In fact, this equation is frequently taken as the definition of a
tensor (at least in older texts). In other words, according to this approach, any
quantity with this transformation property is defined to be a tensor.)
In particular, the components vi of a vector v = vi eá transform as

või = biévj

while the components åá of a covector å = åáøi transform as

åùá = åéajá .

We leave it to the reader to verify that these transformation laws lead to the
self-consistent formulas v = vieá = võj eõé and å = åáøi = åùéøùj as we should
expect (see Exercise 11.1.4).
We point out that these transformation laws are the origin of the terms
“contravariant” and “covariant.” This is because the components of a vector
transform oppositely (“contravariant”) to the basis vectors ei, while the com-
ponents of dual vectors transform the same as (“covariant”) these basis vec-
tors.
It is also worth mentioning that many authors use a prime (or some other
method such as a different type of letter) for distinguishing different bases. In
other words, if we have a basis {eá} and we wish to transform to another basis
which we denote by {eiæ}, then this is accomplished by a transformation
matrix (aijæ) so that eiæ = eéajiæ. In this case, we would write øiæ = aiæéøj where
(aiæé) is the inverse of (aijæ). In this notation, the transformation law for the
tensor T used above would be written as

Tpæqæræ = bpæibqæjakræTijk .

Note that specifying the components of a tensor with respect to one coor-
dinate system allows the determination of its components with respect to any
other coordinate system. Because of this, we shall frequently refer to a tensor
by its “generic” components. In other words, we will refer to e.g., Tijk, as a
“tensor” and not the more accurate description as the “components of the
tensor T.”
 11.1 DEFINITIONS 551

Example 11.1 For those readers who may have seen a classical treatment of
tensors and have had a course in advanced calculus, we will now show how
our more modern approach agrees with the classical.
If {xi} is a local coordinate system on a differentiable manifold X, then a
(tangent) vector field v(x) on X is defined as the derivative function v =
vi($/$xi), so that v(f ) = vi($f/$xi) for every smooth function f: X ‘ ® (and
where each vi is a function of position x ∞ X, i.e., vi = vi(x)). Since every
vector at x ∞ X can in this manner be written as a linear combination of the
$/$xi, we see that {$/$xi} forms a basis for the tangent space at x.
We now define the differential df of a function by df(v) = v(f) and thus
df(v) is just the directional derivative of f in the direction of v. Note that

dxi(v) = v(xi) = vj($xi/$xj) = vj∂ié = vi

and hence df(v) = vi($f/$xi) = ($f/$xi)dxi(v). Since v was arbitrary, we obtain

the familiar elementary formula df = ($f/$xi)dxi. Furthermore, we see that

dxi($/$xj) = $xi/$xj = ∂ié

so that {dxi} forms the basis dual to {$/$xi}. In summary then, relative to the
local coordinate system {xi}, we define a basis {eá = $/$xi} for a (tangent)
space V along with the dual basis {øj = dxj} for the (cotangent) space V*.
If we now go to a new coordinate system {xõi} in the same coordinate
patch, then from calculus we obtain

$/$xõi = ($xj/$xõi)$/$xj

so that the expression eõá = eéajá implies ajá = $xj/$xõi. Similarly, we also have

dxõi = ($xõi/$xj)dxj

so that øùi = biéøj implies bié = $xõi/$xj. Note that the chain rule from calculus
shows us that

aiÉbké = ($xi/$xõk)($xõk/$xj) = $xi/$xj = ∂ié

and thus (bié) is indeed the inverse matrix to (aié).

Using these results in the above expression for Täpqr, we see that
552 MULTILINEAR MAPPINGS AND TENSORS

pq !x p !x q !x k ij
T r= T k
!x i !x j !x r

which is just the classical definition of the transformation law for a tensor of
type (1™).
We also remark that in older texts, a contravariant vector is defined to
have the same transformation properties as the expression dxõi = ($xõi/$xj)dxj,
while a covariant vector is defined to have the same transformation properties
as the expression $/$xõi = ($xj/$xõi)$/$xj. ∆

Finally, let us define a simple classical tensor operation that is frequently

quite useful. To begin with, we have seen that the result of operating on a
vector v = vieá ∞ V with a dual vector å = åéøj ∞ V* is just Óå, vÔ =
åéviÓøj, eáÔ = åévi∂já = åávi. This is sometimes called the contraction of å with
v. We leave it to the reader to show that the contraction is independent of the
particular coordinate system used (see Exercise 11.1.5).
If we start with tensors of higher order, then we can perform the same sort
of operation. For example, if we have S ∞ T2 ¡ with components Sijk and T ∞
T 2 with components Tpq, then we can form the (1™) tensor with components
SiéÉTjq, or a different (1™) tensor with components SiéÉTpj and so forth. This
operation is also called contraction. Note that if we start with a (1¡) tensor T,
then we can contract the components of T to obtain the scalar Tiá. This is
called the trace of T.

Exercises

1. (a) Prove Theorem 11.1.

(b) Prove Theorem 11.2.

2. Prove the four associative and distributive properties of the tensor product
given in the text following Theorem 11.1.

3. If the nonsingular transition matrix from a basis {eá} to a basis {eõá} is

given by A = (aij), show that the transition matrix from the corresponding
dual bases {øi} and {øùi} is given by (Aî)T.

4. Using the transformation matrices (aié) and (bié) for the bases {eá} and {eõá}
and the corresponding dual bases {øi} and {øùi}, verify that v = vieá = võj eõé
and å = åáøi = åùéøùj.
 11.1 DEFINITIONS 553

5. If v ∞ V and å ∞ V*, show that Óå, vÔ is independent of the particular

basis chosen for V. Generalize this to arbitrary tensors.

6. Let Ai be a covariant vector field (i.e., Ai = Ai(x)) with the transformation

rule
!x j
Ai = i A j !!.
!x

Show that the quantity $jAi = $Ai/$xj does not define a tensor, but that
Fij = $iAj - $jAi is in fact a second-rank tensor.

11.2 SPECIAL TYPES OF TENSORS

In order to obtain some of the most useful results concerning tensors, we turn
our attention to the space Tr of covariant tensors on V. Generalizing our
earlier definition for bilinear forms, we say that a tensor S ∞ Tr is symmetric
if for each pair (i, j) with 1 ¯ i, j ¯ r and all vá ∞ V we have

S(vè, . . . , vá, . . . , vé, . . . , vr) = S(vè, . . . , vé, . . . , vá, . . . , vr) .

Similarly, A ∞ Tr is said to be antisymmetric (or skew-symmetric or alter-

nating) if

A(vè, . . . , vá, . . . , vé, . . . , vr) = -A(vè, . . . , vé, . . . , vá, . . . , vr) .

Note this definition implies that A(vè, . . . , vr) = 0 if any two of the vá are
identical. In fact, this was the original definition of an alternating bilinear
form. Furthermore, we also see that A(vè, . . . , vr) = 0 if any vá is a linear
combination of the rest of the vé. In particular, this means that we must always
have r ¯ dim V if we are to have a nonzero antisymmetric tensor of type (rº) on
V.
It is easy to see that if Sè, Sì ∞ Tr are symmetric, then so is aSè + bSì
where a, b ∞ F. Similarly, aAè + bAì is antisymmetric. Therefore the symmet-
ric tensors form a subspace of Tr which we denote by ∑r(V), and the anti-
symmetric tensors form another subspace of Tr which is denoted by „r(V)
(some authors denote this space by „r(V*)). Elements of „r(V) are generally
called exterior r-forms, or simply r-forms. According to this terminology,
the basis vectors {øi} for V* are referred to as basis 1-forms. Note that the
only element common to both of these subspaces is the zero tensor.
554 MULTILINEAR MAPPINGS AND TENSORS

A particularly important example of an antisymmetric tensor is the

determinant function det ∞ Tn(®n) (see Theorem 4.9 and the discussion pre-
ceding it). Note also that the definition of a symmetric tensor translates into
the obvious requirement that (e.g., in the particular case of T2) Sáé = Séá, while
an antisymmetric tensor obeys Aáé = -Aéá. These definitions can also be
extended to include contravariant tensors, although we shall have little need to
do so.
It will be extremely convenient for us to now incorporate the treatment of
permutation groups given in Section 1.2. In terms of any permutation ß ∞ Sr,
we may rephrase the above definitions as follows. We say that S ∞ Tr is sym-
metric if for every collection vè, . . . , vr ∞ V and each ß ∞ Sr we have

S(vè, . . . , vr) = S(vß1 , . . . , vßr) .

Similarly, we say that A ∞ Tr is antisymmetric (or alternating) if either

A(vè, . . . , vr) = (sgn ß)A(vß1 , . . . , vßr)

or
A(vß1 , . . . , vßr) = (sgn ß)A(vè, . . . , vr)

where the last equation follows from the first since (sgn ß)2 = 1. Note that
even if S, T ∞ ∑r(V) are both symmetric, it need not be true that S · T be
symmetric (i.e., S · T ! ∑r+r(V)). For example, if Sáé = Séá and Tpq = Tqp, it
does not necessarily follow that SáéTpq = SipTjq. It is also clear that if A, B ∞
„r(V), then we do not necessarily have A · B ∞ „r+r(V).

Example 11.2 Suppose å ∞ „n(V), let {eè, . . . , eñ} be a basis for V, and for
each i = 1, . . . , n let vá = eéaji where aji ∞ F. Then, using the multilinearity of
å, we may write

å(vè, . . . , vn) = ajè1 ~ ~ ~ ajñnå(ejè , . . . , ejñ)

where the sums are over all 1 ¯ jÉ ¯ n. But å ∞ „n(V) is antisymmetric, and
hence (ejè , . . . , ejñ) must be a permutation of (eè, . . . , eñ) in order that the ejÉ
all be distinct (or else å(ejè , . . . , ejñ) = 0). This means that we are left with

å(vè, . . . , vñ) = ÍJajè1 ~ ~ ~ ajñn å(ejè , . . . , ejñ)

where ÍJ denotes the fact that we are summing over only those values of jÉ
such that (jè, . . . , jñ) is a permutation of (1, . . . , n). In other words, we have
 11.2 SPECIAL TYPES OF TENSORS 555

å(vè, . . . , vñ) = Íß´Sñ aß11 ~ ~ ~ aßnn å(eß1 , . . . , eßn) .

But now, by the antisymmetry of å, we see that å(eß1 , . . . , eßn) =

(sgn ß)å(eè, . . . , eñ) and hence we are left with

å(vè, . . . , vñ) = Íß´Sñ (sgn ß) aß11 ~ ~ ~ aßnn å(e1 , . . . , en) . (*)

Using the definition of determinant and the fact that å(eè, . . . , eñ) is just some
scalar, we finally obtain

å(vè, . . . , vñ) = det(aji) å(eè, . . . , eñ) .

Referring back to Theorem 4.9, let us consider the special case where
å(eè, . . . , eñ) = 1. Note that if {øj} is a basis for V*, then

øßj(vá) = øßj(ekaki) = aki øßj(ek) = aki ∂ßjk = aßji .

Using the definition of tensor product, we can therefore write (*) as

det(aji) = å(vè, . . . , vñ) = Íß´Sñ (sgn ß)øß1 · ~ ~ ~ · øßn(vè, . . . , vñ)

which implies that the determinant function is given by

å = Íß´Sñ (sgn ß)øß1 · ~ ~ ~ · øßn .

In other words, if A is a matrix with columns given by vè, . . . , vñ then det A =

å(vè, . . . , vñ).
While we went through many detailed manipulations in arriving at these
equations, we will assume from now on that the reader understands what was
done in this example, and henceforth leave out some of the intermediate steps
in such calculations. ∆

At the risk of boring some readers, let us very briefly review the meaning
of the binomial coefficient (rˆ) = n!/[r!(n - r)!]. The idea is that we want to
know the number of ways of picking r distinct objects out of a collection of n
distinct objects. In other words, how many combinations of n things taken r at
a time are there? Well, to pick r objects, we have n choices for the first, then
n - 1 choices for the second, and so on down to n - (r - 1) = n - r + 1 choices
for the rth . This gives us

n(n - 1) ~ ~ ~ (n - r + 1) = n!/(n - r)!

556 MULTILINEAR MAPPINGS AND TENSORS

as the number of ways of picking r objects out of n if we take into account the
order in which the r objects are chosen. In other words, this is the number of
injections INJ(r, n) (see Section 4.6). For example, to pick three numbers in
order out of the set {1, 2, 3, 4}, we might choose (1, 3, 4), or we could choose
(3, 1, 4). It is this kind of situation that we must take into account. But for
each distinct collection of r objects, there are r! ways of arranging these, and
hence we have over-counted each collection by a factor of r!. Dividing by r!
then yields the desired result.
If {eè, . . . , eñ} is a basis for V and T ∞ „r(V), then T is determined by its
values T(eiè , . . . , ei‹) for iè < ~ ~ ~ < ir. Indeed, following the same procedure
as in Example 11.2, we see that if vá = eéaji for i = 1, . . . , r then

T(vè, . . . , vr) = aiè1 ~ ~ ~ ai‹rT(eiè , . . . , ei‹)

where each sum is over 1 ¯ iÉ ¯ n. Furthermore, each collection {eiè , . . . , ei‹}

must consist of distinct basis vectors in order that T(eiè , . . . , ei‹) ≠ 0. But the
antisymmetry of T tells us that for any ß ∞ Sr, we must have

T(eßiè , . . . , eßi‹) = (sgn ß)T(eiè , . . . , ei‹)

where we may choose iè < ~ ~ ~ < ir. Thus, since the number of ways of choos-
ing r distinct basis vectors {eiè , . . . , ei‹} out of the basis {eè, . . . , eñ} is (rˆ), it
follows that
dim „r(V) = (rˆ) = n!/[r!(n - r)!] .

We will prove this result again when we construct a specific basis for „r(V)
(see Theorem 11.8 below).
In order to define linear transformations on Tr that preserve symmetry (or
antisymmetry), we define the symmetrizing mapping S: Tr ‘Tr and alter-
nation mapping A: Tr ‘Tr by

(ST)(vè, . . . , vr) = (1/r!)Íß´S‹ T(vß1 , . . . , vßr)

and
(AT)(vè, . . . , vr) = (1/r!)Íß´S‹ (sgn ß)T(vß1 , . . . , vßr)

where T ∞ Tr (V) and vè, . . . , vr ∞ V. That these are in fact linear transforma-
tions on Tr follows from the observation that the mapping Tß defined by

Tß(vè, . . . , vr) = T(vß1 , . . . , vßr)

is linear, and any linear combination of such mappings is again a linear trans-
 11.2 SPECIAL TYPES OF TENSORS 557

formation.
Given any ß ∞ Sr, it will be convenient for us to define the mapping
ßÄ: Vr ‘ Vr by
ßÄ(vè, . . . , vr) = (vß1 , . . . , vßr) .

This mapping permutes the order of the vectors in its argument, not the labels
(i.e. the indices), and hence its argument must always be (v1, v2, . . . , vr) or
(w1, w2, . . . , wr) and so forth. Then for any T ∞ Tr (V) we define ßT ∞ Tr (V)
by
ßT = T ı ßÄ

which is the mapping Tß defined above. It should be clear that ß(Tè + Tì) =
ßTè + ßTì. Note also that if we write

ßÄ(vè, . . . , vr) = (vß1 , . . . , vßr) = (wè, . . . , wr)

then wá = vßi and therefore for any other † ∞ Sr we have

†Ä ı ßÄ(v1, . . . , vr) = †Ä(w1, . . . , wr)

= (w†1, . . . , w†r)
= (v߆1, . . . , v߆r)
= `ßı† (v1, . . . , vr) .

This shows that

†Ä ı ßÄ = `ßı†
and hence

ß(†T) = ß(T ı †Ä) = T ı (†Ä ı ßÄ) = T ı (`ßı†) = (ß ı †)T .

Note also that in this notation, the alternation mapping is defined as

AT = (1/r!)Íß´S‹ (sgn ß)(ßT) .

Theorem 11.3 The linear mappings A and S have the following properties:
(a) T ∞ „r(V) if and only if AT = T, and T ∞ ∑r(V) if and only if ST = T.
(b) A(Tr (V)) = „r(V) and S(Tr (V)) = ∑r(V).
(c) A2 = A and S2 = S, i.e., A and S are projections.

Proof Since the mapping A is more useful, we will prove the theorem only
for this case, and leave the analogous results for S to the reader (see Exercise
11.2.1). Furthermore, all three statements of the theorem are interrelated, so
558 MULTILINEAR MAPPINGS AND TENSORS

we prove them together.

First suppose that T ∞ „r(V). From the definition of antisymmetric tensor
we have T(vß1 , . . . , vßr) = (sgn ß)T(vè, . . . , vr), and thus using the fact that
the order of Sr is r!, we see that

AT (v1,!… ,!vr ) = (1/r!)!" #Sr (sgn " )T (v" 1,!… , v" r )

= (1/r!)!" #Sr T (v1,!…!,!vr )
= T (v1,!…!,!vr )!!.

This shows that T ∞ „r(V) implies AT = T.

Next, let T be any element of Tr (V). We may fix any particular element
œ ∞ Sr and then apply AT to the vectors vœ1 , . . . , vœr to obtain

AT (v!1,!… ,!v! r ) = AT! (v1,!…!,!vr )

= (1/r!)"# $Sr (sgn # )T! (v# 1,!…!,!v# r )
= (1/r!)"# $Sr (sgn # )T (v#! 1,!…!,!v#! r )!!.

Now note that sgn ß = (sgn ß)(sgn œ)(sgn œ) = (sgn ßœ)(sgn œ), and that Sr =
{ƒ = ßœ: ß ∞ Sr} (this is essentially what was done in Theorem 4.3). We now
see that the right hand side of the above equation is just

(1/r!)(sgn ! )"# $Sr (sgn #! )T (v#! 1,!…!,!v#! r )

= (1/r!)(sgn ! )"% $Sr (sgn % )T (v%1,!…!,!v% r )
= (sgn ! ) AT (v1,!…!,!vr )

which shows (by definition) that AT is antisymmetric. In other words, this

shows that AT ∞ „r(V) for any T ∞ Tr (V), or A(Tr (V)) ™ „r(V).
Since the result of the earlier paragraph showed that T = AT ∞ A(Tr (V))
for every T ∞ „r(V), we see that „r(V) ™ A(Tr (V)), and therefore A(Tr (V))
= „r(V). This also shows that if AT = T, then T is necessarily an element of
„r(V). It then follows that for any T ∞ Tr (V) we have AT ∞ „r(V), and
hence A2T = A(AT) = AT so that A2 = A. ˙

Suppose Aiè ~ ~ ~ i‹ and Tiè ~ ~ ~ i‹ (where r ¯ n = dim V and 1 ¯ ik ¯ n) are

both antisymmetric tensors, and consider their contraction Aiè ~ ~ ~ i‹ Tiè ~ ~ ~ i‹.
For any particular set of indices i1, . . . , ir there will be r! different ordered
sets (i1, . . . , ir). But by antisymmetry, the values of Aiè ~ ~ ~ i‹ corresponding to
each ordered set will differ only by a sign, and similarly for Tiè ~ ~ ~ i‹. This
 11.2 SPECIAL TYPES OF TENSORS 559

means that the product of Aiè ~ ~ ~ i‹ times Tiè ~ ~ ~ i‹ summed over the r! ordered
sets (i1, . . . , ir) is the same as r! times a single product which we choose to be
the indices i1, . . . , ir taken in increasing order. In other words, we have

Aiè ~ ~ ~ i‹ Tiè ~ ~ ~ i‹ = r! A \iè ~ ~ ~ i‹\ Tiè ~ ~ ~ i‹

where \i1 ~ ~ ~ ir\ denotes the fact that we are summing over increasing sets of
indices only. For example, if we have antisymmetric tensors Aijk and Tijk in
®3, then
AijkTijk = 3!A\ijk\ Tijk = 6A123T123

(where, in this case of course, Aijk and Tijk can only differ by a scalar).
There is a simple but extremely useful special type of antisymmetric
tensor that we now wish to define. Before doing so however, it is first
convenient to introduce another useful notational device. Note that if T ∞ Tr
and we replace vè, . . . , vr in the definitions of S and A by basis vectors eá,
then we obtain an expression in terms of components as

ST1 ~ ~ ~ r = (1/r!)Íß´S‹Tß1 ~ ~ ~ ßr
and
AT1 ~ ~ ~ r = (1/r!)Íß´S‹ (sgn ß)Tß1 ~ ~ ~ ßr .

We will write T(1 ~ ~ ~ r) = ST1 ~ ~ ~ r and T[1 ~ ~ ~ r] = AT1 ~ ~ ~ r. For example, we

have
T(ij) = (1/2!)(Tij + Tji)
and
T[ij] = (1/2!)(Tij - Tji) .

A similar definition applies to any mixed tensor such as

T k( pq)[ij ] = (1/2!){T k( pq)ij ! T k( pq) ji }

= (1/4){T kpq ij + T kqp ij ! T kpq ji ! T kqp ji }!!.

Note that if T ∞ ∑r(V), then T(iè ~ ~ ~ i‹) = Tiè ~ ~ ~ i‹ , while if T ∞ „r(V), then
T[iè ~ ~ ~ i‹] = Tiè ~ ~ ~ i‹.
Now consider the vector space ®3 with the standard orthonormal basis
{eè, eì, e3}. We define the antisymmetric tensor ´ ∞ „3(®3) by the require-
ment that
!123 = ! (e1,!e2 ,!e3 ) = +1!!.
560 MULTILINEAR MAPPINGS AND TENSORS

Since dim „3(®3) = 1, this defines all the components of ´ by antisymmetry:

´213 = -´231 = ´321 = -1 etc. If {eõá = eéajá} is any other orthonormal basis for ®3
related to the first basis by an (orthogonal) transition matrix A = (aji) with
determinant equal to +1, then it is easy to see that

´(eõè, eõì, eõ3) = det A = +1

also. This is because ´(eá, eé, eÉ) = sgn ß where ß is the permutation that takes
(1, 2, 3) to (i, j, k). Since ´ ∞ „3(®3), we see that ´[ijk] = ´ijk. The tensor ´ is
frequently called the Levi-Civita tensor. However, we stress that in a non-
orthonormal coordinate system, it will not generally be true that ´123 = +1.
While we have defined the ´ijk as the components of a tensor, it is just as
common to see the Levi-Civita (or permutation) symbol ´ijk defined simply
as an antisymmetric symbol with ´123 = +1. In fact, from now on we shall use
it in this way as a convenient notation for the sign of a permutation. For nota-
tional consistency, we also define the permutation symbol ´ijk to have the
same values as ´ijk. A simple calculation shows that ´ijk ´ijk = 3! = 6.
It should be clear that this definition can easily be extended to an arbitrary
number of dimensions. In other words, we define

#+1 if (i1,!…!,!is ) is an even permutation of (1, 2,!…!,!n)

%
!i1!is = $"1 if (i1,!…!,!is ) is an odd permutation of (1, 2,!…!,!n) !!.
% ! 0 otherwise
&

This is just another way of writing sgn ß where ß ∞ Sn. Therefore, using this
symbol, we have the convenient notation for the determinant of a matrix A =
(aij) ∞ Mn(F) as
det A = ´iè ~ ~ ~ iñ aiè1 ~ ~ ~ aiñn .

We now wish to prove a very useful result. To keep our notation from
getting too cluttered, it will be convenient for us write ! ijk i j k
pqr = ! p!q!r . Now

note that ! pqr = 6"[123]

pqr . To see that this true, simply observe that both sides are
antisymmetric in (p, q, r), and are equal to 1 if (p, q, r) = (1, 2, 3). (This also
gives us a formula for ´pqr as a 3 x 3 determinant with entries that are all
Kronecker delta’s. See Exercise 11.2.4) Using ´123 = 1 we may write this as
´123 ´pqr = 6 ![123]
pqr . But now the antisymmetry in (1, 2, 3) yields the general
result
! ijk! pqr = 6"[ijk
pqr
]
(1)
 11.2 SPECIAL TYPES OF TENSORS 561

which is what we wanted to show. It is also possible to prove this in another

manner that serves to illustrate several useful techniques in tensor algebra.

Example 11.3 Suppose we have an arbitrary tensor A ∞ „3(®3), and hence

A[ijk] = Aijk. As noted above, the fact that dim „3(®3) = 1 means that we
must have Aijk = ¬´ijk for some scalar ¬ ∞ ®. Then

Aijk! ijk! pqr = "!ijk! ijk! pqr = 6 "! pqr = 6 "!ijk# ip#qj#rk !!.

Because ´ijk = ´[ijk], we can antisymmetrize over the indices i, j, k on the right
hand side of the above equation to obtain 6 !"ijk#[ijk ]
pqr (write out the 6 terms if
you do not believe it, and see Exercise 11.2.7). This gives us

Aijk! ijk! pqr = 6 "!ijk#[ijk ] [ijk ]

pqr = 6Aijk# pqr !!.

Since the antisymmetric tensor Aijk is contracted with another antisymmetric

tensor on both sides of this equation, the discussion following the proof of
Theorem 11.3 shows that we may write

A|ijk|! ijk! pqr = 6A|ijk|"[ijk

pqr
]

or
A123!123! pqr = 6A123"[123]
pqr !!.

Cancelling A123 then yields !123! pqr = 6"[123]

pqr as we had above, and hence (1)
again follows from this.
In the particular case that p = k, we leave it to the reader to show that

! ijk! kqr = 6"kqr

[ijk ] ij
= "qr # "qrji = "qi"rj # "qj"ri

which is very useful in manipulating vectors in ®3. As an example, consider

the vectors Aë, Bë, Cë ∞ ®3 equipped with a Cartesian coordinate system.
Abusing our notation for simplicity (alternatively, we will see formally in
Example 11.12 that Ai = Aá for such an Aë), we have

(Bë ª Cë)i = ´áéÉBj Ck

and hence

Aë  (Bë ª Cë) = Ai´áéÉBjCk = +´éÉáBjCkAi = Bë  (Cë ª Aë) .

562 MULTILINEAR MAPPINGS AND TENSORS

Other examples are given in the exercises. ∆

Exercises

1. Prove Theorem 11.3 for the symmetrizing mapping S.

2. Using the Levi-Civita symbol, prove the following vector identities in ®3

equipped with a Cartesian coordinate system (where the vectors are actu-
ally vector fields where necessary, f is differentiable, and #i = $i = $/$xi):
(a) Aë ª (Bë ª C ë) = (Aë Â C ë)Bë - (Aë Â Bë)Cë
(b) (Aë ª Bë) Â (Cë ª Dë) = (Aë Â Cë)(Bë Â Dë) - (Aë Â Dë)(Bë Â Cë)
(c) # ª #f = 0
(d) # Â (# ª Aë) = 0
(e) # ª (# ª Aë) = #(# Â Aë) - #2 Aë
(f ) # Â (Aë ª Bë) = Bë Â (# ª Aë) - Aë Â (# ª Bë)
(g) # ª (Aë ª Bë) = Aë(# Â Bë) - Bë(# Â Aë) + (Bë Â #)Aë - (Aë Â #)Bë

3. Using the divergence theorem (ÈV # Â Aë d3 x = ÈS Aë Â nï da), prove that

! ! !
#V ! " A d 3x = # S n " A da !!.
[Hint: Let Cë be a constant vector and show that
! ! ! ! ! ! ! !
C i # ! " A d 3x = # S (n " A)i C da = C i # S n " A da !!.]
V

4. (a) Find the expression for ´pqr as a 3 x 3 determinant with all Kronecker
delta’s as entries.
(b) Write ´ijk´pqr as a 3 x 3 determinant with all Kronecker delta’s as
entries.

5. Suppose V = ®3 and let Aij be antisymmetric and Sij be symmetric. Show

that AijSij = 0 in two ways:
(a) Write out all terms in the sum and show that they cancel in pairs.
(b) Justify each of the following equalities:

AijSij = AijSji = -AjiSji = -AijSij = 0 .

6. Show that a second-rank tensor Tij can be written in the form Tij = T(ij) +
T[ij], but that a third-rank tensor can not. (The complete solution for ten-
 11.2 SPECIAL TYPES OF TENSORS 563

sors of rank higher than two requires a detailed study of representations of

the symmetric group and Young diagrams.)

7. Let Aiè ~ ~ ~ i‹ be antisymmetric, and suppose Tiè ~ ~ ~ i‹ ~ ~ ~ ~ ~ ~ is an arbitrary

tensor. Show that

Aiè ~ ~ ~ i‹ Tiè ~ ~ ~ i‹ ~ ~ ~ ~ ~ ~ = Aiè ~ ~ ~ i‹ T[iè ~ ~ ~ i‹] ~ ~ ~ ~ ~ ~ .

8. (a) Let A = (aij) be a 3 x 3 matrix. Show

´ijk aip ajq akr = (det A)´pqr .

(b) Let A be a linear transformation, and let y(i) = Ax(i). Show

det[y(1), . . . , y(n)] = (det A) det[x(1), . . . , x(n)] .

9. Show that under orthogonal transformations A = (aié) in ®3, the vector

cross product xï = yï ª zï transforms as xõi = ´ijk yõj zõk = (det A)aiéxj. Discuss
the difference between the cases det A = +1 and det A = -1.

11.3 THE EXTERIOR PRODUCT

We have seen that the tensor product of two elements of „r(V) is not gen-
erally another element of „r+r(V). However, using the mapping A we can
define another product on „r(V) that turns out to be of great use. We adopt
the convention of denoting elements of „r(V) by Greek letters such as å, ∫
etc., which should not be confused with elements of the permutation group Sr.
If å ∞ „r(V) and ∫ ∞ „s(V), we define their exterior product (or wedge
product) å°∫ to be the mapping from „r(V) ª „s(V) ‘ „r+s(V) given by

(r + s)!
!"# = A(! $ # )!!.
r!s!

In other words, the wedge product is just an antisymmetrized tensor product.

The reader may notice that the numerical factor is just the binomial coeffi-
cient ( Âr± Í) = ( Âs± Í). It is also worth remarking that many authors leave off this
coefficient entirely. While there is no fundamental reason for following either
convention, our definition has the advantage of simplifying expressions
involving volume elements as we shall see later.
564 MULTILINEAR MAPPINGS AND TENSORS

A very useful formula for computing exterior products for small values of
r and s is given in the next theorem. By way of terminology, a permutation
ß ∞ Sr+s such that ß1 < ~ ~ ~ < ßr and ß(r + 1) < ~ ~ ~ < ß(r + s) is called an
(r, s)-shuffle. The proof of the following theorem should help to clarify this
definition.

Theorem 11.4 Suppose å ∞ „r(V) and ∫ ∞ „s(V). Then for any collection
of r + s vectors vá ∞ V (with r + s ¯ dim V), we have

å°∫(vè, . . . , vr+s) = Í* (sgn ß)å(vß1 , . . . , vßr)∫(vß(r+1) , . . . , vß(r+s))

where Í* denotes the sum over all permutations ß ∞ Sr+s such that ß1 < ~ ~ ~ <
ßr and ß(r + 1) < ~ ~ ~ < ß(r + s) (i.e., over all (r, s)-shuffles).

Proof The proof is simply a careful examination of the terms in the definition
of å°∫. By definition, we have

! " # (v1,!!…!,!vr+s )
= [(r + s)!/r!s!]A(! $ # )(v1,!…!,!vr+s ) (*)
= [1/r!s!]%& (sgn & )! (v& 1,!…!,!v& r )# (v& (r+1) ,!…!,!v& (r+s) )

where the sum is over all ß ∞ Sr+s. Now note that there are only ( Âr± Í) distinct
collections {ß1, . . . , ßr}, and hence there are also only ( Âs± Í) = ( Âr± Í) distinct
collections {ß(r + 1), . . . , ß(r + s)}. Let us call the set {vß1 , . . . , vßr} the “å-
variables,” and the set {vß(r+1), . . . , vß(r+s)} the “∫-variables.” For any of the
( Âr± Í) distinct collections of å- and ∫-variables, there will be r! ways of order-
ing the å-variables within themselves, and s! ways of ordering the ∫-variables
within themselves. Therefore, there will be r!s! possible arrangements of the
å- and ∫-variables within themselves for each of the ( Âr± Í) distinct collections.
Let ß ∞ Sr+s be a permutation that yields one of these distinct collections, and
assume it is the one with the property that ß1 < ~ ~ ~ < ßr and ß(r + 1) < ~ ~ ~ <
ß(r + s). The proof will be finished if we can show that all the rest of the r!s!
members of this collection are the same.
Let T denote the term in (*) corresponding to our chosen ß. Then T is
given by
T = (sgn ß)å(vß1 , . . . , vßr)∫(vß(r+1) , . . . , vß(r+s)) .

This means that every other term t in the distinct collection containing T will
be of the form

t = (sgn œ)å(vœ1 , . . . , vœr)∫(vœ(r+1) , . . . , vœ(r+s))

 11.3 THE EXTERIOR PRODUCT 565

where the permutation œ ∞ Sr+s is such that the set {œ1, . . . , œr} is the same
as the set {ß1, . . . , ßr} (although possibly in a different order), and similarly,
the set {œ(r + 1), . . . , œ(r + s)} is the same as the set {ß(r + 1), . . . , ß(r + s)}.
Thus the å- and ∫-variables are permuted within themselves. But we may then
write œ = ƒß where ƒ ∞ Sr+s is again such that the two sets {ß1, . . . , ßr} and
{ß(r + 1), . . . , ß(r + s)} are permuted within themselves. Because none of the
transpositions that define the permutation ƒ interchange å- and ∫-variables,
we may use the antisymmetry of å and ∫ separately to obtain

t = (sgn !" )# (v!" 1,!…!,!v!" r )$ (v!" (r+1) ,!…!,!v!" (r+s) )

= (sgn !" )(sgn ! )# (v" 1,!…!,!v" r )$ (v" (r+1) ,!…!,!v" (r+s) )
= (sgn " )# (v" 1,!…!,!v" r )$ (v" (r+1) ,!…!,!v" (r+s) )
= T !!.

(It was in bringing out only a single factor of sgn ƒ that we used the fact that
there is no mixing of å- and ∫-variables.) In other words, the original sum
over all (r + s)! possible permutations ß ∞ Sr+s has been reduced to a sum
over ( Âr± Í) = (r + s)!/r!s! distinct terms, each one of which is repeated r!s!
times. We are thus left with

å°∫(vè, . . . , vr+s) = Í* (sgn ß)å(vß1 , . . . , vßr)∫(vß(r+1) , . . . , vß(r+s))

where the sum is over the (r + s)!/r!s! distinct collections {vß1 , . . . , vßr} and
{vß(r+1) , . . . , vß(r+s)} subject to the requirements ß1 < ~ ~ ~ < ßr and ß(r + 1)
< ~ ~ ~ < ß(r + s). ˙

Let us introduce some convenient notation for handling multiple indices.

Instead of writing the ordered set (iè, . . . , ir), we simply write I where the
exact range will be clear from the context. Furthermore, we write I ö to denote
the increasing sequence (iè < ~ ~ ~ < ir). Similarly, we shall also write vI instead
of (viè , . . . , vi‹). To take full advantage of this notation, we first define the
generalized permutation symbol ´ by

#+1 if ( j1,!…!,! jr ) is an even permutation of (i1,!…!,!ir )

! jr %
!i1j1!ir
= $"1 if ( j1,!…!,! jr ) is an odd permutation of (i1,!…!ir ) !!.
%& ! 0 otherwise

For example, ´2£3∞5™ = +1, ´3¢4£1¡ = -1, ´2¡3¢1™ = 0 etc. In particular, if A = (aji) is an
n x n matrix, then

i1 !in 1
det A = !1!n a i1 !!a n in = !1!n i1 in
i1 !in a 1 !!a n
566 MULTILINEAR MAPPINGS AND TENSORS

because
j1 ! jn
!1!n = ! j1! jn = ! j1! jn !!.

Using this notation and Theorem 11.4, we may write the wedge product of
å and ∫ as

! " # (vi1 ,!…!,!vir+s ) = % $ij !!i j !k i! k ! (v j ,!…!,!v j )# (vk ,!…!,!vk )!

1
1
r
r 1
r+s
s
1 r 1 s
J" , K
"

or most simply in the following form, which we state as a corollary to

Theorem 11.4 for easy reference.

Corollary Suppose å ∞ „r(V) and ∫ ∞ „s(V). Then for any collection of

r + s vectors vá ∞ V (with r + s ¯ dim V) we have

! " # (vI ) = % $ IJK ! (vJ )# (vK )!!.

J! , K
!

Example 11.4 Suppose dim V = 5 and {eè, . . . , e5} is a basis for V. If å ∞

„2(V) and ∫ ∞ „1(V), then

! " # (e5 ,!e2 ,!e3 )

j1 j2 k
= $ j1 < j2 , k% 523 ! (e j1 ,!e j2 )# (ek )
235 253 352
= % 523 ! (e2 ,!e3 )# (e5 ) + % 523! (e2 ,!e5 )# (e3 ) + % 523 ! (e3 ,!e5 )# (e2 )
= ! (e2 ,!e3 )# (e5 ) & ! (e2 ,!e5 )# (e3 ) + ! (e3 ,!e5 )# (e2 )!!.!!!

Our next theorem is a useful result in many computations. It is simply a

contraction of indices in the permutation symbols.

Theorem 11.5 Let I = (iè, . . . , iq), J = (jè, . . . , jr+s), K = (kè, . . . , kr) and
L = (lè, . . . , ls). Then
IJ KL IKL
"!1! q+r+s ! J = !1! q+r+s
J"
where I, K and L are fixed quantities, and J is summed over all increasing
subsets jè < ~ ~ ~ < jr+s of {1, . . . , q + r + s}.

Proof The only nonvanishing terms on the left hand side can occur when J is
a permutation of KL (or else ´J ö ü = 0), and of these possible permutations, we
only have one in the sum, and that is for the increasing set J ö. If J is an even
permutation of KL, then ´J ö ü = +1, and ´1ä` ï` ` q+r+s = ´1ä ` ö `ü ` q+r+s since an even
 11.3 THE EXTERIOR PRODUCT 567

number of permutations is required to go from J to KL. If J is an odd permu-

tation of KL, then ´J ö ü = -1, and ´1ä` ï` ` q+r+s = -´1ä ` ö `ü ` q+r+s since an odd
number of permutations is required to go from J to KL. The conclusion then
follows immediately. ˙

Note that we could have let J = (j1, . . . , jr) and left out L entirely in Theorem
11.5. The reason we included L is shown in the next example.

Example 11.5 Let us use Theorem 11.5 to give a simple proof of the asso-
ciativity of the wedge product. In other words, we want to show that

å°(∫°©) = (å°∫)°©

for any å ∞ „q(V), ∫ ∞ „r(V) and © ∞ „s(V). To see this, let I = (iè, . . . , iq),
J = (jè, . . . , jr+s), K = (kè, . . . , kr) and L = (lè, . . . , ls). Then we have

IJ
! " (# " $ )(v1,!…!,!vq+r+s ) = %!I , J! &1" q+r+s! (vI )(# " $ )(vJ )
IJ
= %!I , J! &1" q+r+s! (vI )%K & KL # (vK )$ (vL )
! , L! J
IKL
= %!I , K! , L! &1" q+r+s! (vI )# (vK )$ (vL )!!.

It is easy to see that had we started with (å°∫)°©, we would have arrived at
the same sum.
As was the case with the tensor product, we simply write å°∫°© from
now on. Note also that a similar calculation can be done for the wedge product
of any number of terms. ∆

We now wish to prove the basic algebraic properties of the wedge product.
This will be facilitated by a preliminary result on the alternation mapping.

Theorem 11.6 If S ∞ Tr and T ∞ Ts , then

A((AS) · T) = A(S · T) = A(S · (AT)) .

Proof Using the bilinearity of the tensor product and the definition of AS we
may write
(AS) · T = (1/r!)Íß´S‹ (sgn ß)[(ßS) · T] .

For each ß ∞ Sr, let G ™ Sr+s be the set of permutations ƒ defined by

(ƒ1, . . . , ƒ(r + s)) = (ß1, . . . , ßr, r + 1, . . . , r + s) .

568 MULTILINEAR MAPPINGS AND TENSORS

In other words, G consists of all permutations ƒ ∞ Sr+s that have the same
effect on 1, . . . , r as ß ∞ Sr, but leave the remaining terms r + 1, . . . , r + s
unchanged. This means that sgn ƒ = sgn ß, and ƒ(S · T) = (ßS) · T (see
Exercise 11.3.1). We then have

(AS) · T = (1/r!)̓´G (sgn ƒ)ƒ(S · T)

and therefore

A(( AS) ! T ) = [1/(r + s)!]"# $Sr+s (sgn # )# ((1/r!)"% $G (sgn % )% (S ! T ))

= [1/(r + s)!](1/r!)"% $G "# $Sr+s (sgn # )(sgn % )#% (S ! T )!!.

But for each ƒ ∞ G, we note that Sr+s = {œ = †ƒ: † ∞ Sr+s}, and hence

[1/(r + s)!]!" #Sr+s (sgn " )(sgn $ )"$ (S % T )

= [1/(r + s)!]!" #Sr+s (sgn "$ )"$ (S % T )
= [1/(r + s)!]!& #Sr+s (sgn & )& (S % T )
= A(S % T )!!.

Since this is independent of the particular ƒ ∞ G and there are r! elements in

G, we then have

A(( AS) ! T ) = (1/r!)"# $G A(S ! T )

= A(S ! T )(1/r!)"# $G 1
= A(S ! T )!!.

The proof that A(S · T) = A(S · (AT)) is similar, and we leave it to the
reader (see Exercise 11.3.2). ˙

Note that in defining the wedge product å°∫, there is really nothing that
requires us to have å ∞ „r(V) and ∫ ∞ „s(V). We could just as well be more
general and let å ∞ Tr (V) and ∫ ∞ Ts (V). However, if this is the case, then
the formula given in Theorem 11.4 most certainly is not valid. However, we
do have the following corollary to Theorem 11.6.

Corollary For any S ∞ Tr and T ∞ Ts we have AS°T = S°T = S° AT.

Proof This follows directly from Theorem 11.6 and the wedge product
definition S°T = [(r + s)!/r!s!] A(S · T). ˙

We are now in a position to prove some of the most important properties

of the wedge product. Note that this next theorem is stated in terms of the
 11.3 THE EXTERIOR PRODUCT 569

more general definition of the wedge product.

Theorem 11.7 Suppose å, åè, åì ∞ Tq (V), ∫, ∫è, ∫ì ∞ Tr (V), © ∞ Ts (V) and

a ∞ F. Then
(a) The wedge product is bilinear. That is,

(åè + åì)°∫ = åè°∫ + åì°∫

å°(∫è + ∫ì) = å°∫è + å°∫ì
(aå)°∫ = å°(a∫) = a(å°∫)

(b) å°∫ = (-1)qr∫°å.

(c) The wedge product is associative. That is,

å°(∫°©) = (å°∫)°© = [(q + r + s)!/q!r!s!]A(å · ∫ · ©) .

Proof (a) This follows from the definition of wedge product, the fact that ·
is bilinear and A is linear. This result may also be shown directly in the case
that å, åè, åì ∞ „q(V) and ∫, ∫è, ∫ì ∞ „r(V) by using the corollary to
Theorem 11.4 (see Exercise 11.3.3).
(b) This can also be shown directly from the corollary to Theorem 11.4
(see Exercise 11.3.4). Alternatively, we may proceed as follows. First note
that for ß ∞ Sr, we see that (since for any other † ∞ Sr we have †(ßå) =
(† ı ß)å, and hence †(ßå)(vè, . . . , vr) = å(v†ß1 , . . . , v†ßr))

A(!" )(v1,!…!,!vr ) = (1/r!)#$ %Sr (sgn $ )$ (!" )(v1,!…!,!vr )

= (1/r!)#$ %Sr (sgn $ )" (v$! 1,!…!,!v$! r )
= (1/r!)#$ %Sr (sgn $! )(sgn ! )" (v$! 1,!…!,!v$! r )
= (sgn ! )(1/r!)#& %Sr (sgn & )" (v& 1,!…!,!v& r )
= (sgn ! ) A! (v1,!…!,!vr )!!.

Hence A(ßå) = (sgn ß) Aå.

Now define ßà ∞ Sq+r by

ßà(1, . . . , q + r) = (q + 1, . . . , q + r, 1, . . . , q) .

Since ßà is just a product of qr transpositions, it follows that sgn ßà = (-1)qr.

We then see that

! " # (v1,!…!,!vq+r ) = (# " ! )(v$ 0 1,!…!,!v$ 0 (q+r) )

= $ 0 (# " ! )(v1,!…!,!vq+r )!!.
570 MULTILINEAR MAPPINGS AND TENSORS

Therefore (ignoring the factorial multiplier which will cancel out from both
sides of this equation),

! " # = A(! $ # ) = A(% 0 (# $ ! )) = (sgn % 0 ) A(# $ ! )

= (&1)qr # " ! !!.

(c) Using Theorem 11.6, we simply calculate

(! " # ) " $ = [(q + r + s)!/(q + r)!s!] A((! " # ) % $ )

= [(q + r + s)!/(q + r)!s!][(q + r)!/q!r!] A( A(! % # ) % $ )
= [(q + r + s)!/(q!r!s!] A(! % # % $ )!!.

Similarly, we find that å°(∫°©) yields the same result. We are therefore justi-
fied (as we also saw in Example 11.5) in writing simply å°∫°©. Furthermore,
it is clear that this result can be extended to any finite number of products. ˙

Example 11.6 Suppose å ∞ Tr and ∫ ∞ Ts. Since å°∫ = (-1)rs∫°å, we see

that if either r or s is even, then å°∫ = ∫°å, but if both r and s are odd, then
å°∫ = -∫°å. Therefore if r is odd we have å°å = 0, but if r is even, then
å°å is not necessarily zero. In particular, any 1-form å always has the
property that å°å = 0. ∆

Example 11.7 If å1 , . . . , å5 are 1-forms on ®5, let us define

∫ = å1°å3 + å3°å5
and
© = 2å2°å4°å5 - å1°å2°å4 .

Using the properties of the wedge product given in Theorem 11.7 we then
have

∫°© = (å1°å3 + å3°å5)°(2å2°å4°å5 - å1°å2°å4)

= 2å1°å3°å2°å4°å5 - å1°å3°å1°å2°å4
+ 2å3°å5°å2°å4°å5 - å3°å5°å1°å2°å4
= -2å1°å2°å3°å4°å5 - 0 + 0 - å1°å2°å3°å4°å5
= -3å1°å2°å3°å4°å5 . ∆

Example 11.8 Suppose å1 , . . . , år ∞ „1(V) and vè, . . . , vr ∞ V. Using

Theorem 11.5, it is easy to generalize the corollary to Theorem 11.4 to obtain
(see Exercise 11.3.5)
 11.3 THE EXTERIOR PRODUCT 571

i1 ! ir
!1 "! " ! r (v1, …, vr ) = #i1 ! ir $1! r !1 (vi1 ) ! ! r (vir )
= det(!i (v j ))!!.

(Note that the sum is not over any increasing indices because each åi is only a
1-form.) As a special case, suppose {eá} is a basis for V and {øj} is the cor-
responding dual basis. Then øj(eá) = ∂já and hence

! kr i1
! i1 "! " ! ir (e j1 , …,!e jr ) = #k1 ! kr $ kj11 ! ir
jr ! (ek1 ) ! ! (ekr )

= $ ij1 ! ir
! j !!.
1 r

In particular, if dim V = n, choosing the indices (iè, . . . , in) = (1, . . . , n) =

(jè, . . . , jn), we see that

ø1° ~ ~ ~ °øn (eè, . . . , eñ) = 1 . ∆

Exercises

1. Show that ƒ(S · T) = (ßS) · T in the proof of Theorem 11.6.

2. Finish the proof of Theorem 11.6 by showing that

A(S · T) = A(S · (AT)).

3. Using åá ∞ „q(V) and ∫á ∞ „r(V), prove Theorem 11.7(a) directly from

the corollary to Theorem 11.4.

4. Use the corollary to Theorem 11.4 to prove Theorem 11.7(b).

5. Suppose åè, . . . , år ∞ „1(V) and vè, . . . , vr ∞ V. Show that

åè° ~ ~ ~ °år(vè, . . . , vr) = det(åá(vé)) .

6. Suppose {eè, . . . , eñ} is a basis for V and {ø1, . . . , øn} is the corre-
sponding dual basis. If å ∞ „r(V) (where r ¯ n), show that

å = ÍIö å(eI)øI = Íiè< ~ ~ ~ <i‹ å(eiè, . . . , ei‹)øiè° ~ ~ ~ °øi‹

by applying both sides to (ejè, . . . , ej‹). (See also Theorem 11.8.)

572 MULTILINEAR MAPPINGS AND TENSORS

7. (Interior Product) Suppose å ∞ „r(V) and v, vì, . . . , vr ∞ V. We

define the (r - 1)-form ivå by

iv! = 0 if r = 0.
iv! = ! (v) if r = 1.
iv! (v2 ,!…!,!vr ) = ! (v,!v2 ,!…!,!vr ) if r > 1.

(a) Prove that iu+v = iu + iv.

(b) If å ∞ „r(V) and ∫ ∞ „s(V), prove that iv: „r+s(V) ‘ „r+s-1(V) is
an anti-derivation, i.e.,

iv(å°∫) = (iv å)°∫ + (-1)rå°(iv ∫) .

(c) If v = vieá and å = ÍIö aiè ~ ~ ~ i‹øiè° ~ ~ ~ °øi‹ where {øi} is the basis
dual to {ei}, show that

ivå = Íiì< ~ ~ ~ <i‹biì ~ ~ ~ i‹øiì°~ ~ ~°øi‹

where
biì ~ ~ ~ i‹ = Íévj ajiì ~ ~ ~ i‹ .

(d) If å = f1° ~ ~ ~ °fr, show that

r
iv! = $ ("1)k"1 f k (v) f 1 #! # f k"1 # f k+1 #! # f r
k=1
r
"
= $ ("1)k"1 f k (v) f 1 #! # f k #! # f r
k=1

where the Ä means that the term fk is to be deleted from the expression.

8. Let V = ®n have the standard basis {eá}, and let the corresponding dual
basis for V* be {øi}.
(a) If u, v ∞ V, show that
i j ui vi
! " ! (u,!v) =
uj vj

and that this is ± the area of the parallelogram spanned by the projection
of u and v onto the xi xj-plane. What do you think is the significance of
the different signs?
 11.3 THE EXTERIOR PRODUCT 573

(b) Generalize this to øiè° ~ ~ ~ °øi‹ where r ¯ n.

9. Suppose V = Fn, and let v1, . . . , vn ∞ V have components relative to the

standard basis {eá} defined by vi = ejvji. For any 1 ¯ r < n, let s = n - r
and define the r-form å by
v11 ! v1r
! (v1,!…!,!vr ) = " " !!!
v r1 ! v r r
and the s-form ∫ by
v1r+1 ! v1n
! (v1,!…!,!vs ) = " " !!.
v s r+1 ! v s n

(a) Use Theorem 4.9 to show that å°∫ is the determinant function D on
Fn.
(b) Show that the sign of an (r, s)-shuffle is given by

i1 ! ir j1 ! js i1 + ! +ir +r(r+1)/2
!1! r !r+1! r+s = ("1)

where i1, . . . , ir and j1, . . . , js are listed in increasing order.

(c) If A = (aij) ∞ Mn(F), prove the Laplace expansion formula

ai1 1 " ai1 r a j1 r+1 " a j1 n

det A = !!I ("1)i1 + " +ir +r(r+1)/2 # # # #
air 1 " air r a jr r+1 " a jr n

where I = {i1, . . , ir} and J = {j1, . . . , js} are “complementary” sets of

indices, i.e., I ⁄ J = Å and I ¡ J = {1, 2, . . . , n}.

10. Let B = r!A where A: Tr ‘Tr is the alternation mapping. Define å°∫ in
terms of B. What is B(f1 · ~ ~ ~ · fr) where fi ∞ V*?

11. Let I = (i1, . . . , iq), J = (j1, . . . , jp), and K = (k1, . . . , kq). Prove the
following generalization of Example 11.3:

JI 1! p+q i ]
"!1! p+q! JK = ! KI = n!#k[i 1 ! #kq
1 q
J"
574 MULTILINEAR MAPPINGS AND TENSORS

11.4 TENSOR ALGEBRAS

We define the direct sum of all tensor spaces Tr (V) to be the (infinite-
dimensional) space T0 (V) • T1(V) • ~ ~ ~ • Tr (V) • ~ ~ ~ , and T (V) to be all
elements of this space with finitely many nonzero components. This means
that every element T ∞ T(V) has a unique expression of the form (ignoring
zero summands)
T = T(1)iè + ~ ~ ~ + T(r)i‹

where each T(k)iÉ ∞ TiÉ (V) and iè < ~ ~ ~ < ir. The tensors T(k)iÉ are called the
graded components of T. In the special case that T ∞ Tr (V) for some r, then
T is said to be of order r. We define addition in T (V) componentwise, and we
also define multiplication in T (V) by defining · to be distributive on all of
T (V). We have therefore made T (V) into an associative algebra over F which
is called the tensor algebra.
We have seen that „r(V) is a subspace of Tr (V) since „r(V) is just the
image of Tr (V) under A. Recall also that „0(V) = T0 (V) is defined to be the
scalar field F. As might therefore be expected, we define „(V) to be the
direct sum

„(V) = „0(V) • „1(V) • ~ ~ ~ • „r(V) • ~ ~ ~ ™ T (V) .

Note that „r(V) = 0 if r > dim V.

It is important to realize that if å ∞ „r(V) ™ Tr (V) and ∫ ∞ „s(V) ™
Ts (V), then even though å · ∫ ∞ Tr+s (V), it is not generally true that
å · ∫ ∞ „r+s(V). Therefore „(V) is not a subalgebra of T (V). However, the
wedge product is a mapping from „r(V) ª „s(V) ‘ „r+s(V), and hence if
we extend this product in the obvious manner to a bilinear mapping „(V) ª
„(V) ‘ „(V), then „(V) becomes an algebra over F = „0(V). In other
words, if å = åè + ~ ~ ~ + år with each åá ∞ „rá(V), and ∫ = ∫è + ~ ~ ~ + ∫s with
each ∫á ∞ „sá(V), then we define
r s
! " # = $ $ !i " # j !!.
i=1 j=1
This algebra is called the Grassmann (or exterior) algebra.
The astute reader may be wondering exactly how we add together the ele-
ments åè ∞ „rè(V) and åì ∞ „rì(V) (with rè ≠ rì) when none of the opera-
tions (åè + åì)(vè, . . . , vrè), (åè + åì)(vè, . . . , vrì) nor (åè + åì)(vè, . . . ,
vrè+rì) makes any sense. The answer is that for purposes of the Grassmann
algebra, we consider both åè and åì to be elements of „(V). For example, if
 11.4 TENSOR ALGEBRAS 575

åè is a 1-form and åì is a 2-form, then we write åè = 0 + åè + 0 + 0 + ~ ~ ~ and

åì = 0 + 0 + åì + 0 + ~ ~ ~ , and hence aèåè + aìåì (where aá ∞ F) makes sense
in „(V). In this way, every element of „(V) has a degree (recall that an r-
form is said to be of degree r), and we say that „(V) is a graded associative
algebra.
Unlike the infinite-dimensional algebra T (V), the algebra „(V) is finite-
dimensional. This should be clear from the discussion following Example 11.2
where we showed that dim „r(V) = (rˆ) (where n = dim V), and hence that
dim „r(V) = 0 if r > n. Let us now prove this result again by constructing a
specific basis for „r(V).

Theorem 11.8 Suppose dim V = n. Then for r > n we have „r(V) = {0}, and
if 0 ¯ r ¯ n, then dim „r(V) = (rˆ). Therefore dim „(V) = 2n. Moreover, if
{ø1, . . . , øn} is a basis for V* = „1(V), then a basis for „r(V) is given by
the set
{øiè° ~ ~ ~ °øi‹: 1 ¯ iè < ~ ~ ~ < ir ¯ n} .

Proof Suppose å ∞ „r(V) where r > dim V = n. By multilinearity, å is

determined by its values on a basis {eè, . . . , eñ} for V (see Example 11.2).
But then we must have å(eiè , . . . , ei‹) = 0 because at least two of the eiÉ are
necessarily the same and å is antisymmetric. This means that å(vè, . . . , vr) =
0 for all vá ∞ V, and hence å = 0. Thus „r(V) = {0} if r > n.
Now suppose that {ø1, . . . , øn} is the basis for V* dual to {eá}. From
Theorem 11.2, we know that {øiè · ~ ~ ~ · øi‹: 1 ¯ iè , . . . , ir ¯ n} forms a
basis for Tr (V), and since the alternation mapping A maps Tr (V) onto „r(V)
(Theorem 11.3(b)), it follows that the image of the basis {øiè · ~ ~ ~ · øi‹} for
Tr (V) must span „r(V). If å ∞ „r(V), then å ∞ Tr (V), and hence

å = åiè ~ ~ ~ i‹ øiè · ~ ~ ~ · øi‹

where the sum is over all 1 ¯ iè, . . . , ir ¯ n and åiè ~ ~ ~ i‹ = å(eiè , . . . , ei‹).
Using Theorems 11.3(a) and 11.7(c) we have

! = A! = !i1! ir A(" i1 # ! # " ir )

= !i1! ir (1/r!)" i1$!$ " ir

where the sum is still over all 1 ¯ iè, . . . , ir ¯ n. However, by the antisymme-
try of the wedge product, the collection {iè, . . . , ir} must all be different, and
hence the sum can only be over the (rˆ) distinct such combinations. For each
576 MULTILINEAR MAPPINGS AND TENSORS

such combination there will be r! permutations ß ∞ Sr of the basis vectors. If

we write each of these permutations in increasing order iè < ~ ~ ~ < ir, then the
wedge product changes by a factor sgn ß, as does åiè ~ ~ ~ i‹ = å(eiè , . . . , ei‹).
Therefore the signs cancel and we are left with

å = å\iè ~ ~ ~ i‹\ øiè° ~ ~ ~ °øi‹

where, as mentioned previously, we use the notation å\ iè ~ ~ ~ i‹\ to mean that the
sum is over increasing sets iè < ~ ~ ~ < ir. Thus we have shown that the (rˆ) ele-
ments øiè° ~ ~ ~ °øi‹ with 1 ¯ iè < ~ ~ ~ < ir ¯ n span „r(V). We must still show
that they are linearly independent.
Suppose å\iè ~ ~ ~ i‹\ øiè° ~ ~ ~ °øi‹ = 0. Then for any set {ejè , . . . , ej‹} with
1 ¯ jè < ~ ~ ~ < jr ¯ n we have (using Example 11.8)

0 = !|i1 ! ir |" i1 #! # " ir (e j1 , …,!e jr )

= !|i1 ! ir |$ ij1 !
!j
ir
1 r

= ! j1 ! jr

since the only nonvanishing term occurs when {iè, . . . , ir} is a permutation of
{jè, . . . , jr} and both are increasing sets. This proves linear independence.
Finally, using the binomial theorem, we now see that

n n " %
n
dim„(V ) = ! dim„r (V ) = ! $ ' = (1+1)n = 2 n !!.!!˙
r=0
r
r=0 # &

Example 11.9 Another useful result is the following. Suppose dim V = n,

and let {ø1, . . . , øn} be a basis for V*. If å1, . . . , ån are any other 1-forms in
„1(V) = V*, then we may expand each åi in terms of the øj as åi = aiéøj. We
then have

!1 "! " ! n = a1in ! a n in # i1 "! " # in

i1 ! in 1
= a1in ! a n in $1! n # "! " #
n

= det (ai j )#1 "! " # n !!.

Recalling Example 11.1, if {øi = dxi} is a local basis for a cotangent space V*
and {åi = dyi} is any other local basis, then dyi = ($yi/$xj)dxj and
 11.4 TENSOR ALGEBRAS 577

!(y1 !y n )
det (ai j ) =
!(x1 !x n )

is just the usual Jacobian of the transformation. We then have

"(y1 !y n ) 1
dy1 !! ! dy n = 1 n
dx !! ! dx n !!.
"(x !x )

The reader may recognize dx1° ~ ~ ~ °dxn as the volume element on ®n, and
hence differential forms are a natural way to describe the change of variables
in multiple integrals. ∆

Theorem 11.9 If å1, . . . , år ∞ „1(V), then {å1, . . . , år} is a linearly

dependent set if and only if å1° ~ ~ ~ °år = 0.

Proof If {å1, . . . , år} is linearly dependent, then there exists at least one
vector, say å1, such that !1 = " j#1a j! j . But then

!1 "! " ! r = ( # j$1a j! j ) " ! 2 "! " ! r

= # j$1a j (! j " ! 2 "! " ! r )
=0

since every term in the sum contains a repeated 1-form and hence vanishes.
Conversely, suppose that å1, . . . , år are linearly independent. We can
then extend them to a basis {å1, . . . , ån} for V* (Theorem 2.10). If {eá} is the
corresponding dual basis for V, then å1° ~ ~ ~ °ån(eè, . . . , eñ) = 1 which
implies that å1° ~ ~ ~ °år ≠ 0. Therefore {å1, . . . , år} must be linearly depen-
dent if å1° ~ ~ ~ °år = 0. ˙

11.5 THE TENSOR PRODUCT OF VECTOR SPACES

We now discuss the notion of the tensor product of vector spaces. Our reason
for presenting this discussion is that it provides the basis for defining the
Kronecker (or direct) product of two matrices, a concept which is very useful
in the theory of group representations.
It should be remarked that there are many ways of defining the tensor
product of vector spaces. While we will follow the simplest approach, there is
another (somewhat complicated) method involving quotient spaces that is also
578 MULTILINEAR MAPPINGS AND TENSORS

frequently used. This other method has the advantage that it includes infinite-
dimensional spaces. The reader can find a treatment of this alternative method
in, e.g., in the book by Curtis (1984).
By way of nomenclature, we say that a mapping f: U ª V ‘ W of vector
spaces U and V to a vector space W is bilinear if f is linear in each variable.
This is exactly the same as we defined in Section 9.4 except that now f takes
its values in W rather than the field F. In addition, we will need the concept of
a vector space generated by a set. In other words, suppose S = {sè, . . . , sñ} is
some finite set of objects, and F is a field. While we may have an intuitive
sense of what it should mean to write formal linear combinations of the form
aèsè + ~ ~ ~ + añsñ, we should realize that the + sign as used here has no
meaning for an arbitrary set S. We now go through the formalities involved in
defining such terms, and hence make the set S into a vector space T over F.
The basic idea is that we want to recast each element of S into the form of
a function from S to F. This is because we already know how to add functions
as well as multiply them by a scalar. With these ideas in mind, for each sá ∞ S
we define a function sá: S ‘ F by

sá(sé) = 1∂áé

where 1 is the multiplicative identity of F. Since addition in F is well-defined

as is the addition of functions and multiplication of functions by elements of
F, we see that for any a, b ∞ F and sá, sé ∞ S we have

(a + b)si (s j ) = (a + b)!ij = a!ij + b!ij = asi (s j ) + bsi (s j )

= (asi + bsi )(s j )

and therefore (a + b)sá = asá + bsá. Similarly, it is easy to see that a(bsá) =
(ab)sá.
We now define T to be the set of all functions from S to F. These func-
tions can be written in the form aèsè + ~ ~ ~ + añsñ with aá ∞ F. It should be
clear that with our definition of the terms aásá, T forms a vector space over F.
In fact, it is easy to see that the functions 1sè , . . . , 1sñ are linearly
independent. Indeed, if 0 denotes the zero function, suppose aèsè + ~ ~ ~ + añsñ
= 0 for some set of scalars aá. Applying this function to sá (where 1 ¯ i ¯ n) we
obtain aá = 0. As a matter of course, we simply write sá rather than 1sá.
The linear combinations just defined are called formal linear combina-
tions of the elements of S, and T is the vector space generated by the set S. T
is therefore the vector space of all such formal linear combinations, and is
sometimes called the free vector space of S over F.
 11.5 THE TENSOR PRODUCT OF VECTOR SPACES 579

Theorem 11.10 Let U, V and W be finite-dimensional vector spaces over F.

Then there exists a finite-dimensional vector space over F denoted by T and a
bilinear mapping t: U ª V ‘ T denoted by t(u, v) = u · v satisfying the fol-
lowing properties:
(a) For every bilinear mapping f: U ª V ‘ W, there exists a unique linear
transformation f ÿ: T ‘ W such that f = f ÿı t. In other words, for all u ∞ U and
v ∞ V we have
f(u, v) = f ÿ(t(u, v)) = f ÿ(u · v) .

(b) If {uè, . . . , um} is a basis for U and {vè, . . . , vñ} is a basis for V, then
{uá · vé} is a basis for T and therefore

dim T = mn = (dim U)(dim V) .

Proof Let {uè, . . . , um} be a basis for U and let {vè, . . . , vñ} be a basis for
V. For each pair of integers (i, j) with 1 ¯ i ¯ m and 1 ¯ j ¯ n we let táé be a
letter (i.e., an element of some set). We now define T to be the vector space
over F consisting of all formal linear combinations of the elements táé. In other
words, every element of T is of the form aij táé where aij ∞ F.
Define the bilinear map t: U ª V ‘ T by

ui · vj ^ t(ui, vj) = tij

and hence to all of U ª V by “bilinear extension.” In particular, if u = xiuá ∞ U

and v = yjvé ∞ V, let us define u · v to be that element of T given by

u · v = t(u, v) = xiyj táé .

It should be obvious that this does indeed define a bilinear map.

Now suppose that f: U ª V ‘ W is any bilinear map, and remember that
every element of T is a linear combination of the táé. According to Theorem
5.1, we may define a unique linear transformation f ÿ: T ‘ W by

f ÿ(táé) = f(uá, vé) .

Using the bilinearity of f and the linearity of f ÿ we then have

f (u,!v) = f (x i ui ,!y j v j ) = x i y j f (ui ,!v j ) = x i y j f! (tij ) = f! (x i y j tij )

= f! (u ! v) = f! (t(u,!v))!!.
580 MULTILINEAR MAPPINGS AND TENSORS

This proves the existence and uniqueness of the mapping f ÿ such that f = f ÿı t
as specified in (a).
We have defined T to be the vector space generated by the mn elements
táé = uá · vé where {uè, . . . , um} and {vè, . . . , vñ} were particular bases for U
and V respectively. We now want to show that in fact {uæá · væé} forms a basis
for T where {uæá} and {væé} are arbitrary bases for U and V. For any u =
xæi uæá ∞ U and v = yæj væé ∞ V, we have (using the bilinearity of ·)

u · v = xæi yæj (uæá · væé)

which shows that the mn elements uæá · væé span T. If these mn elements are
linearly dependent, then dim T < mn which contradicts the fact that the mn
elements táé form a basis for T. Hence {uæá · væé} is a basis for T. ˙

The space T defined in this theorem is denoted by U · V and called the

tensor product of U and V. Note that T can be any mn dimensional vector
space. For example, if m = n we could take T = T 2(V) with basis tij = øi · øj,
1 ¯ i, j ¯ n. The map t(ui, vj) = ui · vj then defines ui · vj = øi · øj.

Example 11.10 To show how this formalism relates to our previous treat-
ment of tensors, consider the following example of the mapping f ÿ defined in
Theorem 11.10. Let {eá} be a basis for a real inner product space U, and let us
define the real numbers gáé = Óeá, eéÔ. If eõá = eépjá is another basis for U, then

gõáé = Óeõá, eõéÔ = prápséÓer , esÔ = prápsé grs

so that the gáé transform like the components of a covariant tensor of order 2.
This means that we may define the tensor g ∞ T2(U) by g(u, v) = Óu, vÔ. This
tensor is called the metric tensor on U (see Section 11.10).
Now suppose that we are given a positive definite symmetric bilinear form
(i.e., an inner product) g = Ó , Ô: U ª U ‘ F. Then the mapping gÿ is just the
metric because
g ÿ(eá · eé) = g(eá, eé) = Óeá, eéÔ = gáé .

Therefore, if u = uieá and v = vjeé are vectors in U, we see that

g ÿ(u · v) = g(u, v) = Óu, vÔ = ui vj Óeá, eéÔ = gáéui vj .

If {øi} is the basis for U* dual to {eá}, then according to our earlier formal-
ism, we would write this as g ÿ = gáéøi · øj. ∆
 11.5 THE TENSOR PRODUCT OF VECTOR SPACES 581

Some of the main applications in mathematics and physics (e.g., in the

theory of group representations) of the tensor product of two vector spaces are
contained in the next two results. While the ideas are simple enough, the nota-
tion becomes somewhat awkward because of the double indices.

Theorem 11.11 Let U and V have the respective bases {uè, . . . , um} and
{vè, . . . , vñ}, and suppose the linear operators S ∞ L(U) and T ∞ L(V) have
matrix representations A = (aié) and B = (bié) respectively. Then there exists a
linear transformation S · T: U · V ‘ U · V such that for all u ∞ U and v ∞
V we have (S · T)(u · v) = S(u) · T(v).
Furthermore, the matrix C of S · T relative to the ordered basis

{uè · vè, . . . , uè · vñ, uì · vè, . . . , uì · vñ, . . . , um · vè, . . . , um · vñ}

for U · V is the mn x mn matrix given in block matrix form as

! a1 B a1 B ! a1 B $
# 1 2 m
&
C =# " " " &!!.
# m m m &
" a 1B a 2 B ! a m B %

The matrix C is called the Kronecker (or direct or tensor) product of the
matrices A and B, and will also be written as C = A · B.

Proof Since S and T are linear and · is bilinear, it is easy to see that the
mapping f: U ª V ‘ U · V defined by f(u, v) = S(u) · T(v) is bilinear.
Therefore, according to Theorem 11.10, there exists a unique linear transfor-
mation f ÿ ∞ L(U · V) such that f ÿ(u · v) = S(u) · T(v). We denote the map-
ping f ÿ by S · T. Thus, (S · T)(u · v) = S(u) · T(v).
To find the matrix C of S · T is straightforward enough. We have S(uá) =
ué ajá and T(vá) = vébjá, and hence

(S · T)(uá · vé) = S(uá) · T(vé) = aribsj(ur · vs) .

Now recall that the ith column of the matrix representation of an operator is
just the image of the ith basis vector under the transformation (see Theorem
5.11). In the present case, we will have to use double pairs of subscripts to
label the matrix elements. Relative to the ordered basis

{uè · vè, . . . , uè · vñ, uì · vè, . . . , uì · vñ, . . . , um · vè, . . . , um · vñ}

582 MULTILINEAR MAPPINGS AND TENSORS

for U · V, we then see that, for example, the (1, 1)th column of C is the
vector (S · T)(uè · vè) = arèbsè(ur · vs) given by

(a11b11, . . . , a11bn1, a21b11, . . . , a21bn1, . . . , am1b11, . . . , am1bn1)

and in general, the (i, j)th column is given by

(a1ib1j, . . . , a1ibnj, a2ib1j, . . . , a2ibnj, . . . , amib1j, . . . , amibnj) .

This shows that the matrix C has the desired form. ˙

Theorem 11.12 Let U and V be finite-dimensional vector spaces over F.

(a) If Sè, Sì ∞ L(U) and Tè, Tì ∞ L(V), then

(Sè · Tè)(Sì · Tì) = SèSì · TèTì .

Moreover, if Aá and Bá are the matrix representations of Sá and Tá respectively

(relative to some basis for U · V), then (Aè · Bè)(Aì · Bì) = AèAì · BèBì.
(b) If S ∞ L(U) and T ∞ L(V), then Tr(S · T) = (Tr S)(Tr T).
(c) If S ∞ L(U) and T ∞ L(V), and if Sî and Tî exist, then

(S · T)î = Sî · Tî .

Conversely, if (S · T)î exists, then Sî and Tî also exist, and (S · T)î =

Sî · Tî.

Proof (a) For any u ∞ U and v ∞ V we have

(S1 ! T1 )(S2 ! T2 )(u ! v) = (S1 ! T1 )(S2 (u) ! T2 (v))

= S1S2 (u) ! T1T2 (v)
= (S1S2 ! T1T2 )(u ! v)!!.

As to the matrix representations, simply note that Aá · Bá is the representation

of Sá · Tá, and AèAì · BèBì is the representation of SèSì · TèTì (since the
representation of a product of linear operators is the product of their matrices).
(b) Recall that the trace of a linear operator is defined to be the trace of
any matrix representation of the operator (see Theorem 5.19). Therefore, if
A = (aié) is the matrix of S and B = (bié) is the matrix of T, we see from
Theorem 11.11 that the diagonal blocks of A · B are the matrices a1èB, . . . ,
ammB and hence the diagonal elements of A · B are a1èb1è, . . . , a1èbnñ, . . . ,
ammb1è, . . . , ammbnñ. Therefore the sum of these diagonal elements is just
 11.5 THE TENSOR PRODUCT OF VECTOR SPACES 583

Tr(A ! B) = a11 ( "i bi i ) +!+ a m m ( "i bi i )

= ( " j a j j )( "i bi i )
= (Tr A)(Tr B)!!.

(c) We first note that if 1 denotes the identity transformation, then

(1 · 1)(u · v) = u · v

and hence 1 · 1 = 1. Next note that u · v = (u + 0) · v = u · v + 0 · v, and

hence 0 · v = 0. Similarly, it is clear that u · 0 = 0. This then shows that

(S · 0)(u · v) = S(u) · 0 = 0

so that S · 0 = 0, and similarly 0 · T = 0.

Now, if S and T are invertible, then by part (a) we see that

(Sî · Tî)(S · T) = SSî · TîT = 1 · 1 = 1

and similarly for (S · T)(Sî · Tî). Therefore (S · T)î = Sî · Tî.

Conversely, suppose that S · T is invertible. To prove that S and T are
also invertible we use Theorem 5.9. In other words, a surjective linear
operator is invertible if and only if its kernel is zero. Since S · T is invertible
we must have T ≠ 0, and hence there exists v ∞ V such that T(v) ≠ 0. Suppose
u ∞ U is such that S(u) = 0. Then

0 = S(u) · T(v) = (S · T)(u · v)

which implies that u · v = 0 (since S · T is invertible). But v ≠ 0, and hence

we must have u = 0. This shows that S is invertible. Similarly, had we started
with S ≠ 0, we would have found that T is invertible. ˙

Exercises

1. Give a direct proof of the matrix part of Theorem 11.12(a) using the
definition of the Kronecker product of two matrices.

2. Suppose A ∞ L(U) and B ∞ L(V) where dim U = n and dim V = m. Show

that
det(A · B) = (det A)m (det B)n .
584 MULTILINEAR MAPPINGS AND TENSORS

11.6 VOLUMES IN ®3

Instead of starting out with an abstract presentation of volumes, we shall first

go through an intuitive elementary discussion beginning with ®2, then going
to ®3, and finally generalizing to ®n in the next section.
First consider a parallelogram in ®2 (with the usual norm) defined by the
vectors X and Y as shown.

Y A1
A1 h A2
Y
! b
X

Note that h = ˜ Y ˜ sin œ and b = ˜ Y ˜ cos œ, and also that the area of each tri-
angle is given by Aè = (1/2)bh. Then the area of the rectangle is given by Aì =
( ˜ X ˜ - b)h, and the area of the entire parallelogram is given by

A = 2A1 + A2 = bh + (˜X˜! ! !!b)h =!˜X˜h =!˜X˜!˜Y ˜sin" ! !!. (1)

The reader should recognize this as the magnitude of the elementary “vector
cross product” X ª Y of the ordered pair of vectors (X, Y) that is defined to
have a direction normal to the plane spanned by X and Y, and given by the
“right hand rule” (i.e., out of the plane in this case).
If we define the usual orthogonal coordinate system with the x-axis
parallel to the vector X, then

X = (x1, x2) = ( ˜ X ˜ , 0)
and
Y = (y1, y2) = ( ˜ Y ˜ cos œ, ˜ Y ˜ sin œ)

and hence we see that the determinant with columns formed from the vectors
X and Y is just

x1 y1 ˜X˜ ˜Y˜! cos!

= = ˜X˜!˜Y˜! sin! = A!!. (2)
x2 y2 0 ˜Y˜! sin!

Notice that if we interchanged the vectors X and Y in the diagram, then the
determinant would change sign and the vector X ª Y (which by definition has
a direction dependent on the ordered pair (X, Y)) would point into the page.
 11.6 VOLUMES IN ®3 585

Thus the area of a parallelogram (which is always positive by definition)

defined by two vectors in ®2 is in general given by the absolute value of the
determinant (2).
In terms of the usual inner product (or “dot product”) Ó , Ô on ®2, we have
ÓX, XÔ = ˜ X ˜ 2 and ÓX, YÔ = ÓY, XÔ = ˜ X ˜ ˜ Y ˜ cos œ, and hence

A 2 = ˜X˜2 ˜Y ˜2 sin 2 !
= ˜X˜2 ˜Y ˜2 (1 " cos 2 ! )
= ˜X˜2 ˜Y ˜2 " ÓX,!YÔ2 !!.

Therefore we see that the area is also given by the positive square root of the
determinant
ÓX,! XÔ ÓX,!YÔ
A2 = !!. (3)
ÓY ,! XÔ ÓY ,!YÔ

It is also worth noting that the inner product may be written in the form
ÓX, YÔ = x1y1 + x2y2, and thus in terms of matrices we may write

!ÓX,! XÔ ÓX,!YÔ$ ! x1 x 2 $ ! x1 y1 $
# &=# &# &!!.
"ÓY ,! XÔ ÓY ,!YÔ% #" y1 &#
y2 %" x 2
&
y2 %

Hence taking the determinant of this equation (using Theorems 4.8 and 4.1),
we find (at least in ®2) that the determinant (3) also implies that the area is
given by the absolute value of the determinant in equation (2).
It is now easy to extend this discussion to a parallelogram in ®3. Indeed, if
X = (x1, x2, x3) and Y = (y1, y2, y3) are vectors in ®3, then equation (1) is
unchanged because any two vectors in ®3 define the plane ®2 spanned by the
two vectors. Equation (3) also remains unchanged since its derivation did not
depend on the specific coordinates of X and Y in ®2. However, the left hand
part of equation (2) does not apply (although we will see below that the three-
dimensional version determines a volume in ®3).
As a final remark on parallelograms, note that if X and Y are linearly
dependent, then aX + bY = 0 so that Y = -(a/b)X, and hence X and Y are co-
linear. Therefore œ equals 0 or π so that all equations for the area in terms of
sin œ are equal to zero. Since X and Y are dependent, this also means that the
determinant in equation (2) equals zero, and everything is consistent.
586 MULTILINEAR MAPPINGS AND TENSORS

We now take a look at volumes in ®3. Consider three linearly independent

vectors X = (x1, x2, x3), Y = (y1, y2, y3) and Z = (z1, z2, z3), and consider the
parallelepiped with edges defined by these three vectors (in the given order
(X, Y, Z)).

Z U
Y
X

We claim that the volume of this parallelepiped is given by both the positive
square root of the determinant

ÓX,! XÔ ÓX,!YÔ ÓX,!ZÔ

ÓY ,! XÔ ÓY ,!YÔ ÓY ,!ZÔ (4)
ÓZ,! XÔ ÓZ,!YÔ ÓZ,!ZÔ

and the absolute value of the determinant

x1 y1 z1
x2 y2 z 2 !!. (5)
3 3 3
x y z

To see this, first note that the volume of the parallelepiped is given by the
product of the area of the base times the height, where the area A of the base
is given by equation (3) and the height ˜ U ˜ is just the projection of Z onto the
orthogonal complement in ®3 of the space spanned by X and Y. In other
words, if W is the subspace of V = ®3 spanned by X and Y, then (by Theorem
2.22) V = WÊ • W, and hence by Theorem 2.12 we may write

Z = U + aX + bY

where U ∞ WÊ and a, b ∞ ® are uniquely determined (the uniqueness of a and

b actually follows from Theorem 2.3 together with Theorem 2.12).
By definition we have ÓX, UÔ = ÓY, UÔ = 0, and therefore
 11.6 VOLUMES IN ®3 587

ÓX,!ZÔ = a˜X˜2 + bÓX,!YÔ

ÓY ,!ZÔ = aÓY ,! XÔ + b˜Y ˜2 (6)
ÓU,!ZÔ = ˜U˜2 !!.

We now wish to solve the first two of these equations for a and b by Cramer’s
rule (Theorem 4.13). Note that the determinant of the matrix of coefficients is
just equation (3), and hence is just the square of the area A of the base of the
parallelepiped. Applying Cramer’s rule we have

ÓX,!ZÔ ÓX,!YÔ ÓX,!YÔ ÓX,!ZÔ

aA 2 = =!
ÓY ,!ZÔ ÓY ,!YÔ ÓY ,!YÔ ÓY ,!ZÔ
ÓX,! XÔ ÓX,!ZÔ
bA 2 = !!.
ÓY ,! XÔ ÓY ,!ZÔ

Denoting the volume by Vol(X, Y, Z), we now have (using the last of equa-
tions (6) together with U = Z - aX - bY)

Vol2(X, Y, Z) = A2 ˜ U ˜ 2 = A2ÓU, ZÔ = A2(ÓZ, ZÔ - aÓX, ZÔ - bÓY, ZÔ)

so that substituting the expressions for A2, aA2 and bA2, we find

ÓX,! XÔ ÓX,!YÔ ÓX,!YÔ ÓX,!ZÔ

Vol2 (X,!Y ,!Z ) = ÓZ,!ZÔ + ÓX,!ZÔ
ÓY ,! XÔ ÓY ,!YÔ ÓY ,!YÔ ÓY ,!ZÔ
ÓX,! XÔ ÓX,!ZÔ
! ÓY ,!ZÔ !!.
ÓY ,! XÔ ÓY ,!ZÔ

Using ÓX, YÔ = ÓY, XÔ etc., we see that this is just the expansion of a determi-
nant by minors of the third row, and hence (using det AT = det A)

ÓX,! XÔ ÓY ,! XÔ ÓZ,! XÔ
2
Vol (X,!Y ,!Z ) = ÓX,!YÔ ÓY ,!YÔ ÓZ,!YÔ
ÓX,!ZÔ ÓY ,!ZÔ ÓZ,!ZÔ
2
x1 x2 x 3 x1 y1 z1 x1 y1 z1
= y1 y2 y3 x2 y2 z2 = x2 y2 z 2 !!.
z1 z2 z3 x3 y3 z3 x3 y3 z3
588 MULTILINEAR MAPPINGS AND TENSORS

We remark that if the collection {X, Y, Z} is linearly dependent, then the

volume of the parallelepiped degenerates to zero (since at least one of the
parallelograms that form the sides will have zero area). This agrees with the
fact that the determinant (5) will vanish if two rows are linearly dependent.
We also note that the area of the base is given by

|X ! Y | = ˜X˜˜Y ˜ sin !(X,!Y )

where the direction of the vector X ª Y is up (in this case). Therefore the pro-
jection of Z in the direction of X ª Y is just Z dotted into a unit vector in the
direction of X ª Y, and hence the volume of the parallelepiped is given by the
number Z Â (X ª Y). This is the so-called scalar triple product that should be
familiar from elementary courses. We leave it to the reader to show that the
scalar triple product is given by the determinant (5) (see Exercise 11.6.1).
Finally, note that if any two of the vectors X, Y, Z in equation (5) are
interchanged, then the determinant changes sign even though the volume is
unaffected (since it must be positive). This observation will form the basis for
the concept of “orientation” to be defined later.

Exercises

1. Show that Z Â (X ª Y) is given by the determinant in equation (5).

2. Find the area of the parallelogram whose vertices are:

(a) (0, 0), (1, 3), (-2, 1), and (-1, 4).
(b) (2, 4), (4, 5), (5, 2), and (7, 3).
(c) (-1, 3), (1, 5), (3, 2), and (5, 4).
(d) (0, 0, 0), (1, -2, 2), (3, 4, 2), and (4, 2, 4).
(e) (2, 2, 1), (3, 0, 6), (4, 1, 5), and (1, 1, 2).

3. Find the volume of the parallelepipeds whose adjacent edges are the
vectors:
(a) (1, 1, 2), (3, -1, 0), and 5, 2, -1).
(b) (1, 1, 0), (1, 0, 1), and (0, 1, 1).

4. Prove both algebraically and geometrically that the parallelogram with

edges X and Y has the same area as the parallelogram with edges X and
Y + aX for any scalar a.
 11.6 VOLUMES IN ®3 589

5. Prove both algebraically and geometrically that the volume of the paral-
lelepiped in ®3 with edges X, Y and Z is equal to the volume of the paral-
lelepiped with edges X, Y and Z + aX + bY for any scalars a and b.

6. Show that the parallelepiped in ®3 defined by the three vectors (2, 2, 1),
(1, -2, 2) and (-2, 1, 2) is a cube. Find the volume of this cube.

11.7 VOLUMES IN ®n

Now that we have a feeling for volumes in ®3 expressed as determinants, let

us prove the analogous results in ®n. To begin with, we note that parallelo-
grams defined by the vectors X and Y in either ®2 or ®3 contain all points
(i.e., vectors) of the form aX + bY for any a, b ∞ [0, 1]. Similarly, given three
linearly independent vectors X, Y, Z ∞ ®3, we may define the parallelepiped
with these vectors as edges to be that subset of ®3 containing all vectors of the
form aX + bY + cZ where 0 ¯ a, b, c ¯ 1. The corners of the parallelepiped are
the points ∂1X + ∂ìY + ∂3Z where each ∂á is either 0 or 1.
Generalizing these observations, given any r linearly independent vectors
Xè, . . . , Xr ∞ ®n, we define an r-dimensional parallelepiped as the set of all
vectors of the form a1Xè + ~ ~ ~ + ar Xr where 0 ¯ aá ¯ 1 for each i = 1, . . . , r. In
®3, by a 1-volume we mean a length, a 2-volume means an area, and a 3-
volume is just the usual volume. To define the volume of an r-dimensional
parallelepiped we proceed by induction on r. In particular, if X is a nonzero
vector (i.e., a 1-dimensional parallelepiped) in ®n, we define its 1-volume to
be its length ÓX, XÔ1/2. Proceeding, suppose the (r - 1)-dimensional volume of
an (r - 1)-dimensional parallelepiped has been defined. If we let Pr denote the
r-dimensional parallelepiped defined by the r linearly independent vectors Xè,
. . . , Xr, then we say that the base of Pr is the (r - 1)-dimensional paral-
lelepiped defined by the r - 1 vectors Xè, . . . , Xr-1, and the height of Pr is the
length of the projection of Xr onto the orthogonal complement in ®n of the
space spanned by Xè, . . . , Xr-1. According to our induction hypothesis, the
volume of an (r - 1)-dimensional parallelepiped has already been defined.
Therefore we define the r-volume of Pr to be the product of its height times
the (r - 1)-dimensional volume of its base.
The reader may wonder whether or not the r-volume of an r-dimensional
parallelepiped in any way depends on which of the r vectors is singled out for
projection. We proceed as if it does not and then, after the next theorem, we
shall show that this is indeed the case.
590 MULTILINEAR MAPPINGS AND TENSORS

Theorem 11.13 Let Pr be the r-dimensional parallelepiped defined by the r

linearly independent vectors Xè, . . . , Xr ∞ ®n. Then the r-volume of Pr is the
positive square root of the determinant

ÓX1,! X1Ô ÓX1,! X2Ô ! ÓX1,! XrÔ

ÓX2 ,! X1Ô ÓX2 ,! X2Ô ! ÓX2 ,! XrÔ
!!. (7)
" " "
ÓXr ,! X1Ô ÓXr ,! X2Ô ! ÓXr ,! XrÔ

Proof For the case of r = 1, we see that the theorem is true by the definition
of length (or 1-volume) of a vector. Proceeding by induction, we assume the
theorem is true for an (r - 1)-dimensional parallelepiped, and we show that it
is also true for an r-dimensional parallelepiped. Hence, let us write

ÓX1,! X1Ô ÓX1,! X2Ô ! ÓX1,! Xr!1Ô

ÓX2 ,! X1Ô ÓX2 ,! X2Ô ! ÓX2 ,! Xr!1Ô
A 2 = Vol2 (Pr!1 ) =
" " "
ÓXr!1,! X1Ô ÓXr!1,! X2Ô ! ÓXr!1,! Xr!1Ô

for the volume of the (r - 1)-dimensional base of Pr. Just as we did in our
discussion of volumes in ®3, we write Xr in terms of its projection U onto the
orthogonal complement of the space spanned by the r - 1 vectors Xè, . . . , Xr.
This means that we can write

Xr = U + aè Xè + ~ ~ ~ + ar-1Xr-1

where ÓU, XáÔ = 0 for i = 1, . . . , r - 1, and ÓU, XrÔ = ÓU, UÔ. We thus have the
system of equations

a1ÓX1,! X1Ô +a2ÓX1,! X2Ô +!!!+ ar!1ÓX1,! Xr!1Ô = ÓX1,! XrÔ

a1ÓX2 ,! X1Ô +a2ÓX2 ,! X2Ô +!!!+ ar!1ÓX2 ,! Xr!1Ô = ÓX2 ,! XrÔ
" " " "
a1ÓX r-1,! X1Ô + a2ÓX r-1,! X2Ô +!!!+ ar!1ÓX r-1,! Xr!1Ô = ÓX r-1,! XrÔ

We write Mè, . . . , Mr-1 for the minors of the first r - 1 elements of the last
row in (7). Solving the above system for the aá using Cramer’s rule, we obtain
 11.7 VOLUMES IN ®n 591

A 2 a1 = (!1)r!2 M 1
A 2 a2 = (!1)r!3 M 2
!
A 2 ar!1 = M r!1

where the factors of (-1)r-k-1 in A2aÉ result from moving the last column of
(7) over to become the kth column of the kth minor matrix.
Using this result, we now have

A 2U = A 2 (!a1 X1 ! a2 X2 !! ! ar!1 Xr!1 + Xr )

= (!1)r!1 M 1 X1 + (!1)r!2 M 2 X2 +!+ (!1)M r!1 Xr!1 + A 2 Xr

and hence, using ˜ U ˜ 2 = ÓU, UÔ = ÓU, XrÔ, we find that (since (-1)-k = (-1)k)

A 2 ˜U˜2 = A 2ÓU,! XrÔ

= (!1)r!1 M 1ÓX r ,! X1Ô + (!1)(!1)r!1 M 2ÓXr ,! X2Ô
+!+ A 2ÓXr ,! XrÔ
= (!1)r!1[M 1ÓXr ,! X1Ô ! M 2ÓXr ,! X2Ô
+!+ (!1)r!1 A 2ÓXr ,! XrÔ]!!.

Now note that the right hand side of this equation is precisely the expansion of
(7) by minors of the last row, and the left hand side is by definition the square
of the r-volume of the r-dimensional parallelepiped Pr. This also shows that
the determinant (7) is positive. ˙

This result may also be expressed in terms of the matrix (ÓXá, XéÔ) as

Vol(Pr) = [det(ÓXá, XéÔ)]1/2 .

The most useful form of this theorem is given in the following corollary.

Corollary The n-volume of the n-dimensional parallelepiped in ®n defined

by the vectors Xè, . . . , Xñ where each Xá has coordinates (x1i , . . . , xni) is the
absolute value of the determinant of the matrix X given by
592 MULTILINEAR MAPPINGS AND TENSORS

! x1 x12 ! x1n $
# 1 &
# x2 x22 ! x2n &
X =# 1 &!!.
# " " " &
# n &
"x 1 xn2 ! xnn %

Proof Note that (det X)2 = (det X)(det XT) = det XXT is just the determinant
(7) in Theorem 11.13, which is the square of the volume. In other words,
Vol(Pñ) = \det X\. ˙

Prior to this theorem, we asked whether or not the r-volume depended on

which of the r vectors is singled out for projection. We can now easily show
that it does not. Suppose that we have an r-dimensional parallelepiped defined
by r linearly independent vectors, and let us label these vectors Xè, . . . , Xr.
According to Theorem 11.13, we project Xr onto the space orthogonal to the
space spanned by Xè, . . . , Xr-1, and this leads to the determinant (7). If we
wish to project any other vector instead, then we may simply relabel these r
vectors to put a different one into position r. In other words, we have made
some permutation of the indices in (7). However, remember that any permuta-
tion is a product of transpositions (Theorem 1.2), and hence we need only
consider the effect of a single interchange of two indices.
Notice, for example, that the indices 1 and r only occur in rows 1 and r as
well as in columns 1 and r. And in general, indices i and j only occur in the ith
and jth rows and columns. But we also see that the matrix corresponding to
(7) is symmetric about the main diagonal in these indices, and hence an inter-
change of the indices i and j has the effect of interchanging both rows i and j
as well as columns i and j in exactly the same manner. Thus, because we have
interchanged the same rows and columns there will be no sign change, and
therefore the determinant (7) remains unchanged. In particular, it always
remains positive. It now follows that the volume we have defined is indeed
independent of which of the r vectors is singled out to be the height of the
parallelepiped.
Now note that according to the above corollary, we know that Vol(Pñ) =
Vol(Xè, . . . , Xñ) = \det X\ which is always positive. While our discussion
just showed that Vol(Xè, . . . , Xñ) is independent of any permutation of
indices, the actual value of det X can change sign upon any such permutation.
Because of this, we say that the vectors (Xè, . . . , Xñ) are positively oriented
if det X > 0, and negatively oriented if det X < 0. Thus the orientation of a
set of vectors depends on the order in which they are written. To take into
account the sign of det X, we define the oriented volume Volo(Xè, . . . , Xñ)
to be +Vol(Xè, . . . , Xñ) if det X ˘ 0, and -Vol(Xè, . . . , Xñ) if det X < 0. We
will return to a careful discussion of orientation in a later section. We also
 11.7 VOLUMES IN ®n 593

remark that det X is always nonzero as long as the vectors (Xè, . . . , Xñ) are
linearly independent. Thus the above corollary may be expressed in the form

Volo(Xè, . . . , Xñ) = det (Xè, . . . , Xñ)

where det(Xè, . . . , Xñ) means the determinant as a function of the column

vectors Xá.

Exercises

1. Find the 3-volume of the three-dimensional parallelepipeds in ®4 defined

by the vectors:
(a) (2, 1, 0, -1), (3, -1, 5, 2), and (0, 4, -1, 2).
(b) (1, 1, 0, 0), (0, 2, 2, 0), and (0, 0, 3, 3).

2. Find the 2-volume of the parallelogram in ®4 two of whose edges are the
vectors (1, 3, -1, 6) and (-1, 2, 4, 3).

3. Prove that if the vectors X1, X2, . . . , Xr are mutually orthogonal, the r-
volume of the parallelepiped defined by them is equal to the product of
their lengths.

4. Prove that r vectors X1, X2, . . . , Xr in ®n are linearly dependent if and

only if the determinant (7) is equal to zero.

11.8 LINEAR TRANSFORMATIONS AND VOLUMES

One of the most useful applications of Theorem 11.13 and its corollary relates
to linear mappings. In fact, this is the approach usually followed in deriving
the change of variables formula for multiple integrals. Let {eá} be an ortho-
normal basis for ®n, and let Cñ denote the unit cube in ®n. In other words,

Cñ = {tèeè + ~ ~ ~ + tñeñ ∞ ®n: 0 ¯ tá ¯ 1} .

This is similar to the definition of Pr given previously.

Now let A: ®n ‘ ®n be a linear transformation. Then the matrix of A
relative to the basis {eá} is defined by A(eá) = eéajá. Let us write the image of eá
as Xá, so that Xá = A(eá) = eéajá. This means that the column vector Xá has
594 MULTILINEAR MAPPINGS AND TENSORS

components (a1á , . . . , aná). Under the transformation A, the image of Cñ

becomes
A(Cñ) = A(Í táeá) = Í táA(eá) = Í táXá

(where 0 ¯ tá ¯ 1) which is just the parallelepiped Pñ spanned by the vectors

(Xè, . . . , Xñ). Therefore the volume of Pñ = A(Cñ) is given by

\det(Xè , . . . , Xñ)\ = \det (ajá)\ .

Recalling that the determinant of a linear transformation is defined to be the

determinant of its matrix representation, we have proved the next result.

Theorem 11.14 Let Cñ be the unit cube in ®n spanned by the orthonormal

basis vectors {eá}. If A: ®n ‘ ®n is a linear transformation and Pñ = A(Cñ),
then Vol(Pñ) = Vol A(Cñ) = \det A\.

It is quite simple to generalize this result somewhat to include the image

of an n-dimensional parallelepiped under a linear transformation A. First, we
note that any parallelepiped Pñ is just the image of Cñ under some linear
transformation B. Indeed, if Pñ = {t1Xè + ~ ~ ~ + tñ Xñ: 0 ¯ t á ¯ 1} for some set
of vectors Xá, then we may define the transformation B by B(eá) = Xá, and
hence Pñ = B(Cñ). Thus

A(Pñ) = A(B(Cñ)) = (A ı B)(Cñ)

and therefore (using Theorem 11.14 along with the fact that the matrix of the
composition of two transformations is the matrix product)

Vol A(Pn ) = Vol[(A ! B)(Cn )] = |det(A ! B)| = |det A||det B|

= |det A|Vol(Pn )!!.

In other words, \det A\ is a measure of how much the volume of the parallel-
epiped changes under the linear transformation A. See the figure below for a
picture of this in ®2.
We summarize this discussion as a corollary to Theorem 11.14.

Corollary Suppose Pñ is an n-dimensional parallelepiped in ®n, and let

A: ®n ‘ ®n be a linear transformation. Then Vol A(Pñ) = \det A\Vol(Pñ).
 11.8 LINEAR TRANSFORMATIONS AND VOLUMES 595

Cì A(Xì) A(Pì)
Pì
B A
eì Xì A(Xè)
Xè
eè Xè = B(eè)
Xì = B(eì)

Now that we have an intuitive grasp of these concepts, let us look at this
material from the point of view of exterior algebra. This more sophisticated
approach is of great use in the theory of integration.
Let U and V be real vector spaces. Recall from Theorem 9.7 that given a
linear transformation T ∞ L(U, V), we defined the transpose mapping T* ∞
L(V*, U*) by
T*ø = ø ı T

for all ø ∞ V*. By this we mean if u ∞ U, then T*ø(u) = ø(Tu). As we then

saw in Theorem 9.8, if U and V are finite-dimensional and A is the matrix
representation of T, then AT is the matrix representation of T*, and hence
certain properties of T* follow naturally. For example, if Tè ∞ L(V, W) and
Tì ∞ L(U, V), then (Tè ı Tì)* = Tì* ı Tè* (Theorem 3.18), and if T is
nonsingular, then (Tî)* = (T*)î (Corollary 4 of Theorem 3.21).
Now suppose that {eá} is a basis for U and {fé} is a basis for V. To keep
the notation simple and understandable, let us write the corresponding dual
bases as {ei} and {fj}. We define the matrix A = (aji) of T by Teá = féajá. Then
(just as in the proof of Theorem 9.8)

(T *f i )e j = f i (Te j ) = f i ( fk a k j ) = a k j f i ( fk ) = a k j! i k = ai j = ai k! k j
= ai k e k (e j )

which shows that

T *f i = ai k e k !!. (8)

We will use this result frequently below.

We now generalize our definition of the transpose. If ƒ ∞ L(U, V) and T ∞
Tr (V), we define the pull-back ƒ* ∞ L(Tr (V), Tr (U)) by

(ƒ*T )(uè, . . . , ur) = T(ƒ(uè), . . . , ƒ(ur))

596 MULTILINEAR MAPPINGS AND TENSORS

where uè, . . . , ur ∞ U. Note that in the particular case of r = 1, the mapping ƒ*

is just the transpose of ƒ. It should also be clear from the definition that ƒ* is
indeed a linear transformation, and hence

ƒ*(aTè + bTì) = aƒ*Tè + bƒ*Tì .

We also emphasize that ƒ need not be an isomorphism for us to define ƒ*.

The main properties of the pull-back are given in the next theorem.

Theorem 11.15 If ƒ ∞ L(U, V) and √ ∞ L(V, W), then

(a) (√ ı ƒ)* = ƒ* ı √*.
(b) If I ∞ L(U) is the identity map, then I* is the identity in L(Tr(U)).
(c) If ƒ is an isomorphism, then so is ƒ*, and (ƒ*)î = (ƒî)*.
(d) If Tè ∞ Trè(V) and Tì ∞ Trì(V), then

ƒ*(Tè · Tì) = (ƒ*Tè) · (ƒ*Tì) .

(e) Let U have basis {eè, . . . , em}, V have basis {fè, . . . , fñ} and suppose
that ƒ(eá) = féajá. If T ∞ Tr (V) has components Tiè ~ ~ ~ i‹ = T(fiè , . . . , fi‹), then
the components of ƒ*T relative to the basis {eá} are given by

(ƒ*T )jè~ ~ ~j‹ = Tiè~ ~ ~i‹aièjè ~ ~ ~ ai‹j‹ .

Proof (a) Note that √ ı ƒ: U ‘ W, and hence (√ ı ƒ)*: Tr (W) ‘Tr (U).
Thus for any T ∞ Tr (W) and uè, . . . , ur ∞ U we have

((! ! " )*T )(u1,!…!,ur ) = T (! (" (u1 )),!…!,!! (" (ur )))
= (! *T )(" (u1 ),!…!,!" (ur ))
= ((" * ! ! *)T )(u1,!…!,!ur )!!.

(b) Obvious from the definition of I*.

(c) If ƒ is an isomorphism, then ƒî exists and we have (using (a) and (b))

ƒ* ı (ƒî)* = (ƒî ı ƒ)* = I* .

Similarly (ƒî)* ı ƒ* = I*. Hence (ƒ*)î exists and is equal to (ƒî)*.

(d) This follows directly from the definitions (see Exercise 11.8.1).
(e) Using the definitions, we have
 11.8 LINEAR TRANSFORMATIONS AND VOLUMES 597

(! *T ) j1 ! jr = (! *T )(e j1 ,!…,!e jr )
= T (! (e j1 ), …,!! (e jr ))
= T ( fi1 ai1 j1 , …,! fir air jr )
= T ( fi1 , …,! fir )ai1 j1 ! air jr
= Ti1 ! ir ai1 j1 ! air jr !!.

Alternatively, if {ei}and {fj} are the bases dual to {eá} and {fé} respec-
tively, then T = Tiè~ ~ ~i‹ eiè · ~ ~ ~ · ei‹ and consequently (using the linearity of
ƒ*, part (d) and equation (8)),

! *T = Ti1 ! ir ! *ei1 " ! " ! *eir

= Ti1 ! ir ai1 j1 ! air jr f j1
"!" f jr

which therefore yields the same result. ˙

For our present purposes, we will only need to consider the pull-back as
defined on the space „r(V) rather than on Tr (V). Therefore, if ƒ ∞ L(U, V)
then ƒ* ∞ L(Tr (V), Tr (U)), and hence we see that for ø ∞ „r(V) we have
(ƒ*ø)(uè, . . . , ur) = ø(ƒ(uè), . . . , ƒ(ur)). This shows that ƒ*(„r(V)) ™ „r(U).
Parts (d) and (e) of Theorem 11.15 applied to the space „r(V) yield the fol-
lowing special cases. (Recall that \i1 ~ ~ ~ ir\ means the sum is over increasing
indices iè < ~ ~ ~ < ir.)

Theorem 11.16 Suppose ƒ ∞ L(U, V), å ∞ „r(V) and ∫ ∞ „s(V). Then

(a) ƒ*(å°∫) = (ƒ*å)°(ƒ*∫).
(b) Let U and V have bases {eá} and {fá} respectively, and let U* and V*
have bases {ei} and {fi}. If we write ƒ(eá) = féajá and ƒ*(fi ) = aiéej, and if å =
a\ iè~ ~ ~i‹\ f iè°~ ~ ~°f i‹ ∞ „r(V), then

ƒ*å = aÄ\ kè~ ~ ~k‹\ ekè° ~ ~ ~ °ek‹

where
! jr i1
â|k1 ! kr | = a|i1 ! ir | ! kj1 ! ir
k a j1 ! a jr !!.
1 r

Thus we may write

aÄkè~ ~ ~ k‹ = a\ iè~ ~ ~i‹\ det(aI K)

where
598 MULTILINEAR MAPPINGS AND TENSORS

ai1 k1 ! ai1 kr
det(a I K ) = " " !!.
ir ir
a k1 ! a kr

Proof (a) For simplicity, let us write (ƒ*å)(uJ) = å(ƒ(uJ)) instead of

(ƒ*å)(uè, . . . , ur) = å(ƒ(uè), . . . , ƒ(ur)) (see the discussion following
Theorem 11.4). Then, in an obvious notation, we have

[! *(" # $ )](u I ) = (" # $ )(! (u I ))

= % J! , K! & IJK (" (! (u J ))$ (! (uK ))
= % J! , K! & IJK (! *" )(u J )(! *$ )(uK )
= [(! *" ) # (! *$ )](u I )!!.

By induction, this also obviously applies to the wedge product of a finite

number of forms.
(b) From å = a\ iè~ ~ ~i‹\ f iè° ~ ~ ~ °f i‹ and ƒ*(f i) = ai é ej, we have (using part
(a) and the linearity of ƒ*)

! *" = a|i1 ! ir |! *( f i1 ) #! # ! *( f ir )
= a|i1 ! ir |ai1 j1 ! air jr e j1 #! # e jr !!.

But
! jr k1
e j1 !! ! e jr = "K" # kj1 ! k e !! ! e
kr
1 r

and hence we have

! jr i1
! *" = a|i1 ! ir |#K" $ kj11 ! ir k1
kr a j1 ! a jr e %! % e
kr

= â|k1 ! kr |ek1 %! % ekr

where
! jr i1
â|k1 ! kr | = a|i1 ! ir | ! kj1 ! ir
k a j1 ! a jr !!.
1 r

Finally, from the definition of determinant we see that

ai1 k1 ! ai1 kr
! jr i1
! kj11 ! ir
kr a j1 ! a jr = " " !!.!!˙
ir ir
a k1 ! a kr
 11.8 LINEAR TRANSFORMATIONS AND VOLUMES 599

Example 11.11 (This is a continuation of Example 11.1.) An important

example of ƒ*å is related to the change of variables formula in multiple
integrals. While we are not in any position to present this material in detail,
the idea is this. Suppose we consider the spaces U = ®3(u, v, w) and V =
®3(x, y, z) where the letters in parentheses tell us the coordinate system used
for that particular copy of ®3. Note that if we write (x, y, z) = (x1, x2, x3) and
(u, v, w) = (u1, u2, u3), then from elementary calculus we know that dxi =
($xi/$uj)duj and $/$ui = ($xj/$ui)($/$xj).
Now recall from Example 11.1 that at each point of ®3(u, v, w), the tan-
gent space has the basis {eá} = {$/$ui} and the cotangent space has the corre-
sponding dual basis {ei} = {dui}, with a similar result for ®3(x, y, z). Let us
define ƒ: ®3(u, v, w) ‘ ®3(x, y, z) by

ƒ($/$ui) = ($xj/$ui)($/$xj) = ajá($/$xj) .

It is then apparent that (see equation (8))

ƒ*(dxi) = aiéduj = ($xi/$uj)duj

as we should have expected. We now apply this to the 3-form

å = a123 dx1°dx2°dx3 = dx°dy°dz ∞ „3(V) .

Since we are dealing with a 3-form in a 3-dimensional space, we must

have
ƒ*å = aÄ du°dv°dw

where aÄ = aÄ123 consists of the single term given by the determinant

a11 a12 a13 !x1 /!u1 !x1 /!u 2 !x1 /!u 3

a 21 a 2 2 a 2 3 = !x 2 /!u1 !x 2 /!u 2 !x 2 /!u 3
a 31 a 32 a 33 !x 3 /!u1 !x 3 /!u 2 !x 3 /!u 3

which the reader may recognize as the so-called Jacobian of the transforma-
tion. This determinant is usually written as $(x, y, z)/$(u, v, w), and hence we
see that
#(x,!y,!z)
! *(dx " dy " dz) = du " dv " dw!!.
#(u,!v,!w)
600 MULTILINEAR MAPPINGS AND TENSORS

This is precisely how volume elements transform (at least locally), and
hence we have formulated the change of variables formula in quite general
terms. ∆

This formalism allows us to define the determinant of a linear transforma-

tion in an interesting abstract manner. To see this, suppose ƒ ∞ L(V) where
dim V = n. Since dim „n(V) = 1, we may choose any nonzero øà ∞ „n(V) as
a basis. Then ƒ*: „n(V) ‘ „n(V) is linear, and hence for any ø = càøà ∞
„n(V) we have

ƒ*ø = ƒ*(càøà) = càƒ*øà = càcøà = c(càøà) = cø

for some scalar c (since ƒ*øà ∞ „n(V) is necessarily of the form cøà). Noting
that this result did not depend on the scalar cà and hence is independent of ø =
càøà, we see that the scalar c must be unique. We therefore define the deter-
minant of ƒ to be the unique scalar, denoted by det ƒ, such that

ƒ*ø = (det ƒ)ø .

It is important to realize that this definition of the determinant does not

depend on any choice of basis for V. However, let {eá} be a basis for V, and
define the matrix (aié) of ƒ by ƒ(eá) = eéajá. Then for any nonzero ø ∞ „n(V)
we have
(ƒ*ø)(eè, . . . , eñ) = (det ƒ)ø(eè, . . . , eñ) .

On the other hand, Example 11.2 shows us that

(! *" )(e1,!…!,!en ) = " (! (e1 ),!…!,!! (en ))

= ai1 1 ! ain n" (ei1 ,!…!, ein )
= (det(ai j ))" (e1,!…!, en )!!.

Since ø ≠ 0, we have therefore proved the next result.

Theorem 11.17 If V has basis {eè, . . . , eñ} and ƒ ∞ L(V) has the matrix
representation (aié) defined by ƒ(eá) = eéajá, then det ƒ = det(aié).

In other words, our abstract definition of the determinant is exactly the

same as our earlier classical definition. In fact, it is now easy to derive some
of the properties of the determinant that were not exactly simple to prove in
the more traditional manner.
 11.8 LINEAR TRANSFORMATIONS AND VOLUMES 601

Theorem 11.18 If V is finite-dimensional and ƒ, √ ∞ L(V, V), then

(a) det(ƒ ı √) = (det ƒ)(det √).
(b) If ƒ is the identity transformation, then det ƒ = 1.
(c) ƒ is an isomorphism if and only if det ƒ ≠ 0, and if this is the case, then
det ƒî = (det ƒ)î.

Proof (a) By definition we have (ƒ ı √)*ø = det(ƒ ı √)ø. On the other hand,
by Theorem 11.15(a) we know that (ƒ ı √)* = √* ı ƒ*, and hence

(! ! " )*# = " *(! *# ) = " *[(det ! )# ] = (det ! )" *#

= (det ! )(det " )# !!.

(b) If ƒ = 1 then ƒ* = 1 also (by Theorem 11.15(b)), and thus ø = ƒ*ø =

(det ƒ)ø implies det ƒ = 1.
(c) First assume that ƒ is an isomorphism so that ƒî exists. Then by parts
(a) and (b) we see that

1 = det(ƒƒî) = (det ƒ)(det ƒî)

which implies det ƒ ≠ 0 and det ƒî = (det ƒ)î. Conversely, suppose that ƒ is
not an isomorphism. Then Ker ƒ ≠ 0 and there exists a nonzero eè ∞ V such
that ƒ(eè) = 0. By Theorem 2.10, we can extend this to a basis {eè, . . . , eñ} for
V. But then for any nonzero ø ∞ „n(V) we have

(det ! )" (e1,!…!, en ) = (! *" )(e1,!…!, en )

= " (! (e1 ),!…!, ! (en ))
= " (0,!! (e2 ),!…!,!! (en ))
=0

and hence we must have det ƒ = 0. ˙

Exercises

1. Prove Theorem 11.15(d).

2. Show that the matrix ƒ*T defined in Theorem 11.15(e) is just the r-fold
Kronecker product A · ~ ~ ~ · A where A = (aij).
602 MULTILINEAR MAPPINGS AND TENSORS

The next three exercises are related.

3. Let ƒ ∞ L(U, V) be an isomorphism, and suppose T ∞ Tr Í(U). Define the

push-forward ƒ§ ∞ L(Tr Í(U), Tr Í(V)) by

ƒ§T(å1, . . . , ås, u1, . . . , ur) = T(ƒ*å1, . . . , ƒ*ås, ƒîu1, . . . , ƒîur)

where å1, . . . , ås ∞ U* and u1, . . . , ur ∞ U. If √ ∞ L(V, W) is also an iso-

morphism, prove the following:
(a) (√ ı ƒ)§ = √§ ı ƒ§.
(b) If I ∞ L(U) is the identity map, then so is I§ ∞ L(Tr Í(U)).
(c) ƒ§ is an isomorphism, and (ƒ§)î = (ƒî)§.
(d) If T1 ∞ T r1 s1 (U ) and T2 ∞ T r2 s2 (U ) , then

ƒ§(T1 · T2) = (ƒ§T1) · (ƒ§T2) .

4. Let ƒ ∞ L(U, V) be an isomorphism, and let U and V have bases {ei} and
{fi} respectively. Define the matrices (aij) and (bij) by ƒ(ei) = fjaji and
ƒî(fi) = ejbji. Suppose T ∞ Tr Í(U) has components Tiè ~ ~ ~ i› jè ~ ~ ~ j‹ relative
to {ei}, and S ∞ Tr Í(V) has components Siè ~ ~ ~ i› jè ~ ~ ~ j‹ relative to {fi}.
Show that the components of ĤT and ĤS are given by

(!§T )i1 ! is j1 ! jr = ai1 p1 ! ais ps T p1 ! ps q1 ! qr b q1 j1 ! b qr jr

(!§S)i1 ! is j1 ! jr = bi1 p1 ! bis ps S p1 ! ps q1 ! qr a q1 j1 ! a qr jr !!.

5. Let {øi} be the basis dual to {ei} for ®2. Let

T = 2e1 · ø1 - e2 · ø1 + 3e1 · ø2

and suppose ƒ ∞ L(®2) and √ ∞ L(®3, ®2) have the matrix representations

# !!2 1& # 0 !1 "1 &

! =% ( and ) =% (!!.
$ "1 1' $ 1 !0 !!2 '

Compute Tr T, ƒ*T, √*T, Tr(√*T), and ƒ§T.

 11.9 ORIENTATIONS AND VOLUMES 603

11.9 ORIENTATIONS AND VOLUMES

Suppose dim V = n and consider the space „n(V). Since this space is 1-
dimensional, we consider the n-form

ø = e1° ~ ~ ~ °en ∞ „n(V)

where the basis {ei} for V* is dual to the basis {eá} for V. If {vá = ejvji} is any
set of n linearly independent vectors in V then, according to Examples 11.2
and 11.8, we have

ø(vè, . . . , vñ) = det(vji)ø(eè, . . . , eñ) = det(vji) .

However, from the corollary to Theorem 11.13, this is just the oriented n-
volume of the n-dimensional parallelepiped in ®n spanned by the vectors {vá}.
Therefore, we see that an n-form in some sense represents volumes in an n-
dimensional space. We now proceed to make this definition precise, beginning
with a careful definition of the notion of orientation on a vector space.
In order to try and make the basic idea clear, let us first consider the space
®2 with all possible orthogonal coordinate systems. For example, we may
consider the usual “right-handed” coordinate system {eè, eì} shown below, or
we may consider the alternative “left-handed” system {eæè, eæì} also shown.

eæè

eì eæì

eè
In the first case, we see that rotating eè into eì through the smallest angle
between them involves a counterclockwise rotation, while in the second case,
rotating eæè into eæì entails a clockwise rotation. This effect is shown in the
elementary vector cross product, where the direction of eè ª eì is defined by
the “right-hand rule” to point out of the page, while eæè ª eæì points into the
page.
We now ask whether or not it is possible to continuously rotate eæè into eè
and eæì into eì while maintaining a basis at all times. In other words, we ask if
these two bases are in some sense equivalent. Without being rigorous, it
should be clear that this can not be done because there will always be one
point where the vectors eæè and eæì will be co-linear, and hence linearly
dependent. This observation suggests that we consider the determinant of the
matrix representing this change of basis.
604 MULTILINEAR MAPPINGS AND TENSORS

In order to formulate this idea precisely, let us take a look at the matrix
relating our two bases {eá} and {eæá} for ®2. We thus write eæá = eéajá and
investigate the determinant det(aié). From the above figure, we see that

e! 1 = e1a11 + e2 a 21 where a11 < 0 and a 21 > 0

e! 2 = e1a12 + e2 a 2 2 where a12 < 0 and a 2 2 > 0

and hence det(aié) = a1è a2ì - a1ì a2è < 0.

Now suppose that we view this transformation as a continuous modifica-
tion of the identity transformation. This means we consider the basis vectors
eæá to be continuous functions eæá(t) of the matrix ajá(t) for 0 ¯ t ¯ 1 where ajá(0)
= ∂já and ajá(1) = ajá, so that eæá(0) = eá and eæá(1) = eæá. In other words, we write
eæá(t) = eéajá(t) for 0 ¯ t ¯ 1. Now note that det(aié(0)) = det(∂ié) = 1 > 0, while
det(aié(1)) = det(aié) < 0. Therefore, since the determinant is a continuous
function of its entries, there must be some value tà ∞ (0, 1) where det(aié(tà)) =
0. It then follows that the vectors eæá(tà) will be linearly dependent.
What we have just shown is that if we start with any pair of linearly inde-
pendent vectors, and then transform this pair into another pair of linearly
independent vectors by moving along any continuous path of linear transfor-
mations that always maintains the linear independence of the pair, then every
linear transformation along this path must have positive determinant. Another
way of saying this is that if we have two bases that are related by a transfor-
mation with negative determinant, then it is impossible to continuously trans-
form one into the other while maintaining their independence. This argument
clearly applies to ®n and is not restricted to ®2.
Conversely, suppose we had assumed that eæá = eéajá, but this time with
det(aié) > 0. We want to show that {eá} may be continuously transformed into
{eæá} while maintaining linear independence all the way. We first assume that
both {eá} and {eæá} are orthonormal bases. After treating this special case, we
will show how to take care of arbitrary bases.
(Unfortunately, the argument we are about to give relies on the topological
concept of path connectedness. Since a complete discussion of this topic
would lead us much too far astray, we shall be content to present only the
fundamental concepts in Appendix C. Besides, this discussion is only motiva-
tion, and the reader should not get too bogged down in the details of this
argument. Those readers who know some topology should have no trouble
filling in the necessary details if desired.)
Since {eá} and {eæá} are orthonormal, it follows from Theorem 10.6
(applied to ® rather than ç) that the transformation matrix A = (aié) defined by
eæá = eéajá must be orthogonal, and hence det A = +1 (by Theorem 10.8(a) and
 11.9 ORIENTATIONS AND VOLUMES 605

the fact that we are assuming {eá} and {eæá} are related by a transformation
with positive determinant). By Theorem 10.19, there exists a nonsingular
matrix S such that SîAS = Mœ where Mœ is the block diagonal canonical form
consisting of +1’s, -1’s, and 2 x 2 rotation matrices R(œá) given by

# cos!i "sin !i &

R(!i ) = % (!!.
$ sin !i !!cos!i '

It is important to realize that if there are more than two +1’s or more than
two -1’s, then each pair may be combined into one of the R(œá) by choosing
either œá = π (for each pair of -1’s) or œá = 0 (for each pair of +1’s). In this
manner, we view Mœ as consisting entirely of 2 x 2 rotation matrices, and at
most a single +1 and/or -1. Since det R(χ) = +1 for any χ, we see that (using
Theorem 4.14) det Mœ = +1 if there is no -1, and det Mœ = -1 if there is a
single -1. From A = SMœSî, we see that det A = det Mœ, and since we are
requiring that det A > 0, we must have the case where there is no -1 in Mœ.
Since cos œá and sin œá are continuous functions of œá ∞ [0, 2π) (where the
interval [0, 2π) is a path connected set), we note that by parametrizing each œá
by œá(t) = (1 - t)œá, the matrix Mœ may be continuously connected to the
identity matrix I (i.e., at t = 1). In other words, we consider the matrix Mœ(t)
where Mœ(0) = Mœ and Mœ(1) = I. Hence every such Mœ (i.e., any matrix of the
same form as our particular Mœ, but with a different set of œá’s) may be
continuously connected to the identity matrix. (For those readers who know
some topology, note all we have said is that the torus [0, 2π) ª ~ ~ ~ ª [0, 2π) is
path connected, and hence so is its continuous image which is the set of all
such Mœ.)
We may write the (infinite) collection of all such Mœ as M = {Mœ}.
Clearly M is a path connected set. Since A = SMœSî and I = SISî, we see
that both A and I are contained in the collection SMSî = {SMœSî}. But
SMSî is also path connected since it is just the continuous image of a path
connected set (matrix multiplication is obviously continuous). Thus we have
shown that both A and I lie in the path connected set SMSî, and hence A may
be continuously connected to I. Note also that every transformation along this
path has positive determinant since det SMœSî = det Mœ = 1 > 0 for every
Mœ ∞ M.
If we now take any path in SMSî that starts at I and goes to A, then
applying this path to the basis {eá} we obtain a continuous transformation
from {eá} to {eæá} with everywhere positive determinant. This completes the
proof for the special case of orthonormal bases.
Now suppose that {vá} and {væá} are arbitrary bases related by a transfor-
mation with positive determinant. Starting with the basis {vá}, we first apply
606 MULTILINEAR MAPPINGS AND TENSORS

the Gram-Schmidt process (Theorem 2.21) to {vá} to obtain an orthonormal

basis {eá} = {vébjá}. This orthonormalization process may be visualized as a
sequence vá(t) = vébjá(t) (for 0 ¯ t ¯ 1) of continuous scalings and rotations that
always maintain linear independence such that vá(0) = vá (i.e., bjá(0) = ∂já) and
vá(1) = eá (i.e., bjá(1) = bjá). Hence we have a continuous transformation bjá(t)
taking {vá} into {eá} with det(bjá(t)) > 0 (the transformation starts with
det(bjá(0)) = det I > 0, and since the vectors are always independent, it must
maintain det((bjá(t)) ≠ 0). Similarly, we may transform {væá} into an orthonor-
mal basis {eæá} by a continuous transformation with positive determinant.
(Alternatively, it was shown in Exercise 5.4.14 that the Gram-Schmidt process
is represented by an upper-triangular matrix with all positive diagonal ele-
ments, and hence its determinant is positive.) Now {eá} and {eæá} are related
by an orthogonal transformation that must also have determinant equal to +1
because {vá} and {væá} are related by a transformation with positive determi-
nant, and both of the Gram-Schmidt transformations have positive determi-
nant. This reduces the general case to the special case treated above.
With this discussion as motivation, we make the following definition. Let
{vè, . . . , vñ} and {væè, . . . , væñ} be two ordered bases for a real vector space
V, and assume that væá = véajá. These two bases are said to be similarly
oriented if det(aié) > 0, and we write this as {vá} — {væá}. In other words,
{vá} — {væá} if væá = ƒ(vá) with det ƒ > 0. We leave it to the reader to show that
this defines an equivalence relation on the set of all ordered bases for V (see
Exercise 11.9.1). We denote the equivalence class of the basis {vá} by [vá].
It is worth pointing out that had we instead required det(aié) < 0, then this
would not have defined an equivalence relation. This is because if (bié) is
another such transformation with det(bié) < 0, then

det(aiébjÉ) = det(aié)det(bjÉ) > 0 .

Intuitively this is quite reasonable since a combination of two reflections

(each of which has negative determinant) is not another reflection.
We now define an orientation of V to be an equivalence class of ordered
bases. The space V together with an orientation [vá] is called an oriented
vector space (V, [vá]). Since the determinant of a linear transformation that
relates any two bases must be either positive or negative, we see that V has
exactly two orientations. In particular, if {vá} is any given basis, then every
other basis belonging to the equivalence class [vá] of {vá} will be related to
{vá} by a transformation with positive determinant, while those bases related
to {vá} by a transformation with negative determinant will be related to each
other by a transformation with positive determinant (see Exercise 11.9.1).
 11.9 ORIENTATIONS AND VOLUMES 607

Now recall we have seen that n-forms seem to be related to n-volumes in

an n-dimensional space V. To precisely define this relationship, we formulate
orientations in terms of n-forms. To begin with, the nonzero elements of the 1-
dimensional space „n(V) are called volume forms (or sometimes volume
elements) on V. If øè and øì are volume forms, then øè is said to be equiv-
alent to øì if øè = cøì for some real c > 0, and in this case we also write øè —
øì. Since every element of „n(V) is related to every other element by a rela-
tionship of the form øè = aøì for some real a (i.e., -Ÿ < a < Ÿ), it is clear that
this equivalence relation divides the set of all nonzero volume forms into two
distinct groups (i.e., equivalence classes). We can relate any ordered basis
{vá} for V to a specific volume form by defining

ø = v1° ~ ~ ~ °vn

where {vi} is the basis dual to {vá}. That this association is meaningful is
shown in the next result.

Theorem 11.19 Let {vá} and {võá} be bases for V, and let {vi} and {või} be
the corresponding dual bases. Define the volume forms

ø = v1° ~ ~ ~ °vn
and
øù = võ1° ~ ~ ~ °võn .

Then {vá} — {võá} if and only if ø — øù.

Proof First suppose that {vá} — {võá}. Then võá = ƒ(vá) where det ƒ > 0, and
hence (using

ø(vè , . . . , vñ) = v1° ~ ~ ~ °vn (vè , . . . , vñ) = 1

as shown in Example 11.8)

! (v1,!…!, vn ) = ! (" (v1 ),!…!, " (vn ))

= (" *! )(v1,!…!, vn )
= (det " )! (v1,!…!, vn )
= det " !!.

If we assume that ø = cøù for some -Ÿ < c < Ÿ, then using øù(võè , . . . , võñ) = 1
we see that our result implies c = det ƒ > 0 and thus ø — øù.
608 MULTILINEAR MAPPINGS AND TENSORS

Conversely, if ø = c øù where c > 0, then assuming that võá = ƒ(vá), the

above calculation shows that det ƒ = c > 0, and hence {vá} — { võá}. ˙

What this theorem shows us is that an equivalence class of bases uniquely

determines an equivalence class of volume forms and conversely. Therefore it
is consistent with our earlier definitions to say that an equivalence class [ø] of
volume forms on V defines an orientation on V, and the space V together
with an orientation [ø] is called an oriented vector space (V, [ø]). A basis
{vá} for (V, [ø]) is now said to be positively oriented if ø(vè, . . . , vñ) > 0.
Not surprisingly, the equivalence class [-ø] is called the reverse orientation,
and the basis {vá} is said to be negatively oriented if ø(vè, . . . , vñ) < 0. Note
that if the ordered basis {vè, vì , . . . , vñ} is negatively oriented, then the basis
{vì, vè , . . . , vñ} will be positively oriented because ø(vì, vè , . . . , vñ) =
- ø(vè , vì , . . . , vñ) > 0. By way of additional terminology, the standard
orientation on ®n is that orientation defined by either the standard ordered
basis {eè, . . . , eñ}, or the corresponding volume form e1° ~ ~ ~ °en.
In order to proceed any further, we must introduce the notion of a metric
on V. This is the subject of the next section.

Exercises

1. (a) Show that the collection of all similarly oriented bases for V defines
an equivalence relation on the set of all ordered bases for V.
(b) Let {vá} be a basis for V. Show that all other bases related to {vá} by a
transformation with negative determinant will be related to each other by a
transformation with positive determinant.

2. Let (U, ø) and (V, µ) be oriented vector spaces with chosen volume ele-
ments. We say that ƒ ∞ L(U, V) is volume preserving if ƒ*µ = ø. If
dim U = dim V is finite, show that ƒ is an isomorphism.

3. Let (U, [ø]) and (V, [µ]) be oriented vector spaces. We say that ƒ ∞
L(U, V) is orientation preserving if ƒ*µ ∞ [ø]. If dim U = dim V is
finite, show that ƒ is an isomorphism. If U = V = ®3, give an example of a
linear transformation that is orientation preserving but not volume
preserving.
 11.10 THE METRIC TENSOR AND VOLUME FORMS 609

11.10 THE METRIC TENSOR AND VOLUME FORMS

We now generalize slightly our definition of inner products on V. In particu-

lar, recall from Section 2.4 (and the beginning of Section 9.2) that property
(IP3) of an inner product requires that Óu, uÔ ˘ 0 for all u ∞ V and Óu, uÔ = 0 if
and only if u = 0. If we drop this condition entirely, then we obtain an
indefinite inner product on V. (In fact, some authors define an inner product
as obeying only (IP1) and (IP2), and then refer to what we have called an
inner product as a “positive definite inner product.”) If we replace (IP3) by the
weaker requirement

(IP3æ) Óu, vÔ = 0 for all v ∞ V if and only if u = 0

then our inner product is said to be nondegenerate. (Note that every example
of an inner product given in this book up to now has been nondegenerate.)
Thus a real nondegenerate indefinite inner product is just a real nondegenerate
symmetric bilinear map. We will soon see an example of an inner product
with the property that Óu, uÔ = 0 for some u ≠ 0 (see Example 11.13 below).
Throughout the remainder of this chapter, we will assume that our inner
products are indefinite and nondegenerate unless otherwise noted. We further-
more assume that we are dealing exclusively with real vector spaces.
Let {eá} be a basis for an inner product space V. Since in general we will
not have Óeá, eéÔ = ∂áé, we define the scalars gáé by

gáé = Óeá, eé Ô .

In terms of the gáé, we have for any X, Y ∞ V

ÓX, YÔ = Óxieá, yjeéÔ = xi yjÓeá, eéÔ = gáéxiyj .

If {eõá} is another basis for V, then we will have eõá = eéajá for some nonsingular
transition matrix A = (ajá). Hence, writing gõáé = Óeõá, eõéÔ we see that

gõáé = Óeõá, eõéÔ = Óerari, esasjÔ = ariasjÓer, esÔ = ariasjgrs

which shows that the gáé transform like the components of a second-rank
covariant tensor. Indeed, defining the tensor g ∞ T2 (V) by

g(X, Y) = ÓX, YÔ
results in
g(eá, eé) = Óeá, eéÔ = gáé
610 MULTILINEAR MAPPINGS AND TENSORS

as it should. We are therefore justified in defining the (covariant) metric

tensor
g = gáéøi · øj ∞ T 2 (V)

(where {øi} is the basis dual to {eá}) by g(X, Y) = ÓX, YÔ. In fact, since the
inner product is nondegenerate and symmetric (i.e., ÓX, YÔ = ÓY, XÔ), we see
that g is a nondegenerate symmetric tensor (i.e., gáé = géá).
Next, we notice that given any vector A ∞ V, we may define a linear func-
tional ÓA, Ô on V by the assignment B ’ ÓA, BÔ. In other words, for any A ∞
V, we associate the 1-form å defined by å(B) = ÓA, BÔ for every B ∞ V. Note
that the kernel of the mapping A ’ ÓA, Ô (which is easily seen to be a vector
space homomorphism) consists of only the zero vector (since ÓA, BÔ = 0 for
every B ∞ V implies that A = 0), and hence this association is an iso-
morphism. Given any basis {eá} for V, the components aá of å ∞ V* are given
in terms of those of A = aieá ∞ V by

aá = å(eá) = ÓA, eáÔ = Óaj eé, eáÔ = ajÓeé, eáÔ = ajgéá

Thus, to any contravariant vector A = ai eá ∞ V, we can associate a unique

covariant vector å ∞ V* by

å = aáøi = (ajgéá)øi

where {øi} is the basis for V* dual to the basis {eá} for V. In other words, we
write
aá = ajgéá

and we say that aá arises by lowering the index j of aj.

Example 11.12 If we consider the space ®n with a Cartesian coordinate sys-

tem {eá}, then we have gáé = Óeá, eé Ô = ∂áé, and hence aá = ∂áéaj = ai. Therefore,
in a Cartesian coordinate system, there is no distinction between the compo-
nents of covariant and contravariant vectors. This explains why 1-forms never
arise in elementary treatments of vector analysis. ∆

Since the metric tensor is nondegenerate, the matrix (gáé) must be nonsin-
gular (or else the mapping aj ’ aá would not be an isomorphism). We can
therefore define the inverse matrix (gij ) by

gijgéÉ = gÉégji = ∂iÉ .

 11.10 THE METRIC TENSOR AND VOLUME FORMS 611

Using (gij ), we see that the inverse of the mapping aj ’ aá is given by

gijaé = ai .

This is called, naturally enough, raising an index. We will show below that
the gij do indeed form the components of a tensor.
It is worth remarking that the “tensor” gié = gikgÉé = ∂ié (= ∂éi ) is unique in
that it has the same components in any coordinate system. Indeed, if {eá} and
{eõá} are two bases for a space V with corresponding dual bases {øi} and {øùi},
then eõá = eéajá and øùj = bjáøi = (aî)jáøi (see the discussion following Theorem
11.2). Therefore, if we define the tensor ∂ to have the same values in the first
coordinate system as the Kronecker delta, then ∂ié = ∂(øi, eé). If we now define
the symbol ∂äié by ∂äié = ∂(øùi, eõé), then we see that

! i j = ! (" i ,!e j ) = ! ((a#1 )i k " k ,!er a r j ) = (a#1 )i k a r j! (" k ,!er )

= (a#1 )i k a r j! k r = (a#1 )i k a k j = ! i j !!.

This shows that the ∂ié are in fact the components of a tensor, and that these
components are the same in any coordinate system.
We would now like to show that the scalars gij are indeed the components
of a tensor. There are several ways that this can be done. First, let us write
gáégjk = ∂ki where we know that both gáé and ∂ki are tensors. Multiplying both
sides of this equation by (aî)rÉais and using (aî)rÉais∂ki = ∂rs we find

gáégjk(a î)rÉais = ∂rs .

Now substitute gáé = gát∂té = gitatq(aî)qé to obtain

[ais atq git][(aî)qé(aî)rÉgjk] = ∂rs .

Since git is a tensor, we know that aisatq git = gõsq. If we write

gõq r = (aî)qé(aî)rÉgjk

then we will have defined the gjk to transform as the components of a tensor,
and furthermore, they have the requisite property that gõsq gõqr = ∂rs. Therefore
we have defined the (contravariant) metric tensor G ∞ T 0 ™(V) by
612 MULTILINEAR MAPPINGS AND TENSORS

G = gijeá · eé
where gijgéÉ = ∂iÉ.
There is another interesting way for us to define the tensor G. We have
already seen that a vector A = ai eá ∞ V defines a unique linear form å =
aéøj ∞ V* by the association å = gáéaiøj. If we denote the inverse of the matrix
(gáé) by (gij) so that gijgéÉ = ∂iÉ, then to any linear form å = aáøi ∞ V* there
corresponds a unique vector A = ai eá ∞ V defined by A = gijaáeé. We can now
use this isomorphism to define an inner product on V*. In other words, if Ó , Ô
is an inner product on V, we define an inner product Ó , Ô on V* by

Óå, ∫Ô = ÓA, BÔ

where A, B ∞ V are the vectors corresponding to the 1-forms å, ∫ ∞ V*.

Let us write an arbitrary basis vector eá in terms of its components relative
to the basis {eá} as eá = ∂jáeé. Therefore, in the above isomorphism, we may
define the linear form eÄá ∞ V* corresponding to the basis vector eá by

eÄá = géÉ∂jáøk = gikøk

and hence using the inverse matrix, we find that

øk = gkieÄá .

Applying our definition of the inner product in V* we have ÓeÄá, eÄéÔ = Óeá, eéÔ =
gáé, and therefore we obtain

Ó! i ,!! jÔ = Ógir êr ,!g js êsÔ = gir g jsÓêr ,! êsÔ = gir g js grs = gir" j r = gij

which is the analogue in V* of the definition gáé = Óeá, eéÔ in V.

Lastly, since øùj = (aî)jáøi, we see that

g ij = Ó! i ,!! jÔ = Ó(a"1 )i r ! r ,!(a"1 ) j s ! sÔ = (a"1 )i r (a"1 ) j s Ó! r ,!! sÔ

= (a"1 )i r (a"1 ) j s g rs

so the scalars gij may be considered to be the components of a symmetric

tensor G ∞ T2 º(V) defined as above by G = gijeá · eé.
Now let g = Ó , Ô be an arbitrary (i.e., possibly degenerate) real symmetric
bilinear form on the inner product space V. It follows from the corollary to
Theorem 9.14 that there exists a basis {eá} for V in which the matrix (gáé) of g
takes the unique diagonal form
 11.10 THE METRIC TENSOR AND VOLUME FORMS 613

" Ir %
$ '
gij =!$ !I s '
$ '
# 0t &

where r + s + t = dim V = n. Thus

# !!1 for 1 ! i ! r
%
g(ei ,!ei ) = $"1 for r +1 ! i ! r + s !!.
%& !!0 for r + s +1 ! i ! n

If r + s < n, the inner product is degenerate and we say that the space V is
singular (with respect to the given inner product). If r + s = n, then the inner
product is nondegenerate, and the basis {eá} is orthonormal. In the orthonor-
mal case, if either r = 0 or r = n, the space is said to be ordinary Euclidean,
and if 0 < r < n, then the space is called pseudo-Euclidean. Recall that the
number r - s = r - (n - r) = 2r - n is called the signature of g (which is
therefore just the trace of (gáé)). Moreover, the number of -1’s is called the
index of g, and is denoted by Ind(g). If g = Ó , Ô is to be a metric on V, then by
definition, we must have r + s = n so that the inner product is nondegenerate.
In this case, the basis {eá} is called g-orthonormal.

Example 11.13 If the metric g represents a positive definite inner product on

V, then we must have Ind(g) = 0, and such a metric is said to be Riemannian.
Alternatively, another well-known metric is the Lorentz metric used in the
theory of special relativity. By definition, a Lorentz metric η has Ind(η) = 1.
Therefore, if η is a Lorentz metric, an η-orthonormal basis {eè, . . . , eñ}
ordered in such a way that η(eá, eá) = +1 for i = 1, . . . , n - 1 and η(eñ, eñ) =
-1 is called a Lorentz frame.
Thus, in terms of a g-orthonormal basis, a Riemannian metric has the form

!1 0 ! 0$
#0 1 ! 0&
(gij ) = # &
#" " "&
"0 0 ! 1%

while in a Lorentz frame, a Lorentz metric takes the form

#1 0 ! !!0 &
%0 1 ! !!0 (
(!ij ) = % (!!.
%" " !!" (
$0 0 ! "1 '
614 MULTILINEAR MAPPINGS AND TENSORS

It is worth remarking that a Lorentz metric is also frequently defined as

having Ind(η) = n - 1. In this case we have η(eè, eè) = 1 and η(eá, eá) = -1 for
each i = 2, . . . , n. We also point out that a vector v ∞ V is called timelike if
η(v, v) < 0, lightlike (or null) if η(v, v) = 0, and spacelike if η(v, v) > 0. Note
that a Lorentz inner product is clearly indefinite since, for example, the
nonzero vector v with components v = (0, 0, 1, 1) has the property that Óv, vÔ =
η(v, v) = 0. ∆

We now show that introducing a metric on V leads to a unique volume

form on V.

Theorem 11.20 Let g be a metric on an n-dimensional oriented vector space

(V, [ø]). Then, corresponding to the metric g, there exists a unique volume
form µ = µ(g) ∞ [ø] such that µ(eè , . . . , eñ) = 1 for every positively oriented
g-orthonormal basis {eá} for V. Moreover, if {vá} is any (not necessarily g-
orthonormal) positively oriented basis for V with dual basis {vi}, then

µ = \det(g(vá, vé))\1/2 v1° ~ ~ ~ °vn .

In particular, if {vá} = {eá} is a g-orthonormal basis, then µ = e1° ~ ~ ~ °en.

Proof Since ø ≠ 0, there exists a positively oriented g-orthonormal basis {eá}

such that ø(eè, . . . , eñ) > 0 (we can multiply eè by -1 if necessary in order
that {eá} be positively oriented). We now define µ ∞ [ø] by

µ(eè, . . . , eñ) = 1 .

That this defines a unique µ follows by multilinearity. We claim that if {fá} is

any other positively oriented g-orthonormal basis, then µ(fè, . . . , fñ) = 1 also.
To show this, we first prove a simple general result.
Suppose {vá} is any other basis for V related to the g-orthonormal basis
{eá} by vá = ƒ(eá) = eéajá where, by Theorem 11.17, we have det ƒ = det(aié).
We then have g(vá, vé ) = aráaség(er, es) which in matrix notation is [g]v =
AT[g]eA, and hence

det(g(vi ,!v j )) = (det ! )2 det(g(er ,!es ))!!. (9)

However, since {eá} is g-orthonormal we have g(er, es) = ±∂rs, and therefore
\det(g(er, es))\ = 1. In other words

|det(g(vi ,!v j ))|1/2 = |det ! |!!. (10)

 11.10 THE METRIC TENSOR AND VOLUME FORMS 615

Returning to our problem, we have det(g(fá, fé)) = ±1 also since {fá} =

{ƒ(eá)} is g-orthonormal. Thus (10) implies that \det ƒ\ = 1. But {fá} is posi-
tively oriented so that µ(fè, . . . , fñ) > 0 by definition. Therefore

0 < µ ( f1,!…!,! fn ) = µ (! (e1 ),!…! µ (! (en )) = (! *µ )(e1,!…!, en )

= (det ! )µ (e1,!…!,!en ) = det !

so that we must in fact have det ƒ = +1. In other words, µ(fè, . . . , fñ) = 1 as
claimed.
Now suppose that {vá} is an arbitrary positively oriented basis for V such
that vá = ƒ(eá). Then, analogously to what we have just shown, we see that
µ(vè, . . . , vñ) = det ƒ > 0. Hence (10) shows that (using Example 11.8)

µ (v1,!…!,!vn ) = det !
= \det(g(vi ,!v j ))|1/2
= |det(g(vi ,!v j ))|1/2 v1 "!" v n (v1,!…!,!vn )

which implies
µ = \det(g(vá, vé))\1/2 v1° ~ ~ ~ °vn . ˙

The unique volume form µ defined in Theorem 11.20 is called the g-

volume, or sometimes the metric volume form. A common (although rather
careless) notation is to write \det(g(vá, vé))\1/2 = “ \”g”\” . In this notation, the g-
volume is written as
“ \”g”\” v1° ~ ~ ~ °vn

where {vè, . . . , vñ} must be positively oriented. If the basis {vè, vì, . . . , vñ}
is negatively oriented, then clearly {vì, vè, . . . , vñ} will be positively
oriented. Furthermore, even though the matrix of g relative to each of these
oriented bases will be different, the determinant actually remains unchanged
(see the discussion following the corollary to Theorem 11.13). Therefore, for
this negatively oriented basis, the g-volume is

“ \”g”\” v2°v1°~ ~ ~°vn = -“ \”g”\” v1°v2° ~ ~ ~ °vn .

We thus have the following corollary to Theorem 11.20.

Corollary Let {vá} be any basis for the n-dimensional oriented vector space
(V, [ø]) with metric g. Then the g-volume form on V is given by

±“ \”g”\” v1°~ ~ ~°vn

616 MULTILINEAR MAPPINGS AND TENSORS

where the “+” sign is for {vá} positively oriented, and the “-” sign is for {vá}
negatively oriented.

Example 11.14 From Example 11.13, we see that for a Riemannian metric g
and g-orthonormal basis {eá} we have det(g(eá, eé)) = +1. Hence, from equa-
tion (9), we see that det(g(vá, vé)) > 0 for any basis {vá = ƒ(eá)}. Thus the g-
volume form on a Riemannian space is given by ±“g” v1° ~ ~ ~ °vn.
For a Lorentz metric we have det(¨(eá, eé)) = -1 in a Lorentz frame, and
therefore det(g(vá, vé)) < 0 in an arbitrary frame. Thus the g-volume in a
Lorentz space is given by ±“-”g” v1° ~ ~ ~ °vn.
Let us point out that had we defined Ind(η) = n - 1 instead of Ind(η) = 1,
then det(η(eá, eé)) < 0 only in an even dimensional space. In this case, we
would have to write the g-volume as in the above corollary. ∆

Example 11.15 (This example is a continuation of Example 11.11.) We

remark that these volume elements are of great practical use in the theory of
integration on manifolds. To see an example of how this is done, let us use
Examples 11.1 and 11.11 to write the volume element as (remember that this
applies only locally, and hence the metric depends on the coordinates)

d† = “ \”g”\” dx1° ~ ~ ~ °dxn .

If we go to a new coordinate system {xõi}, then

!x r !x s
gij = grs
!x i !x j

so that \gõ\ = (Jî)2\g\ where Jî = det($xr/$xõi) is the determinant of the inverse

Jacobian matrix of the transformation. But using dxõi = ($xõi/$xj)dxj and the
properties of the wedge product, it is easy to see that

"x 1 "x n i1
dx 1 !!! dx n = i1
!!! in
dx !!! dx in
"x "x
# "x i & 1
= det %% j (( dx !!! dx n
$ "x '

and hence
dxõ1°~ ~ ~°dxõn = J dx1° ~ ~ ~ °dxn
 11.10 THE METRIC TENSOR AND VOLUME FORMS 617

where J is the determinant of the Jacobian matrix. (Note that the proper
transformation formula for the volume element in multiple integrals arises
naturally in the algebra of exterior forms.) We now have

d! = |g| dx 1 "!" dx n = J #1 |g| J dx1 "!" dx n

= |g| dx1 "!" dx n = d!

and hence d† is a scalar called the invariant volume element. In the case of
®4 as a Lorentz space, this result is used in the theory of relativity. ∆

Exercises

1. Suppose V has a metric gáé defined on it. Show that for any A, B ∞ V we
have ÓA, BÔ = aábi = aibá.

2. According to the special theory of relativity, the speed of light is the same
for all unaccelerated observers regardless of the motion of the source of
light relative to the observer. Consider two observers moving at a constant
velocity ∫ with respect to each other, and assume that the origins of their
respective coordinate systems coincide at t = 0. If a spherical pulse of light
is emitted from the origin at t = 0, then (in units where the speed of light is
equal to 1) this pulse satisfies the equation x2 + y2 + z2 - t2 = 0 for the first
observer, and xõ2 + yõ2 + zõ2 - t ä2 = 0 for the second observer. We shall use
the common notation (t, x, y, z) = (x0, x1, x2, x3) for our coordinates, and
hence the Lorentz metric takes the form

$#1 '
& 1 )
!µ" =&
1 )
& )
% 1(

where 0 ¯ µ, ¥ ¯ 3.
(a) Let the Lorentz transformation matrix be Ò so that xõµ = Òµ¥ x¥. Show
that the Lorentz transformation must satisfy ÒT ηÒ = η.
(b) If the {xõµ} system moves along the x1-axis with velocity ∫, then it
turns out that the Lorentz transformation is given by

xõ0 = ©(x0 - ∫x1)

xõ1 = ©(x1 - ∫x0)
618 MULTILINEAR MAPPINGS AND TENSORS

xõ2 = x2
xõ3 = x3

where ©2 = 1/(1 - ∫2). Using Òµ¥ = $xõµ/$x¥, write out the matrix (Òµ¥),
and verify explicitly that ÒTηÒ = η.
(c) The electromagnetic field tensor is given by

#0 "E x "E y "Ez &

% (
Ex !0 !Bz "By (
Fµ! = % !!.
% Ey "Bz !0 !Bx (
% (
$ Ez !By "Bx !0 '

Using this, find the components of the electric field E ë and magnetic field
Bë in the {xõµ} coordinate system. In other words, find Fäµ¥ . (The actual
definition of Fµ¥ is given by Fµ¥ = $µA¥ - $¥Aµ where $µ = $/$xµ and
Aµ = (ƒ, A1, A2, A3) is related to Eë and Bë through the classical equations
Eë = -#ƒ - $Aë/$t and Bë = # ª Aë. See also Exercise 11.1.6.)

3. Let V be an n-dimensional vector space with a Lorentz metric η, and let

W be an (n - 1)-dimensional subspace of V. Note that

WÊ = {v ∞ V: η(v, w) = 0 for all w ∞ W}

is the 1-dimensional subspace of all normal vectors for W. We say that W

is timelike if every normal vector is spacelike, null if every normal vector
is null, and spacelike if every normal vector is timelike. Prove that η
restricted to W is
(a) Positive definite if W is spacelike.
(b) A Lorentz metric if W is timelike.
(c) Degenerate if W is null.

4. (a) Let D be a 3 x 3 determinant considered as a function of three contra-

variant vectors Ai(1), Ai(2), and Ai(3). Show that under a change of coordi-
nates, D does not transform as a scalar, but that D“ \”g”\” does transform as a
proper scalar. [Hint: Use Exercise 11.2.8.]
(b) Show that ´ijk“ \”g”\” transforms like a tensor. (This is the Levi-Civita
tensor in general coordinates. Note that in a g-orthonormal coordinate
system this reduces to the Levi-Civita symbol.)
(c) What is the contravariant version of the tensor in part (b)?
 CHAPTER 12

Hilbert Spaces

The material to be presented in this chapter is essential for all advanced work
in physics and analysis. We have attempted to present several relatively diffi-
cult theorems in sufficient detail that they are readily understandable by
readers with less background than normally might be required for such results.
However, we assume that the reader is quite familiar with the contents of
Appendices A and B, and we will frequently refer to results from these
appendices. Essentially, this chapter serves as an introduction to the theory of
infinite-dimensional vector spaces. Throughout this chapter we let E, F and G
denote normed vector spaces over the real or complex number fields only.

12.1 MATHEMATICAL PRELIMINARIES

This rather long first section presents the elementary properties of limits and
continuous functions. While most of this material properly falls under the
heading of analysis, we do not assume that the reader has already had such a
course. However, if these topics are familiar, then the reader should briefly
scan the theorems of this section now, and return only for details if and when
it becomes necessary.
For ease of reference, we briefly repeat some of our earlier definitions and
results. By a norm on a vector space E, we mean a mapping ˜ ˜: E ‘ ® satis-
fying:

619
620 HILBERT SPACES

(N1) ˜u˜ ˘ 0 for every u ∞ E and ˜u˜ = 0 if and only if u = 0 (positive

definiteness).
(N2) ˜cu˜ = \c\ ˜u˜ for every u ∞ E and c ∞ F.
(N3) ˜u + v˜ ¯ ˜u˜ + ˜v˜ (triangle inequality).

If there is more than one norm on E under consideration, then we may denote
them by subscripts such as ˜ ˜ì etc. Similarly, if we are discussing more than
one space, then the norm associated with a space E will sometimes be denoted
by ˜ ˜E . We call the pair (E, ˜ ˜) a normed vector space.
If E is a complex vector space, we define the Hermitian inner product as
the mapping Ó , Ô: E ª E ‘ ç such that for all u, v, w ∞ E and c ∞ ç we have:

(IP1) Óu, v + wÔ = Óu, vÔ + Óu, wÔ.

(IP2) Ócu, vÔ = c*Óu, vÔ.
(IP3) Óu, vÔ = Óv, uÔ*.
(IP4) Óu, uÔ ˘ 0 and Óu, uÔ = 0 if and only if u = 0.

where * denotes complex conjugation. A Hermitian inner product is some-

times called a sesquilinear form. Note that (IP2) and (IP3) imply

Óu, cvÔ = Ócv, uÔ* = cÓv, uÔ* = cÓu, vÔ

and that (IP3) implies Óv, vÔ is real.

As usual, if Óu, vÔ = 0 we say that u and v are orthogonal, and we some-
times write this as u ‡ v. If we let S be a subset of E, then the collection

{u ∞ E: Óu, vÔ = 0 for every v ∞ S}

is a subspace of E called the orthogonal complement of S, and is denoted by

SÊ.
It should be remarked that many authors define Ócu, vÔ = cÓu, vÔ rather than
our (IP2), and the reader must be careful to note which definition is being fol-
lowed. Furthermore, there is no reason why we could not have defined a map-
ping E ª E ‘ ®, and in this case we have simply an inner product on E
(where obviously there is now no complex conjugation).
The most common example of a Hermitian inner product is the standard
inner product on çn = ç ª ~ ~ ~ ª ç defined for all x = (xè, . . . , xñ) and y =
(yè, . . . , yñ) in çn by
n
Óx,!yÔ = ! xi *yi !!.
i=1
 12.1 MATHEMATICAL PRELIMINARIES 621

We leave it to the reader to verify conditions (IP1) - (IP4). Before defining a

norm on çn, we again prove (in a slightly different manner from that in
Chapter 2) the Cauchy-Schwartz inequality.

Example 12.1 Let E be a complex (or real) inner product space, and let u,
v ∞ E be nonzero vectors. Then for any a, b ∞ ç we have

0 ¯ Óau + bv, au + bvÔ = \a\2Óu, uÔ + a*bÓu, vÔ + b*aÓv, uÔ + \b\2Óv, vÔ .

Now note that the middle two terms are complex conjugates of each other, and
hence their sum is 2Re(a*bÓu, vÔ). Therefore, letting a = Óv, vÔ and b = -Óv, uÔ,
we have
0 ¯ Óv, vÔ2Óu, uÔ - 2Óv, vÔ\Óu, vÔ\2 + \Óu, vÔ\2Óv, vÔ

which is equivalent to

Óv, vÔ\Óu, vÔ\2 ¯ Óv, vÔ2Óu, uÔ .

Since v ≠ 0 we have Óv, vÔ ≠ 0, and hence dividing by Óv, vÔ and taking the
square root yields the desired result

\Óu, vÔ\ ¯ Óu, uÔ1/2 Óv, vÔ1/2 . ∆

If a vector space E has an inner product defined on it, then we may define
a norm on E by
˜v˜ = Óv, vÔ1/2

for all v ∞ E. Properties (N1) and (N2) for this norm are obvious, and (N3)
now follows from the Cauchy-Schwartz inequality and the fact that ReÓu, vÔ ¯
\Óu, vÔ\:
˜u + v˜2 = Óu + v,!u + vÔ
= ˜u˜2 +2 ReÓu,!vÔ + ˜v˜2
! ˜u˜2 +2|Óu,!vÔ| + ˜v˜2
! ˜u˜2 + 2˜u˜˜v˜ + ˜v˜2
= (˜u˜ + ˜v˜)2 !!.

We leave it to the reader (Exercise 12.1.1) to prove the so-called parallel-

ogram law in an inner product space (E, Ó , Ô):

˜u + v˜2 + ˜u - v˜2 = 2˜u˜2 + 2˜v˜2 .

622 HILBERT SPACES

The geometric meaning of this formula in ®2 is that the sum of the squares of
the diagonals of a parallelogram is equal to the sum of the squares of the sides.
If Óu, vÔ = 0, then the reader can also easily prove the Pythagorean theorem:

˜u + v˜2 = ˜u˜2 + ˜v˜2 if u ‡ v .

In terms of the standard inner product on çn, we now define a norm on çn

by
n
˜ x˜ = Óx,!xÔ = ! |xi |2 !!.
2

i=1
The above results now show that this does indeed satisfy the requirements of a
norm.
Continuing, if (E, ˜ ˜) is a normed space, then we may make E into a
metric space (E, d) by defining

d(u, v) = ˜u - v˜ .

Again, the only part of the definition of a metric space (see Appendix A) that
is not obvious is (M4), and this now follows from (N3) because

d(u,!v) = ˜u ! v˜ = ˜u ! w + w ! v˜ " ˜u ! w˜ + ˜w ! v˜
= d(u,!w) + d(w,!v)!!.

The important point to get from all this is that normed vector spaces form
a special class of metric spaces. This means that all the results from Appendix
A and many of the results from Appendix B will carry over to the case of
normed spaces. In Appendix B we presented the theory of sequences and
series of numbers. As we explained there however, many of the results are
valid as well for normed vector spaces if we simply replace the absolute value
by the norm.
For example, suppose A ™ E and let v ∞ E. Recall that v is said to be an
accumulation point of A if every open ball centered at v contains a point of
A distinct from v. In other words, given ´ > 0 there exists u ∞ A, u ≠ v, such
that ˜u - v˜ < ´. As expected, if {vñ} is a sequence of vectors in E, then we say
that {vñ} converges to the limit v ∞ E if given ´ > 0, there exists an integer
N > 0 such that n ˘ N implies ˜vñ - v˜ < ´. As usual, we write lim vñ =
limn‘Ÿvñ = v. If there exists a neighborhood of v (i.e., an open ball contain-
ing v) such that v is the only point of A in this neighborhood, then we say that
v is an isolated point of A.
 12.1 MATHEMATICAL PRELIMINARIES 623

Example 12.2 Suppose lim vñ = v. Then for every ´ > 0, there exists N such
that n ˘ N implies ˜v - vñ˜ < ´. From Example 2.11 we then see that

\ ˜v˜ - ˜vñ˜ \ ¯ ˜v - vñ˜ < ´

and hence directly from the definition of lim ˜vñ˜ we have

˜ lim vñ ˜ = ˜v˜ = lim ˜vñ˜ .

This result will be quite useful in several later proofs. ∆

Note that if v is an accumulation point of A, then for every n > 0 there

exists vñ ∞ A, vn ≠ v, such that ˜vñ - v˜ < 1/n. In particular, for any ´ > 0,
choose N so that 1/N < ´. Then for all n ˘ N we have ˜vñ - v˜ < 1/n < ´ so that
{vñ} converges to v. Conversely, it is clear that if vñ ‘ v with vñ ∞ A, vn ≠ v,
then v must necessarily be an accumulation point of A. This proves the fol-
lowing result.

Theorem 12.1 If A ™ E, then v is an accumulation point of A if and only if it

is the limit of some sequence in A - {v}.

A function f: (X, dX) ‘ (Y, dY) is said to be continuous at xà ∞ X if for each

´ > 0 there exists ∂ > 0 such that dX(x, xà) < ∂ implies dY(f(x), f(xà)) < ´. Note
though, that for any given ´, the ∂ required will in general be different for
each point xà chosen. If f is continuous at each point of X, then we say that f is
“continuous on X.”
A function f as defined above is said to be uniformly continuous on X if
for each ´ > 0, there exists ∂ > 0 such that for all x, y ∞ X with dX(x, y) < ∂
we have dY(f(x), f(y)) < ´. The important difference between continuity and
uniform continuity is that for a uniformly continuous function, once ´ is
chosen, there is a single ∂ (which will generally still depend on ´) such that
this definition applies to all points x, y ∞ X subject only to the requirement
that dX(x, y) < ∂. It should be clear that a uniformly continuous function is
necessarily continuous, but the converse is not generally true. We do though
have the next very important result. However, since we shall not have any
occasion to refer to it in this text, we present it only for its own sake and as an
(important and useful) illustration of the concepts involved.

Theorem 12.2 Let A ™ (X, dX) be compact, and let f: A ‘ (Y, dY) be con-
tinuous. Then f is uniformly continuous. In other words, a continuous function
on a compact set is uniformly continuous.
624 HILBERT SPACES

Proof Fix ´ > 0. Since f is continuous on A, for each point x ∞ A there exists
∂x > 0 such that for all y ∞ A, dX(x, y) < ∂í implies dY(f(x), f(y)) < ´/2. The
collection {B(x, ∂x/2): x ∞ A} of open balls clearly covers A, and since A is
compact, a finite number will cover A. Let {xè, . . . , xñ} be the finite collec-
tion of points such that {B(xá, ∂xá/2)}, i = 1, . . . , n covers A, and define ∂ =
(1/2)min({∂xá}). Since each ∂xá > 0, ∂ must also be > 0. (Note that if A were
not compact, then ∂ = inf({∂í}) taken over all x ∞ A could be equal to 0.)
Now let x, y ∞ A be any two points such that dX(x, y) < ∂. Since the
collection {B(xá, ∂xá/2)} covers A, x must lie in some B(xá, ∂xá/2), and hence
dX(x, xá) < ∂xá/2 for this particular xá. Then we also have

dX(y, xá) ¯ dX(x, y) + dX(x, xá) < ∂ + ∂xá/2 ¯ ∂xá .

But f is continuous at xá, and ∂xá was defined so that the set of points z for
which dX(z, xá) < ∂xá satisfies dY(f(z), f(xá)) < ´/2. Since we just showed that x
and y satisfy dX(x, xá) < ∂xá/2 < ∂xá and dX(y, xá) < ∂xá , we must have

dY(f(x), f(y)) ¯ dY(f(x), f(xá)) + dY(f(y), f(xá)) < ´/2 + ´/2 = ´ .

In other words, for our given ´, we found a ∂ such that for all x, y ∞ A
with dX(x, y) < ∂, we have dY(f(x), f(y)) < ´. ˙

Example 12.3 Consider the function f: E ‘ ® defined by f(u) = ˜u˜. In other

words, f is just the norm function on E. Referring to the above discussion, we
say that a function g is uniformly continuous if given ´ > 0, there exists ∂ > 0
such that ˜u - v˜ < ∂ implies that \g(u) - g(v)\ < ´ (note that the norm on E is
˜ ˜ while the norm on ® is \ \). But for our norm function f and for any ´ > 0,
we see that for all u, v ∞ E, if we choose ∂ = ´ then ˜u - v˜ < ∂ = ´ implies

\f(u) - f(v)\ = \ ˜u˜ - ˜v˜ \ ¯ ˜u - v˜ < ´

(where we used Example 2.11). Thus the norm is in fact uniformly continuous
on E.
We leave it to the reader (see Exercise 12.1.2) to show (using the Cauchy-
Schwartz inequality) that the inner product on E is also continuous in both
variables. ∆

There is an equivalent way to define continuous functions in terms of

limits that is also of great use. Let X and Y be metric spaces, and suppose f:
A ™ X ‘ Y. Then if xà is an accumulation point of A, we say that a point L ∞
 12.1 MATHEMATICAL PRELIMINARIES 625

Y is the limit of f at xà if, given ´ > 0, there exists ∂ > 0 (which may depend
on f, xà and ´) such that for all x ∞ A we have 0 < dX(x, xà) < ∂ implies
dY(f(x), L) < ´. This is written as limx‘xà f(x) = L or simply “f(x) ‘ L as
x ‘ xà.”
Note that while xà is an accumulation point of A, xà is not necessarily an
element of A, and hence f(xà) might not be defined. In addition, even if xà ∞
A, it is not necessarily true that limx‘xà f(x) = f(xà). However, we do have the
following result.

Theorem 12.3 If f: A ™ (X, dX) ‘ (Y, dY) and xà ∞ A is an accumulation

point of A, then f is continuous at xà if and only if

limx‘xà f(x) = f(xà) .

Proof This obvious by comparison of the definition of continuity of f at xà,

and the definition of the limit of f at xà. ˙

Before we can prove the basic properties of continuous functions, we must

prove some elementary properties of limits. First we need a definition. A
product on E ª F ‘ G is a mapping denoted by (u, v) ’ uv that is bilinear
and satisfies ˜uv˜G ¯ ˜u˜E ˜v˜F. For example, using the Cauchy-Schwartz
inequality, we see that the usual inner product on ®n is just a product on ®n ª
®n ‘ ®.

Example 12.4 We say that a function f: S ‘ F is bounded if there exists

M > 0 such that ˜f(x)˜ ¯ M for all x ∞ S. Now consider the space E = B(S, ®)
of real-valued bounded functions on any nonempty set S. Let us define a norm
˜ ˜Ÿ on B(S, ®) by
˜f˜Ÿ = supx´S \f(x)\

for any f ∞ B(S, ®). This important norm is called the sup norm. For any f,
g ∞ B(S, ®) suppose ˜f˜Ÿ = Cè and ˜g˜Ÿ = Cì. Then it follows that \f(x)\ ¯ Cè
and \g(x)\ ¯ Cì for all x ∞ S. But then for all x ∞ S we have

\f(x)g(x)\ = \f(x)\ \g(x)\ ¯ CèCì = ˜f˜Ÿ ˜g˜Ÿ

so that the usual product of (real-valued) functions is also bounded. Therefore

we see that
˜fg˜Ÿ ¯ ˜f˜Ÿ˜g˜Ÿ

and since the usual product is obviously bilinear, we have a (general) product
on E ª E ‘ E. ∆
626 HILBERT SPACES

With the notion of a product carefully defined, we can repeat parts (a) -
(c) of Theorem B2 in a more general form as follows. The proof is virtually
identical to that of Theorem B2 except that here we replace the absolute value
by the norm.

Theorem 12.4 (a) Let uñ ‘ u and vñ ‘ v be convergent sequences of vec-

tors in E. Then limn‘Ÿ(uñ + vñ) = u + v.
(b) Let vñ ‘ v be a convergent sequence of vectors in E, and let c be a
scalar. Then limn‘Ÿ(cvñ) = cv.
(c) Let uñ ‘ u and vñ ‘ v be convergent sequences of vectors in E and F
respectively, and let E ª F ‘ G be a product. Then limn‘Ÿ(uñvñ) = uv.

Theorem 12.5 (a) Suppose that A ™ E and v is an accumulation point of A.

Let f and g be mappings of A into F, and assume that limu ‘v f(u) = wè and
limu‘v g(u) = wì. Then

limu ‘v(f + g)(u) = wè + wì .

(b) Let A be a subset of some normed space, and let v be an accumulation

point of A. Let f: A ‘ E and g: A ‘ F be mappings, and assume further that
limu‘v f(u) = wè and limu‘v g(u) = wì. If E ª F ‘ G is a product, then

limu‘v f(u)g(u) = w1wì .

Proof (a) Given ´ > 0, there exists ∂è > 0 such that if u ∞ A with ˜u - v˜ < ∂è
then \f(u) - wè\ < ´/2. Similarly, there exists ∂ì > 0 such that ˜u - v˜ < ∂ì
implies \g(u) - wì\ < ´/2. Choosing ∂ = min{∂è, ∂ì} we see that if u ∞ A and
˜u - v˜ < ∂ we have

˜( f + g)(u) ! (w1 + w2 )˜ = ˜ f (u) ! w1 + g(u) ! w2 ˜

" ˜ f (u) ! w1˜ + ˜g(u) ! w2 ˜
< # /2 + # /2
= # !!.

(b) Given ´ > 0, there exists ∂è > 0 such that ˜u - v˜ < ∂è implies

˜f(u) - wè˜ < ´/[2(1 + ˜wì˜)] .

Similarly, there exists ∂ì > 0 such that ˜u - v˜ < ∂ì implies

˜g(u) - wì˜ < ´/[2(1 + ˜wè˜)] .

 12.1 MATHEMATICAL PRELIMINARIES 627

From the definition of limit, given ´ = 1 there exists ∂3 > 0 such that ˜u - v˜ <
∂3 implies
˜f(u) - wè˜ < 1 .

But from Example 2.11 we see that

˜f(u)˜ - ˜wè˜ ¯ ˜f(u) - wè˜ < 1

which implies
˜f(u)˜ < 1 + ˜wè˜ .

If we let ∂ = min{∂è, ∂ì, ∂3}, then for all u ∞ A with ˜u - v˜ < ∂ we have

˜ f (u)g(u) ! w1w2 ˜ = ˜ f (u)g(u) ! f (u)w2 + f (u)w2 ! w1w2 ˜

" ˜ f (u)[g(u) ! w2 ]˜ + ˜[ f (u) ! w1 ]w2 ˜
" ˜ f (u)˜ ˜g(u) ! w2 ˜ + ˜ f (u) ! w1˜ ˜w2 ˜
< (1+ ˜w1˜)# /[2(1+ ˜w1˜)] + # /[2(1+ ˜w2 ˜)] ˜w2 ˜
< # /2 + # /2
= # !!.!!˙

The reader should realize that the norms used in the last proof are not
defined on the same normed space. However, it would have been too cluttered
for us to distinguish between them, and this practice is usually followed by
most authors.
It will also be of use to formulate the limit of a composition of mappings.

Theorem 12.6 Suppose A ™ E and B ™ F, and let f: A ‘ B and g: B ‘ G

be mappings. Assume that u is an accumulation point of A and that
limx‘u f(x) = v. Assume also that v is an accumulation point of B and that
limy‘v g(y) = w. Then

limx‘u (g ı f )(x) = limx‘u g(f(x)) = w .

Proof Given ´ > 0, there exists ∂è > 0 such that for all y ∞ B with ˜y - v˜ < ∂è,
we have ˜g(y) - w˜ < ´. Then given this ∂è, there exists ∂ì > 0 such that for all
x ∞ A with ˜x - u˜ < ∂ì, we have ˜f(x) - v˜ < ∂è. But now letting y = f(x), we
see that for such an x ∞ A we must have ˜g(f(x)) - w˜ < ´. ˙

We are now in a position to prove the basic properties of continuous func-

tions on a normed space.
628 HILBERT SPACES

Theorem 12.7 (a) If A ™ E and f, g: A ‘ F are continuous at v ∞ A, then

the sum f + g is continuous at v.
(b) Let f: A ‘ E and g: A ‘ F be continuous at v ∞ A and suppose that
E ª F ‘ G is a product. Then the product map fg is continuous at v.
(c) Suppose A ™ E, B ™ F, and let f: A ‘ B and g: B ‘ G be mappings.
Assume that f is continuous at v ∞ A with f(v) = w, and assume that g is con-
tinuous at w. Then the composite map g ı f is continuous at v.
(d) A mapping f: E ‘ F is continuous at v if and only if for every
sequence {vñ} in E we have vñ ‘ v implies f(vñ) ‘ f(v). In other words, we
have lim f(vn) = f(lim vn).

Proof (a) If v is an isolated point there is nothing to prove, so assume that v

is an accumulation point. Then by Theorems 12.5 and 12.3 we have

lim u!v ( f + g)(u) = lim u!v f (u) + lim u!v g(u)

= f (v) + g(v)
= ( f + g)(v)!!.

(b) This also follows from Theorems 12.5 and 12.3.

(c) Left to the reader (see Exercise 12.1.3).
(d) We first assume that f is continuous at v, and that the sequence {vñ}
converges to v. We must show that f(vñ) ‘ f(v). Now, since f is continuous,
we know that given ´ > 0, there exists ∂ > 0 such that ˜u - v˜ < ∂ implies
˜f(u) - f(v)˜ < ´. Furthermore, the convergence of {vñ} means that given ∂ > 0,
there exists N such that ˜vñ - v˜ < ∂ for every n ˘ N. Therefore, for every n ˘
N we have ˜vñ - v˜ < ∂ implies ˜f(vñ) - f(v)˜ < ´.
We now prove that if f is not continuous at v, then there exists a conver-
gent sequence vñ ‘ v for which f(vñ) ‘ Ö f(v). It will be notationally simpler
for us to formulate this proof in terms of open balls defined by the induced
metric (see Appendix A). If f is not continuous at v, then there exists B(f(v), ´)
with no corresponding B(v, ∂) such that f(B(v, ∂)) ™ B(f(v), ´). Consider the
sequence of open balls {B(v, 1/n)} for n = 1, 2, . . . . Since f is not continu-
ous, we can find vñ ∞ B(v, 1/n) such that f(vñ) ! B(f(v), ´). It is clear that the
sequence {vñ} converges to v (given ´, choose n ˘ N = 1/´), but that f(vñ)
does not converge to f(v) since by construction B(f(v), ´) contains none of the
f(vñ). ˙

Since the notion of open sets is extremely important in much of what fol-
lows, it is natural to wonder whether different norms defined on a space lead
to different open sets (through their induced metrics). We shall say that two
 12.1 MATHEMATICAL PRELIMINARIES 629

norms ˜ ˜è and ˜ ˜ì defined on E are equivalent if there exists a number C such

that for all u ∞ E we have

Cî˜u˜è ¯ ˜u˜ì ¯ C˜u˜1 .

We leave it to the reader to show that this defines an equivalence relation on

the set of all norms on E (see Exercise 12.1.4).

Example 12.5 It is easy to see that this definition does exactly what we want
it to do. For example, suppose U ™ E is open relative to a norm ˜ ˜è. This
means that for any u ∞ U, there exists ´è > 0 such that ˜u - v˜è < ´è implies v ∞
U. We would like to show that given an equivalent norm ˜ ˜ì, then there exists
´ì > 0 such that ˜u - v˜ì < ´ì implies v ∞ U. We know there exists C > 0 such
that Cî˜ ˜è ¯ ˜ ˜ì ¯ C ˜ ˜è, and hence choosing ´ì = ´è/C, it follows that for all
v ∞ E with ˜u - v˜ì < ´ì we have

˜u - v˜è ¯ C ˜u - v˜ì < C´2 = ´è

so that v ∞ U. Therefore we have shown that if a set is open with respect to

one norm, then it is open with respect to any equivalent norm. ∆

Example 12.6 It is also easy to give an example of two non-equivalent norms

defined on a space. To see this, consider the space E of all real-valued contin-
uous functions defined on [0, 1]. We define a norm on E by means of the
scalar product. Thus for any f, g ∞ E we define the scalar product by

1
Óf ,!gÔ = ! 0 f (x)g(x) dx
and the associated norm by ˜f˜ì = Óf, f Ô1/2. This norm is usually called the L2-
norm. Alternatively, we note that any continuous real function defined on
[0, 1] must be bounded (Theorems A8 and A14). Hence we may also define
the sup norm ˜f˜Ÿ by
˜f˜Ÿ = sup \f(x)\

where the sup is taken over all x ∞ [0, 1].

Now suppose f ∞ E and write ˜f˜Ÿ = C. Then we have

1 1
! 0 [ f (x)]2 dx " ! 0 C 2 dx = C 2
and hence ˜f˜ì ¯ ˜f˜Ÿ. However, this is only half of the inequalities required by
the definition. Consider the peaked function defined on [0, 1] by
630 HILBERT SPACES

If we let this function become arbitrarily narrow while maintaining the height,
it is clear that the sup norm will always be equal to 1, but that the L2-norm can
be made arbitrarily small. ∆

The source of the problem that arose in this example is a result of the fact
that the space E of continuous functions defined on [0, 1] is infinite-
dimensional. In fact, we will soon prove that this can not occur in finite-
dimensional spaces. In other words, we will see that all norms are equivalent
in a finite-dimensional space.
The reader may wonder whether or not the limits we have defined depend
in any way on the particular norm being used. It is easy to show that if the
limit of a sequence exists with respect to one norm, then it exists with respect
to any other equivalent norm, and in fact the limits are equal (see Exercise
12.1.5). It should now be clear that a function that is continuous at a point v
with respect to a norm ˜ ˜è is also continuous at v with respect to any equiva-
lent norm ˜ ˜ì.
Now recall from Appendix B that a metric space in which every Cauchy
sequence converges to a point in the space is said to be complete. It was also
shown there that the space ®n is complete with respect to the standard norm
(Theorem B8), and hence so is the space çn (since çn may be thought of as
®n ª ®n = ®2n). Recall also that a Banach space is a normed vector space
(E, ˜ ˜) that is complete as a metric space (where as usual, the metric is that
induced by the norm). If an inner product space (E, Ó , Ô) is complete as a
metric space (again with the metric defined by the norm induced by the inner
product), then E is called a Hilbert space.
It is natural to wonder whether a space that is complete relative to one
norm is necessarily complete relative to any other equivalent norm. This is
answered by the next theorem. In the proof that follows, it will be convenient
to use the nonstandard norm ˜ ˜Ñ defined on ®n (or çn) by
n
˜(u ,!…!,!u )˜N = ! |u i |
1 n

i=1
 12.1 MATHEMATICAL PRELIMINARIES 631

where (u1, . . . , un) is a vector n-tuple in ®n (or çn). In ®2, the unit ball
{(uè, uì): ˜(u1, u2)˜Ñ ¯ 1} looks like

(0, 1)

(-1, 0) (1,0)

(0, -1)

Theorem 12.8 Let E be a finite-dimensional vector space over either ® or ç.

Then
(a) There exists a norm on E.
(b) All norms on E are equivalent.
(c) All norms on E are complete.

Proof Let {eè, . . . , eñ} be a basis for E so that any u ∞ E may be written as
u = Íui eá.
(a) We define the norm ˜ ˜è on E by
n
˜u ˜1 = ! |u i |!!.
i=1
Properties (N1) and (N2) are trivial to verify, and if v = Ívieá is any other
vector in E, then u + v = Í(ui + vi)eá, and hence

˜u + v˜1 = ! | u i + vi | " !(| u i | + | vi |) = !| u i | +!| vi |

= ˜u ˜1 + ˜v˜1

so that (N3) is also satisfied.

This norm is quite convenient for a number of purposes. Note that it yields
the same result for any v = Ívi eá ∞ E as does the nonstandard norm ˜ ˜Ñ for
the corresponding (v1, . . . , vn) ∞ ®n (or çn).
(b) Let ˜ ˜ì be any other norm on E, and let u, v ∞ E be arbitrary. Using
Example 2.11 and properties (N3) and (N2), we see that
632 HILBERT SPACES

|˜u ˜2 ! ˜v˜2 | " ˜ u ! v˜2

= ˜ #(u i ! vi )ei ˜2
" # ˜(u i ! vi )ei ˜2
(*)
= #| u i ! vi | ˜ei ˜2
" max1"i"n {˜ei ˜2 } #| u i ! vi |
= max1"i"n {˜ei ˜2 }˜(u1,!…!,!u n ) ! (v1,!…!,!v n )˜N

Define the mapping f: çn ‘ ® by x = (x1, . . . , xn) ’ ˜Íxieá˜ì ∞ [0, Ÿ). To

say that f is uniformly continuous on çn with respect to the norm ˜ ˜Ñ means
that given ´ > 0, we can find ∂ > 0 such that for all x, y ∞ çn with

˜x - y˜Ñ = ˜(x1, . . . , xn) - (y1, . . . , yn)˜Ñ < ∂

we have
\f(x) - f(y)\ = \ ˜Íxieá˜ì - ˜Íyieá˜ì \ < ´ .

If we define B = max1¯i¯n{˜eá˜ì}, then choosing ∂ = ´/B, we see (*) shows that

f is (uniformly) continuous with respect to the norm ˜ ˜Ñ on çn.
We now note that the unit sphere S = {x ∞ çn: ˜x˜Ñ = 1} is closed and
bounded, and hence S is compact (Theorem A14). The restriction of the func-
tion f to S is then continuous and strictly positive (by (N1)), and hence
according to Theorem A15, f attains both its minimum m and maximum M on
S. In other words, for every x = (x1, . . . , xn) ∞ çn with ˜x˜Ñ = 1 we have 0 <
m ¯ ˜Íxieá˜ì ¯ M. Since ˜x˜Ñ = ˜ (x1, . . . , xn) ˜Ñ = 1, we may write

m˜(x1, . . . , xn)˜Ñ ¯ ˜Íxieá˜ì ¯ M˜(x1, . . . , xn)˜Ñ

or, using part (a) with u = Íxieá ∞ E, we find that

m˜u˜è ¯ ˜u˜ì ¯ M˜u˜è .

Choosing C = max{1/m, M}, we see that ˜ ˜è and ˜ ˜ì are equivalent. The fact
that ˜ ˜ì was arbitrary combined with the fact that equivalent norms form an
equivalence class completes the proof that all norms on E are equivalent.
(c) It suffices to show that E is complete with respect to any particular
norm on E. This is because part (b) together with the fact that a sequence that
converges with respect to one norm must converge with respect to any equiv-
alent norm then shows that E will be complete with respect to any norm.
 12.1 MATHEMATICAL PRELIMINARIES 633

Recall from the corollary to Theorem 2.8 that E is isomorphic to either ®n or

çn. The result then follows from Theorem B8 or its obvious extension to çn
(see Exercise 12.1.6). ˙

We shall see that closed subspaces play an important role in the theory of
Hilbert spaces. Because of this, we must make some simple observations.
Suppose that Y is a closed subset of a complete space (X, d), and let {xñ} be a
Cauchy sequence in Y. Then {xñ} is also obviously a Cauchy sequence in X,
and hence xñ ‘ x ∞ X. But this means that x ∞ Cl Y = Y (Theorem B13(b) or
B14(a)) so that {xñ} converges in Y.
On the other hand, suppose that Y is a complete subset of an arbitrary
metric space (X, d) and let {xñ} be any sequence in Y that converges to an
element x ∞ X. We claim that in fact x ∞ Y which will prove that Y is closed
(Theorem B14(a)). Since xñ ‘ x ∞ X, it follows that {xñ} is a Cauchy
sequence in X (since any convergent sequence is necessarily Cauchy). In other
words, given ´ > 0 there exists N > 0 such that m, n ˘ N implies ˜xm - xñ˜ < ´.
But then {xñ} is just a Cauchy sequence in Y (which is complete), and hence
xñ ‘ x ∞ Y.
This discussion proves the next result.

Theorem 12.9 Any closed subset of a complete metric space is also a com-
plete metric space. On the other hand, if a subset of an arbitrary metric space
is complete, then it is closed.

Corollary A finite-dimensional subspace of any real or complex normed

vector space is closed.

Proof This follows from Theorems 12.8 and 12.9. ˙

Now suppose that A ™ E and that we have a mapping f: A ‘ F where F =

Fè ª ~ ~ ~ ª Fñ is the Cartesian product of normed spaces. Then for any v ∞ A
we have f(v) = (fè(v), . . . , fñ(v)) where each fá: A ‘ Fá is called the ith coor-
dinate function of f. In other words, we write f = (fè, . . . , fñ).
If w = (wè, . . . , wñ) ∞ F, then one possible norm on F, the sup norm, is
defined by
˜w˜ = sup1¯i¯n{˜wá˜}

where ˜wᘠdenotes the norm in Fá. However, this is not the only possible
norm. Recall that if x = (xè, . . . , xñ) ∞ ®n = ® ª ~ ~ ~ ª ®, then the standard
norm in ®n is given by ˜x˜2 = Í \xá\2. The analogous “Pythagorean” norm ˜ ˜p
on F would then be defined by ˜w˜p2 = Í ˜wá˜2. Alternatively, we could also
634 HILBERT SPACES

define the “nonstandard” norm ˜w˜Ñ = Í ˜wá˜. We leave it to the reader to

show that these three norms are equivalent on F (see Exercise 12.1.7).
The next result should come as no surprise. Note that we will use the sup
norm on F as defined above.

Theorem 12.10 Suppose A ™ E and let f: A ‘ F = Fè ª ~ ~ ~ ª Fñ be a map-

ping. If v is an accumulation point of A, then limu‘v f(u) exists if and only if
limu‘v fá(u) exists for each i = 1, . . . , n. If this is the case and if limu‘v f(u)
= w = (wè, . . . , wñ), then limu‘v fá(u) = wá for each i = 1, . . . , n.

Proof First assume that limu‘v f(u) = w = (wè, . . . , wñ). This means that
given ´ > 0, there exists ∂ such that ˜u - v˜ < ∂ implies ˜f(u) - w˜ < ´. If we
write f(u) = (fá(u), . . . , fñ(u)), then for all u ∞ A with ˜u - v˜ < ∂, the defini-
tion of sup norm tells us that

˜fá(u) - wᘠ¯ ˜f(u) - w˜ < ´ .

This proves that limu‘v fá(u) = wá.

Conversely, suppose limu‘v fá(u) = wá for each i = 1, . . . , n. Then given
´ > 0, there exists ∂á such that ˜u - v˜ < ∂á implies ˜fá(u) - wᘠ< ´. Defining ∂ =
min{∂á}, we see that for all u ∞ A with ˜u - v˜ < ∂, we have ˜fá(u) - wᘠ< ´ for
each i = 1, . . . , n and therefore

˜f(u) - w˜ = sup{˜fá(u) - wá˜} < ´ .

This shows that limu‘v f(u) = w. ˙

Corollary The mapping f defined in Theorem 12.10 is continuous if and

only if each fá is continuous.

Proof Obvious from the definitions. ˙

Exercises

1. If u, v ∞ (E, Ó , Ô) prove:
(a) The parallelogram law: ˜u + v˜2 + ˜u - v˜2 = 2˜u˜2 + 2˜v˜2.
(b) The Pythagorean theorem: ˜u + v˜2 = ˜u˜2 + ˜v˜2 if u ‡ v.

2. Show that an inner product on E is continuous in both variables by

showing that limy‘yà Óx, yÔ = Óx, y0Ô and limx‘xà Óx, yÔ = Óx0, yÔ.
 12.1 MATHEMATICAL PRELIMINARIES 635

3. Prove part (c) of Theorem 12.7.

4. Show that equivalent norms define an equivalence relation on the set of all
norms on E.

5. (a) Suppose {vñ} is a sequence in a normed space E. Show that if vñ ‘ v

with respect to one norm on E, then vñ ‘ v with respect to any equivalent
norm on E.
(b) Show that if a function is continuous at a point with respect to one
norm, then it is continuous at that point with respect to any equivalent
norm.

6. Fill in the details in the proof of Theorem 12.8(c).

7. Let F = Fè ª ~ ~ ~ ª Fñ be a Cartesian product of normed spaces, and sup-

pose w = (wè, . . . , wñ) ∞ F. If ˜wᘠdenotes the norm on Fá, show that the
norms ˜w˜ = sup1¯i¯n{˜wá˜}, ˜w˜p2 = Íi ˆ=1˜wá˜2 and ˜w˜N = Íi ˆ=1˜wᘠare
equivalent on F.

8. Show that the set B(S, E) of all bounded functions from a nonempty set S
to a normed vector space E forms a vector space (over the same field as
E).

12.2 OPERATOR NORMS

Suppose E and F are normed spaces, and let A: E ‘ F be a linear map. If

there exists a number M > 0 such that ˜Av˜F ¯ M˜v˜E for all v ∞ E, then A is
said to be bounded, and the number M is called a bound for A. In other
words, to say that A is bounded means that it takes bounded values on
bounded sets. Note that we labeled our norms in a way that denotes which
space they are defined on. From now on though, we shall not complicate our
notation by this designation unless it is necessary. However, the reader should
be careful to note that the symbol ˜ ˜ may mean two different things within a
single equation.
Recall also that a linear map A: E ‘ F is said to be continuous at và ∞ E
if given ´ > 0, there exists ∂ > 0 such that for all v ∞ E with ˜và - v˜ < ∂, we
have ˜Avà - Av˜ = ˜A(và - v) ˜ < ´.
636 HILBERT SPACES

Theorem 12.11 Let E be a finite-dimensional normed space, and let A: E ‘

F be a linear map of E into a normed space F (not necessarily finite-
dimensional). Then A is bounded.

Proof Let {eè, . . . , eñ} be a basis for E so that any v ∞ E may be written in
the form v = Ívieá. Using the defining properties of the norm and the linearity
of A, we then have

˜Av˜ = ˜AÍvieᘠ= ˜ÍviAeᘠ¯ Í ˜viAeᘠ= Í \vi\ ˜Aeᘠ.

Since all norms on E are equivalent (Theorem 12.8), we use the norm ˜ ˜è
defined by ˜v˜è = Í\vi\. Thus any other norm ˜ ˜ì on E will be related to ˜ ˜è by
Cî˜ ˜ì ¯ ˜ ˜è ¯ C˜ ˜ì for some number C. Since ˜Aeᘠ< Ÿ for each i, we define
the real number M = max{˜Aeá˜}. Then

˜Av˜ ¯ MÍ\vi\ = M˜v˜1 ¯ MC˜v˜2 . ˙

Our next result is quite fundamental, and will be referred to again several
times.

Theorem 12.12 Let A: E ‘ F be a linear map of normed spaces. If A is

bounded, then A is uniformly continuous, and if A is continuous at 0 then A is
bounded.

Proof If A is bounded, then there exists M > 0 such that ˜Av˜ ¯ M˜v˜ for
every v ∞ E. Then for any ´ > 0, we choose ∂ = ´/M so that for all u, v ∞ E
with ˜u - v˜ < ∂, we have

˜Au - Av˜ = ˜A(u - v) ˜ ¯ M˜u - v˜ < ´ .

This proves that A is uniformly continuous.

Conversely, suppose A is continuous at 0 ∞ E. Then given ´ = 1, there
exists ∂ > 0 such that ˜v˜ < ∂ implies ˜Av˜ < ´ = 1. In particular, we see that for
any nonzero v ∞ E we have ˜∂v/(2˜v˜)˜ = ∂/2 < ∂ which implies ˜A(∂v/(2˜v˜))˜
< 1. Taking out the constants yields ˜Av˜ < (2/∂)˜v˜. This shows that A is
bounded with bound M = 2/∂. ˙

As shown in Exercise 12.2.1, there is nothing special about the continuity

of A at the origin.
 12.2 OPERATOR NORMS 637

Corollary 1 A linear map A: E ‘ F is bounded if and only if A is continu-

ous.

Proof Obvious. ˙

Corollary 2 Let E be finite-dimensional, and let A: E ‘ F be a linear map.

Then A is uniformly continuous.

Proof This follows directly from Theorems 12.11 and 12.12. ˙

Let E and F be normed spaces, and let A: E ‘ F be a continuous (and

hence bounded) linear map (if E is finite-dimensional, then the continuity
requirement is redundant). We define the operator norm of A by

˜A˜ = sup{˜Av˜/˜v˜:!v ! E,!v " 0}

= sup{˜Av˜:!˜v˜ = 1}!!.

If ˜v˜ ¯ 1, then we may write v = cvÄ where ˜vĘ = 1 and \c\ ¯ 1. Then ˜Av˜ =
\c\ ˜AvĘ ¯ ˜AvĘ and therefore, since we are using the sup, an equivalent defini-
tion of ˜A˜ is
˜A˜ = sup{˜Av˜: ˜v˜ ¯ 1} .

From the first definition, we see that for any v ∞ E we have ˜Av˜/˜v˜ ¯ ˜A˜,
and hence we have the important result

˜Av˜ ¯ ˜A˜ ˜v˜ .

This shows that another equivalent definition of ˜A˜ is

˜A˜ = inf{M > 0: ˜Av˜ ¯ M˜v˜ for all v ∞ E} .

Another useful result follows by noting that if A: E ‘ F and B: F ‘ G,

then for any v ∞ E we have

˜(B ı A)v˜ = ˜B(Av) ˜ ¯ ˜B˜ ˜Av˜ ¯ ˜B˜ ˜A˜ ˜v˜

and hence from the definition of the operator norm we have

˜B ı A˜ ¯ ˜B˜ ˜A˜ .
638 HILBERT SPACES

We denote the space of all continuous linear maps from E to F by L(E, F).
That L(E, F) is in fact a vector space will be shown below. Since for any A ∞
L(E, F) we have ˜A˜ = sup{˜Av˜: ˜v˜ ¯ 1}, we see that by restricting A to the
unit ball in E, the space L(E, F) is just a subspace of the space B(S, F) of all
bounded maps from S into F that was defined in Example 12.4 (where S is just
the unit ball in E).

Theorem 12.13 The space L(E, F) with the operator norm is a normed
vector space. Moreover, if F is a Banach space, then so is L(E, F).

Proof Suppose that A ∞ L(E, F). We first verify requirements (N1) - (N3)
for a norm. From the definitions, it is obvious that ˜A˜ ˘ 0 and ˜0˜ = 0. In addi-
tion, if ˜A˜ = 0 then for any v ∞ E we have ˜Av˜ ¯ ˜A˜ ˜v˜ = 0 which implies
that A = 0. This verifies (N1). If c ∞ F, then

˜cA˜ = sup{˜(cA)v˜:!˜v˜ ! 1}
= |c|sup{˜Av˜:!˜v˜ = 1}
= |c|˜A˜

which verifies (N2). Now let A, B ∞ L(E, F). Then using Theorem 0.5 we see
that (leaving out the restriction on ˜v˜)

˜A + B˜ = sup{˜(A + B)v˜}
= sup{˜Av + Bv˜}
! sup{˜Av˜ + ˜Bv˜}
= sup{˜Av˜} + sup{˜Bv˜}
= ˜A˜ + ˜B˜

which proves (N3). That L(E, F) is in fact a vector space follows from
Theorem 12.7(a) and (b).
Now suppose that F is a Banach space and let {Añ} be a Cauchy sequence
in L(E, F). This means that for every ´ > 0 there exists N such that m, n ˘ N
implies ˜Am - Añ˜ < ´. In particular, for any v ∞ E and ´ > 0, there exists N
such that for all m, n ˘ N we have

˜Amv - Añv˜ = ˜(Am - Añ)v˜ ¯ ˜Am - Añ˜ ˜v˜ < (´/˜v˜)˜v˜ = ´

so that {Añv} is a Cauchy sequence in F. Since F is a Banach space this

sequence converges, and hence we define Av ∞ F by

Av = limn‘Ÿ Añv .
 12.2 OPERATOR NORMS 639

This defines a map A: E ‘ F. Since each Añ is linear, it should be clear

(using Theorem 12.4) that A is linear. We must still show that A is continuous
(so that A ∞ L(E, F)) and that Añ ‘ A.
Given ´ > 0, there exists N such that ˜Am - Añ˜ < ´ for all m, n ˘ N. If v ∞
E is such that ˜v˜ ¯ 1, then

˜Amv - Añv˜ ¯ ˜Am - Añ˜ ˜v˜ < ´ .

But {Amv} converges to Av, and hence letting m ‘ Ÿ yields

˜(A - Añ)v˜ = ˜Av - Añv˜ ¯ ´

for every v ∞ E with ˜v˜ ¯ 1. This shows that A - Añ is continuous at 0, and

hence A - Añ is in L(E, F) (by Theorem 12.12 and its Corollary 1). Thus A ∞
L(E, F) (since each Añ is). Finally, since A - Añ ∞ L(E, F), we may apply the
definition of operator norm to obtain

˜A - Añ˜ = sup{˜(A - Añ)v˜: ˜v˜ ¯ 1} ¯ ´

for every n ˘ N, and hence Añ ‘ A. ˙

Exercises

1. Let A: E ‘ F be linear and continuous a some point v0 ∞ E. Prove

directly that A is continuous at every v ∞ E.

2. (Linear Extension Theorem) Let E be a normed vector space, F a sub-

space of E, and G a Banach space. Suppose A ∞ L(F, G) and assume that
˜A˜ = M. Prove:
(a) The closure Fä of F in E is a subspace of E.
(b) There exists a unique extension Aä ∞ L(F ä, G) of A. [Hint: If v ∞ F ä,
then there exists {vñ} ∞ F such that vñ ‘ v (why?). Show that {Avñ} is
Cauchy in G, and converges to a unique limit that is independent of {vñ}.
Define Aäv = lim Avñ, and show that Aä is linear. Also show that Aäv = Av
for any v ∞ F. Next, show that Aä is bounded so that Aä ∞ L(F ä, G) (why?),
and finally show that the Aä so defined is unique.]
(c) ˜Aä˜ = ˜A˜.
640 HILBERT SPACES

3. Let E be a normed vector space. A completion (Eä, A) of E is a Banach

space Eä together with a continuous injective linear mapping A: E ‘ Eä
that preserves the norm and is such that A(E) is dense in Eä. Show that
(Eä, A) is unique in the sense that if (F, B) is another completion of E, then
there exists a unique invertible element C ∞ L(Eä, F) such that B = C ı A.
[Hint: Apply the previous exercise to the mappings B ı Aî and A ı Bî.]

12.3 HILBERT SPACES

Discussing infinite-dimensional vector spaces requires a certain amount of

care that was not needed in our treatment of finite-dimensional spaces. For
example, how are we to express an arbitrary vector as a linear combination of
basis vectors? For that matter, how do we define a basis in an infinite-
dimensional space? As another example, recall that in our treatment of
operator adjoints, we restricted our discussion to finite-dimensional spaces
(see Theorems 10.1 and 10.2). While we cannot define the adjoint in an
arbitrary infinite-dimensional space (e.g., a Banach space), we shall see that it
is nevertheless possible to make such a definition in a Hilbert space.
Unfortunately, a thorough treatment of Hilbert spaces requires a knowl-
edge of rather advanced integration theory (i.e., the Lebesgue theory).
However, it is quite easy to present a fairly complete discussion of many of
the basic and important properties of Hilbert spaces without using the general
theory of integration.
As in the previous sections of this chapter, we consider only vector spaces
over the real and complex fields. In fact, unless otherwise noted we shall
always assume that our scalars are complex numbers. Recall from Section
12.1 that a Hilbert space is an inner product space which is complete as a
metric space. We shall generally denote a Hilbert space by the letter H.
To begin with, we recall that a linear space E is n-dimensional if it con-
tains a set of n linearly independent vectors, but every set of n + 1 vectors is
linearly dependent. If E contains n linearly independent vectors for every pos-
itive integer n, then we say that E is infinite-dimensional. Let us rephrase
some of our earlier discussion of (infinite) series in a terminology that fits in
with the concept of an infinite-dimensional space.
We have already seen that a sequence {vñ} of vectors in a space (E, ˜ ˜)
converges to v ∞ E if for each ´ > 0, there exists an N such that n ˘ N implies
˜vñ - v˜ < ´. We sometimes write this as vñ ‘ v or ˜vñ - v˜ ‘ 0. Similarly,
we say that an infinite linear combination Ík~ = 1aÉwÉ of vectors in E con-
verges if the sequence of partial sums vñ = Íkˆ= 1aÉwÉ converges. In other
words, to write v = Ík~ = 1aÉwÉ means that vñ ‘ v. If no explicit limits on the
sum are given, then we assume that the sum is over an infinite number of
vectors.
 12.3 HILBERT SPACES 641

Just as we did with finite linear combinations, we define the addition of

infinite linear combinations componentwise. Thus, if x = Íanvñ and y =
Íbñvñ, then we define the sum x + y by x + y = Í(añ + bñ)vñ. If xñ =
Íkˆ=1aÉvÉ converges to x, and yñ = Íkˆ=1bÉvÉ converges to y, then it is quite
easy to see that xñ + yñ = Íkˆ=1(aÉ + bÉ)vÉ converges to x + y (see Exercise
12.3.1). Furthermore, we define scalar multiplication of an infinite linear
combination x = Íañvñ by cx = Í(cañ)vñ. It is also easy to see that if the nth
partial sum xñ converges to x, then cxñ converges to cx.
In our next two examples we define the general Banach spaces lpˆ and lp,
and we then show that both l2ˆ and l2 may be made into Hilbert spaces. (In
more advanced work, the space lp may be generalized to include measure
spaces.) Remember that our scalars may be either real or complex numbers.

Example 12.7 If p is any real number such that 1 ¯ p < Ÿ, we let lpˆ denote
the space of all scalar n-tuples x = (xè, . . . , xñ) with the norm ˜ ˜p defined by

"n %1/p
p
˜x˜ p = $$! |xi | '' !!.
# i=1 &

We first show that this does indeed define a norm on lpˆ. Properties (N1) and
(N2) of the norm are obvious, so it remains to show that property (N3) is also
obeyed. To show this, we will prove two general results that are of importance
in their own right. In the derivation to follow, if p occurs by itself, it is defined
as above. If the numbers p and q occur together, then q is defined the same
way as p, but we also assume that 1/p + 1/q = 1. (If p and q satisfy the relation
1/p + 1/q = 1, then p and q are said to be conjugate exponents. Note that in
this case both p and q are strictly greater than 1.)
Let å and ∫ be real numbers ˘ 0. We first show that

!1/p "1/q # ! /p + " /q!!. (1)

This result is clear if either å or ∫ is zero, so assume that both å and ∫ are
greater than zero. For any real k ∞ (0, 1) define the function f(t) for t ˘ 1 by

f(t) = k(t - 1) - tk + 1 .

From elementary calculus, we see that fæ(t) = k(1 - tk-1), and hence fæ(t) ˘ 0
for every t ˘ 1 and k ∞ (0, 1). Since f(1) = 0, this implies that f(t) ˘ 0, and thus
the definition of f(t) shows that
642 HILBERT SPACES

tk ¯ k(t - 1) + 1 = kt + (1 - k) .

If å ˘ ∫, we let t = å/∫ and k = 1/p to obtain

(å/∫)1/p ¯ å/∫p + (1 - 1/p) = å/∫p + 1/q .

Multiplying through by ∫ and using ∫1-1/p = ∫1/q yields the desired result.
Similarly, if å < ∫ we let t = ∫/å and k = 1/q.
To help see the meaning of (1), note that taking the logarithm of both sides
of (1) yields
1 1 $! " '
log ! + log " # log & + )!!.
p q % p q(

The reader should recognize this as the statement that the logarithm is a
“convex function” (see the figure below).

log t
(1/p)log å + (1/q)log ∫

å t ∫
t = å/p + ∫/q

We now use (1) to prove H!!

older’s inequality:
n
"|xi yi | ! ˜x˜ p ˜y˜q !!.
i=1
Again, we assume that x and y are both nonzero. Define åá = (\xá\ / ˜x˜p)p and
∫á = (\yá\ / ˜y˜q)q . From (1) we see that

\xáyá\ / (˜x˜p ˜y˜q) ¯ åá/p + ∫á/q .

Using the definition of ˜ ˜p, it follows that Í iˆ=1åá = 1 and similarly for ∫á.
Hence summing the previous inequality over i = 1, . . . , n and using the fact
 12.3 HILBERT SPACES 643

that 1/p + 1/q = 1 yields HoÜlder’s inequality. We remark that the particular
case of p = q = 2 yields
n #n &1/2 # n &1/2
2 2
"| xi yi | ! %%"| xi | (( %%"| yi | ((
i=1 $ i=1 ' $ i=1 '

which is called Cauchy’s inequality.

Finally, we use HoÜlder’s inequality to prove Minkowski’s inequality:

˜x + y˜p ¯ ˜x˜p + ˜y˜p .

If p = 1 this is obvious since \xá + yá\ ¯ \xá\ + \yá\, so we may assume that p > 1.
In this case we have

n
(˜x + y˜ p ) p = ! | xi + yi | p
i=1
n
= ! | xi + yi |!| xi + yi | p"1 (2)
i=1
n
# ! (| xi | + | yi |)| xi + yi | p"1 !!.
i=1

Using HoÜlder’s inequality with yá replaced by \xá + yá\p/q results in the

inequality
n
!| xi |!| xi + yi | p/q " ˜ x˜ p (˜ x + y˜ p ) p/q
i=1
with a similar result if we interchange xá and yá. Since 1/p + 1/q = 1 implies
p/q = p - 1, we now see that (2) yields

(˜x + y˜p)p ¯ (˜x˜p + ˜y˜p)(˜x + y˜p)p -1 .

Dividing this by (˜x + y˜p)p -1 yields Minkowski’s inequality.

We now see that Minkowski’s inequality is just the requirement (N3), and
thus we have shown that our norm on lpˆ is indeed a norm. Finally, it follows
from Theorem 12.8 that lpˆ is complete and is thus a Banach space.
We now consider the particular case of l2ˆ, and define an inner product in
the expected manner by
n
Óx,!yÔ = ! xi * yi !!.
i=1
644 HILBERT SPACES

Defining the norm by

"n %1/2
1/2 2
˜x˜ = Óx,!xÔ = $$! | xi | ''
# i=1 &

it is easy to see that l2ˆ satisfies all of the requirements for a Hilbert space. ∆

In our next example, we generalize this result to the case of infinite-

dimensional spaces.

Example 12.8 As in the previous example, let p be any real number such that
1 ¯ p < Ÿ. We let lp denote the space of all sequences x = {xè, xì, . . . } of
scalars such that Ík~ = 1 \xÉ\p < Ÿ, and we define a norm on lp by

#! &1/p
p
˜ x˜ p = %% " | xk | (( !!.
$ k=1 '

We must show that this definition also satisfies the properties of a norm,
which means that we need only verify the not entirely obvious condition (N3).
From the previous example, we may write Minkowski’s inequality for the
space lpˆ in the form

" n %1/ p " n %1/ p " n %1/ p

p p p
$$ ! | xk + yk | '' ( $$ ! | xk | '' + $$ ! | yk | '' !!.
# k=1 & # k=1 & # k=1 &

Now, if x, y ∞ lp, then both (Ík~ = 1 \xÉ\p)1/p and (Ík~ = 1 \yÉ\p)1/p exist since
they are convergent by definition of lp . Hence taking the limit of
Minkowski’s inequality as n ‘ Ÿ shows that this equation also applies to
infinite series as well. (This requires the observation that the pth root is a
continuous function so that, by Theorem 12.7(d), the limit may be taken inside
the root.) In other words, the equation ˜x + y˜p ¯ ˜x˜p + ˜y˜p also applies to the
space lp . This shows that our definition of a norm is satisfactory. It should
also be clear that HoÜlder’s inequality similarly applies to the space lp.
It is more difficult to show that lp is complete as a metric space. The origin
of the problem is easily seen by referring to Theorems B3 and B8. In these
theorems, we showed that a Cauchy sequence {xÉ} in ®n led to n distinct
Cauchy sequences {xÉj} in ®, each of which then converged to a number xj by
the completeness of ®. This means that for each j = 1, . . . , n there exists an Né
such that \xÉj - xj\ < ´/“n” for all k ˘ Né . Letting N = max{Né}, we see that
 12.3 HILBERT SPACES 645

n
˜xk ! x˜2 = " | xk j ! x j |2 < n(# 2 /n) = # 2
j=1
for all k ˘ N, and hence xÉ ‘ x. However, the case of lp we cannot take the
max of an infinite number of integers. To circumvent this problem we may
proceed as follows.
To keep the notation as simple as possible and also consistent with most
other authors, we let x = {xè, xì, . . . } be an element of lp with components xá,
and we let {x(n)} be a sequence in lp. Thus, the kth component of the vector
x(n) ∞ lp is given by xÉ(n). Note that this is the opposite of our notation in the
finite-dimensional case.
Let {x(n)} be a Cauchy sequence in lp. This means that for any ´ > 0, there
exists M > 0 such that m, n ˘ M implies ˜x(m) - x(n) ˜p < ´. Then, exactly as in
the finite-dimensional case, for any k = 1, 2, . . . we have

$
(m) (n) p
| xk ! xk | " % | x j (m) ! x j (n) | p = (˜ x(m) ! x(n) ˜ p ) p < # p
j=1

and hence \xÉ(m) - xÉ(n)\ < ´. Therefore, for each k the sequence {xÉ(n)} of the
kth component forms a Cauchy sequence. Since ® (or ç) is complete, these
sequences converge to a number which we denote by xÉ. In other words, for
every k we have
limn ‘Ÿ xÉ(n) = xÉ .

To show that lp is complete, we will show that the sequence x = {xÉ} is an

element of lp , and that in fact ˜x(n) ‘ x˜p ‘ 0.
Using Minkowski’s inequality, we have for every N and any n,

"N %1/ p " N %1/ p

p (n) (n) p
$$ ! | xk | '' = $$ ! | xk ( xk + xk | ''
# k=1 & # k=1 &
"N %1/ p "N %1/ p
(n) p
) $$ ! | xk ( xk | '' + (n) p
$$ ! | xk | '' (3)
# k=1 & # k=1 &
"N %1/ p
) $$ ! | xk ( xk | '' + ˜ x (n) ˜ p !!.
(n) p

# k=1 &
646 HILBERT SPACES

Now write the nth term of the sequence {x(n)} as x(n) - x(m) + x(m) to obtain

˜x(n)˜p = ˜x(n) - x(m) + x(m)˜p ¯ ˜x(n) - x(m)˜p + ˜x(m)˜p .

Since {x(n)} is a Cauchy sequence , we know that given any ´ > 0, there exists
Mà such that m, n ˘ Mà implies ˜x(n) - x(m) ˜p < ´. Thus for any fixed m ˘
Mà, the set {˜x(n)˜p: n ˘ Mà} of real numbers is bounded by ´ + ˜x(m)˜p .
Moreover, we may take the max of the (finite) set of all ˜x(n)˜p with n < Mà. In
other words, we have shown that the norms of every term in any Cauchy
sequence are bounded, and hence we may write (3) as

"N %1/ p " N %1/ p

p (n) p
$$ ! | xk | '' ( $$ ! | xk ) xk | '' + B (4)
# k=1 & # k=1 &

where ˜x(n)˜p ¯ B for all n.

Since xÉ(n) ‘ xÉ for each of the finite number of terms k = 1, . . . , N, we
can choose n sufficiently large (but depending on N) that the first term on the
right hand side of (4) is ¯ 1, and hence for every N we have

N
" | xk | p ! (1+ B) p !!.
k=1

This shows that the series Ík~ = 1 \xÉ\p converges, and thus by definition of lp ,
the corresponding sequence x = {xÉ} is an element of lp . We must still show
that x(n) ‘ x.
Since {x(n)} is a Cauchy sequence, it follows that given ´ > 0, there exists
M such that m, n ˘ M implies ˜x(m) - x(n) ˜p < ´. Then for any N and all m,
n ˘ M we have (using the Minkowski inequality again)

#N &1/ p
(n) p
%% " | xk ! xk | ((
$ k=1 '
#N &1/ p #N &1/ p
(m) p (m) (n) p
) %% " | xk ! xk | (( + %% " | xk ! xk | ((
$ k=1 ' $ k=1 '
#N &1/ p
) %% " | xk ! xk | (( + ˜ x (m) ! x (n) ˜ p
(m) p

$ k=1 '
 12.3 HILBERT SPACES 647

$N '1/ p
(m) p
! && # | xk " xk | )) + * !!. (5)
% k=1 (

But xÉ(m) ‘ xÉ for each k = 1, . . . , N and hence (by the same argument used
above) we can choose m sufficiently large that the first term in the last line of
(5) is < ´. This means that for every N and all n ˘ m (where m is independent
of N) we have
#N &1/ p
(n) p
%% " | xk ! xk | (( < 2)
$ k=1 '

and hence taking the limit as N ‘ Ÿ yields

$" '1/ p
(n) (n) p
˜ x ! x ˜ p = && # | xk ! xk | )) * 2+ !!.
% k=1 (

Since this inequality holds for all n ˘ M, we have shown that ˜x - x(n)˜p ‘ 0
or, alternatively, that x(n) ‘ x. We have therefore shown that the space lp is
complete, i.e., it is a Banach space.
It is now easy to show that l2 is a Hilbert space. To see this, we define the
inner product on l2 in the usual way by
!
Óx,!yÔ = " xk * yk !!.
k=1
Using the infinite-dimensional version of HoÜlder’s inequality with p = q = 2
(i.e., Cauchy’s inequality), we see that this series converges absolutely, and
hence the series converges to a complex number (see Theorem B20). This
shows that the inner product so defined is meaningful. The rest of the verifi-
cation that l2 is a Hilbert space is straightforward and left to the reader (see
Exercise 12.3.2). ∆

Recall that a subset A of a metric space X is said to be dense if Cl A = X.

Intuitively, this simply means that any neighborhood of any x ∞ X contains
points of A (see Theorems B13 and B15). A space is said to be separable if it
contains a countable dense subset. An important class of Hilbert spaces are
those that are separable.

Example 12.9 Let us show that the space l2 is actually separable. In other
words, we shall show that l2 contains a countable dense subset. To see this, we
say that a point x = {xè, xì, . . . } ∞ l2 is a rational point if xñ ≠ 0 for only a
648 HILBERT SPACES

finite number of the indices n, and if each of these nonzero components is a

rational (complex) number. It should be clear that the set of all such rational
points is countably infinite. We must now show that any neighborhood of any
point in l2 contains at least one rational point.
To do this, we show that given any x ∞ l2 and any ´ > 0, there exists a
rational point r = {rè, rì, . . . , 0, 0, . . . } ∞ l2 such that ˜r - x˜2 < ´. Since x ∞
l2 the series Ík~ = 1 \xÉ\2 converges, and hence there exists N such that

!
" | xk |2 < # 2 /2
k=N +1

(see Theorem B17). Next, for each k = 1, . . . , N we find a rational number rÉ

with the property that
"
|rk ! xk | < !!.
2N

(That this can be done follows from Theorem 0.4 applied to both the real and
imaginary parts of xÉ.) Then the distance between x and the rational point r =
{rè, rì, . . . , rN, 0, 0, . . . } is given by

$N # '1/2
2 2
˜r ! x˜2 = && " |rk ! xk | + " | xk | ))
% k=1 k=N +1 (
< [N(* 2 /2N ) + * 2 /2]1/2 = * !!.!!!

As the last remark of this section, the reader should note that the proof of
the Cauchy-Schwartz inequality in Example 12.1 made no reference whatso-
ever to any components, and thus it clearly holds in any Hilbert space, as does
the parallelogram law. Furthermore, as mentioned in Example 12.3, the
Cauchy-Schwartz inequality also shows that the inner product is continuous in
each variable. Indeed, applying Theorem 12.7(d) we see that if xñ ‘ x and
yñ ‘ y, then

|Óxn ,!ynÔ ! Óx,!yÔ| = |Óxn ! x,!yn ! yÔ + Óxn ! x,!yÔ + Óx,!yn ! yÔ|

" ˜xn ! x˜ ˜yn ! y˜ + ˜xn ! x˜! ˜y˜ + ˜ x˜ ˜yn ! y˜ # 0

This is sometimes expressed by saying that the inner product is jointly

continuous. Alternatively, we can note that

\Óxè, yÔ - Óxì, yÔ\ = \Óxè - xì, yÔ\ ¯ ˜xè - xì˜ ˜y˜

 12.3 HILBERT SPACES 649

which shows that the map x ‘ Óx, yÔ is actually uniformly continuous, with
the same result holding for y ‘ Óx, yÔ.

Exercises

1. If xñ = Íkˆ=1aÉvÉ ‘ x and yñ = Íkˆ=1bÉvÉ ‘ y, show that xñ + yñ ‘ x + y.

2. Complete the proof (begun in Example 12.8) that l2 is a Hilbert space.

[Hint: Note that if x = {xè, xì, . . . } and y = {yè, yì, . . . } are vectors in l2,
then you must show that x + y ∞ l2 also.]

3. Prove that every compact metric space (X, d) is separable. [Hint: For each
integer n ˘ 1 consider the collection Uñ of open spheres

Uñ = {B(x, 1/n): x ∞ X} .]

4. Let H be a Hilbert space and suppose A ∞ L(H) is a positive symmetric

operator. Prove the generalized Schwartz inequality:

\ÓAx, yÔ\2 ¯ ÓAx, xÔÓAy, yÔ

where x, y ∞ H. [Hint: Let c be a real number and consider the vector z =

x + cÓAx, yÔy.]

5. Let lŸ denote the linear space consisting of all bounded sequences x =

{x1, x2, . . . , xn, . . . } of scalars with norm ˜x˜ = sup \xn\. Show that lŸ is a
Banach space.

12.4 CLOSED SUBSPACES

Since the norm on a vector space induces a metric topology on the space (i.e.,
defines the open sets in terms of the induced metric), it makes sense to define
a closed subspace as a subspace which is a closed set relative to the metric
topology. In view of Theorem B14, we say that a set A of vectors is closed if
every convergent sequence of vectors in A converges to a vector in A. If E is a
vector space, many authors define a linear manifold to be a subset S ™ E of
vectors such that S is also a linear space. In this case, a subspace is defined to
be a closed linear manifold. From the corollary to Theorem 12.9, we then see
that any finite-dimensional linear manifold over either ç or ® is a subspace.
We mention this terminology only in passing, and will generally continue to
650 HILBERT SPACES

use the word “subspace” in our previous context (i.e., as a linear manifold).
As a simple example, let V = ® be a vector space over the field Œ. Then the
subspace W ™ V defined by W = Œ is not closed (why?).
Recall from Theorem 2.22 that if W is a subspace of a finite-dimensional
inner product space V, then V = W • WÊ. We now wish to prove that if M is
a closed subspace of a Hilbert space H, then H = M • MÊ. Unfortunately, this
requires that we prove several preliminary results along the way. We begin
with a brief discussion of convex sets.
We say that a subset S of a vector space V is convex if for every pair
x, y ∞ S and any real number t ∞ [0, 1], the vector

z = (1 - t)x + ty

is also an element of S. Intuitively, this is just says that the straight line
segment from x to y in V is in fact contained in S. It should be obvious that
the intersection of any collection of convex sets is convex, and that every sub-
space of V is necessarily convex.
It follows by induction that if S is convex and xè, . . . , xñ ∞ S, then the
vector tèxè + ~ ~ ~ + tñxñ where 0 ¯ tá ¯ 1 and tè + ~ ~ ~ + tñ = 1 is also in S.
Conversely, the set of all such linear combinations forms a convex set. It is
trivial to verify that if S is convex, then so is any translate

S + z = {x + z: z ∞ V is fixed and x ∞ S} .

Moreover, if ¬: V ‘ W is a linear map and S ™ V is convex, then ¬(S) is a

convex subset of W, and if T ™ W is convex, then ¬î(T) is convex in V. We
leave the proofs of these elementary facts to the reader (see Exercise 12.4.1).
The main result dealing with convex sets that we shall need is given in the
next theorem.

Theorem 12.14 Every nonempty closed convex subset S of a Hilbert space

H contains a unique vector of smallest norm. In other words, there exists a
unique xà ∞ S with the property that ˜xà˜ ¯ ˜x˜ for every x ∞ S.

Proof Let ∂ = inf{˜x˜: x ∞ S}. By definition of inf, this implies the existence
of a sequence {xñ} of vectors in S such that ˜xñ˜ ‘ ∂. Since S is convex,
(xñ + xm)/2 is also in S (take t = 1/2 in the definition of convex set), and hence
˜(xñ + xm)/2˜ ˘ ∂ or ˜xñ + xm˜ ˘ 2∂. Applying the parallelogram law we see
that
 12.4 CLOSED SUBSPACES 651

˜ xn ! xm ˜2 = 2˜ xn ˜2 + 2˜ xm ˜2 ! ˜ xn + xm ˜2
" 2˜ xn ˜2 + 2˜ xm ˜2 ! 4# 2 !!.

Taking the limit of the right hand side of this equation as m, n ‘ Ÿ shows
that ˜xñ - xm˜ ‘ 0, and hence {xñ} is a Cauchy sequence in S. By Theorem
12.9, S is complete, and thus there exists a vector x ∞ S such that xñ ‘ x.
Since the norm is a continuous function, we see that (see Examples 12.2 and
12.3 or Theorem 12.7(d))

˜x˜ = ˜lim xñ˜ = lim ˜xñ˜ = ∂ .

Thus x ∞ S is a vector with smallest norm ∂ = inf{˜x˜: x ∞ S}.

To show that this x is unique, suppose that y ∞ S is such that ˜y˜ = ∂.
Applying the parallelogram law again to the vectors x/2 and y/2 yields

˜x - y˜2/4 = ˜x˜2/2 + ˜y˜2/2 - ˜(x + y)/2˜ˆ2 .

But (x + y)/2 ∞ S implies ˜(x + y)/2˜ ˘ ∂, and thus we have

˜x - y˜2 ¯ 2˜x˜2 + 2˜y˜2 - 4∂2 .

If ˜x˜ = ˜y˜ = ∂, then this equation implies that x = y. ˙

The notion of orthogonality is extremely important in the theory of Hilbert

spaces. We recall from Section 12.1 that two vectors x and y in a Hilbert
space H are said to be orthogonal if Óx, yÔ = 0, and we write this as x ‡ y. If S
is a (nonempty) subset of H and x ∞ H is orthogonal to every y ∞ S, then we
express this by writing x ‡ S. Thus the orthogonal complement SÊ of S is
defined by SÊ = {x ∞ H: x ‡ S}.
As an example, we consider the orthogonal complement xÊ of any x ∞ H.
If x ‡ y and x ‡ z, then x ‡ (y + z) and x ‡ (åy) for any scalar å. Therefore
xÊ is actually a subspace of H. If we define a continuous linear map fx: H ‘
ç by fx(y) = Óx, yÔ, then xÊ = {y ∞ H: fx(y) = 0}. In fact, if {yñ} is a sequence
in xÊ that converges to an element y ∞ H, then the continuity of the inner
product yields
Óx, yÔ = Óx, lim yñÔ = lim Óx, yñÔ = 0

and hence y ∞ xÊ also. This proves that xÊ is in fact a closed subspace of H.

Carrying this idea a little farther, if S is a subset of H, then we can clearly
write SÊ = ⁄x´S xÊ. Since this shows that SÊ is the intersection of closed
subspaces, it follows that SÊ must also be a closed subspace (see Exercise
12.4.2). Alternatively, if y ∞ S and {xñ} ™ SÊ with xñ ‘ x, then we again
652 HILBERT SPACES

have Óx, yÔ = lim Óxñ, yÔ = 0 so that x ∞ SÊ and SÊ is therefore closed. We

leave it to the reader to prove the following (see Exercise 12.4.3):

0Ê = H and HÊ = 0.
S ⁄ SÊ ™ {0}.
S ™ SÊÊ = (SÊ)Ê.
Sè ™ Sì implies SìÊ ™ SèÊ.

Furthermore, using the next theorem, it is not hard to show that a subset M of
a Hilbert space H is closed if and only if MÊÊ = M (see Exercise 12.4.6).

Theorem 12.15 Let M be a proper closed subspace of a Hilbert space H (i.e.,

M ≠ H). Then there exists a nonzero vector xà ∞ H such that xà ‡ M.

Proof Suppose x ∞ H and x ! Μ. Since any subspace is automatically

convex, it follows that the set x - M = {x - y: y ∞ M} is closed and convex.
By Theorem 12.14, this set contains a unique vector xà = x - yà ∞ x - M of
smallest norm. By definition, this means that ˜x - yà˜ = inf{˜x - y˜: y ∞ M}. If
we had ˜x - yà˜ = 0, then x would be an accumulation point of M, contra-
dicting the assumption that M is closed and x ! M. Thus we must have xà ≠ 0,
and we claim that xà ‡ M.
Since xà is of smallest norm, we see that for any y ∞ M and any å ∞ ç we
have
˜xà˜2 ¯ ˜xà + åy˜2 .

Expanding this out in terms of the inner product on H we find that

0 ¯ 2 Re{åÓxà, yÔ} + \å\2 ˜y˜2 .

In particular, if we let å = cÓy, xàÔ where c ∞ ® is nonzero, then this equation

becomes
0 ¯ c\Óxà, yÔ\2 (2 + c˜y˜2) .

If y ∞ M is such that Óxà, yÔ ≠ 0, then the fact that this equation holds for all
nonzero c ∞ ® leads to a contradiction if we choose c such that -2/˜y˜2 < c <
0. It therefore follows that we must have Óxà, yÔ = 0 for every y ∞ M, and
hence xà ‡ M. ˙

We are now in a position to prove our earlier assertion. After the proof we
shall give some background as to why this result is important.
 12.4 CLOSED SUBSPACES 653

Theorem 12.16 Let M be a closed subspace of a Hilbert space H. Then H =

M • MÊ.

Proof We first show that M + MÊ is a closed subspace of H. To see this, we

note that M ⁄ MÊ = {0} (since M is a subspace and hence contains the zero
vector), and every z ∞ M + MÊ may be written in the unique form z = x + y
with x ∞ M and y ∞ MÊ. (See the proof of Theorem 2.12. Note also that we
have not yet shown that every z ∞ H is of this form.) Now let {zñ} =
{xñ + yñ} be any sequence in M + MÊ that converges to an element z ∞ H.
We must show that z ∞ M + MÊ. Using the Pythagorean theorem we see that

˜zm ! zn ˜ 2 = ˜(xm + ym ) ! (xn + yn )˜ 2

= ˜(xm ! xn ) + (ym ! yn )˜ 2
= ˜ xm ! xn ˜ 2 + ˜ ym ! yn ˜ 2

and therefore {zñ} is a Cauchy sequence in M + MÊ if and only if {xñ} is a

Cauchy sequence in M and {yñ} is a Cauchy sequence in MÊ. Since both M
and MÊ are closed they are complete (Theorem 12.9). Therefore, since {zñ} is
a convergent sequence in H it is a Cauchy sequence in H, and in fact it is a
Cauchy sequence in M + MÊ since every zñ = xñ + yñ ∞ M + MÊ. But then
{xñ} and {yñ} are Cauchy sequences in M and MÊ which must converge to
points x ∞ M and y ∞ MÊ. Hence

z = lim zñ = lim(xñ + yñ) = lim xñ + lim yñ = x + y ∞ M + MÊ .

This shows that M + MÊ is a closed subspace of H.

We now claim that H = M + MÊ. Since we already know that M ⁄ MÊ =
{0}, this will complete the proof that H = M • MÊ. If H ≠ M + MÊ, then
according to Theorem 12.15 there exists a nonzero zà ∞ H with the property
that zà ‡ (M + MÊ). But this implies that zà ∞ MÊ and zà ∞ MÊÊ, and hence
˜zà˜2 = Ózà, zàÔ = 0 (or observe that z0 ∞ MÊ ⁄ MÊÊ = {0}) which contradicts
the assumption that zà ≠ 0. ˙

To gain a little insight as to why this result is important, we recall our dis-
cussion of projections in Section 7.8. In particular, Theorem 7.27 shows that a
linear transformation E on a finite-dimensional vector space V is idempotent
(i.e., E2 = E) if and only if V = U • W where E is the projection of V on U =
Im E in the direction of W = Ker E. In order to generalize this result to
Banach spaces, we define an operator on a Banach space B to be an element
of L(B, B). In other words, an operator on B is a continuous linear trans-
formation of B into itself. A projection on B is an idempotent operator on B.
654 HILBERT SPACES

Thus, in order that an operator P on B be idempotent, it must obey both the

algebraic requirement that P2 = P as well as the topological condition of
continuity. The generalization of Theorem 7.27 to Banach spaces is given by
the next two theorems, the proofs of which are left to the reader since we will
not be needing them again.

Theorem 12.17 Let P be a projection on a Banach space B and let M = Im P

and N = Ker P. Then M and N are closed subspaces of B, and B = M • N.

Proof See Exercise 12.4.5. ˙

Theorem 12.18 Let B be a Banach space and let M, N be closed subspaces

of B such that B = M • N. Then for any z = x + y ∞ M • N, the mapping P
defined by P(z) = x is a projection on B with Im P = M and Ker P = N.

Proof The only difficult part of this theorem is the proof that P is continuous.
While this may be proved using only what has been covered in this book
(including the appendices), it is quite involved since it requires proving both
Baire’s theorem and the open mapping theorem. Since these are essentially
purely topological results whose proofs are of no benefit to us at this point, we
choose to refer the interested reader to, e.g., the very readable treatment by
Simmons (1963). ˙

As mentioned in Section 7.8, if we are given a space V and subspace U,

there may be many subspaces W with the property that V = U • W. Thus, if
we are given a closed subspace M of a Banach space B, then there could be
many algebraic projections defined on B with image M, and in fact none of
them may be projections as defined above (i.e., they may not be continuous).
In other words, there may not exist any closed subspace N such that B = M •
N. However, Theorem 12.16 together with Theorem 12.18 shows that if we
have a Hilbert space H together with a closed subspace M, then there always
exists a projection P defined on H = M • MÊ with Im P = M and Ker P = MÊ.

Exercises

1. Let V and W be vector spaces, and let S ™ V be convex.

(a) Show that the intersection of any collection of convex sets is convex.
(b) Show that every subspace of V is convex.
 12.4 CLOSED SUBSPACES 655

(c) Show that any translate S + z = {x + z: z ∞ V is fixed and x ∞ S} is

convex.
(d) If ¬: V ‘ W is linear, show that ¬(S) is a convex subset of W, and if
T ™ W is convex, then ¬î(T) is convex in V.

2. Let H be a Hilbert space and S a nonempty subset of H. Show that SÊ =

⁄x´S xÊ is a closed subspace of H.

3. Let H be a Hilbert space and S a nonempty subset of H. Prove the

following:
(a) 0Ê = H and HÊ = 0.
(b) S ⁄ SÊ ™ {0}.
(c) S ™ SÊÊ.
(d) Sè ™ Sì implies SìÊ ™ SèÊ.

4. Show that a subset M of a Hilbert space H is closed if and only if MÊÊ =

M.

5. Prove Theorem 12.17.

12.5 HILBERT BASES

Let us now turn our attention to the infinite-dimensional analogue of the

expansion of a vector in terms of a basis. (We recommend that the reader first
review Sections 0.3 and 0.4 before continuing on with this material.) Suppose
that x is a vector in a Hilbert space H such that ˜x˜ ≠ 0, and let y ∞ H be arbi-
trary. We claim there exists a unique scalar c such that y - cx is orthogonal to
x. Indeed, if (y - cx) ‡ x, then

0 = Óx, y - cxÔ = Óx, yÔ - cÓx, xÔ

implies that
c = Óx, yÔ/Óx, xÔ

while if c = Óx, yÔ/Óx, xÔ, then reversing the argument shows that (y - cx) ‡ x.
The scalar c is usually called the Fourier coefficient of y with respect to (or
relative to) x.
To extend this idea to finite sets of vectors, let {xá} = {xè, . . . , xñ} be a
collection of vectors in H. Furthermore assume that the xá are mutually
orthogonal, i.e., Óxá, xéÔ = 0 if i ≠ j. If cá = Óxá, yÔ/Óxá, xáÔ is the Fourier coeffi-
cient of y ∞ H with respect to xá, then
656 HILBERT SPACES

n n
Óxi ,!y ! " c j x jÔ = Óxi ,!yÔ ! " c jÓxi ,!x jÔ
j=1 j=1
= Óxi ,!yÔ ! ciÓxi ,!xiÔ
=0

which shows that y - Íj ˆ=1 céxé is orthogonal to each of the xá. Geometrically,
this result says that if we subtract off the components of a vector y in the
direction of n orthogonal vectors xá, then the resulting vector is orthogonal to
each of the vectors xá.
We can easily simplify many of our calculations be requiring that our
finite set {xá} be orthonormal instead of just orthogonal. In other words, we
assume that Óxá, xéÔ = ∂áé, which is equivalent to requiring that i ≠ j implies that
xi ‡ xé and ˜xᘠ= 1. Note that given any xá ∞ H with ˜xá˜≠ 0, we can normalize
xá by forming the vector eá = xá/˜xá˜. It is then easy to see that the above cal-
culations remain unchanged except that now we simply have cá = Óxá, yÔ. We
will usually denote such an orthonormal set by {eá}, and hence we write

Óeá, eéÔ = ∂áé .

Suppose {eá} is a finite orthonormal set in a Hilbert space H and x is any

element of H. We claim that the expression
n
˜ x ! " ak ek ˜
k=1
achieves its minimum value in the case where each of the scalars aÉ is equal to
the Fourier coefficient cÉ = ÓeÉ, xÔ. To see this, we note that the above discus-
sion showed that x - Íkˆ=1 cÉeÉ is orthogonal to each eá for i = 1, . . . , n and
hence we may apply the Pythagorean theorem to obtain

n n n
˜ x ! " ak ek ˜ 2 = ˜ x ! " ck ek + " (ck ! ak )ek ˜ 2
k=1 k=1 k=1
n n
= ˜ x ! " ck ek ˜ 2 + ˜ " (ck ! ak )ek ˜ 2 !!.
k=1 k=1

It is clear that the right hand side of this equation takes its minimum value at
aÉ = cÉ for k = 1, . . . , n and hence we see that in general
n n
˜ x ! # ck ek ˜ " ˜ x ! # ak ek ˜
k=1 k=1
for any set of scalars aÉ. Moreover, we see that (using cÉ = ÓeÉ, xÔ)
 12.5 HILBERT BASES 657

n
0 ! ˜ x " # ck ek ˜ 2
k=1
n n
= Óx " # ck ek ,!x " # cr erÔ
k=1 r=1
n n n
= ˜ x˜ 2 " # ck *Óek ,!xÔ " # crÓx,!erÔ + # |ck | 2
k=1 r=1 k=1
n
= ˜ x˜ 2 " # |ck | 2
k=1

which implies
n n
! |ck | = ! |Óek ,!xÔ|2 "!˜ x˜ 2 !!.
2

k=1 k=1

This relationship is frequently called Bessel’s inequality, although this des-

ignation also applies to the infinite-dimensional version to be proved below.
We now seek to generalize these last two results to the case of arbitrary
(i.e., possibly uncountable) orthonormal sets. We begin with a simple
theorem.

Theorem 12.19 Let {eá}, i ∞ I (where I may be uncountable) be an arbitrary

orthonormal set in a Hilbert space H. Then if x is any vector in H, the set S =
{eá: Óeá, xÔ ≠ 0} is countable (but possibly empty).

Proof For each n ∞ Û+ define the set

Sñ = {eá: \Óeá, xÔ\2 > ˜x˜2/n} .

We claim that each Sñ can contain at most n - 1 vectors. To see this, suppose
Sñ contains N vectors, i.e., Sñ = {eè, . . . , eN}. Then from the definition of Sñ
we have
N
!|Óei ,!xÔ|2 > (˜ x˜ 2 /n)N
i=1
while Bessel’s inequality shows that
N
!|Óei ,!xÔ|2 " ˜ x˜ 2 !!.
i=1
Thus we must have N < n which is the same as requiring that N ¯ n - 1. The
theorem now follows if we note that each Sñ consists of a finite number of
vectors, and that S = ¡n~=1Sñ since Sñ ‘ S as n ‘ Ÿ. ˙
658 HILBERT SPACES

Theorem 12.19 now allows us to prove the general (i.e., infinite-

dimensional) version of Bessel’s inequality. Keep in mind that an arbitrary
orthonormal set may consist of an uncountable number of elements, and in
this case we do not write any limits in the sum Í\Óeá, xÔ\2.

Theorem 12.20 If {eá} is any orthonormal set in a Hilbert space H and x is

any vector in H, then Í\Óeá, xÔ\2 ¯ ˜x˜2.

Proof First note that if eå ∞ {eá} is such that Óeå, xÔ = 0, then this eå will not
contribute to Í\Óeá, xÔ\2. As in Theorem 12.19, we again consider the set

S = {eá: Óeá, xÔ ≠ 0} .

If S = Å, then we have Í\Óeá, xÔ\2 = 0 and the conclusion is obviously true. If S

≠ Å, then according to Theorem 12.19 it must contain a countable number of
vectors. If S is in fact finite, then we write S = {eè, . . . , eñ} and the theorem
follows from the finite-dimensional Bessel inequality proved above. Thus we
need only consider the case where S is countably infinite.
We may consider the vectors in S to be arranged in any arbitrary (but now
fixed) order {eè, eì, . . . , eñ, . . . }. From the corollary to Theorem B20 we
know that if Íi~=1\Óeá, xÔ\2 converges, then this sum is independent of any
rearrangement of the terms in the series. This then gives an unambiguous
meaning to the expression Í\Óeá, xÔ\2 = Íi~=1\Óeá, xÔ\2 . Therefore we see that the
sum is a nonnegative (extended) real number that depends only on the set S
and not on the order in which the vectors in S are written. If we let
n
sn = ! |Óei ,!xÔ|2
i=1
be the nth partial sum of the series, then the finite-dimensional version of
Bessel’s inequality shows that sñ ¯ ˜x˜2 for every n, and hence we must have
!
"|Óei ,!xÔ|2 # ˜ x˜ 2 !!.!!˙
i=1

Theorem 12.21 Let {eá} be an orthonormal set in a Hilbert space H, and let x
be any vector in H. Then (x - ÍÓeá, xÔeá) ‡ eé for each j.

Proof Just as we did in the proof of Theorem 12.20, we must first make
precise the meaning of the expression ÍÓeá, xÔeá. Therefore we again define the
set S = {eá: Óeá, xÔ ≠ 0}. If S = Å, then we have ÍÓeá, xÔeá = 0 so that our
theorem is obviously true since the definition of S then means that x ‡ eé for
every j. If S is finite but nonempty, then the theorem reduces to the finite case
 12.5 HILBERT BASES 659

proved in the discussion prior to Theorem 12.19. Thus, by Theorem 12.19, we

are again left with the case where S is countably infinite. We first prove the
result for a particular ordering S = {eè, eì, . . . }, and afterwards we will show
that our result is independent of the ordering chosen.
Let sñ = Íi ˆ=1Óeá, xÔeá. Since the Pythagorean theorem may be applied to
any finite collection of orthogonal vectors, the fact that {eá} is an orthonormal
set allows us to write (for m > n)

m m
˜sm ! sn ˜ 2 = ˜ " Óei ,!xÔei ˜ 2= " |Óei ,!xÔ|2 !!.
i=n+1 i=n+1

Now, Bessel’s inequality shows that Íi~=1\Óeá, xÔ\2 must converge, and hence
for any ´ > 0 there exists N such that m > n ˘ N implies Í i ˜= n+1 \Óeá, xÔ\2 < ´2
(this is just Theorem B17). This shows that {sñ} is a Cauchy sequence in H,
and thus the fact that H is complete implies that sñ ‘ s = Íi~=1Óeá, xÔeá ∞ H. If
we define ÍÓeá, xÔeá = Íi~=1Óeá, xÔeá = s, then the continuity of the inner product
yields

Óe j ,!x ! sÔ = Óe j ,!xÔ ! Óe j ,!sÔ = Óe j ,!xÔ ! Óe j ,!lim snÔ

n
= Óe j ,!xÔ ! limÓe j ,!snÔ = Óe j ,!xÔ ! lim " Óei ,!xÔÓe j ,!eiÔ
i=1
= Óej ,!xÔ ! Óe j ,!xÔ = 0!!.

Thus we have shown that (x - s) ‡ eé for every j.

We now show that this result is independent of the particular order chosen
for the {eá} in the definition of s. Our proof of this fact is similar to the proof
of the corollary to Theorem B20. Let {eæá} be any other arrangement of the set
{eá}, and let sæñ = Íi ˆ=1Óeæá, xÔ eæá. Repeating the above argument shows that sæñ
converges to a limit sæ = Íi~=1Óeæá, xÔeæá. We must show that sæ = s. Since {sñ},
{sæñ} and Íi~=1\Óeá, xÔ\2 all converge, we see that for any ´ > 0, there exists N >
0 such that n ˘ N implies

˜sñ - s˜ < ´

˜sæñ - sæ˜ < ´

and
Íi~=N+1 \Óeá, xÔ\2 < ´2

(this last inequality follows by letting m ‘ Ÿ in Theorem B17). We now note

that since there are only a finite number of terms in sN and {eæá} is just a
660 HILBERT SPACES

rearrangement of {eá}, there must exist an integer M > N such that every term
in sN also occurs in sæM . Then sæM - sN contains a finite number of terms,
each of which is of the form Óeá, xÔeá for i = N + 1, N + 2, . . . . We therefore
have
$
˜ s!M " SN ˜ 2 # % |Óei ,!xÔ|2 < & 2
i=N +1
and hence ˜sæM - sN˜ < ´. Putting all of this together, we have

˜sæ - s˜ ¯ ˜sæ - sæM˜ + ˜sæM - sN˜ + ˜sN - s˜ < 3´

and hence sæ = s. ˙

At last we are in a position to describe the infinite-dimensional analogue

of the expansion of a vector in terms of a basis. Let H be a nonzero Hilbert
space, and let {xá} be an arbitrary collection of vectors in H such that ˜xᘠ≠ 0
for each i. (We required H to be nonzero so that such a collection will exist.)
For each finite subcollection {xiè , . . . , xiñ} of {xá}, we can form the vector
space spanned by this subcollection of vectors. In other words, we can con-
sider the space consisting of all linear combinations of the form c1xiè + ~ ~ ~ +
cnxiñ where each cá is a complex number. In order to simplify our notation, we
will generally omit the double indices and write simply {xè , . . . , xñ}.
Now consider the union of all vector spaces generated by such finite sub-
collections of {xá}. This union is clearly a vector space itself, and is called the
subspace generated by the collection {xá}. Let us denote this space by E. We
say that the collection {xá} is total in H if E is dense in H (i.e., Cl E = H). In
other words, {xá} is total in H if every vector in H is the limit of a sequence of
vectors in E (see Theorem B14(b)). A total orthonormal set {eá} is called a
Hilbert basis (or an orthonormal basis). Be careful to note however, that
this is not the same as a basis in a finite-dimensional space. This is because
not every vector in H can be written as a linear combination of a finite number
of elements in a Hilbert basis.
An equivalent way of formulating this property that is frequently used is
the following. Consider the family of all orthonormal subsets of a nonzero
Hilbert space H. We can order this family by ordinary set inclusion, and the
result is clearly a partially (but not totally) ordered set. In other words, if Sè
and Sì are orthonormal sets, we say that Sè ¯ Sì if Sè ™ Sì. We say that an
orthonormal set {eá} is complete if it is maximal in this partially ordered set.
This means that there is no nonzero vector x ∞ H such that if we adjoin e =
x/˜x˜ to {eá}, the resulting set {eá, e} is also orthonormal and contains {eá} as a
proper subset. We now show the equivalence of this approach to the previous
paragraph.
 12.5 HILBERT BASES 661

Let {eá} be a complete orthonormal set in a Hilbert space H, and let E be

the subspace generated by {eá}. If Cl E ≠ H, then by Theorem 12.15 there
exists a nonzero vector x ∞ H such that x ‡ Cl E. In particular, this means that
x ‡ E and hence the set {eá, e = x/˜x˜} would be a larger orthonormal set than
{eá}, contradicting the maximality of {eá}.
Conversely, suppose that {eá} is a Hilbert basis for H (i.e., a total ortho-
normal set). If {eá} is not complete, then there exists a nonzero vector x ∞ H
such that {eá, e = x/˜x˜} is an orthonormal set that contains {eá} as a proper
subset. Then e ‡ {eá}, and hence the subspace E generated by {eá} must be a
subset of eÊ. Since eÊ is closed, it follows that Cl E ™ eÊ. But then e ‡ Cl E
which contradicts the assumption that Cl E = H.

Theorem 12.22 Every nonzero Hilbert space H contains a complete ortho-

normal set. Alternatively, every such H has a Hilbert basis.

Proof Note that every chain of orthonormal sets in H has an upper bound
given by the union of the sets in the chain. By Zorn’s lemma, the set of all
orthonormal sets thus has a maximal element. This shows that H contains a
complete orthonormal set. That H has an orthonormal basis then follows from
the above discussion on the equivalence of a complete orthonormal set and a
Hilbert basis. ˙

Some of the most important basic properties of Hilbert spaces are con-
tained in our next theorem.

Theorem 12.23 Let {eá} be an orthonormal set in a Hilbert space H. Then

the following conditions are equivalent:
(1) {eá} is complete.
(2) x ‡ {eá} implies x = 0.
(3) For any x ∞ H we have x = ÍÓeá, xÔeá.
(4) For any x ∞ H we have ˜x˜2 = Í\Óeá, xÔ\2.

Proof (1) ¶ (2): If (2) were not true, then there would exist a nonzero vector
e = x/˜x˜ ∞ H such that e ‡ {eá}, and hence {eá, e} would be an orthonormal
set larger than {eá}, contradicting the completeness of {eá}.
(2) ¶ (3): By Theorem 12.21, the vector y = x - ÍÓeá, xÔeá is orthogonal
to {eá}, and hence (2) implies that y = 0.
(3) ¶ (4): Using the joint continuity of the inner product (so that the sum
as a limit of partial sums can be taken outside the inner product), we simply
calculate
662 HILBERT SPACES

˜ x˜ 2 = Óx,!xÔ = ÓÍÓei ,!xÔei ,!ÍÓe j ,!xÔe jÔ

= ÍÓei ,!xÔ*ÍÓe j ,!xÔÓei ,!e jÔ
= ÍÓei ,!xÔ*Óei ,!xÔ
= Í|Óei ,!xÔ|2 !!.

(4) ¶ (1): If {eá} is not complete, then there exists e ∞ H such that {eá, e}
is a larger orthonormal set in H. Since this means that e ‡ {eá}, statement (4)
yields ˜e˜2 = Í\Óeá, eÔ\2 = 0 which contradicts the assumption that ˜e˜ = 1. ˙

Note that the equivalence of (1) and (3) in this theorem is really just our
earlier statement that an orthonormal set is complete if and only if it is a
Hilbert basis. We also remark that statement (4) is sometimes called
Parseval’s equation, although this designation also applies to the more gen-
eral result
Óx, yÔ = ÍÓx, eáÔÓeá, yÔ
(see Exercise 12.5.1).
It should be emphasized that we have so far considered the general case
where an arbitrary Hilbert space H has a possibly uncountable orthonormal
set. However, if H happens to be separable (i.e., H contains a countable dense
subset), then we can show that every orthonormal set in H is in fact countable.

Theorem 12.24 Every orthonormal set {eá} in a separable Hilbert space H

contains at most a countable number of elements.

Proof We first note that by the Pythagorean theorem we have

˜eá - eé˜2 = ˜eá˜2 + ˜eé˜2 = 2

and hence ˜eá - eé˜ = “2” for every i ≠ j. If we consider the set {B(eá, 1/2)} of
open balls of radius 1/2, then the fact that 2(1/2) = 1 < “2” implies that these
balls are pairwise disjoint. Now let {xñ} be a countable dense subset of H.
This means that any neighborhood of any element of H must contain at least
one of the xñ. In particular, each of the open balls B(ei, 1/2) must contain at
least one of the xñ, and hence there can be only a countable number of such
balls (since distinct balls are disjoint). Therefore the set {eá} must in fact be
countable. ˙

It is worth remarking that if we are given any countable set of linearly

independent vectors {xá} in a Hilbert space H, then the Gram-Schmidt proce-
dure (see the corollary to Theorem 2.21) may be applied to yield a countable
orthonormal set {eá} such that for any n, the space spanned by {eè , . . . , eñ} is
 12.5 HILBERT BASES 663

the same as the space spanned by {xè , . . . , xñ}. It then follows that {eá} is
complete if and only if {xá} is complete.
Finally, suppose that we have a countable (but not necessarily complete)
orthonormal set {eá} in a Hilbert space H. From Bessel’s inequality, it follows
that a necessary condition for a set of scalars cè, cì, . . . to be the Fourier coef-
ficients of some x ∞ H is that Ík~ = 1\cÉ\2 ¯ ˜x˜2. In other words, the series
Ík~=1\cÉ\2 must converge. That this is also a sufficient condition is the content
of our next result, which is a special case of the famous Riesz-Fischer
theorem.

Theorem 12.25 (Riesz-Fischer) Let {eá} be an orthonormal set in a Hilbert

space H, and let {cá} be a collection of scalars such that the series Ík~=1\cÉ\2
converges. Then there exists a vector x ∞ H with {cá} as its Fourier coeffi-
cients. In other words, Ík~=1\cÉ\2 = ˜x˜2 where cÉ = ÓeÉ, xÔ.

Proof For each n, define the vector

n
xn = ! ck ek
k=1
and note that cÉ = ÓeÉ, xñÔ for k ¯ n. Since Ík~=1\cÉ\2 converges, it follows from
Theorem B17 that for each ´ > 0, there exists N such that n > m ˘ N implies
n
! |ck |2 < " !!.
k=m+1
Using the Pythagorean theorem, we then see that n > m ˘ N implies

n n n
˜ xn ! x m ˜ 2 = ˜ " ck ek ˜ 2 = " ˜ck ek ˜ 2 = " |ck |2 < #
k=m+1 k=m+1 k=m+1

and hence {xñ} is a Cauchy sequence in H. Since H is complete, there exists a

vector x ∞ H such that ˜xñ - x˜ ‘ 0. In addition, we note that we may write

Óek ,!xÔ = Óek ,!xnÔ + Óek ,!x ! xnÔ (6)

where the first term on the right hand side is just cÉ. From the Cauchy-
Schwartz inequality we see that

\ÓeÉ, x - xñÔ\ ¯ ˜eɘ ˜x - xñ˜ = ˜x - xñ˜

and thus letting n ‘ Ÿ shows that ÓeÉ, x - xñÔ ‘ 0. Since the left hand side of
(6) is independent of n, we then see that
664 HILBERT SPACES

ÓeÉ, xÔ = cÉ .

Using this result, we now let n ‘ Ÿ to obtain (since ˜x - xñ˜ ‘ 0)

n n
2
˜ x ! xn ˜ = Óx ! " ck ek ,!x ! " ck ekÔ
k=1 k=1
n
= ˜ x˜ 2! " |ck | 2 # 0!!.
k=1

In other words, we have

n "
lim n!" # |ck | = # |ck | 2 = ˜ x˜ 2 !!.!!˙
2

k=1 k=1

Exercises

1. If {eá} is a complete orthonormal set in a Hilbert space H and x, y ∞ H,

prove that Óx, yÔ = ÍáÓx, eáÔÓeá, yÔ.

2. Let en denote the sequence with a 1 in the nth position and 0’s elsewhere.
Show that {e1, e2, . . . , en, . . . } is a complete orthonormal set in l2.

3. Prove that a Hilbert space H is separable if and only if every orthonormal

set in H is countable.

4. (a) Show that an orthonormal set in a Hilbert space is linearly indepen-

dent.
(b) Show that a Hilbert space is finite-dimensional if and only if every
complete orthonormal set is a basis.

5. Let S be a nonempty set, and let l2(S) denote the set of all complex-valued
functions f defined on S with the property that:
(i) {s ∞ S: f(s) ≠ 0} is countable (but possibly empty).
(ii) Í\f(s)\2 < Ÿ.
It should be clear that l2(S) forms a complex vector space with respect to
pointwise addition and scalar multiplication.
 12.5 HILBERT BASES 665

(a) Show that l2(S) becomes a Hilbert space if we define the norm and
inner product by ˜f˜ = (Í\f(s)\2)1/2 and Óf, gÔ = Íf(s)*g(s).
(b) Show that the subset of l2(S) consisting of functions that have the
value 1 at a single point and 0 elsewhere forms a complete orthonormal
set.
(c) Now let S = {ei} be a complete orthonormal set in a Hilbert space H.
Each x ∞ H defines a function f on S by f(ei) = Óei, xÔ. Show that f is in
l2(S).
(d) Show that the mapping x ’ f is an isometric (i.e., norm preserving)
isomorphism of H onto l2(S).

12.6 BOUNDED OPERATORS ON A HILBERT SPACE

One of the most important concepts in quantum theory is that of self-adjoint

operators on a Hilbert space. We now begin a discussion on the existence of
operator adjoints. While the existence of the adjoint in a finite-dimensional
space was easy enough to prove, the infinite-dimensional analogue requires
slightly more care. Therefore, our first goal is to prove one version of a
famous result known as the Riesz representation theorem, which is the Hilbert
space analogue of Theorem 10.1.
As usual, we let E* denote the dual space (which is also frequently called
the conjugate space) to the Banach space E. In other words, E* is just the
space L(E, ç) of continuous linear maps of E into ç. Elements of E* are
called functionals, and it is important to remember that this designation
implies that the map is continuous (and hence bounded). If f ∞ E*, we may
define the norm of f as usual by

˜f˜ = sup{\f(x)\: ˜x˜ = 1} .

Since ç is clearly a Banach space, it follows from Theorem 12.13 that E* is

also a Banach space (even if E is not).
If y is any (fixed) vector in a Hilbert space H, then we define the function
fy: H ‘ ç by fy: x ’ Óy, xÔ. Since the inner product is continuous, it follows
that fy is continuous. Furthermore, we note that for any xè, xì ∞ H and å ∞ ç
we have

fy(xè + xì) = Óy, xè + xìÔ = Óy, xèÔ + Óy, xìÔ = fy(xè) + fy(xì)

and
fy(åxè) = Óy, åxèÔ = åÓy, xèÔ = åfy(xè)
666 HILBERT SPACES

and hence fy is linear. This shows that fy ∞ H* = L(H, ç), and therefore we
may define
˜fy˜ = sup{\fy(x)\: ˜x˜ = 1} .

Using Example 12.1, we see that \Óy, xÔ\ ¯ ˜y˜ ˜x˜, and thus (by definition
of sup)
˜fó˜ = sup{\Óy, xÔ\: ˜x˜ = 1} ¯ ˜y˜ .

On the other hand, we see that y = 0 implies ˜fy˜ = 0 = ˜y˜, while if y ≠ 0 then
(again by the definition of sup)

˜fy˜ = sup{\fy(x)\: ˜x˜ = 1} ˘ \fy(y/˜y˜)\ = \Óy, y/˜y˜Ô\ = ˜y˜ .

We thus see that in fact ˜fy˜ = ˜y˜, and hence the map y ’ fy preserves the
norm.
However, the mapping y ’ fy is not linear. While it is true that

fyè+ yì(x) = Óyè + yì, xÔ = (fyè + fyì)(x)

and hence fyè+ yì = fyè + fyì , we also have

fåy(x) = Óåy, xÔ = å*Óy, xÔ = å*fy(x)

so that fåy = å*fy. This shows that the map y ’ fy is really a norm preserving
antilinear mapping of H into H*. We also note that

˜fyè - fyì˜ = ˜fyè- yì˜ = ˜yè - yì˜

which shows that the map y ’ fy is an isometry.

What we have shown so far is that given any y ∞ H, there exists a linear
functional fy ∞ H* where the association y ’ fy = Óy, Ô is a norm preserving
antilinear map. It is of great importance that this mapping is actually an iso-
morphism of H onto H*. In other words, any element of H* may be written in
the form fy = Óy, Ô for a unique y ∞ H. We now prove this result, which is a
somewhat restricted form of the Riesz representation theorem.

Theorem 12.26 (Riesz Representation Theorem) Let H be a Hilbert space.

Then given any f ∞ H*, there exists a unique y ∞ H such that

f (x) = Óy,!xÔ (7)

for every x ∞ H. Moreover we have ˜y˜ = ˜f˜.

 12.6 BOUNDED OPERATORS ON A HILBERT SPACE 667

Proof Assuming the existence of such a y ∞ H, it is easy to show that it must

be unique. To see this, we simply note that if f(x) = Óyè, xÔ and f(x) = Óyì, xÔ
for every x ∞ H, then 0 = Óyè, xÔ - Óyì, xÔ implies 0 = Óyè - yì, xÔ. But this
holds for all x ∞ H, and hence we must have yè - yì = 0 which implies yè = yì.
(If Óy, xÔ = 0 for all x ∞ H, then in particular, 0 = Óy, yÔ = ˜y˜2 implies y = 0.)
We now prove that such a y does indeed exist. First note that if f = 0, then
we may choose y = 0 to satisfy the theorem. Therefore we now assume that
f ≠ 0. Let M = Ker f. We know that Ker f is a subspace of H, and since f ≠ 0
we must have M ≠ H. Furthermore, if {xñ} is a sequence in M that converges
to some x ∞ H, then the continuity of f shows that

f(x) = f(lim xñ) = lim f(xñ) = 0

and hence x ∞ M. This shows that M is a proper closed subspace of H (by

Theorem B14(a)). By Theorem 12.15, there exists a nonzero yà ∞ H such that
yà ‡ M. We claim that y = åyà will satisfy our requirements for a suitably
chosen scalar å.
First note that for any scalar å and any x ∞ M, we have f(x) = 0 on the one
hand, while Óåyà, xÔ = å*Óyà, xÔ = 0 on the other (since yà ‡ x). This shows
that (7) is true for every x ∞ M no matter how we choose å. However, if we
now require that (7) hold for the vector x = yà (where y0 ! M by definition),
then we must also have

f(yà) = Óåyà, yàÔ = å*˜yà˜2

which leads us to choose å = f(yà)*/ ˜yà˜2. With this choice of å, we have then
shown that (7) holds for all x ∞ M and for the vector x = yà. We now show
that in fact (7) holds for every x ∞ H.
We observe that any x ∞ H may be written as

x = x - [f(x)/f(yà)]yà + [f(x)/f(yà)]yà

where x - [f(x)/f(yà)]yà ∞ M. In other words, any x ∞ H may be written in the

form x = m + ∫yà where m ∞ M and ∫ = f(x)/f(yà). Since f is linear, we now
see that our previously shown special cases result in (setting y = åyà)

f (x) = f (m + ! y0 ) = f (m) + ! f (y0 ) = Óy,!mÔ + !Óy,!y0Ô

= Óy,!m + ! y0Ô = Óy,!xÔ

and hence f(x) = Óy, xÔ for every x ∞ H.

668 HILBERT SPACES

Finally, the fact that ˜y˜ = ˜f˜ was shown in the discussion prior to the
theorem. ˙

If H is a Hilbert space, we define an inner product on H* by

Ófy , fxÔ = Óx, yÔ .

Note the order of the vectors x and y in this definition. This is to ensure that
the inner product on H* has the correct linearity properties. In other words,
using the fact that the mapping y ’ fy is antilinear, we have

Óåfó, fíÔ = Ófå*y, fíÔ = Óx, å*yÔ = å*Óx, yÔ = å*Ófó, fíÔ

and
Ófó, åfíÔ = Ófó, få*xÔ = Óå*x, yÔ = åÓx, yÔ = åÓfó, fíÔ .

Using fyè + fyì = fyè+ yì it is trivial to verify that

Ófyè + fyì, fxÔ = Ófyè, fxÔ + Ófyì, fxÔ .

This inner product induces a norm on H* in the usual way.

We claim that H* is also a Hilbert space. To see this, let {fxá} be a Cauchy
sequence in H*. Given ´ > 0, there exists N such that m > n ˘ N implies that
˜fxm ! fxn ˜ < " . Then, since every fxñ corresponds to a unique xñ (the kernel of
the mapping x ‘ fí is {0}), it should be obvious that {xñ} will be a Cauchy
sequence in H. However, we can also show this directly as follows:

˜ f xm ! f xn ˜ 2 = Ó f xm ! f xn ,! f xm ! f xn Ô
= Ó f xm ,! f xm Ô ! Ó f xm ,! f xn Ô ! Ó f xn ,! f xm Ô + Ó f xn ,! f xn Ô
= Óxm ,!xmÔ - Óxm ,!xnÔ - Óxn ,!xmÔ + Óxn ,!xnÔ
= Óxm ! xn ,!xmÔ - Óxm ! xn ,!xnÔ
= Óxm ! xn ,!xm ! xnÔ
= ˜ xm ! xn ˜ 2 !!.

This shows that {xñ} is a Cauchy sequence in H, and hence xñ ‘ x ∞ H. But

then fxñ ‘ fx ∞ H* which shows that H* is complete, and hence H* is also a
Hilbert space.
 12.6 BOUNDED OPERATORS ON A HILBERT SPACE 669

Example 12.10 Recalling the Banach space lpˆ defined in Example 12.7, we
shall show that if 1 < p < Ÿ and 1/p + 1/q = 1, then (lpˆ)* = lqˆ. By this equality
sign, we mean there exists a norm preserving isomorphism of lqˆ onto (lpˆ)*. If
{eá} is the standard basis for ®n, then any x = (xè , . . . , xñ) ∞ lpˆ may be
written in the form
n
x = ! xi ei !!.
i=1
Now let f be a linear mapping of lpˆ into any normed space (although we shall
be interested only in the normed space ç). By Corollary 2 of Theorem 12.12,
we know that f is (uniformly) continuous. Alternatively, we can show this
directly as follows. The linearity of f shows that
n
f (x) = ! xi f (ei )
i=1
and hence
n n n
˜ f (x)˜ = ˜" xi f (ei )˜ ! "| xi | ˜ f (ei )˜ ! max{˜ f (ei )˜}"| xi |!!.
i=1 i=1 i=1
But
n
| xi | ! " | xi | p = (˜ x˜ p ) p
p

i=1
and therefore \ xá\ ¯ ˜x˜p. If we write K = max{˜f(eá)˜}, then this leaves us with

˜f(x) ˜ ¯ nK ˜x˜p

which shows that f is bounded and hence continuous (Theorem 12.12).

We now restrict ourselves to the particular case that f: lpˆ ‘ ç, and we
then see that the set of all such f’s is just the dual space (lpˆ)*. Since f is
bounded, we can define the norm of f in the usual way by

˜f˜ = inf{K > 0: \f(x)\ ¯ K˜x˜p for all x ∞ lpˆ} .

Now note that for each i = 1, . . . , n the result of f applied to eá is just some
scalar yá = f(eá) ∞ ç. Since f(x) = Í iˆ=1xáf(eá) = Í iˆ=1xáyá, we see that speci-
fying each of the yá’s will determine f, and conversely, f determines each of
the yá’s. We therefore have an isomorphism y = (yè, . . . , yñ) ‘ f of the space
of all n-tuples y = (yè, . . . , yñ) ∞ ç of scalars onto the space (lpˆ)* of all linear
functionals f on lpˆ defined by f(x) = Íi ˆ=1xáyá. Because of this isomorphism,
we want to know what norm to define on the set of all such y’s so that the
mapping y ‘ f is an isometry.
For any x ∞ lpˆ, HoÜlder’s inequality (see Example 12.7) yields
670 HILBERT SPACES

n n
| f (x)| = ! xi yi " !| xi yi |
i=1 i=1

#n &1/ p # n &1/q
p q
" %%! | xi | (( %%! | yi | ((
$ i=1 ' $ i=1 '
=˜ x˜ p ˜ y˜q !!.

By definition, this implies that ˜f˜ ¯ ˜y˜q (since ˜f˜ is just the greatest lower
bound of the set of all bounds for f). We will show that in fact ˜f˜ = ˜y˜q.
Consider the vector x = (xè, . . . , xñ) defined by xá = 0 if yá = 0, and xá =
\yá\q/yá if yá ≠ 0. Then using the fact that 1/p = 1 - 1/q we find

" n | y | pq %1/ p " n %1/q

q
˜ x˜ p ˜ y˜q = $$! i p '' $$! | yi | ''
# i=1 | yi | & # i=1 &
"n %1/ p " n %1/q
q q
= $$! | yi | '' $$! | yi | ''
# i=1 & # i=1 &
"n %1(1/q " n %1/q
q q
= $$! | yi | '' $$! | yi | ''
# i=1 & # i=1 &
n
= !| yi |q !!.
i=1

On the other hand, we also see that for this x we have

n n n
q
| f (x)| = ! xi yi = !| yi | = !| yi |q !!.
i=1 i=1 i=1

Thus, for this particular x, we find that \f(x)\ = ˜x˜p ˜y˜q, and hence in general
we must have ˜f˜ = ˜y˜q (since it should now be obvious that nothing smaller
than K = ˜y˜q can satisfy \f(x)\ ¯ K ˜x˜p for all x).
In summary, defining a norm on the space of all n-tuples y = (yè, . . . , yñ)
by ˜y˜q, we have constructed a norm preserving isomorphism of lqˆ onto (lpˆ)*
as desired.
 12.6 BOUNDED OPERATORS ON A HILBERT SPACE 671

In the particular case of p = q = 2, we know that l2ˆ is a Hilbert space, and

hence (l2ˆ)* = l2ˆ . Note also that in general,

(lpˆ)** = (lqˆ)* = lpˆ .

Any normed linear space E for which E** = E is said to be reflexive. Thus we
have shown that lpˆ is a reflexive Banach space, and hence l2ˆ is a reflexive
Hilbert space. In fact, it is not difficult to use the Riesz representation theorem
to show that any Hilbert space is reflexive (see Exercise 12.6.1). ∆

Example 12.11 Recall the space lŸ defined in Exercise 12.3.5, and let c0
denote the subspace consisting of all convergent sequences with limit 0. In
other words, x = {x1, x2, . . . , xn, . . . } has the property that xn ‘ 0 as n ‘ Ÿ.
We shall show that c0** = l1* = lŸ, and hence c0 is not reflexive.
Let us first show that any bounded linear functional f on c0 is expressible
in the form
!
f (x) = " fi xi
i=1
where
!
" | fi | < !!!.
i=1

To see this, let ei = {0, 0, . . . , 1, 0, . . . } be the sequence with a 1 in the ith

position and 0’s elsewhere. Now let f(x) be any bounded linear functional on
c0, and define fi = f(ei). Note that if

x = {x1,!x2 ,!…!,!xn ,!0,!…!} (*)

then
x = x1e1 + x2e2 + ~ ~ ~ + xnen

and
n
f (x) = ! fi xi !!.
i=1

Observe that if Íi~= 1\fi\ = Ÿ, then for every real B it would be possible to find
an integer N such that
N
! | fi | > B!!.
i=1
672 HILBERT SPACES

But then consider an element x defined by

# !!1 if i ! N and fi > 0

%
%"1 if i ! N and fi < 0
xi = $ !!.
% !!0 if i ! N and fi = 0
%& !!0 if i > N

Clearly ˜x˜ = sup \xi\ = 1, and

N N
| f (x)| = ! fi xi = ! | fi | > B = B˜ x˜
i=1 i=1

which contradicts the assumed boundedness of f. Therefore we must have

!
" | fi | < !
i=1
It is not hard to see that the set of all elements of the form (*) is dense in
c0. Indeed, suppose we are given any z = {z1, z2, . . . , zn, . . . } ∞ c0. Then
given ´ > 0, we must find an x of the above form with the property that

˜z - x˜ = supi \zi - xi\ < ´ .

Since any sequence x ∞ c0 has the property that xn ‘ 0 as n ‘ Ÿ, it follows

that given ´ > 0, there exists M such that \xn\ < ´ for all n ˘ M. If we choose
x = {z1, z2, . . . , zM, 0, . . . }, then clearly we will have ˜z - x˜ < ´.
By Corollary 1 of Theorem 12.12 any bounded linear functional is con-
tinuous. Together with Theorem 12.7(d), this shows that any bounded linear
functional on c0 is uniquely defined by its values on the dense set of elements
of the form (*), and hence for every x ∞ c0 we must have
!
f (x) = " fi xi !!.
i=1
We now claim that the norm of any such linear functional is given by
!
˜ f ˜ = " | fi |!!.
i=1
First note that
" "
| f (x)| ! # | fi | !| xi | ! ˜ x˜# | fi | = ˜ x˜a
i=1 i=1
where
 12.6 BOUNDED OPERATORS ON A HILBERT SPACE 673

!
a = " | fi |!!.
i=1
Then
"
| f (x)|
˜ x˜
! # | fi | = a
i=1

and hence
" | f (x)| %
˜ f ˜ = sup # :!x ! 0 & ( a!!.
$ ˜ x˜ '

On the other hand, it follows from Theorem B17 (see Appendix B) that given
´ > 0, there exists N such that
N
a ! " !!<!!# | fi |!!.
i=1
If we define x again by

# !!1 if i ! N and fi > 0

%
%"1 if i ! N and fi < 0
xi = $ !!.
% !!0 if i ! N and fi = 0
%& !!0 if i > N
then ˜x˜ = 1 and
! N
f (x) = " fi xi = " | fi |
i=1 i=1

so that \f(x)\ > a - ´. Therefore \f(x)\/˜x˜ ˘ a since ´ > 0 was arbitrary. But then
˜f˜ ˘ a, and hence we must have ˜f˜ = a as claimed.
In summary, we have shown that c0* = l1. In Exercise 12.6.2 the reader is
asked to show that l1* = lŸ, and hence this shows that c0** = lŸ. ∆

We now proceed to define the adjoint of an operator exactly as we did in

Section 10.1. Therefore, let H be a Hilbert space and let T ∞ L(H, H) be a
linear operator on H. If y ∞ H is an arbitrary but fixed vector, then the
mapping x ’ Óy, TxÔ is just a linear functional on H. We can thus apply the
Riesz representation theorem to conclude that there exists a unique vector z ∞
H such that Óy, TxÔ = Óz, xÔ. We now define the adjoint T¿ of T by T¿y = z.
Since y was arbitrary, we see that the definition of T¿ may be stated as

ÓT¿y, xÔ = Óy, TxÔ

for all x, y ∞ H.
674 HILBERT SPACES

To prove T¿ is unique, suppose that Tæ ∞ L(H, H) is defined by ÓTæy, xÔ =

Óy, TxÔ. Then ÓTæy, xÔ = ÓT¿y, xÔ and hence ÓTæy - T¿y, xÔ = 0. But this must
hold for all x ∞ H, and therefore Tæy - T¿y = 0, i.e., Tæy = T¿y. Since this is
true for all y, it then follows that Tæ = T¿.
Let us show that T¿ as we have defined it is really an element of L(H, H).
In other words, we must show that T¿ is both linear and continuous. But this is
easy since for any x, y, z ∞ H and å ∞ ç we have

ÓT † (x + y),!zÔ = Óx + y,!TzÔ = Óx,!TzÔ + Óy,!TzÔ = ÓT † x,!zÔ + ÓT † y,!zÔ

= ÓT † x + T † y,!zÔ
and

ÓT¿(åx), yÔ = Óåx, TyÔ = å*Óx, TyÔ = å*ÓT¿x, yÔ = Ó(åT¿)x, yÔ .

Therefore
T¿(x + y) = T¿x + T¿y

and
T¿(åx) = (åT¿)x .

To prove the continuity of T¿, we first show that it is bounded. Using the
Cauchy-Schwartz inequality we have

˜T¿x˜2 = ÓT¿x, T¿xÔ = Óx, TT¿xÔ ¯ ˜x˜ ˜TT¿x˜ ¯ ˜x˜ ˜T˜ ˜T¿x˜

which shows that ˜T¿x˜ ¯ ˜T˜ ˜x˜ for all x ∞ H. Since ˜T˜ < Ÿ, this shows that
T¿ is continuous (Theorem 12.12). We can therefore define the norm of T¿ in
the usual manner to obtain

˜T¿˜ = sup{˜T¿x˜: ˜x˜ = 1} ¯ ˜T˜ .

In fact, we will show in the next theorem that ˜T¿˜ = ˜T˜.

Theorem 12.27 Suppose S and T are operators on a Hilbert space H. Then

there exists a unique linear operator T¿ on H defined by ÓT¿y, xÔ = Óy, TxÔ for
all x, y ∞ H. Moreover, this operator has the following properties:
(a) (S + T)¿ = S¿ + T¿.
(b) (åT)¿ = å*T¿.
(c) (ST)¿ = T ¿S¿.
(d) T¿¿ = (T¿)¿ = T.
(e) ˜T¿ ˜ = ˜T˜.
 12.6 BOUNDED OPERATORS ON A HILBERT SPACE 675

(f) ˜T¿T˜ = ˜T˜2.

(g) 0¿ = 0 and 1¿ = 1.
(h) If T is invertible, then (T ¿)î = (Tî)¿.

Proof The existence and uniqueness of T¿ was shown in the above discus-
sion. Properties (a) - (d) and (g) - (h) follow exactly as in Theorem 10.3. As
to property (e), we just showed that ˜T¿˜ ¯ ˜T˜, and hence together with
property (d), this also shows that ˜T˜ = ˜ (T¿)¿˜ ¯ ˜T¿˜. To prove (f), we first
note that the basic properties of the norm along with property (e) show that

˜T¿T˜ ¯ ˜T¿˜ ˜T˜ = ˜T˜2 .

To show that ˜T˜2 ¯ ˜T¿T˜, we observe that the Cauchy-Schwartz inequality

yields
˜Tx˜2 = ÓTx, TxÔ = ÓT¿Tx, xÔ ¯ ˜T¿Tx˜ ˜x˜ ¯ ˜T¿T˜ ˜x˜2

which (by definition of ˜T˜) implies ˜T˜ ¯ ˜T¿T˜1/2 . ˙

While we have defined the adjoint in the most direct manner possible, we
should point out that there is a more general approach that is similar to our
discussion of the transpose mapping defined in Theorem 9.7. This alternative
method shows that a linear operator T defined on a Banach space E leads to a
“conjugate” operator T* defined on the dual space E*. Furthermore, the map-
ping T ‘ T* is a norm preserving isomorphism of L(E, E) into L(E*, E*).
However, in the case of a Hilbert space, Theorem 12.26 gives us an isomor-
phism between H and H*, and hence we can consider T* to be an operator on
H itself, and we therefore define T¿ = T*. For the details of this approach, the
reader is referred to, e.g., the very readable treatment by Simmons (1963).

Exercises

1. Let H be a Hilbert space. We define a mapping H ‘ H** by x ’ Fx

where Fx ∞ H** is defined by Fx(f) = f(x). We can also consider the com-
posite mapping H ‘ H* ‘ H** defined by x ’ fx ’ Ffí where fx(y) =
Óy, xÔ and Ffí(f) = Óf, fxÔ.
(a) Show that H** is a Hilbert space with the inner product ÓFf, FgÔ =
Óg, f Ô.
(b) By considering the two mappings defined above, show that H is
reflexive.
(c) Show ÓFx, FyÔ = Óx, yÔ.
676 HILBERT SPACES

2. (a) Show (l1 ˆ)* = lŸˆ and (lŸˆ)* = l1 .

(b) Show l1* = lŸ and lŸ* = l1. [Hint: Refer to Example 12.11.]

3. Let V be infinite-dimensional with orthonormal basis {ei}. Define T ∞

L(V) by Tei = ei-1. Show T¿ei = ei+1.

12.7 HERMITIAN, NORMAL AND UNITARY OPERATORS

Let us denote the space L(H, H) of continuous linear maps of H into itself by
L(H). In other words, L(H) consists of all operators on H. As any physics
student knows, the important operators A ∞ L(H) are those for which A¿ = A.
These operators are called self-adjoint or Hermitian. In fact, we now show
that the set of all Hermitian operators on H is a closed subspace of L(H).

Theorem 12.28 The set of all Hermitian operators on a Hilbert space H

forms a real closed subspace of L(H). Moreover, this subspace is a real
Banach space containing the identity transformation on H.

Proof We showed in Theorem 12.27 that 0 and 1 are Hermitian. If A, B ∞

L(H) are Hermitian operators and å, ∫ ∞ ®, then

(åA + ∫B)¿ = (åA)¿ + (∫B)¿ = åA¿ + ∫B¿ = åA + ∫B

so that åA + ∫B is also Hermitian, and hence the set of Hermitian operators

forms a real subspace of L(H). If {Añ} is a sequence of Hermitian operators
with the property that Añ ‘ A ∞ L(H), then (using Añ¿ = Añ and Theorem
12.27(e))
˜ A ! A† ˜ " ˜ A ! An ˜ + ˜(An ! A)† ˜
= ˜ A ! An ˜ + ˜ An ! A˜
= 2˜ An ! A˜!! !!.

Since this shows that ˜A - A¿˜ ‘ 0 as n ‘ Ÿ, we see that A = A¿ and hence

A is also Hermitian. Therefore the subspace of all Hermitian operators on H is
closed (Theorem B14(a)).
Finally, since L(H) is a Banach space (Theorem 12.13), the fact that the
closed subspace of Hermitian operators forms a real Banach space follows
from Theorem 12.9. ˙
 12.7 HERMITIAN, NORMAL AND UNITARY OPERATORS 677

It should be clear by now that most of the basic properties of Hermitian

operators on an infinite-dimensional Hilbert space are exactly the same as in
the finite-dimensional case discussed in Chapter 10. In particular, the proofs
of Theorems 10.4, 10.9(a) and 10.11(a) all carry over verbatim to the infinite-
dimensional case.
Recall from Section 10.3 that an operator N on H is said to be normal if N
and N¿ commute, i.e., if N¿N = NN¿. It should be obvious that any Hermitian
operator is necessarily normal, and that åN is normal for any scalar å.
However, even if Nè and Nì are normal, it is not generally true that either
Nè + Nì or NèNì are normal, and hence the subset of L(H) consisting of
normal operators is not a subspace. We do however have the following two
results.

Theorem 12.29 The set of all normal operators on H is a closed subset of

L(H) that is also closed under scalar multiplication.

Proof All that remains to be shown is that if {NÉ} is a sequence of normal

operators that converges to an operator N ∞ L(H), then N is normal. Since
NÉ ‘ N, it follows from Theorem 12.27(a) and (e) that NÉ¿ ‘ N¿. We thus
have (using the fact that each NÉ is normal)

˜ NN † ! N † N˜ " ˜ NN † ! N k N k † ˜ + ˜ N k N k † ! N k † N k ˜ + ˜ N k † N k ! N † N˜
= ˜ NN † ! N k N k † ˜ + ˜ N k † N k ! N † N˜ # 0

which shows that NN¿ = N¿N, and hence N is normal. ˙

Theorem 12.30 Let Nè and Nì be normal operators with the property that
one of them commutes with the adjoint of the other. Then Nè + Nì and NèNì
are normal.

Proof Suppose that NèNì¿ = Nì¿Nè. Taking the adjoint of both sides of this
equation then shows that NìNè¿ = Nè¿Nì. In other words, the hypothesis of the
theorem is equivalent to the statement that both operators commute with the
adjoint of the other. The rest of the proof is left to the reader (see Exercise
12.7.1). ˙

Probably the most important other type of operator that is often defined on
a Hilbert space is the unitary operator. We recall that unitary and isometric
operators were defined in Section 10.2, and we suggest that the reader again
go through that discussion. Here we will repeat the essential content of that
earlier treatment in a concise manner.
678 HILBERT SPACES

We say that an operator Ø ∞ L (H) is isometric if ˜Øx˜ = ˜x˜ for every x ∞

H. Note that we do not require that Ø map H onto H, and hence Øî need not
exist (at least if we assume that Øî must be defined on all of H). The defini-
tion of an isometric operator shows that ˜Øx˜ = 0 if and only if x = 0, and
hence an isometric operator Ø is a one-to-one mapping of H into H (since
Øx = Øy implies ˜Ø(x - y)˜ = 0 which then implies x = y).

Theorem 12.31 If Ø ∞ L(H), then the following conditions are equivalent:

(a) Ø¿Ø = 1.
(b) ÓØx, ØyÔ = Óx, yÔ for all x, y ∞ H.
(c) ˜Øx˜ = ˜x˜.

Proof Let x, y ∞ H be arbitrary.

(a) ¶ (b): ÓØx, ØyÔ = Óx, Ø¿ØyÔ = Óx, 1yÔ = Óx, yÔ.
(b) ¶ (c): ˜Øx˜2 = ÓØx, ØxÔ = Óx, xÔ = ˜x˜2.
(c) ¶ (a): ˜Øx˜2 = ˜x˜2 implies ÓØx, ØxÔ = Óx, xÔ or Óx, Ø¿ØxÔ = Óx, xÔ, and
hence Óx, (Ø¿Ø - 1)xÔ = 0. It now follows from Theorem 10.4(b) that Ø¿Ø -
1 = 0, and hence Ø¿Ø = 1. ˙

Isometric operators are sometimes defined by the relationship Ø¿Ø = 1,

and we saw in Section 10.2 that in a finite-dimensional space this implies that
ØØ¿ = 1 also. However, in an infinite-dimensional space, the property ØØ¿ =
1 must be imposed as an additional condition on Ø in one way or another. A
unitary operator U ∞ L(H) is an operator that satisfies U¿U = UU¿ = 1. Since
inverses are unique (if they exist), this implies that an equivalent definition of
unitary operators is that they map H onto itself and satisfy U¿ = Uî.
Alternatively, our next theorem shows that we can define a unitary operator as
an isometric operator that maps H onto H.

Theorem 12.32 An operator U ∞ L(H) is unitary if and only if it is a one-to-

one isometry of H onto itself. In other words, U ∞ L(H) is unitary if and only
if it is a bijective isometry.

Proof If U is unitary then it maps H onto itself, and since U¿U = 1, we see
from Theorem 12.31 that ˜Ux˜ = ˜x˜. Therefore U is an isometric isomorphism
of H onto H.
Conversely, if U is an isomorphism of H onto H then Uî exists, and the
fact that U is isometric shows that U¿U = 1 (Theorem 12.31). Multiplying
from the right by Uî shows that U¿ = Uî, and hence U¿U = UU¿ = 1 so that
U is unitary. ˙
 12.7 HERMITIAN, NORMAL AND UNITARY OPERATORS 679

One reassuring fact about unitary operators in a Hilbert space is that they
also obey the analogue of Theorem 10.6. In other words, an operator U on a
Hilbert space H is unitary if and only if {Ueá} is a complete orthonormal set
whenever {eá} is (see Exercise 12.7.2 for a proof).
There is no all-encompassing treatment of eigenvalues (i.e., like Theorems
10.21 or 10.26) for Hermitian or unitary operators in an infinite-dimensional
space even close to that for finite-dimensional spaces. Unfortunately, most of
the general results that are known are considerably more difficult to treat in
the infinite-dimensional case. In fact, a proper treatment involves a detailed
discussion of many subjects which the ambitious reader will have to study on
his or her own.

Exercises

1. Finish the proof of Theorem 12.30.

2. Prove that U ∞ L(H) is unitary if and only if {Ueá} is a complete orthonor-

mal set if {eá} is.
 APPENDIX A

Metric Spaces

For those readers not already familiar with the elementary properties of metric
spaces and the notion of compactness, this appendix presents a sufficiently
detailed treatment for a reasonable understanding of this subject matter.
However, for those who have already had some exposure to elementary point
set topology (or even a solid introduction to real analysis), then the material in
this appendix should serve as a useful review of some basic concepts. Besides,
any mathematics or physics student should become thoroughly familiar with
all of this material.
Let S be any set. Then a function d: S ª S ‘ ® is said to be a metric on S
if it has the following properties for all x, y, z ∞ S:

(M1) d(x, y) ˘ 0;
(M2) d(x, y) = 0 if and only if x = y;
(M3) d(x, y) = d(y, x);
(M4) d(x, y) + d(y, z) ˘ d(x, z).

The real number d(x, y) is called the distance between x and y, and the set S
together with a metric d is called a metric space (S, d).
As a simple example, let S = ® and let d(x, y) = \x - y\ for all x, y ∞ ®.
From the properties of the absolute value, conditions (M1) - (M3) should be
obvious, and (M4) follows by simply noting that

680
 METRIC SPACES 681

\x - z\ = \x - y + y - z\ ¯ \x - y\ + \y - z\ .

For our purposes, we point out that given any normed vector space (V, ˜ ˜)
we may treat V as a metric space by defining

d(x, y) = ˜x - y˜

for every x, y ∞ V. Using Theorem 2.17, the reader should have no trouble
showing that this does indeed define a metric space (V, d). In fact, it is easy to
see that ®n forms a metric space relative to the standard inner product and its
associated norm.
Given a metric space (X, d) and any real number r > 0, the open ball of
radius r and center x0 is the set Bd(xà, r) ™ X defined by

Bd(xà, r) = {x ∞ X: d(x, xà) < r} .

Since the metric d is usually understood, we will generally leave off the sub-
script d and simply write B(xà, r). Such a set is frequently referred to as an r-
ball. We say that a subset U of X is open if, given any point x ∞ U, there
exists r > 0 and an open ball B(x, r) such that B(x, r) ™ U.
Probably the most common example of an open set is the open unit disk
Dè in ®2 defined by
Dè = {(x, y) ∞ ®2: x2 + y2 < 1} .

We see that given any point xà ∞ Dè, we can find an open ball B(xà, r) ™ Dè
by choosing r = 1 - d(xà, 0). The set

D2 = {(x, y) ∞ ®2: x2 + y2 ¯ 1}

is not open because there is no open ball centered on any of the boundary
points x2 + y2 = 1 that is contained entirely within D2.
The fundamental characterizations of open sets are contained in the
following three theorems.

Theorem A1 Let (X, d) be a metric space. Then any open ball is an open
set.

Proof Let B(xà, r) be an open ball in X and let x be any point in B(xà, r). We
must find a B(x, ræ) contained in B(xà, r).
682 APPENDIX A

x
B(x, ræ) ræ

xà r

B(xà, r)

Since d(x, x0) < r, we define ræ = r - d(x, x0). Then for any y ∞ B(x, ræ) we
have d(y, x) < ræ, and hence

d(y, x0) ¯ d(y, x) + d(x, x0) < ræ + d(x, x0) = r

which shows that y ∞ B(xà, r). Therefore B(x, ræ) ™ B(xà, r). ˙

Theorem A2 Let (X, d) be a metric space. Then

(a) Both X and Å are open sets.
(b) The intersection of a finite number of open sets is open.
(c) The union of an arbitrary number of open sets is open.

Proof (a) X is clearly open since for any x ∞ X and r > 0 we have B(x, r) ™
X. The statement that Å is open is also automatically satisfied since for any
x ∞ Å (there are none) and r > 0, we again have B(x, r) ™ Å.
(b) Let {Ui}, i ∞ I, be a finite collection of open sets in X. Suppose {Ui}
is empty. Then ⁄Ui = X because a point is in the intersection of a collection
of sets if it belongs to each set in the collection, so if there are no sets in the
collection, then every point of X satisfies this requirement. Hence ⁄Ui = X is
open by (a). Now assume that {Ui} is not empty, and let U = ⁄Ui. If U = Å
then it is open by (a), so assume that U ≠ Å. Suppose x ∞ U so that x ∞ Ui for
every i ∞ I. Therefore there exists B(x, rá) ™ Ui for each i, and since there are
only a finite number of the ri we may let r = min{ri}. It follows that

B(x, r) ™ B(x, rá) ™ Uá

for every i, and hence B(x, r) ™ ⁄Ui = U. In other words, we have found an
open ball centered at each point of U and contained in U, thus proving that U
is open.
 METRIC SPACES 683

(c) Let {Ui} be an arbitrary collection of open sets. If {Ui} is empty, then
U = ¡Ui = Å is open by (a). Now suppose that {Ui} is not empty and x ∞
¡Ui . Then x ∞ Ui for some i, and hence there exists B(x, rá) ™ Ui ™ ¡Ui so
that ¡Ui is open. ˙

Notice that part (b) of this theorem requires that the collection be finite. To
see the necessity of this, consider the infinite collection of intervals in ® given
by (-1/n, 1/n) for 1 ¯ n < Ÿ. The intersection of these sets is the point {0}
which is not open in ®.
In an arbitrary metric space the structure of the open sets can be very
complicated. However, the most general description of an open set is con-
tained in the following.

Theorem A3 A subset U of a metric space (X, d) is open if and only if it is

the union of open balls.

Proof Assume U is the union of open balls. By Theorem A1 each open ball is
an open set, and hence U is open by Theorem A2(c). Conversely, let U be an
open subset of X. For each x ∞ U there exists at least one B(x, r) ™ U, so that
¡x´U B(x, r) ™ U. On the other hand each x ∞ U is contained in at least
B(x, r) so that U ™ ¡x´U B(x, r). Therefore U = ¡B(x, r). ˙

As a passing remark, note that a set is never open in and of itself. Rather, a
set is open only with respect to a specific metric space containing it. For
example, the set of numbers [0, 1) is not open when considered as a subset of
the real line because any open interval about the point 0 contains points not in
[0, 1). However, if [0, 1) is considered to be the entire space X, then it is open
by Theorem A2(a).
If U is an open subset of a metric space (X, d), then its complement Uc =
X - U is said to be closed. In other words, a set is closed if and only if its
complement is open. For example, a moments thought should convince you
that the subset of ®2 defined by {(x, y) ∞ ®2: x2 + y2 ¯ 1} is a closed set. The
closed ball of radius r centered at xà is the set B[xà, r] defined in the obvious
way by
B[xà, r] = {x ∞ X: d(xà, x) ¯ r} .

We leave it to the reader (see Exercise A.3) to prove the closed set ana-
logue of Theorem A2. The important difference to realize is that the intersec-
tion of an arbitrary number of closed sets is closed, while only the union of a
finite number of closed sets is closed.
If (X, d) is a metric space and Y ™ X, then Y may be considered a metric
space in its own right with the same metric d used on X. In other words, if we
684 APPENDIX A

let d\Y denote the metric d restricted to points in Y, then the space (Y, d\Y) is
said to be a subspace of the metric space (X, d).

Theorem A4 Let (X, d) be a metric space and (Y, d\Y) a metric subspace of
X. Then a subset W ™ Y is open in Y (i.e., open with respect to the metric
d\Y) if and only if W = Y ⁄ U where U is open in X.

Proof Let W ™ Y be open in Y and suppose x ∞ W. Then there exists r > 0

such that the set
Bd\Y(x, r) = {y ∞ Y: (d\Y)(x, y) < r}

is a subset of W. But this is clearly the same as the open set

Bd(x, r) = {y ∞ X: d(x, y) < r}

restricted to only those points y that are in Y. Another way of saying this is
that
Bd\Y(x, r) = Bd(x, r) ⁄ Y .

Since W = ¡x´W Bd\Y(x, r), it follows that (see Exercise 0.1.1(b)) W = U ⁄

Y where U = ¡x´W Bd(x, r) is open in X (by Theorem A2(c)).
On the other hand, let W = Y ⁄ U where U is open in X, and suppose x ∞
W. Then x ∞ U so there exists r > 0 with

Bd(x, r) = {y ∞ X: d(x, y) < r} ™ U .

But Y ⁄ Bd(x, r) is just Bd\Y(x, r) = {y ∞ Y: (d\Y)(x, y) < r} ™ W which

shows that W is open in Y. ˙

Note that all of our discussion on metric spaces also applies to normed
vector spaces where d(x, y) = ˜x - y˜. Because of this, we can equally well
discuss open sets in any normed space V.
Let f: (X, dX) ‘ (Y, dY) be a mapping. We say that f is continuous at
x0 ∞ X if, given any real number ´ > 0, there exists a real number ∂ > 0 such
that dX(f(x), f(xà)) < ´ for every x ∞ X with dY(x, xà) < ∂. Equivalently, f is
continuous at xà if for each B(f(xà), ´) there exists B(xà, ∂) such that
f(B(xà, ∂)) ™ B(f(xà), ´). (Note that these open balls are defined with respect
to two different metrics since they are in different spaces. We do not want to
clutter the notation by adding subscripts such as dX and dY to B.) In words,
“if you tell me how close you wish to get to the number f(xà), then I will tell
you how close x must be to xà in order that f(x) be that close.” If f is defined
 METRIC SPACES 685

on a subset S ™ X, then f is said to be continuous on S if f is continuous at

every point of S.
For example, consider the mapping f: (0, Ÿ) ™ ® ‘ (0, Ÿ) ™ ® defined by
f(x) = 1/x. For any xà ∞ (0, Ÿ) we have (using the absolute value as our
metric)
\f(x) - f(xà)\ = \1/x - 1/xà\ = \xà - x\/\x xà\ .

If x is such that \x - xà\ < ∂, then we see that

\f(x) - f(xà)\ < ∂/\x xà\ = ∂/(x xà) .

In particular, choosing ∂ ¯ xà/2, it follows that x ˘ xà/2 (since \x - xà\ < ∂ =

xà/2), and hence ∂/(x xà) < 2∂/xà2. Therefore, given any ´ > 0, if we pick ∂ =
min{xà/2, ´xà2/2} then we will have \f(x) - f(xà)\ < ´.
Fortunately one can usually tell by inspection (i.e., by drawing a picture if
necessary) whether or not a particular function is continuous without resorting
to clever calculations. The general definition is a powerful technique for prov-
ing theorems about classes of continuous functions satisfying given properties.
Moreover, there is an intrinsic way to characterize continuous mappings that
is of the utmost importance.

Theorem A5 Let f: (X, dX) ‘ (Y, dY). Then f is continuous if and only if
fî(U) is open in X for all open sets U in Y.

Proof Suppose f is continuous and U is an open subset of Y. If x ∞ fî(U),

then f(x) ∞ U so there exists B(f(x), ´) ™ U (since U is open). But the continu-
ity of f then implies that there exists B(x, ∂) such that

f(B(x, ∂)) ™ B(f(x), ´) ™ U .

Therefore B(x, ∂) is an r-ball centered on x and contained in fî(U), and hence

fî(U) is open.
Conversely, assume that fî(U) is open whenever U is, and let x ∞ X be
arbitrary. Then the open ball B(f(x), ´) is an open set, so its inverse image is
an open set containing x. Therefore there exists an open ball B(x, ∂) contained
in this inverse image, and it clearly has the property that f(B(x, ∂)) ™
B(f(x), ´), hence proving that f is continuous. ˙

Corollary If f: (X, dX) ‘ (Y, dY), then f is continuous if and only if fî(F) is
closed in X whenever F is closed in Y.
686 APPENDIX A

Proof It was shown in Exercise 0.2.1 that if A ™ Y, then fî(Ac) = fî(A)c.

Therefore if F ™ Y is closed, then F = Uc for some open set U ™ Y and so by
Theorem A5, fî(F) = fî(Uc) = fî(U)c must be closed if and only if f is con-
tinuous. ˙

Note that if f: X ‘ Y is continuous and U ™ Y is open, then fî(U) is

open, but if A ™ X is open, then it is not necessarily true that f(A) is open. As
a simple example, consider the function f: ® ‘ ®2 defined by f(x) = (x, x2). It
should be clear that the open ball U ™ ®2 shown below is an open set whose
inverse image is an open interval on ® ¡ {Å} (since some points of U are not
the image under f of any point in ®), but that the image under f of an open
interval is part of the parabola y = x2 which is not open as a subset of ®2.

f(x)

| ( | ) ® f ( ) ®
0 x x

Now suppose that (X, d) is a metric space, and let {Uá} be a collection of
open subsets of X such that ¡Uá = X. Such a collection of subsets is called an
open cover of X. A subcollection {Vé} of the collection {Uá} is said to be an
open subcover of X if ¡Vé = X. A space (X, d) is said to be compact if every
open cover has a finite subcover. Similarly, given a subset A ™ X, a collection
{Ui} of open subsets of X with the property that A ™ ¡Ui is said to be an
open cover of A. Equivalently, the collection {Ui} of open subsets of X is an
open cover of A in X if the collection {Ui ⁄ A} is an open cover of the subset
A in the metric d\A (i.e., in the subspace A). We then say that A is compact if
every open cover of A has a finite subcover, or equivalently, A is compact if
the subspace A is compact. While this is not a particularly easy concept to
thoroughly understand and appreciate without detailed study, its importance to
us is based on the following two examples.

Example A1 Consider the subset A = (0, 1) of the real line ®. We define the
collection {Uè, Uì, . . . } of open sets by
 METRIC SPACES 687

Uñ = (1/2n+1, 1 - 1/2n+1) .

Thus Uè = (1/4, 3/4), Uì = (1/8, 7/8) etc. The collection {Uñ} clearly covers A
since for any x ∞ (0, 1) we can always find some Uñ such that x ∞ Uñ.
However, A is not compact since given any finite number of the Uñ there
exists ´ > 0 (so that ´ ∞ (0, 1)) which is not in any of the Uñ. ∆

Example A2 Let us show that the subspace [0, 1] of the real line is compact.
This is sometimes called the Heine-Borel theorem, although we shall prove a
more general version below.
First note that the points 0 and 1 which are included in the subspace [0, 1]
are not in the set (0, 1) discussed in the previous example. However, if we
have positive real numbers a and b with a ¯ b < 1, then the collection {Uñ}
defined above together with the sets [0, a) and (b, 1] does indeed form an open
cover for [0, 1] (the sets [0, a) and (b, 1] are open by Theorem A4). It should
be clear that given the sets [0, a) and (b, 1] we can now choose a finite cover
of [0, 1] by including these sets along with a finite number of the Uñ. To
prove that [0, 1] is compact however, we must show that any open cover has a
finite subcover.
Somewhat more generally, let {Oñ} be any open cover of the interval
[a, b] in ®. Define

A = {x ∞ [a, b]: [a, x] is covered by a finite number of the Oñ} .

We see that A ≠ Å since clearly a ∞ A, and furthermore A is bounded above

by b. Therefore (by the Archimedean axiom) A must have a least upper bound
m = sup A ¯ b. If A is to be compact, then we must have b ∞ A. We will show
that this is true by first proving that m ∞ A, and then that m = b.
Since {Oñ} covers [a, b] and m ∞ [a, b], it follows that m ∞ Om for some
Om ∞ {Oñ}. Now, Om is an open subset of [a, b], and hence there are points in
Om that are less than m, and points in Om that are greater than m.

Om
| ( | | | ) |
a x m y b

Since m = sup A, there is an x < m with x ∞ Om such that the interval [a, x] is
covered by a finite number of the Oñ, while [x, m] is covered by the single set
Om. Therefore [a, m] is covered by a finite number of open sets so that m ∞
A.
688 APPENDIX A

Now suppose that m < b. Then there is a point y with m < y < b such that
[m, y] ™ Om. But we just showed that m ∞ A, so the interval [a, m] is covered
by finitely many Oñ while [m, y] is covered by Om. Therefore y ∞ A which
contradicts the definition of m, and hence we must have m = b. ∆

An important property of metric spaces is the following. Given two dis-

tinct points x, y ∞ X, there exist disjoint open sets U and V in X such that x ∞
U and y ∞ V. That this does indeed hold for metric spaces is easy to prove by
considering open balls of radius d(x, y)/2 centered on each of the points x and
y. This property is called the Hausdorff property. We sometimes refer to a
metric space as a “Hausdorff space” if we wish to emphasize this property.
The following theorems describe some of the most fundamental properties
of compact spaces.

Theorem A6 Any closed subset of a compact space is compact.

Proof Let F ™ X be a closed subset of a compact space X. If {Uá} is any

open cover of F, then (¡Uá) ¡ Fc is an open cover of X. Since X is compact,
we may select a finite subcover by choosing Fc along with a finite number of
the Uá. But then F is covered by this finite subcollection of the Uá, and hence F
is compact. ˙

Theorem A7 Any compact subset of a metric space is closed.

Proof Let F be a compact subset of a metric space X. We will show that Fc is

open. Fix any x ∞ Fc and suppose y ∞ F. Since X is Hausdorff, there exist
open sets Uó and Vó such that x ∞ Uó, y ∞ Vó and Uó ⁄ Vó = Å. As the point
y varies over F, we see that {Vó: y ∞ F} is an open cover for F. Since F is
compact, a finite number, say Vyè , . . . , Vyñ, will cover F. Corresponding to
each Vyá there is a Uyá, and we let U = ⁄áUyá and V = ¡áVyá. By construction
x ∞ U, F ™ V and U ⁄ V = Å. But then U is an open set containing x such
that U ⁄ F = Å, and hence Fc is open. ˙

Theorem A8 Let (X, dX) be a compact space and let f be a continuous func-
tion from X onto a space (Y, dY). Then Y is compact.

Proof Let {Uá} be any open cover of Y. Since f is continuous, each fî(Uá) is
open in X, and hence {fî(Uá)} is an open cover for X. But X is compact, so
that a finite number of the fî(Ui), say {fî(Uiè), . . . , fî(Uiñ)} cover X.
Therefore {Uiè, . . . , Uiñ} form a finite subcover for Y, and hence Y is
compact. ˙
 METRIC SPACES 689

Theorem A9 Let {Ká} be a collection of compact subsets of a metric space

X, such that the intersection of every finite subcollection of {Ká} is nonempty.
Then ⁄Ká is nonempty.

Proof Fix any Kè ∞ {Ká} and assume that K1 ⁄ (⁄i!1K i ) = Å. We will show
that this leads to a contradiction. First note that by our assumption we have
(⁄i!1K i ) ™ K1c, and hence from Example 0.1 and Theorem 0.1 we see that

K1 ™ ⁄i!1K i c = ¡i!1K i c .

Thus {Kic}, i ≠ 1, is an open cover of K1. But K1 is compact so that a finite

number of these sets, say Kièc, . . . , Kiñc cover K1. Then

K1 ! ( "#n =1 K i# c ) = ( $#n =1 K i# )c

which implies
K1 ! (!"n =1 K i" ) = Å .

However, this contradicts the hypothesis of the theorem. ˙

Corollary If {Kñ} is a sequence of nonempty compact sets such that

Kñ ! Kn+1, then ⁄Kñ ≠ Å.

Proof This is an obvious special case of Theorem A9. ˙

As a particular application of this corollary we see that if {Iñ} is a

nonempty sequence of intervals [añ, bñ] ™ ® such that Iñ ! In+1, then ⁄Iñ ≠
Å. While this result is based on the fact that each Iñ is compact (Example A2),
we may also prove this directly as follows. If Iñ = [añ, bñ], we let S = {añ}.
Then S ≠ Å and is bounded above by bè. By the Archimedean axiom, we let
x = sup S. For any m, n ∞ Û+ we have añ ¯ am+n ¯ bm+n ¯ bm so that x ¯ bm
for all m. Since am ¯ x for all m, we must have x ∞ [am, bm] = Im for each
m = 1, 2, . . . so that ⁄Iñ ≠ Å. We now show that this result holds in ®n as
well.
Suppose a, b ∞ ®n where ai < bi for each i = 1, . . . , n. By an n-cell we
mean the set of all points x ∞ ®n such that ai ¯ xi ¯ bi for every i. In other
words, an n-cell is just an n-dimensional rectangle.

Theorem A10 Let {IÉ} be a sequence of n-cells such that IÉ ! Ik+1. Then
⁄IÉ ≠ Å.
690 APPENDIX A

Proof For each k = 1, 2, . . . the n-cell IÉ consists of all points x ∞ ®n with

the property that aÉi ¯ xi ¯ bÉi for every 1 ¯ i ¯ n, so we let IÉi = [aÉi, bÉi].
Now, for each i = 1, . . . , n the sequence {IÉi} satisfies the hypotheses of the
corollary to Theorem A9. Hence for each i = 1, . . . , n there exists zi ∞
[aÉi, bÉi] for every k = 1, 2, . . . . If we define z = (z1, . . . , zn) ∞ ®n, we see
that z ∞ IÉ for every k = 1, 2, . . . . ˙

Theorem A11 Every n-cell is compact.

Proof Let I be an n-cell as defined above, and set ∂ = [Í iˆ=1(bá - aá)2]1/2.

Then if x, y ∞ I we have ˜x - y˜ ¯ ∂ (see Example 2.9). Let {Ui} be any open
cover of I and assume that it contains no finite subcover. We will show that
this leads to a contradiction.
Let cj = (aj + bj )/2 for each j = 1, . . . , n. Then we have 2n n-cells Qá
defined by the intervals [aj, cj] and [cj, bj] such that ¡Qá = I. Since I has no
finite subcover, at least one of the Qá, which we call Iè, can not be covered by
any finite number of the Ui. Next we subdivide Iè into another 2n n-cells and
continue in the same manner. We thus obtain a sequence {Iå} of n-cells with
the following properties:

(a) I ! I1 ! I 2 ! ! ;
(b) Iå is not covered by any finite subcollection of the UÉ;
(c) x, y ∞ Iå implies ˜x - y˜ ¯ 2-å ∂.

By (a) and Theorem A10, there exists z ∞ ⁄Iå , and since {Ui} covers I,
we must have z ∞ Uk for some k. Now, UÉ is an open set in the metric space
®n, so there exists ´ > 0 such that ˜z - y˜ < ´ implies that y ∞ UÉ. If we
choose å sufficiently large that 2-å∂ < ´ (that this can be done follows from
Theorem 0.3), then (c) implies that Iå ™ UÉ which contradicts (b). ˙

We are now in a position to prove the generalized Heine-Borel theorem.

Before doing so however, we first prove a simple result which is sometimes
taken as the definition of a compact set. By way of terminology, any open set
U containing a point x is said to be a neighborhood of x, and the set U - {x}
is called a deleted neighborhood of x. We say that a point x ∞ (X, d) is an
accumulation point of A ™ X if every deleted neighborhood of x intersects
A.

Theorem A12 Any infinite subset A of a compact set K has a point of accu-
mulation in K.
 METRIC SPACES 691

Proof Suppose every point x ∞ K is not an accumulation point of A. Then

there exists a neighborhood Uí of x such that Uí contains at most a single
point of A, namely x itself if x ∞ A. Then clearly no finite subcollection of
{Uí} covers A ™ K so that K can not possibly be compact. ˙

Theorem A13 A subset A of a metric space (X, d) is closed if and only if A

contains all of its accumulation points.

Proof First suppose that A is closed. Let x ∞ X be an accumulation point of

A and assume that x ! A. Then x ∞ Ac which is an open set containing x that
does not intersect A, and hence contradicts the fact that x is an accumulation
point of A. Therefore x must be an element of A.
Conversely, suppose A contains all of its accumulation points. We show
that Ac is open. If x ∞ Ac and hence is not an accumulation point of A, then
there exists an open set U containing x such that A ⁄ U = Å. But then x ∞
U ™ Ac which implies that Ac is open. ˙

We say that a subset A ™ ®n is bounded if it can be enclosed in some n-

cell. The equivalence of (a) and (b) in the next theorem is called the
(generalized) Heine-Borel theorem, while the equivalence of (a) and (c) is a
general version of the Bolzano-Weierstrass theorem.

Theorem A14 Let A be a subset of ®n. Then the following three properties
are equivalent:
(a) A is closed and bounded .
(b) A is compact.
(c) Every infinite subset of A has a point of accumulation in A.

Proof (a) ¶ (b): If (a) holds, then A can be enclosed by some n-cell which is
compact by Theorem A11. But then A is compact by Theorem A6.
(b) ¶ (c): This follows from Theorem A12.
(c) ¶ (a): We assume that every infinite subset of A has an accumulation
point in A. Let us first show that A must be bounded. If A is not bounded,
then for each positive integer k = 1, 2, . . . we can find an xk ∞ A such that
˜xk˜ > k. Then the set {xk} is clearly infinite but contains no point of accumu-
lation in ®n, so it certainly contains none in A. Hence A must be bounded.
We now show that A must be closed. Again assume the contrary. Then
there exists x0 ∞ ®n which is an accumulation point of A but which does not
belong to A (Theorem A13). This means that for each k = 1, 2, . . . there exists
xk ∞ A such that ˜xk - x0˜ < 1/k. The set S = {xk} is then an infinite subset of
692 APPENDIX A

A with x0 as an accumulation point. Since x0 ! A, we will be finished if we

can show that S has no accumulation point in A (because the assumption that
A is not closed then leads to a contradiction with the property described in
(c)).
First note that if a , b ∞ ®n, then Example 2.11 shows us that

˜a + b˜ = ˜a - (-b)˜ ˘ ˜a ˜ - ˜b˜ .

Using this result, if y is any point of ®n other than x0 we have

˜ xk ! y˜ = ˜ xk ! x0 + x0 ! y˜
" ˜ x0 ! y˜ - ˜ xk ! x0 ˜
> ˜ x0 ! y˜ - 1/k!!.

No matter how large (or small) ˜x0 - y˜ is, we can always find a kà ∞ Û+
such that 1/k ¯ (1/2)˜x0 - y˜ for every k ˘ kà (this is just Theorem 0.3). Hence

˜xÉ - y˜ > (1/2)˜x0 - y˜

for every k ˘ kà. This shows that y can not possibly be an accumulation point
of {xk} = S (because the open ball of radius (1/2)˜x0 - y˜ centered at y can
contain at most a finite number of elements of S). ˙

We remark that the implication “(a) implies (b)” in this theorem is not true
in an arbitrary metric space (see Exercise A.5).
Let f be a mapping from a set A into ®n. Then f is said to be bounded if
there exists a real number M such that ˜f(x)˜ ¯ M for all x ∞ A. If f is a con-
tinuous mapping from a compact space X into ®n, then f(X) is compact
(Theorem A8) and hence closed and bounded (Theorem A14). Thus we see
that any continuous function from a compact set into ®n is bounded. On the
other hand, note that the function f: ® ‘ ® defined by f(x) = 1/x is not
bounded on the interval (0, 1). We also see that the function g: ® ‘ ® defined
by g(x) = x for x ∞ [0, 1) never attains a maximum value, although it gets
arbitrarily close to 1. Note that both f and g are defined on non-compact sets.
We now show that a continuous function defined on a compact space takes
on its maximum and minimum values at some point of the space.

Theorem A15 Let f be a continuous real-valued function defined on a com-

pact space X, and let M = supx´X f(x) and m = infx´X f(x). Then there exist
points p, q ∞ X such that f(p) = M and f(q) = m.
 METRIC SPACES 693

Proof The above discussion showed that f(X) is a closed and bounded subset
of ®. Hence by the Archimedean axiom, f(X) must have a sup and an inf. Let
M = sup f(x). This means that given ´ > 0, there exists x ∞ X such that

M - ´ < f(x) < M

(or else M would not be the least upper bound of f(X)). This just says that any
open ball centered on M intersects f(X), and hence M is an accumulation point
of f(X). But f(X) is closed so that Theorem A13 tells us that M ∞ f(X). In
other words, there exists p ∞ X such that M = f(p). The proof for the minimum
is identical. ˙

As an application of these ideas, we now prove the Fundamental Theorem

of Algebra.

Theorem A16 (Fundamental Theorem of Algebra) The complex number

field ç is algebraically closed.

Proof Consider the non-constant polynomial

f(z) = aà + aèz + ~ ~ ~ + añzn ∞ ç[z]

where añ ≠ 0. Recall that we view ç as the set ® ª ® = ®2, and let R be any
(finite) real number. Then the absolute value function \f\: ç ‘ ® that takes
any z ∞ ç to the real number \f(z)\ is continuous on the closed ball B[0, R] of
radius R centered at the origin. But B[0, R] is compact (Theorem A14) so that
\f(z)\ takes its minimum value at some point on the ball (Theorem A15). On
the other hand, if we write f(z) in the form

f(z) = añzn(aà/añzn + aè/añzn-1 + ~ ~ ~ + an-1/añz + 1)

we see that \f(z)\ becomes arbitrarily large as \z\ becomes large. To be precise,
given any real C > 0 there exists R > 0 such that \z\ > R implies \f(z)\ > C.
We now combine these two facts as follows. Let zè be arbitrary, and define
C = \f(zè)\. Then there exists Rà > 0 such that \f(z)\ > \f(zè)\ for all z ∞ ç such
that \z - zè\ > Rà (i.e., for all z outside B[zè, Rà]). Since B[zè, Rà] is compact,
there exists a point zà ∞ B[zè, Rà] such that \f(zà)\ ¯ \f(z)\ for all z ∞ B[zè, Rà].
694 APPENDIX A

Rà
zè Â
zà Â

B[zè, Rà]

In particular, \f(zà)\ ¯ \f(zè)\ and hence we see that \f(zà)\ < \f(z)\ for all z ∞ ç.
In other words, zà is an absolute minimum of \f\. We claim that f(zà) = 0.
To show that f(zà) = 0, we assume that f(zà) ≠ 0 and arrive at a contradic-
tion. By a suitable choice of constants cá, we may write f in the form

f(z) = cà + cè(z - zà) + ~ ~ ~ + cñ(z - zà)n .

If f(zà) ≠ 0 then cà = f(zà) ≠ 0. By assumption, deg f ˘ 1, so we let m be the

smallest integer greater than 0 such that cm ≠ 0. Defining the new variable w =
z - zà , we may define the polynomial function g by

f(z) = g(w) = cà + cmwm + wm+1h(w)

for some polynomial h.

Now let wè be a complex number such that wèm = -cà/cm and consider all
values of w = ¬wè for real ¬ with 0 ¯ ¬ ¯ 1. Then

cm wm = cm ¬m wèm = -cà ¬m
and hence

f (z) = g(! w1 ) = c0 " ! m c0 + ! m+1w1m+1h(! w1 )

= c0 [1 " ! m + ! m+1w1m+1c0 "1h(! w1 )]!!.

But ¬ ∞ [0, 1] which is compact, and hence \wèm+1càîh(¬wè)\ is a continuous

function defined on a compact set. Then the image of this function is a
compact subset of ® (Theorem A8) and so is closed and bounded (Theorem
A14). This means there exists a number B > 0 such that

\wèm+1càîh(¬wè)\ ¯ B

for all ¬ ∞ [0, 1], and therefore (since 0 ¯ ¬ ¯ 1 implies that 0 ¯ ¬m ¯ 1)

 METRIC SPACES 695

| g(! w1 )| = |c0 |!! |1 " ! m + ! m+1w1m+1c0 "1h(! w1 )|

# |c0 |!! {|1 " ! m | + ! m+1|w1m+1c0 "1h(! w1 )|}
# |c0 | (1 " ! m + ! m+1 B)!!.

Now recall that \cà\ = \f(zà)\ ¯ \f(z)\ for all z ∞ ç. If we can show that

0 < 1 - ¬m + ¬m+1B < 1

for sufficiently small ¬ with 0 < ¬ < 1, then we will have shown that \f(z)\ =
\g(¬wè)\ < \cà\, a contradiction. But it is obvious that ¬ can be chosen so that
0 < 1 - ¬m + ¬m+1B. And to require that 1 - ¬m + ¬m+1B < 1 is the same as
requiring that ¬B < 1 which can certainly be satisfied for small enough ¬. ˙

Exercises

1. Show the absolute value function is continuous on ®.

2. Show that the norm on a vector space V defines a metric on V.

3. Let (X, d) be a metric space. Prove:

(a) Both X and Å are closed sets.
(b) The intersection of an arbitrary number of closed sets is closed.
(c) The union of a finite number of closed sets is closed.

4. Let A be the subset of [0, 1] consisting of all x ∞ [0, 1] whose decimal

expansion contains only the digits 4 and 7. Explain whether or not A is
countable, dense in [0, 1], or compact.

5. Show that {x: ˜x˜2 ¯ 1} is closed and bounded but not compact in the
space l2 (see Example 12.8).

6. A metric space is said to be separable if it contains a countable dense

subset. Prove that ®n is separable. [Hint: Consider the set of all points in
®n with rational coordinates.]
 APPENDIX B

Sequences and Series

In this appendix we briefly go through all of the theory necessary for an

understanding of Section 10.6 and Chapter 12. Furthermore, as we mentioned
at the beginning of Appendix A, rather than being simply an overview, we
want the reader to understand this material even if it has not been studied
before. We do assume however, that the reader has studied Appendix A, and
part of this appendix is a direct extension of that material.
A sequence {xn} = {xè, xì, . . . } in a set S is any function from the set Û+
of positive integers into S. If (X, d) is a metric space, then a sequence {xn} in
X is said to converge to the limit x if for each B(x, ´) there exists N ∞ Û+
such that xn ∞ B(x, ´) for every n ˘ N. In other words, given ´ > 0, there exists
a positive integer N such that n ˘ N implies d(xn, x) < ´. This is usually
written as lim xn = x or xn ‘ x. If a sequence {xn} does not converge, then it
is said to diverge. Furthermore, if for every real number M there exists an
integer N such that n ˘ N implies xn ˘ M, then we write xn ‘ +Ÿ. Similarly,
if for every real number M there exists an integer N such that n ˘ N implies
xn ¯ M, then we write xn ‘ -Ÿ.
(We remark that the small, always positive number ´ will be used exten-
sively in many proofs, and it is important to realize that for proofs of
convergence there is no real difference between the number ´ and certain
simple functions of ´ such as 2´. For example, suppose we can show that
given ´, there exists N such that n ˘ N implies d(xn, x) < 2´. We claim this
also proves that xn ‘ x. Indeed, let ´æ = ´/2. Then, by assumption, there exists

696
 SEQUENCES AND SERIES 697

Næ such that n ˘ Næ implies d(xn, x) < 2´æ = ´ which shows that x is the limit of
the sequence. It should be clear that the statement “given ´ > 0” is equivalent
to saying “given C´ > 0” for any finite C > 0.)
The set of all points xn for n = 1, 2, . . . is called the range of the sequence
{xn}. This may be either a finite or an infinite set of points. A set A ™ (X, d)
is said to be bounded if there exists a real number M and a point x0 ∞ X such
that d(x, x0) ¯ M for all x ∞ A. (The point x0 is almost always taken to be the
origin of any given coordinate system in X.) The sequence {xn} is said to be
bounded if its range is a bounded set. It is easy to show that any convergent
sequence is bounded. Indeed, if xn ‘ x then, given 1, there exists N such that
n ˘ N implies d(xn, x) < 1. This shows that {xn: n ˘ N} is bounded. To show
that {xn: n = 1, . . . , N - 1} is bounded, let

r = max{1, d(x1, x), . . . , d(xN-1, x)} .

Since xn ∞ X, it must be true that each d(xn, x) is finite, and hence d(xn, x) ¯ r
for each n = 1, . . . , N - 1.
We now prove several elementary properties of sequences, starting with
the uniqueness of the limit.

Theorem B1 If {xn} is a sequence in a metric space (X, d) such that xn ‘ x

and xn ‘ y, then x = y.

Proof Given ´ > 0, there exists N such that n ˘ N implies d(xn, x) < ´ and
d(xn, y) < ´. But then d(x, y) ¯ d(xn, x) + d(xn, y) < 2´. Since this holds for all
´ > 0, we must have x = y (see Appendix A, definition (M2)). ˙

Theorem B2 Let sn ‘ s and tn ‘ t be convergent sequences of complex

numbers. Then
(a) lim (sn + tn) = s + t.
(b) lim csn = cs and lim (c + sn) = c + s for any c ∞ ç.
(c) lim sntn = st.
(d) lim 1/sn = 1/s if s ≠ 0 and sn ≠ 0 for all n.

Proof (a) Given ´ > 0, there exists N1 and N2 such that n ˘ Nè implies that
\sn - s\ < ´/2 and n ˘ Nì implies that \tn - t\ < ´/2. Let N = max{N1, N2}. Then
n ˘ N implies

\(sn - s) + (tn - t)\ ¯ \sn - s\ + \tn - t\ < ´ .

(b) This is Exercise B.1.

698 APPENDIX B

(c) Note the algebraic identity

sntn - st = (sn - s)(tn - t) + s(tn - t) + t(sn - s) .

By parts (a) and (b) we see that lim s(tn - t) = 0 = lim t(sn - s). Now, given
´ > 0, there exists N1 and N2 such that n ˘ N1 implies

\sn - s\ < “´”

and n ˘ N2 implies
\tn - t\ < “´” .

If N = max{N1, N2}, then n ˘ N implies that

\(sn - s)(tn - t)\ < ´

and hence lim (sntn - st) = lim (sn - s)(tn - t) = 0.

(d) At the end of Section 0.4 we showed that \a\ - \b\ ¯ \a + b\ for any a,
b ∞ ®. Reviewing the proof shows that this applies to complex numbers as
well. Letting b ‘ -b then shows that \a\ - \b\ ¯ \a - b\.
Given \s\/2, there exists m such that n ˘ m implies \sn - s\ < \s\/2 (this
follows from the fact that sn ‘ s). But \s\ - \sn\ ¯ \sn - s\ < \s\/2 implies \sn\ >
\s\/2 .
Alternatively, given ´ > 0, there exists N (which we can always choose
greater than m) such that n ˘ N implies \sn - s\ < \s\2´/2. Combining these
results, we see that for all n ˘ N we have

1 1 sn ! s s ! s 2 sn ! s
! = = n < 2
< " !!.!!˙
sn s ssn s sn s

Intuitively, we expect that a sequence {xÉ} of points in ®n converges to a

point in ®n if and only if each of the n coordinates converges on its own. That
this is indeed true is shown in our next theorem.

Theorem B3 Suppose xk = (xk1, . . . , xkn ) ∞ ®n. Then {xk} converges to x

= (x1, . . . , xn) ∞ ®n if and only if xki ‘ xi for each i = 1, . . . , n.

Proof First note that for any j = 1, . . . , n we have

n
2
x ! y = " (x i ! yi )2 # (x j ! y j )2
i=1
 SEQUENCES AND SERIES 699

which implies \xj - yj\ ¯ \x - y \. Now assume that xk ‘ x. Then given ´ > 0,
there exists N such that k ˘ N implies \xkj - yj\ ¯ \xk - x\ < ´. This shows that
xk ‘ x implies xkj ‘ xj.
Conversely, assume xkj ‘ xj for every j = 1, . . . , n. Then given ´ > 0,
there exists N such that k ˘ N implies \xkj - xj\ < ´/“n” . Hence k ˘ N implies

#n &1/2
xk ! x = %" (xk j ! x j )2 ( < (n) 2 /n)1/2 = )
% (
$ j=1 '

so that xk ‘ x. ˙

A sequence {xk} in a metric space (X, d) is said to be a Cauchy sequence

if given ´ > 0, there exists N such that n, m ˘ N implies d(xn, xm) < ´. It is
easy to see that every convergent sequence is in fact a Cauchy sequence.
Indeed, simply note that if xk ‘ x, then given ´ > 0, there exists N such that
n ˘ N implies d(xn, x) < ´/2. Hence if n, m ˘ N we have

d(xn, xm) ¯ d(xn, x) + d(xm, x) < ´/2 + ´/2 = ´ .

However, it is not true that a Cauchy sequence need converge. For exam-
ple, suppose X = (0, 1] ™ ® and let {xk} = {1/k} for k = 1, 2, . . . . This is a
Cauchy sequence that wants to converge to the point 0 (choose N = 1/´ so that
\1/n - 1/m\ ¯ \1/n\ + \1/m\ < 2´ for all m, n ˘ N). But 0 ! (0, 1] so that the
limit of the sequence is not in the space. This example shows that convergence
is not an intrinsic property of sequences, but rather depends on the space in
which the sequence lies. A metric space in which every Cauchy sequence
converges is said to be complete (see Appendix A).
We have shown that any convergent sequence is necessarily a Cauchy
sequence, but that the converse is not true in general. However, in the case of
®n, it is in fact true that every Cauchy sequence does indeed converge, i.e., ®n
is a complete metric space. This is easy to prove using the fact that any n-cell
in ®n is compact (Theorem A11), and we outline the proof in Exercise B.10.
However, it is worth proving that ®n is complete without using this result. We
begin by proving several other facts dealing with the real number line ®. By
way of terminology, a sequence {xn} of real numbers with the property that
xn ¯ xn+1 is said to be increasing. Similarly, if xn ˘ xn+1 then the sequence is
said to be decreasing. We will sometimes use the term monotonic to refer to
a sequence that is either increasing or decreasing.
700 APPENDIX B

Theorem B4 Let {xk} be an increasing sequence (i.e., xk ¯ xk+1) of real

numbers that is bounded above. Then the least upper bound b of the set {xk}
is the limit of the sequence.

Proof It should be remarked that the existence of the least upper bound is
guaranteed by the Archimedean axiom. Given ´ > 0, the number b - ´/2 is not
an upper bound for {xk} since b is by definition the least upper bound.
Therefore there exists N such that b - ´/2 ¯ xN ¯ b (for otherwise b - ´/2
would be the least upper bound). Since {xk} is increasing, we have

b - ´/2 ¯ xN ¯ xn ¯ b

for every n ˘ N. Rearranging, this is just b - xn ¯ ´/2 < ´ which is the same as
\xn - b\ < ´. ˙

Since the Archimedean axiom also refers to the greatest lower bound of a
set of real numbers, it is clear that Theorem B4 may be applied equally well to
the greatest lower bound of a decreasing sequence.
Let (X, d) be a metric space, and suppose A is a subset of X. Recall from
Appendix A that a point x ∞ X is said to be an accumulation point of A if
every deleted neighborhood of x contains a point of A. The analogous term for
sequences is the following. A number x is said to be a cluster point (or limit
point) of a sequence {xn} if given ´ > 0, there exist infinitely many integers n
such that \xn - x\ < ´. Equivalently, x is a cluster point if given ´ > 0 and given
N, there exists some n ˘ N such that \xn - x\ < ´. Note that this does not say
that there are infinitely many distinct xn such that \xn - x\ < ´. In fact, all the
xn could be identical. It is important to distinguish between the indices n and
the actual elements xn of the sequence. It is also important to realize that a
limit point of a sequence is not the same as the limit of a sequence (why?).
Note also that a sequence in X may be considered to be a subset of X, and in
this context we may also refer to the accumulation points of a sequence.
Our next result is known as the Bolzano-Weierstrass Theorem.

Theorem B5 (Bolzano-Weierstrass) Let {xk} be a sequence of real num-

bers, and let a, b ∞ ® be such that a ¯ xk ¯ b for all positive integers k. Then
there exists a cluster point c of the sequence with a ¯ c ¯ b.

Proof For each n, the sequence {xn, xn+1, . . . } is bounded below (by a), and
hence has a greatest lower bound (whose existence is guaranteed by the
Archimedean axiom) which we denote by cn . Then {cn} forms an increasing
sequence cn ¯ cn+1 ¯ ~ ~ ~ which is bounded above by b. Theorem B4 now
 SEQUENCES AND SERIES 701

shows that the sequence {cn} has a least upper bound c (with a ¯ c ¯ b) which
is in fact the limit of the sequence {cn}. We must show that c is a cluster point
of the sequence {xk}.
To say that c is the limit of the sequence {cn} means that given ´ > 0 and
given any N, there exists some m ˘ N such that

\cm - c\ < ´/2 .

By definition, cm is the greatest lower bound of the set {xm, xm+1, . . . } which
means there exists k ˘ m such that cm ¯ xk < cm + ´/2 or

\xk - cm\ < ´/2 .

Therefore k ˘ m ˘ N and

\xk - c\ = \xk - cm + cm - c\ ¯ \xk - cm\ + \cm - c\ < ´

which shows that c is a cluster point of the sequence {xk}. ˙

Note that Theorem B5 also follows from Theorem A12.

Theorem B6 If {xn} is a Cauchy sequence of numbers, then it is bounded.

Proof By definition of a Cauchy sequence, given ´ = 1 there exists N such

that n ˘ N implies \xn - xN\ < 1. Hence \xn\ - \xN\ ¯ \xn - xN\ < 1 implies \xn\ ¯
\xN\ + 1 for every n ˘ N. Define B = max{\x1\, . . . , \xN\, \xN\ + 1}. Then B is
clearly a bound for {xn}. ˙

Theorem B7 Any Cauchy sequence {xn} of numbers converges.

Proof From Theorem B6 the sequence {xn} has a bound B, and hence we
have -B ¯ xn ¯ B for all n. Hence by Theorem B5, the sequence {xn} has a
cluster point c. We claim that c is the limit of the sequence. Since the
sequence is Cauchy, given ´ > 0 there exists N such that m, n ˘ N implies
\xm - xn\ < ´/2. Using this ´, we see that because c is a cluster point, there
exists m ˘ N such that \xm - c\ < ´/2. Combining these last two results shows
that for all n ˘ N
\xn - c\ ¯ \xn - xm\ + \xm - c\ < ´ . ˙

We are now in a position to prove our principal assertion.

702 APPENDIX B

Theorem B8 ®n is a complete metric space. In other words, every Cauchy

sequence in ®n converges to a point in ®n.

Proof Let {xk} be a Cauchy sequence in ®n. Then \xmj - xnj\ ¯ \xm - xn\ (see
the proof of Theorem B3) so that {xkj} is also a Cauchy sequence in ® for
each j = 1, . . . , n. Hence by Theorem B7 each of the sequences {xkj} also
converges in ®. Therefore (by Theorem B3) the sequence {xk} must converge
in ®n. ˙

We have seen that any convergent sequence is a Cauchy sequence and

hence bounded. However, the converse is not generally true. (For example,
the sequence {1, 2, 1, 2, . . . } is clearly bounded but does not converge to
either 1 or 2.) There is, however, a special case in which the converse is true
that will be of use to us.

Theorem B9 A monotonic sequence {xn} converges if and only if it is

bounded.

Proof We consider increasing sequences. The proof for decreasing sequences

is similar. It was shown above that any convergent sequence is bounded, so
we need only consider the converse. But this was proved in Theorem B4. ˙

Finally, accumulation points are useful in determining whether or not a set

is closed. The principal result relating these concepts is the following.

Theorem B10 A subset A of a metric space (X, d) is closed if and only if A

contains all of its accumulation points.

Proof This is also Theorem A13. ˙

Before continuing, we must make a digression to discuss some more basic

properties of metric spaces. If the reader already knows that a point x in a
subset A of a metric space X is in the closure of A if and only if every neigh-
borhood of x intersects A, then he/she may skip to Theorem B16 below.
Let (X, d) be a metric space, and suppose A ™ X. We define

(a) The closure of A, denoted by Cl A, to be the intersection of all closed

supersets of A;
(b) The interior of A, denoted by Int A (or Ao), to be the union of all
open subsets of A;
 SEQUENCES AND SERIES 703

(c) The boundary of A, denoted by Bd A, to be the set of all x ∞ X such

that every open set containing x contains both points of A and points
of Ac = X - A;
(d) The exterior of A, denoted by Ext A, to be (Cl A)c = X - Cl A;
(e) The derived set of A, denoted by Aæ, to be the set of all accumulation
points of A.

Example B1 Let X = ®2 with the Pythagorean metric. Let A be the open unit
ball defined by A = {(x, y): x2 + y2 < 1}. Then the following sets should be
intuitively clear to the reader:

Cl A = {(x, y): x2 + y2 ¯ 1};

Int A = A;
Bd A = {(x, y): x2 + y2 = 1};
Ext A = {(x, y): x2 + y2 > 1};
Aæ = Cl A. ∆

Theorem B11 Let (X, d) be a metric space and suppose A, B ™ X. Then

(a) A ™ Cl A.
(b) Cl(Cl A) = Cl A.
(c) Cl(A ¡ B) = (Cl A) ¡ (Cl B).
(d) Cl Å = Å.
(e) A is closed if and only if A = Cl A.
(f) Cl(A ⁄ B) ™ (Cl A) ⁄ (Cl B).

Proof Parts (a), (b), (d) and (e) are essentially obvious from the definition of
Cl A, the fact that the intersection of an arbitrary number of closed sets is
closed (see Exercise A.3), the fact that the empty set is a subset of every set,
and the fact that if A is closed, then A is one of the closed sets which contains
A.
(c) First note that if S ™ T, then any closed superset of T is also a closed
superset of S, and therefore Cl S ™ Cl T. Next, observe that A ™ A ¡ B and
B ™ A ¡ B, so that taking the closure of both sides of each of these relations
yields
Cl A ™ Cl(A ¡ B)
and
Cl B ™ Cl(A ¡ B) .

Together these show that (Cl A) ¡ (Cl B) ™ Cl(A ¡ B). Since Cl A and Cl B
are both closed, (Cl A) ¡ (Cl B) must also be closed and contain A ¡ B.
Hence we also have
704 APPENDIX B

Cl(A ¡ B) ™ (Cl A) ¡ (Cl B) .

This shows that Cl(A ¡ B) = (Cl A) ¡ (Cl B).

(f) By (a) we have A ⁄ B ™ A ™ Cl A and A ⁄ B ™ B ™ Cl B, and hence
A ⁄ B is a subset of the closed set (Cl A) ⁄ (Cl B). But by definition of
closure this means that

Cl(A ⁄ B) ™ (Cl A) ⁄ (Cl B) . ˙

Theorem B12 Let (X, d) be a metric space and suppose A ™ X. Then

(a) Cl A = A ¡ Aæ.
(b) Cl A = Int A ¡ Bd A.
(c) Bd A = Bd Aü.
(d) Int A = Cl A - Bd A.

Proof (a) Assume x ∞ Cl A but that x ! A. We first show that x ∞ Aæ. Let U
be any open set containing x. If U ⁄ A = Å, then Uc is a closed superset of A
which implies Cl A ™ Uc. But this contradicts the assumption that x ∞ U since
it was assumed that x ∞ Cl A. Therefore, since x ! A, we must have

(U - {x}) ⁄ A ≠ Å

so that x ∞ Aæ. This shows that Cl A ™ A ¡ Aæ.

Now assume that x ∞ A ¡ Aæ. If x ∞ A, then obviously x ∞ Cl A (since
A ™ Cl A), so suppose x ∞ Aæ. We will show that x is contained in any closed
superset of A. Let F be any closed superset of A not containing x. Then Fc is
an open set containing x and such that (Fc - {x}) ⁄ A = Å which says that
x ! Aæ, a contradiction. Thus x is contained in any closed superset of A so
that x ∞ Cl A. Since this shows that A ¡ Aæ ™ Cl A, it follows that A ¡ Aæ =
Cl A.
(b) We first suppose that x ∞ Cl A but x ! Bd A. Since x ! Bd A, there
exists an open set U containing x such that either U ™ A or U ™ Ac. If it were
true that U ™ Ac, then Uc would be a closed superset of A (see Example 0.1)
so that Cl A ™ Uc which contradicts the assumption that x ∞ Cl A. We must
therefore have U ™ A, and hence x ∞ Int A. Since the assumption that x ∞ Cl
A but x ! Bd A led to the requirement that x ∞ Int A, we must have

Cl A ™ Int A ¡ Bd A .
 SEQUENCES AND SERIES 705

Now assume x ∞ Int A ¡ Bd A, but that x ! Cl A. Note Int A ™ A ™ Cl A

so that x ! Int A, and hence it must be true that x ∞ Bd A. However, since
x ! Cl A, (Cl A)c is an open set containing x with the property that (Cl A)c ⁄
A = Å. But this says that x ! Bd A which contradicts our original assump-
tion. In other words, we must have Int A ¡ Bd A ™ Cl A, so that

Cl A = Int A ¡ Bd A .

(c) If x ∞ Bd A and U is any open set containing x, then U ⁄ A ≠ Å and

U ⁄ Ac ≠ Å. But A = (Ac)c so that we also have U ⁄ (Ac)c ≠ Å. Together
with U ⁄ Ac ≠ Å, this shows that x ∞ Bd Ac. Reversing the argument shows
that if x ∞ Bd Ac, then x ∞ Bd A. Hence Bd A = Bd Ac.
(d) This will follow from part (b) if we can show that Int A ⁄ Bd A = Å.
Now suppose that x ∞ Int A ⁄ Bd A. Then since x ∞ Bd A, it must be true
that every open set containing x intersects Ac. But this contradicts the
assumption that x ∞ Int A (since by definition there must exist an open set U
containing x such that U ™ A), and hence we must have Int A ⁄ Bd A = Å. ˙

It should be remarked that some authors define Bd A as Cl A - Int A so

that our definition of Bd A follows as a theorem. This fact, along with some
additional insight, is contained in the following theorem.

Theorem B13 Let A be a subset of a metric space (X, d), and suppose x ∞
X. Then
(a) x ∞ Int A if and only if some neighborhood of x is a subset of A.
(b) x ∞ Cl A if and only if every neighborhood of x intersects A.
(c) Bd A = Cl A - Int A.

Proof (a) If x ∞ Int A, then by definition of Int A there exists a neighbor-

hood U of x such that U ™ A. On the other hand, if there exists an open set U
such that x ∞ U ™ A, then x ∞ Int A.
(b) By Theorem B12(a), Cl A = A ¡ Aæ. If x ∞ Cl A and x ∞ A, then
clearly every neighborhood of x intersects A. If x ∞ Cl A and x ∞ Aæ, then
every neighborhood of x also intersects A. Conversely, suppose that every
neighborhood of x intersects A. Then either x ∞ A or x ∞ Aæ, and hence x ∞
A ¡ Aæ = Cl A.
(c) By definition, x ∞ Bd A if and only if every open set containing x
contains both points of A and points of Aü. In other words, x ∞ Bd A if and
only if every open set containing x contains points of A but is not a subset of
A. By parts (a) and (b), this is just Bd A = Cl A - Int A. ˙
706 APPENDIX B

Example B2 An elementary fact that will be referred to again is the follow-

ing. Let A be a nonempty set of real numbers that is bounded above. Then by
the Archimedean axiom, A has a least upper bound b = sup A. Given ´ > 0,
there must exist x ∞ A such that b - ´ < x < b, for otherwise b - ´ would be an
upper bound of A. But this means that any neighborhood of b intersects A,
and hence b ∞ Cl A. Thus b ∞ A if A is closed. ∆

Our next example yields an important basic result.

Example B3 Let X = ® with the standard (absolute value) metric, and let
Œ ™ ® be the subset of all rational numbers. In Theorem 0.4 it was shown that
given any two distinct real numbers there always exists a rational number
between them. This may also be expressed by stating that any neighborhood
of any real number always contains a rational number. In other words, we
have Cl Œ = ®. ∆

From Theorems B10 and B12(a), we might guess that there is a relation-
ship between sequences and closed sets. This is indeed the case, and our next
theorem provides a very useful description of the closure of a set.

Theorem B14 (a) A set A ™ (X, d) is closed if and only if for every
sequence {xn} in A that converges, the limit is an element of A.
(b) If A ™ (X, d), then x ∞ Cl A if and only if there is a sequence {xn} in
A such that xn ‘ x.

Proof (a) Suppose that A is closed, and let xn ‘ x. Since any neighborhood
of x must contain all xn for n sufficiently large, it follows from Theorem B10
that x ∞ A.
Conversely, assume that any sequence in A converges to an element of A,
and let x be any accumulation point of A. We will construct a sequence in A
that converges to x. To construct such a sequence, choose xn ∞ B(x, 1/n) ⁄ A.
This is possible since x is an accumulation point of A. Then given ´ > 0,
choose N ˘ 1/´ so that xn ∞ B(x, ´) for every n ˘ N. Hence xn ‘ x so that x ∞
A. Theorem B10 then shows that A is closed.
(b) This is Exercise B.4. ˙

If (X, d) is a metric space, then A ™ X is said to be somewhere dense if

Int(Cl A) ≠ Å. The set A is said to be nowhere dense if it is not somewhere
dense. If Cl A = X, then A is said to be dense in X.
 SEQUENCES AND SERIES 707

Example B4 Let X = ® with the usual metric. The set A = [a, b) has closure
Cl A = [a, b], and therefore Int(Cl A) = (a, b) ≠ Å. Hence A is somewhere
dense. Example B3 showed that the set Œ is dense in ®. Now let A = Û, the set
of all integers. Ûc = ® - Û is the union of open sets of the form (n, n + 1)
where n is an integer, and hence Û is closed. It should also be clear that Ûæ =
Å since there clearly exist deleted neighborhoods of any integer that do not
contain any other integers. By Theorem B13(a), we also see that Int(Cl Û) =
Int Û = Å so that Û is nowhere dense. ∆

Theorem B15 A subset A of a metric space (X, d) is dense if and only if

every open subset U of X contains some point of A.

Proof Suppose A is dense so that Cl A = X. If U ™ X is open, then the fact

that any x ∞ U ™ Cl A implies that every neighborhood of x intersects A
(Theorem B13). In particular, U is a neighborhood of x so that U ⁄ A ≠ Å.
On the other hand, suppose that every open set U intersects A. If x ∞ X, then
every neighborhood U of x must intersect A so that x ∞ Cl A. Since x was
arbitrary, it must be true that Cl A = X. ˙

After this topological digression, let us return to sequences of numbers.

Given a sequence {xn}, we may consider a sequence {nk} of positive integers
that forms a subset of Û+ such that n1 < n2 < ~ ~ ~ . The corresponding subset
{xnÉ} of {xn} is called a subsequence of {xn}. If {xnÉ} converges, then its
limit is called a subsequential limit of {xn}. From the definitions, it should
be clear that any cluster point of {xn} is the limit of a convergent subse-
quence. It should also be clear that a sequence {xn} converges to x if and only
if every subsequence also converges to x (see Exercise B.5).

Theorem B16 The set of all subsequential limits of a sequence {xn} in a

metric space (X, d) forms a closed subset of X.

Proof Let S be the set of all subsequential limits of {xn}. If y is an accumu-

lation point of S, we must show that y ∞ S (Theorem B10), and hence that
some subsequence of {xn} converges to y. Choose n1 such that xnè ≠ y (why
can this be done?), and let ∂ = d(xnè, y). Now suppose that n1, n2, . . . , nk-1
have been chosen. Since y is an accumulation point of S, there exists z ∞ S,
z ≠ y, such that d(z, y) < 2-k∂. But z ∞ S implies that z is a subsequential
limit, and hence there exists nk > nk-1 such that d(z, xnÉ) < 2-k∂. Therefore,
for each k = 1, 2, . . . we have

d(xnÉ, y) ¯ d(xnÉ, z) + d(z, y) < 21-k∂

708 APPENDIX B

so that the subsequence {xnÉ} converges to y. ˙

Given a sequence {añ} of complex numbers (which can be regarded as

points in ®2), we may define the infinite series (generally called simply a
series)
!
" an = a1 + a2 +!
i=1
as the sequence {sn} where
n
sn = ! ak !!.
k=1
If the sequence {sñ} converges to a number s, we say that the series
converges and we write this as Ín~=1añ = s. Note that the number s is the limit
of a sequence of partial sums. For notational convenience, the lower limit in
the sum may be taken to be 0 instead of 1, and we will frequently write just
Íañ when there is no danger of ambiguity and the proper limits are under-
stood.
The reader should note that Ín~=1 añ stands for two very different things.
On the one hand it is used to stand for the sequence {sn} of partial sums, and
on the other hand it stands for limn‘Ÿ sn. This is a common abuse of notation,
and the context usually makes it clear which meaning is being used.
We have seen that any convergent sequence is Cauchy, and in Theorem
B8 we showed that any Cauchy sequence in ®n converges. Thus, a sequence
in ®n converges if and only if it is Cauchy. This is called the Cauchy
criterion. Since the convergence of a series is defined in terms of the
sequence of partial sums, we see that the Cauchy criterion may be restated as
follows.

Theorem B17 A series of numbers Íañ converges if and only if given ´ > 0,
there exists an integer N such that m ˘ n ˘ N implies

m
! ak < " !!.
k=n

Proof If the series Íañ converges, then the sequence {sÉ} of partial sums
k
sÉ = !n=1 añ converges, and hence {sÉ} is Cauchy. Conversely, if the sequence
of partial sums sÉ is Cauchy, then {sÉ} converges (Theorem B8). In either
case, this means that given ´ > 0, there exists N such that p ˘ q ˘ N implies
 SEQUENCES AND SERIES 709

p q p
s p ! sq = " ak ! " ak = " ak < # !!.
k=1 k=1 k=q+1

The result now follows by choosing m = p and n = q + 1. ˙

Another useful way of stating Theorem B17 is to say that a series Íañ
converges if and only if given ´ > 0, there exists N such that k ˘ N implies
that \ak + ~ ~ ~ + ak+p\ < ´ for all positive integers p = 0, 1, 2, . . . .

Corollary If Íañ converges, then given ´ > 0 there exists N such that \añ\ < ´
for all n ˘ N. In other words, if Íañ converges, then limn‘Ÿ añ = 0.

Proof This follows from Theorem B17 by letting m = n. ˙

While this corollary says that a necessary condition for Íañ to converge is
that añ ‘ 0, this is not a sufficient condition (see Example B5 below).
If we have a series of nonnegative real numbers, then each partial sum is
clearly nonnegative, and the sequence of partial sums forms a non-decreasing
sequence. Thus, directly from Theorem B9 we have the following.

Theorem B18 A series of nonnegative real numbers converges if and only if

its partial sums form a bounded sequence.

One consequence of this theorem is the next result.

Theorem B19 Suppose Ían is a series such that aè ˘ aì ˘ ~ ~ ~ ˘ 0. Then Ían

converges if and only if
!
" 2 k a2 k = a1 + 2a2 + 4a4 +!
k=0
converges.

Proof Let sn = aè + aì + ~ ~ ~ + añ and let tk = aè + 2aì + ~ ~ ~ + 2k a2k . Since all

terms in the series Ían are nonnegative we may write for n < 2k

sn ! a1 + (a2 + a3 ) + (a4 + a5 + a6 + a7 ) +!+ (a2 k +!+ a2 k+1 "1 )

since this is just adding the nonnegative term a2k+1 + ~ ~ ~ + a2k+1-1 to s2k ˘ sñ.
But {añ} is a decreasing sequence, so noting that the last term in parentheses
consists of 2k terms, we have
710 APPENDIX B

sn ! a1 + 2a2 + 4a4 +!+ 2 k a2 k = t k !!.

Similarly, if n > 2k we have

sn ! a1 + a2 + (a3 + a4 ) +!+ (a2 k"1 +1 +!+ a2 k )

! (1/2)a1 + a2 + 2a4 +!+ 2 k"1 a2 k
= (1/2)t k !!.

We have now shown that n < 2k implies sñ ¯ tk, and n > 2k implies 2sñ ˘ tk.
Thus the sequences {sñ} and {t k} are either both bounded or both unbounded.
Together with Theorem B18, this completes the proof. ˙

The interesting aspect of this theorem is that the convergence of Íañ is

determined by the convergence of a rather “sparse” subsequence.

Example B5 Let us show that the series Ín-p converges if p > 1 and
diverges if p ¯ 1. Indeed, suppose p > 0. Then by Theorem B19 we consider
the series
# #
$ 2 k ! 2"kp = $ 2 k(1" p) !!.
k=0 k=0
By the corollary to Theorem B17, we must have 1 - p < 0 so that p > 1. In this
case, Í 2k(1-p) = Í(21-p)k is a geometric series which converges as in
Example B1, and hence Theorem B19 shows that Ín-p converges for p > 1. If
p ¯ 0, then Í n-p diverges by the corollary to Theorem B17. ∆

If we are given a series Ían, we could rearrange the terms in this series to
obtain a new series Íaæñ. Formally, we define this rearrangement by letting
{kñ} be a sequence in which every positive integer appears exactly once. In
other words, {kñ} is a one-to-one mapping from Û+ onto Û+. If we now define
aæñ = akñ for n = 1, 2, . . . , then the corresponding series Íaæñ is called a
rearrangement of the series Ían.
For each of the series Ían and Íaæn, we form the respective sequences of
partial sums {sk} and {sæk}. Since these sequences are clearly different in
general, it is not generally the case that they both converge to the same limit.
While we will not treat this problem in any detail, there is one special case
that we will need. This will be given as a corollary to the following theorem.
A series Íañ is said to converge absolutely if the series Í\añ\ converges.

Theorem B20 If Ían converges absolutely, then Ían converges.

 SEQUENCES AND SERIES 711

Proof Note |Ík ˜=n ak| ¯ Ík ˜= n \ak\ and apply Theorem B17. ˙

Corollary If Ían converges absolutely, then every rearrangement of Íañ

converges to the same sum.

Proof Let Íaæñ be a rearrangement with partial sums sæÉ. Since Ían con-
verges absolutely, we may apply Theorem B17 to conclude that for every ´ >
0 there exists N such that m ˘ n ˘ N implies
m
! ai < " !!. (*)
i=n
Using the notation of the discussion preceding the theorem, we let p ∞ Û+ be
such that the integers 1, . . . , N are contained in the collection kè , . . . , kp
(note that we must have p ˘ N). If for any n > p we now form the difference
sñ - sæñ, then the numbers aè, . . . , aN will cancel out (as may some other num-
bers if p > N) and hence, since (*) applies to all m ˘ n ˘ N, we are left with
\sñ - sæñ\ < ´. This shows that {sñ} and {sæñ} both converge to the same sum
(since if sn ‘ s, then \sæn - s\ ¯ \sæn - sn\ + \sn - s\ < 2´ which implies that
sæn ‘ s also). ˙

We remark that Theorems B17 and B20 apply equally well to any com-
plete normed space if we replace the absolute value by the appropriate norm.
Before presenting any examples of series, we first compute the limits of
some commonly occurring sequences of real numbers.

Theorem B21 (a) If p > 0, then limn‘Ÿ 1/np = 0.

(b) If p > 0, then limn‘Ÿ p1/n = 1.
(c) limn‘Ÿ n1/n = 1.
(d) If p > 0 and r is real, then limn‘Ÿ nr/(1 + p)n = 0.
(e) If \x\ < 1, then limn‘Ÿ xn = 0.

Proof (a) Given ´ > 0, we seek an N such that n ˘ N implies 1/np < ´. Then
choose N ˘ (1/´)1/p.
(b) If p = 1, there is nothing to prove. For p > 1, define xñ = p1/n - 1 > 0
so that by the binomial theorem (Example 0.7) we have

p = (1 + xñ)n ˘ 1 + nxn .

Thus 0 < xñ ¯ (p - 1)/n so that lim xñ = 0, and hence lim p1/n = 1. If 0 < p < 1,
we define yñ = (1/p)1/n - 1 > 0. Then
712 APPENDIX B

p = (1 + yñ)n ˘ 1 + nyñ

so that yñ ‘ 0, and hence we again have lim p1/n = 1.

(c) Let xñ = n1/n - 1 ˘ 0, so that using only the quadratic term in the
binomial expansion yields

"n% n(n (1) 2

n = (1+ xn )n ! $ ' xn 2 = xn !!.
#2& 2

Thus (for n ˘ 2) we have 0 ¯ xñ ¯ [2/(n - 1)]1/2 so that xñ ‘ 0. Therefore

lim n1/n = 1.
(d) Let k be any integer > 0 such that k > r. Choosing the kth term in the
binomial expansion we have (since p > 0)

"n % n(n (1) ! (n ( (k (1)) k

(1+ p)n ! $ ' p k = p !!.
# p& k!

If we let n > 2k, then k < n/2 so that n > n/2, n - 1 > n/2, . . . , n - (k - 1) > n/2
and hence (1 + p)n > (n/2)kpk/k! . Thus (for n > 2k)

nr 2 k k! r!k
0< < n !!.
(1+ p)n pk

Since r - k < 0, it follows that nr-k ‘ 0 by (a).

(e) Choose r = 0 in (d). ˙

Corollary If N > 0 is any finite integer, then limn‘Ÿ (nN)1/n = 1.

Proof (nN)1/n = nN/n = (n1/n)N so that (by Theorem B2(c))

lim(n1/n)N = (lim n1/n)N = 1N = 1 . ˙

Example B6 The geometric series Ín~=0 xn converges for \x\ < 1 and
diverges for \x\ ˘ 1. Indeed, from elementary algebra we see that (for x ≠ 1)

1 ! x n+1
1+ x + x 2 + ! + x n = !!.
1! x

If \x\ < 1, we clearly have lim xn+1 = 0, and hence

 SEQUENCES AND SERIES 713

!
1
" x n = 1 # x !!.
n=0
If \x\ > 1, then \x\n+1 ‘ Ÿ and the series diverges. In the case that \x\ = 1, we
see that xn ‘ Ö 0 so the series diverges. ∆

Let {xn} be a sequence of real numbers. In general this sequence may

have many cluster points, in which case it will not converge to a definite limit.
Now define the sequences
Un = supk˘n xk
and
Ln = infk˘n xk .

Note that Un is a decreasing sequence and Ln is an increasing sequence. If å is

the largest cluster point of {xn}, then clearly the Uñ will approach å as n
increases. Furthermore, no matter how large n gets, Uñ will always remain ˘
å. Similarly, if ∫ is the smallest cluster point of {xñ}, then all the Lñ must be
¯ ∫. This situation is represented schematically in the figure below.

Lè L2 ~ ~ ~ Ln Un ~~~ U2 U1
| | |Â Â| | | ®
∫ å

Let U = infnUn. By Theorem B4 and the remarks following it we see that

Un converges to U. The limit U is called the upper limit (or limit superior)
of the sequence {xn} and will be denoted by xõ. In other words,

xõ = infn supk˘n xk = limn‘Ÿ supk˘n xk .

The upper limit is frequently also written as lim sup xn.

Similarly, Ln converges to L = supn Ln. The limit L is called the lower
limit (or limit inferior) of the sequence {xn}, and is denoted by xÕ. Thus

xÕ = supn infk˘n xk = limn‘Ÿ infk˘n xk

which is also written as lim inf xn. Note that either or both xõ and xÕ could be
±Ÿ.

Theorem B22 If xñ ¯ yñ for all n greater than or equal to some fixed N, then
lim sup xñ ¯ lim sup yñ and lim inf xñ ¯ lim inf yñ.
714 APPENDIX B

Proof This is Exercise B.6. ˙

We have already remarked that in general a sequence may have many (or
no) cluster points, and hence will not converge. However, suppose {xn} con-
verges to x, and let lim Un = U. We claim that x = U.
To see this, we simply use the definitions involved. Given ´ > 0, we may
choose N such that for all n ˘ N we have both \x - xn\ < ´ and \U - Un\ < ´.
Since UN = supk˘N xk , we see that given this ´, there exists k ˘ N such that
UN - ´ < xk or UN - xk < ´. But then we have

\U - x\ ¯ \U - UN\ + \UN - xk\ + \xk - x\ < 3´

which proves that U = x. In an exactly analogous way, it is easy to prove that

L = lim Ln = x (see Exercise B.7). We have therefore shown that xn ‘ x
implies lim sup xn = lim inf xn = x. That the converse of this statement is also
true is given in the next theorem. It should be clear however, that all but a
finite number of terms in the sequence {xn} will be caught between U and L,
and hence if U = L it must be true that xn ‘ x = U = L.

Theorem B23 A real-valued sequence {xñ} converges to the number x if

and only if lim sup xñ = lim inf xñ = x.

Proof Let Un = supk˘n xk and Ln = infk˘n xk , and first suppose that lim Uñ =
lim Lñ = x. Given ´ > 0, there exists N such that \Uñ - x\ < ´ for all n ˘ N, and
there exists M such that \Lñ - x\ < ´ for all n ˘ M. These may be written as
(see Example 0.6) x - ´ < Uñ < x + ´ for all n ˘ N, and x - ´ < Lñ < x + ´ for
all n ˘ M. But from the definitions of Uñ and Lñ we know that xn ¯ Un and
Ln ¯ xn. Hence xn < x + ´ for all n ˘ N and x - ´ < xn for all n ˘ M. Therefore
\xn - x\ < ´ for all n ˘ max{N, M} so that xn ‘ x.
The converse was shown in the discussion preceding the theorem. ˙

Define S to be the set of all cluster points of {xn}. Since any cluster point
is the limit of some subsequence, it follows that S is just the set of all
subsequential limits of {xn}. From the figure above, we suspect that sup S = xõ
and inf S = xÕ. It is not hard to prove that this is indeed the case.

Theorem B24 Let {xñ} be a sequence of real numbers and let S, xõ and xÕ be
defined as above. Then sup S = xõ and inf S = xÕ.

Proof This is Exercise B.8. ˙

 SEQUENCES AND SERIES 715

Example B7 Let xñ = (-1)n/(1 + 1/n). Then it should be clear that we have

lim sup xñ = 1 and lim inf xñ = -1. ∆

Our next theorem will be very useful in proving several tests for the con-
vergence of series.

Theorem B25 Let {xñ} be a sequence of real numbers, let S be the set of all
subsequential limits of {xn}, and let xõ = lim sup xn and xÕ = lim inf xn. Then
(a) xõ ∞ S.
(b) If r > xõ, then there exists N such that n ˘ N implies xñ < r.
(c) xõ is unique.
Of course, the analogous results hold for xÕ as well.

Proof We will show only the results for xõ, and leave the case of xÕ to the
reader.
(a) Since S (the set of all subsequential limits) lies in the extended number
system, we must consider three possibilities. If -Ÿ < xõ < +Ÿ, then S is
bounded above so that at least one subsequential limit exists. Then the set S is
closed (by Theorem B16), and hence xõ = sup S ∞ S (see Example B2).
If xõ = +Ÿ, then S is not bounded above so that {xñ} is not bounded above.
Thus there exists a subsequence {xnÉ} such that xnÉ ‘ +Ÿ. But then +Ÿ ∞ S
so that xõ ∞ S.
If xõ = -Ÿ, then there is no finite subsequential limit (since xõ is the least
upper bound of the set of such limits), and hence S consists solely of the
element -Ÿ. This means that given any real M, xñ > M for at most a finite
number of indices n so that xñ ‘ -Ÿ, and hence xõ = -Ÿ ∞ S.
(b) If there existed an r > xõ such that xñ ˘ r for an infinite number of n,
then there would be a subsequential limit xæ of {xñ} such that xæ ˘ r > xõ. This
contradicts the definition of xõ.
(c) Let xõ and yõ be distinct numbers that satisfy (a) and (b), and suppose
xõ < yõ. Let r be any number such that xõ < r < yõ (that such an r exists was shown
in Theorem 0.4). Since xõ satisfies (b), there exists N such that xñ < r for all n ˘
N. But then yõ can not possibly satisfy (a). ˙

We now have the background to prove three basic tests for the conver-
gence of series.

Theorem B26 (a) (Comparison test) If Íbñ converges, and if \añ\ ¯ bñ for
n ˘ Nà (Nà fixed), then Íañ converges. If Ícñ diverges, and if añ ˘ cñ ˘ 0 for
n ˘ Nà, then Íañ diverges.
(b) (Root test) Given the series Íañ, let aõ = lim sup\añ\1/n. If aõ < 1, then
Íañ converges, and if aõ > 1, then Íañ diverges.
716 APPENDIX B

(c) (Ratio test) The series Íañ converges if lim sup \an+1/añ\ < 1, and
diverges if \an+1/añ\ ˘ 1 for n ˘ N (N fixed).

Proof (a) Given ´ > 0, there exists N ˘ Nà such that m ˘ n ˘ N implies that
|Ík ˜=n bÉ| < ´ (Theorem B17). Hence Íañ converges since

m m m m
! ak " ! ak " ! bk = ! bk < # !!.
k=n k=n k=n k=n

By what has just been shown, we see that if 0 ¯ cñ ¯ añ and Íañ converges,
then Ícñ must also converge. But the contrapositive of this statement is then
that if Ícñ diverges, then so must Íañ.
(b) First note that aõ ˘ 0 since \añ\1/n ˘ 0. Now suppose that aõ < 1. By
Theorem B25(b), for any r such that aõ < r < 1, there exists N such that n ˘ N
implies \añ\1/n < r, and thus \añ\ < rn. But Írn converges (Example B5) so that
Íañ must also converge by the comparison test.
If aõ > 1, then by Theorems B22(a) and B14(b), there must exist a sequence
{nÉ} such that \anÉ\1/nÉ ‘ aõ . But this means that \añ\ > 1 for infinitely many n
so that añ ‘ Ö0 and Íañ does not converge (corollary to Theorem B17).
(c) If lim sup\an+1/añ\ < 1 then, by Theorem B25(b), we can find a number
r < 1 and an integer N such that n ˘ N implies \an+1/añ\ < r. We then see that

aN +1 < r aN
aN +2 < r aN +1 < r 2 aN
!
aN + p < r p aN !!.

Therefore, letting n = N + p we have

\añ\ < rn-N\aÑ\ = r-N\aÑ\rn

for n ˘ N, and hence Íañ converges by the comparison test and Example B6.
If \an+1\ ˘ \añ\ for n ˘ N (N fixed), then clearly añ ‘ Ö0 so that Íañ can not
converge (corollary to Theorem B17). ˙

Note that if aõ = 1 when applying the root test we get no information since,
for example, Í1/n and Í1/n2 both have aõ = 1 (corollary to Theorem B21), but
the first diverges whereas the second converges (see Example B5).
 SEQUENCES AND SERIES 717

Example B8 Consider the function ex ( = exp x) defined by

!
x x2 xn
e = 1+ x + + ! = "
2! n=0
n!

where x may be real or complex. To test for convergence, we observe that

\an+1/an\ = \x/(n+1)\ so that

an+1 x
lim sup = lim n!" sup k#n
an k +1
x
= lim n!"
n +1
=0

and hence the series converges by the ratio test.

It is of great use in both mathematics and physics to prove that

x # x &n
e = lim n!" %1+ ( !!.
$ n'

While this can be proved by taking the logarithm of both sides and using
l’Hospital’s rule, we shall follow a direct approach. Let xñ = (1 + x/n)n.
Expanding xñ by the binomial theorem we have (for n > 2)

n
n! " x %k
xn = ( $ '
k=0
k!(n ! k)! # n &
n(n !1) x 2 n(n !1)(n ! 2) x 3 xn
= 1+ x + + + ! + !!.
2! n 2 3! n3 nn

If we write

1 n! 1 1 n(n !1)(n ! 2) ! (n ! (n !1))

= =
n n n! n n n! nn
1 " 1 % " 2 % " n !1 %
= $1 ! ' $1 ! ' ! $1 ! '
n! # n & # n & # n &

then
718 APPENDIX B

1 " 1 % 2 1 " 1 %" 2 % 3

xn = 1+ x + $1 ! ' x + $1 ! ' $1 ! ' x
2! # n & 3! # n & # n &
1 " 1 % " 2 % " k !1 % k
+ ! + $1 ! ' $1 ! ' ! $1 ! 'x
k! # n & # n & # n &
1 " 1 % " 2 % " n !1 % n
+ ! + $1 ! ' $1 ! ' ! $1 ! ' x !!.
n! # n & # n & # n &

We now treat each xn as an infinite series by defining all terms with k > n to
be zero, and we consider the difference

.
x 1 ( " 1 % " 2 % " k !1 %+ k
e ! xn = / *1 ! $1 ! ' $1 ! ' ! $1 ! '-x !!. (*)
k=2
k! ) # n & # n & # n &,

Applying Theorem B17 to the convergent series ex = Í\x\n/n!, we see that for
fixed x and ´ > 0, we can choose an integer m sufficiently large that

! k
x
" k!
< # 2!!.
k=m+1

Writing (*) in the form Ík~ = 2 = Í k˜=2 + Ík~ =m+1 and noting that the coef-
ficient of xk in the (second) sum is ˘ 0 but ¯ 1/k!, we obtain (for n > m)

m
x 1) # 1 &# 2& # k !1 &, k
|e ! xn | " / k!+*1 ! %$1 ! n ('%$1 ! n (' ! %$1 ! (.| x | + 0 /2!! !!.
k=2
n '-

Since the sum in this expression consists of a finite number of terms, each of
which approaches 0 as n ‘ Ÿ, we may choose an N > 0 such that the sum is
less than ´/2 for n > N. Therefore, for n > N we have \ex - xñ\ < ´ which
proves that xñ ‘ ex. ∆

Exercises

1. Prove Theorem B2(b).

2. Let {xñ} and {yñ} be sequences of real numbers such that xñ ¯ yñ for all
n ˘ N where N is fixed. If xñ ‘ x and yñ ‘ y, prove that x ¯ y.
3. If A is a subset of a metric space X and x ∞ X, prove that x ∞ Ext A if
and only if x has some neighborhood disjoint from A.
 SEQUENCES AND SERIES 719

4. Prove Theorem B14(b).

5. Prove that any cluster point is the limit of a subsequence.

6. Prove Theorem B22.

7. If {xn} is a sequence of real numbers converging to x, and Ln = infk˘n xk

converges to L, show that x = L.

8. Prove Theorem B24.

9. Let ®ä denote the extended real number system, and let f: (a, b) ™ ® ‘ ®ä.
Define
limx‘y sup f(x) = inf∂ > 0 sup0< \x-y\ < ∂ f(x)

and suppose that limx‘y f(x) = L (i.e., limx‘y f(x) exists). Show

limx‘y sup f(x) = L .

[Hint: Let S∂ = sup\x-y\ < ∂ f(x) and define S = inf∂ S∂ . Then note that

\S - L\ ¯ \S - S∂\ + \S∂ - f(x)\ + \f(x) - L\ .]

10. (a) Let {xn} be a Cauchy sequence in ®n, and assume that {xn} has a
cluster point c. Prove that {xn} converges to c.
(b) Using this result, prove that any Cauchy sequence in ®n converges to
a point of ®n.

11. Show that Bd A = Cl A ⁄ Cl Ac.

12. If U ™ (X, d) is open and A ™ X is dense, show that Cl U = Cl(U ⁄ A).

13. If {xn} is a Cauchy sequence with a convergent subsequence {xnÉ}, show

that {xn} converges.
 APPENDIX C

Path Connectedness

In order to avoid having to define a general topological space, we shall phrase

this appendix in terms of metric spaces. However, the reader should be aware
that this material is far more general than we are presenting it. We assume that
the reader has studied Appendix A.
In elementary analysis and geometry, one thinks of a curve as a collection
of points whose coordinates are continuous functions of a real variable t. For
example, a curve in the plane ®2 may be specified by giving its coordinates
(x = f(t), y = g(t)) where f and g are continuous functions of the parameter t. If
we require that the curve join two points p and q, then the parameter can
always be adjusted so that t = 0 at p and t = 1 at q. Thus we see that the curve
is described by a continuous mapping from the unit interval I = [0, 1] into the
plane.
Let X be a metric space, and let I = [0, 1] be a subspace of ® with the
usual metric. We define a path in X, joining two points p and q of X, to be a
continuous mapping f: I ‘ X such that f(0) = p and f(1) = q. This path will be
said to lie in a subset A ™ X if f(I) ™ A. It is important to realize that the path
is the mapping f, and not the set of image points f(I). The space X is said to be
path connected if for every p, q ∞ X there exists a path in X joining p and q.
If A ™ X, then A is path connected if every pair of points of A can be joined
by a path in A. (We should note that what we have called path connected is
sometimes called arcwise connected.)

720
 PATH CONNECTEDNESS 721

Let us consider for a moment the space ®n. If xá, xé ∞ ®n, then we let x”i”x”j
denote the closed line segment joining xá and xé. A subset A ™ ®n is said to be
polygonally connected if given any two points p, q ∞ A there are points xà =
p, xè, xì, . . . , xm = q in A such that ¡ i ˜=1x”i”-”1”x”i” ™ A.

x4 = q
Â
 x3

xè Â A ™ ®2

 x2
Â
xà = p

Just because a subset of ®n is path connected does not mean that it is

polygonally connected. For example, the unit circle in ®2 is path connected
since it is actually a path itself, but it is not polygonally connected.

Example C1 The space ®n is path connected. Indeed, if p ∞ ®n has coordi-

nates (x1, . . . , xn) and q ∞ ®n has coordinates (y1, . . . , yn), then we define the
mapping f: I ‘ ®n by f(t) = (f1(t), . . . , fn(t)) where fi(t) = (1 - t)xi + tyi.
This mapping is clearly continuous and satisfies f(0) = p and f(1) = q. Thus f
is a path joining the arbitrary points p and q of ®n, and hence ®n is path con-
nected. ∆

The following is a simple consequence of Theorem A5 that we shall need

for our main result (i.e., Theorem C2).

Theorem C1 Let f: (Xè, dè) ‘ (Xì, dì) and g: (Xì, dì) ‘ (X3, d3) both be
continuous functions. Then g ı f: (Xè, dè) ‘ (X3, d3) is a continuous function.

Proof If U ™ X3 is open, then the continuity of g shows that gî(U) ™ Xì is

open. Therefore (g ı f)î(U) = (fî ı gî)(U) = fî(gî(U)) is open by the con-
tinuity of f. ˙

Theorem C2 Let f be a continuous mapping from a metric space X onto a

metric space Y. Then Y is path connected if X is.
722 APPENDIX C

Proof Let xæ, yæ be any two points of Y. Then (since f is surjective) there
exist x, y ∞ X such that f(x) = xæ and f(y) = yæ. Since X is path connected,
there exists a path g joining x and y such that g(0) = x and g(1) = y. But then
f ı g is a continuous function (Theorem C1) from I into Y such that (f ı g)(0)
= xæ and (f ı g)(1) = yæ. In other words, f ı g is a path joining xæ and yæ, and
hence Y is path connected. ˙

It is an obvious corollary of Theorem C2 that if f is a continuous mapping

from the path connected space X into Y, then f(X) is path connected in Y
since f maps X onto the subspace f(X).
Bibliography

This bibliography lists those books referred to in the text as well as many of
the books that we found useful in our writing.

Abraham, R., Marsden, J. E. and Ratiu, T., Manifolds, Tensor Analysis, and
Applications, Addison-Wesley, Reading, MA, 1983.

Adler, R., Bazin, M. and Schiffer, M., Introduction to General Relativity,

McGraw-Hill, New York, 1965.

Arnold, V. I., Mathematical Methods of Classical Mechanics, Springer-

Verlag, New York, 1978.

Biggs, N. L., Discrete Mathematics, Clarendon Press, Oxford, 1985.

Bishop, R. L. and Goldberg, S. I., Tensor Analysis on Manifolds, Macmillan,

New York, 1968.

Boothby, W. M., An Introduction to Differentiable Manifolds and Riemannian

Geometry, Academic Press, New York, 1975.

723
724 BIBLIOGRAPHY

Byron, F. W. Jr., and Fuller, R. W., Mathematics of Classical and Quantum

Physics, Addison-Wesley, Reading, MA, 1969.

Curtis, C. W., Linear Algebra, Springer-Verlag, New York, 1984.

Curtis, W. D. and Miller, F. R., Differential Manifolds and Theoretical

Physics, Academic Press, Orlando, FL, 1985.

Dennery, P. and Krzywicki, A., Mathematics for Physicists, Harper & Row,
New York, 1967.

Durbin, J. R., Modern Algebra, John Wiley & Sons, New York, 1985.

Flanders, H., Differential Forms, Academic Press, New York, 1963.

Frankel, T., The Geometry of Physics, 2nd edition, Cambridge University

Press, New York, NY, 2004.

Frankel, T., Linear Algebra, unpublished lecture notes, University of

California, San Diego, 1980.

Friedberg, S. H. and Insel, A. J., Introduction to Linear Algebra with

Applications, Prentice-Hall, Englewood Cliffs, NJ, 1986.

Friedberg, S. H., Insel, A. J. and Spence, L. E., Linear Algebra, Prentice-Hall,

Englewood Cliffs, NJ, 1979.

Gemignani, M. C., Elementary Topology, 2nd edition, Addison-Wesley,

Reading, MA, 1972.

Geroch, R., Mathematical Physics, University of Chicago Press, Chicago, IL,

1985.

Goldstein, H., Classical Mechanics, 2nd edition, Addison-Wesley, Reading,

MA, 1980.

Halmos, P. R., Finite-Dimensional Vector Spaces, Springer-Verlag, New

York, 1974.

Herstein, I. N., Topics in Algebra, Xerox College Publishing, Lexington, MA,

1964.
 BIBLIOGRAPHY 725

Hoffman, K. and Kunze, R., Linear Algebra, 2nd edition, Prentice-Hall,

Englewood Cliffs, NJ, 1971.

Jackson, J. D., Classical Electrodynamics, 2nd edition, John Wiley & Sons,
New York, 1975.

Johnson, R. E., Linear Algebra, Prindle, Weber & Schmidt, Boston, MA,
1967.

Knopp, K., Infinite Sequences and Series, Dover Publications, New York,
1956.

Kolmogorov, A. N. and Fomin, S. V., Functional Analysis, Graylock Press,

Rochester, NY, 1957.

Lang, S., Analysis I, Addison-Wesley, Reading, MA, 1968.

Lang, S., Linear Algebra, 2nd edition, Addison-Wesley, Reading, MA, 1971.

Lang, S., Real Analysis, 2nd edition, Addison-Wesley, Reading, MA, 1983.

Lipschutz, S., Linear Algebra, Schaum’s Outline Series, McGraw-Hill, New

York, 1968.

Marcus, M. and Minc, H., A Survey of Matrix Theory and Matrix Inequalities,
Allyn and Bacon, Boston, MA, 1964.

Marcus, M., Finite-Dimensional Multilinear Algebra, Parts I and II, Marcel

Dekker, New York, 1973.

Marcus, M., Introduction to Linear Algebra, Dover Publications, New York,

1988.

Marion, J. B., Classical Dynamics of Particles and Systems, 2nd edition,

Academic Press, Orlando, FL, 1970.

Marsden, J. E., Elementary Classical Analysis, W. H. Freeman, San

Francisco, CA, 1974.

Misner, C. W., Thorne, K. S. and Wheeler, J. A., Gravitation, W. H. Freeman,

San Francisco, CA, 1973.

Munkres, J. R., Topology, Prentice-Hall, Englewood Cliffs, NJ, 1975.

726 BIBLIOGRAPHY

Murdoch, D. C., Linear Algebra, John Wiley & Sons, New York, 1970.

Prugovecki, E., Quantum Mechanics in Hilbert Space, 2nd edition, Academic

Press, New York, 1981.

Reed, M. and Simon, B., Functional Analysis, Revised and Enlarged edition,
Academic Press, Orlando, FL, 1980.

Roach, G. F., Green’s Functions, 2nd edition, Cambridge University Press,

Cambridge, 1982.

Royden, H. L., Real Analysis, 2nd edition, Macmillan, New York, 1968.

Rudin, W., Functional Analysis, 2nd edition, McGraw-Hill, New York, 1974.

Rudin, W., Principles of Mathematical Analysis, 3rd edition, McGraw-Hill,

New York, 1976.

Ryder, L. H., Quantum Field Theory, Cambridge University Press,

Cambridge, 1985.

Schutz, B., Geometrical Methods of Mathematical Physics, Cambridge

University Press, Cambridge, 1980.

Shankar, R., Principles of Quantum Mechanics, Plenum Press, New York,

1980.

Simmons, G. F., Introduction to Topology and Modern Analysis, McGraw-

Hill, New York, 1963.

Talman, J. D., Special Functions, W. A. Benjamin, New York, 1968.

Taylor, J. R., Scattering Theory, John Wiley & Sons, New York, 1972.

Tung, W., Group Theory in Physics, World Scientific, Philadelphia, PA, 1985.
Index

Absolute convergence 710 Basis 77

Absolute value 15, 20 dual 446
Accumulation point 622, 690, 700 g-orthonormal 613
Adjoint negatively oriented 608
classsical 192 ordered 79
of a linear operator 493, 673 positively oriented 608
of a matrix 183, 384 similarly oriented 606
Algebra 227 Basis 1-forms 447, 554
Algebraic number 16 Bessel’s inequality 106, 657, 658
Algebraically closed field 276 Bijective 5
Alternation mapping 556 Bilinear form 461
Annihilator 323 alternating 467
of a set of vectors 453 antisymmetric 467
Anti-derivation 572 nonnegative 479
Antilinear mapping 452 polar form of 472
Archimedean axiom 13 positive definite 479
Archimedean property 13 positive semidefinite 479
Arcwise connected 720 symmetric 469
Augmented matrix 117 Bilinear functional 448
Automorphism 50 Bilinear map 461
Axiom of choice 9 Binomial coefficient 18
Binomial theorem 18
Block matrix 204
Banach space 630 Bolzano-Weierstrass theorem 691, 700
Basic Jordan block 378, 420 Boundary 703

727
728 INDEX

Bounded 625 Contravariant vector 545

mapping 692 Convergent sequence of vectors 640
sequence 697 Convex set 650
set 691, 697 Coordinate function 633
Bounded linear map 635 Coordinates 3, 79
Coprime 28
Coset 61
Cancellation law 31 Covariant vector 545
Canonical form 296 Cramer’s rule 200
Cardinality 11 Critical
Cartesian product 3 point 523
Cauchy criterion 708 value 523
Cauchy sequence 699 Cyclic group 290
Cauchy-Binet theorem 214 order of 290
Cauchy-Schwartz inequality 100, 621 Cyclic subspace 336, 432
Cauchy’s inequality 643 Cyclic vector 433
Cayley transform 508
Cayley-Hamilton theorem 312, 378, 395
Cayley’s theorem 52 De Moivre’s theorem 24
Chain 8 De Morgan formulas 3
Characteristic Dense 13, 647, 706
equation 309 nowhere 706
matrix 309 somewhere 706
of integral domain 58 Derived set 703
polynomial 309 Determinant 171, 600
root 302 of a linear operator 246
value 302 order 171
vector 302 Determinantal divisors 397
Closed ball 683 Diagonal subgroup 34
Closed interval 14 Diagonalizable 248, 317
Closed set 649, 683 Differential of a function 551
Closure of a set 702 Direct sum 90
Cluster point 700 external 89
Cofactor 187 internal 88
Column space 128 of matrices 333
Compact space 686 of operators 332
Companion matrix 412 Division algorithm 26, 258
Comparable 8 Division ring 54
Comparison test 715 Double dual 222, 452
Complete metric space 630, 699 Dual basis 222
Complete orthonormal set 660 Dual space 222, 446, 665
Completely reducible 333
Completeness relation 113
Complex conjugate 20 Eigenspace 306
Complex number 19 Eigenvalue 302
imaginary part 19 spectrum 346, 524
modulus 20 Eigenvector 302
nth root 24 Elementary divisor theorem 439
real part 19 Elementary divisor 398
Conductor 335 Elementary matrix 163
Congruent matrices 465 Elementary row operations 121
Conjugate exponents 641 Empty set 2
Conjugate linear 452 Equivalence class 9
Conjugate space 665 Equivalence relation 9
Continuous 623, 684 Equivalent norms 629
Continuous linear map 635 Equivalent representations 335
 INDEX 729

Euclidean algorithm 27, 265 Heine-Borel theorem 687, 691

Euclidean space 99, 508 Hermitian adjoint 383
Expansion by minors 189 Hermitian form 481
(see also Laplace expansion) associated quadratic form 482
Extended real number system 14 nonnegative semidefinite 482
Exterior of a set 703 positive definite 482
Exterior algebra 574 Hermitian inner product 620
Exterior product 563 Hilbert basis 660
Exterior r-forms 554 Hilbert space 630, 640
Homomorphic groups 49
Homomorphism 49
Factor group 64 kernel of 50
Factor theorem 262 vector space 79
Factorial 18 HoÜlder’s inequality 642
Field 54 Hypercompanion matrix 420
characteristic of 58 Hyperplane 457
Field of quotients 284 Hyperspace 457
Finite field
characteristic of 291
order of 288 Ideal 59
Finite-dimensional 77 generated by 275
Formal linear combinations 578 unit 275
Fourier coefficients 106, 655 Idempotent 157, 232, 353
Free variables 142 Identity mapping 6, 227
Free vector space 579 Identity matrix 136
Function 4 Indeterminate 115, 255
Functional 665 Index set 2
Fundamental Theorem of Algebra 276, 693 Induction 17
Fundamental Theorem of Arithmetic 29 Infimum 8
Infinite series 708
Infinity symbols 14
Gaussian elimination 124 Injective 5
Generalized permutation symbol 565 Inner product 98
Gershgorin’s theorem 307 Hermitian 98
Graded associative algebra 575 indefinite 609
Graded components 574 nondegenerate 451, 609
Gram-Schmidt process 109 Inner product space 99
Gramian 197 complex 99
Grassmann algebra 574 real 99
Greatest common divisor 26, 263 Integers
Greatest lower bound 8 modulo n 57, 285
Group 30 Integral domain 57
abelian 30 Interior of a set 702
center of 67 Interior product 572
class 35 Invariant direct sum decomposition 332
conjugate elements of 35 Invariant factors 398
cyclic 291 Invariant subspace 329
direct product 34 Invariant volume element 617
finite 31 Irrational number 12
homomorphism 49 Irreducible representation 335
infinite 31 Isolated point 622
multiplication 30 Isometry 113, 498
order of 31 Isomorphic 59, 79
representation 33, 335 Isomorphic groups 50
g-volume 615 Isomorphism 50, 79
Hausdorff property 687 inner product space 501
730 INDEX

nullity 225
orientation preserving 608
Jacobian 599 range 225
Jointly continuous 648 rank 225
Jordan form 376, 422 reducible 332
uniqueness of 427 restriction of 330
singular 229
volume preserving 608
Kronecker delta 105 Linearly dependent 75
Kronecker product 581 Linearly independent 75
Lorentz frame 613
Lorentz transformation 616
Lagrange interpolation formula 280 Lower bound 8
Lagrange’s theorem 61 Lower limit 713
Laplace expansion 573 Lowering the index 610
Law of cosines 96
Law of inertia 478
Least common multiple 29, 272 Mapping 4
Least upper bound 8 alternating 180
Least upper bound property 12 associative 6
Left identity 31 bilinear 578
Left inverse 31, 157 commutative 6
Left zero divisor 163 composite 6
Levi-Civita domain 4
symbol 560 image 4
tensor 560 inverse 5
Limit 625 inverse image 4
Limit inferior 713 multilinear 180, 544
Limit point 700 one-to-one 5
Limit superior 713 onto 5
Linear extension theorem 639 range 4
Linear algebra 227 restriction 4
Linear combination 72 Matrix 117
Linear equations 115 adjoint 183, 383
coefficients of 115 anticommutator 156, 184
constant term 115 antisymmetric 156, 184, 469
solution vector 115 block 204
system of 116 canonical form 169
equivalent 118 classical adjoint 191
homogeneous 138 column rank 128
nonhomogeneous 138 columns 117
nontrivial solution 138 commutator 154, 156
trivial solution 138 conjugate transpose 183
Linear functional 221 derogatory 409
Linear manifold 649 diagonal 154, 165
Linear operators 227 direct product 581
Linear span 72 distinguished elements 124
Linear transformation 79, 215 equivalent 515
diagonalizable 318 equivalent over P 393
image 224 Hermitian 383, 482
inverse 228 Hermitian adjoint 482
invertible 228 idempotent 157
kernel 225 inverse 157
matrix representation of 235 invertible 157
negative of 219 irreducible 427
nonsingular 229
 INDEX 731

lower-triangular 156, 162, 177

negative 148 Natural mapping 453
nilpotent 184 Natural numbers 2
nonderogatory 409 n-cell 689
nonsingular 157 Neighborhood 690
normal 383, 515 deleted 690
orthogonal 183, 249, 384, 502 Nilpotent 233
product 148 index of nilpotency 369
rank 136 operator 306, 369
reduced row-echelon form 123 Nondegenerate 462
reducible 333, 427 Nonnegative 2
row equivalent 123 Norm 619
row rank 128 Normal coordinates 489
row-echelon form 123 Normal matrix 515
row-reduced form 125 Normed vector space 101, 620
rows 117 Null space 154, 225
singular 157
size 117
skew-Hermitian 388 1-forms 447
skewsymmetric 156 Open
square 148 ball 681
sum 147 cover 686
supertriangular 376 interval 14
symmetric 156, 184, 384, 470 set 681
tensor product 581 subcover 686
trace 155, 246 Operator 490, 653
transpose 152 adjoint 673
unit 392 anti-Hermitian 508
unitarily similar 515 antisymmetric 511
unitary 183, 383, 502 Hermitian 495, 676
upper-triangular 156, 162, 177 isometric 498, 499, 677
Matrix exponential series 530 nonnegative 508
Matrix of coefficients 116 normal 509, 677
Maximal element 8 orthogonal 499, 501
Metric 104, 680 positive 508
Metric space 680 positive definite 508
complete 699 positive semidefinite 508
Metric tensor 580 self-adjoint 495, 676
contravariant 611 skew-Hermitian 508
covariant 610 symmetric 511
index of 613 unitary 499, 678
Lorentz 613 Operator norm 637
Riemannian 613 Order of a vector 440
Metric volume form 615 Ordered by inclusion 8
Minimal element 9 Orientation 606, 608
Minimal polynomial 299, 313, 326 negative 592, 608
of a vector 323 positive 592, 608
Minkowski’s inequality 643 Oriented vector space 606, 608
Minor 188 Oriented volume 592
Minor matrix 187 Orthogonal 102
Module 69 compliment 105, 620
Multilinear form 544 projection 217, 354
Multiplicity set 105
algebraic 345 Orthonormal set 105
geometric 345
732 INDEX

Parallelogram law 103, 621 Positive integers 2

Parallelpiped Positive transformation 539
base 589 Power set 7
height 589 Preimage 4
r-dimensional 589 Primary decomposition theorem 339
r-volume 589 Prime number 25
Parseval’s equation 662 Prime polynomial 262
Partial isometry 507 Principal ideal 60
Partial ordering 7 generator of 60
bounded above 8 Product 625
induced 8 Projection 157, 232, 352, 654
largest element 8 Pull-back 595
smallest element 8 Push-forward 602
upper bound 8 Pythagorean theorem 95, 103, 622
Partially ordered set 7
Partition 9
induced 10 Quadratic
Path 720 form 471
Path connected 720 diagonal representation 477
Pauli spin matrices 156 polynomial 471
P-elementary operations 390 Quaternions 280
Permutation 35 Quotient 25
even 44 Quotient group 64
odd 44 Quotient ring 64
one-line notation 36 Quotient space 362
sign of 46
two-line notation 35
Permutation group 40 Raising an index 611
cycle 41 Rank
degree of 40 of a bilinear form 465
equivalent elements in 41 of a matrix 135
orbit 41 Ratio test 717
Permutation symbol 560 Rational canonical form 416
Perpendicular 102 Rational numbers 2
Polar decomposition 540 Rayleigh quotient 513
Polar form identities 103 Rearrangement lemma 34
Polygonally connected 721 Reducible representation 333
Polynomial 253 Reflexive space 671
associates 262 Relation 7
coefficients of 253 Relatively prime 28, 263
constant 256 Remainder 25
degree of 256 Remainder theorem 261
factor of 261 Resolution of the identity 524
greatest common divisor 397 r-forms 554
irreducible 262 Riesz representation theorem 666
leading coefficient of 256 Riesz-Fischer theorem 663
minimal 300, 314, 327 Right identity 31
monic 256 Right inverse 31, 157
norm of 397 Right zero divisor 163
reducible 262 Ring 53
root 261, 297 associates 262
zero 261, 297 associative 53
Polynomial equation 261 commutative 53
solution 261 embedded 282
Polynomial function 255 extension 282
homomorphism 56
 INDEX 733

kernel of 59 Simple root 305

isomorphism 59 Smith canonical form 400
with unit element 53 Solution set 116
Ring of sets 4 Space of linear functionals 222
r-linear form 544 Space of linear transformations 220
Root 261 Spectral decomposition 524
multiplicity of 281, 305 Spectral theorem 525
Root test 717 Spectrum 346
Row canonical form 125 degenerate 346
Row space 128 Square root 15
Row-column-equivalent 169 Standard basis 79
Standard inner product 99, 620
Standard orientation 608
Scalar 69 Subdeterminant 185
Scalar mapping 301 Subgroup 33
Scalar multiplication 68 index of 62
Scalar product 94 normal 62
Scalar triple product 588 Submatrix 185, 193, 209
Schur canonical form 384 Subsequence 707
Schur’s lemma 335 Subsequential limit 707
Schwartz’s inequality 20 Subset 2
generalized 649 proper 2
Second dual 222, 452 Subspace 72, 649
Secular equation 309 closed 649
Separable 647, 695 generated by 72, 660
Sequence 696 intersection of 86
Cauchy 699 invariant 243, 329
decreasing 699 irreducible 518
increasing 699 null 618
limit of 622, 696 of a metric space 684
monotonic 699 proper 72
range 697 spacelike 618
Series 708 spanned by 72
rearrangement of 710 sum of 74, 86
Sesquilinear form 620 timelike 618
Set 2 trivial 72
closed 683 Summation convention 545
complement of 2 Sup norm 625, 629, 633
countable 11 Superdiagonal 155, 370
countably infinite 11 Superset 2
disjoint 3 Supremum 8
family of 2 Surjective 5
finite 11 Sylvester’s theorem 478
infinite 11 Symmetric group 37
intersection 3 Symmetrizing mapping 556
open 681
symmetric difference 4
uncountable 11 T-cyclic subspace 432
union 2 generated by 432
Shuffle 563 T-invariant subspace 243
Signature 477 Tensor 545
Signed permutation matrix 389 antisymmetric 553, 554
Similar matrices 184, 245 classical law of transformation 550
Similarity class 329 components 545, 547
Similarity invariants 408 contraction 552
Similarity transformation 184, 245 contravariant order 545
734 INDEX

covariant order 545 Volume forms 607

rank 545 equivalent 607
skew-symmetric 553
symmetric 553, 554
trace 552 Wedge product 462, 563
type 545 Well-defined 4
Tensor algebra 574 Well-ordered 17
Tensor product 462, 464, 547, 580 Weyl’s formula 536
Total 660
Total ordering 8
Trace 155 Zero divisor 57
Transcendental number 17 Zero mapping 219
Transition matrix 243 Zero matrix 148
orthogonal 249 Zorn’s lemma 9
Transpose
of a linear transformation 459
of a matrix 153
Transpositions 44
Triangle inequality 101
Triangular form theorem 367, 376
Two-sided inverse 157

Uniformly continuous 623

Unique factorization theorem 266
Unit (of a ring) 262
Unit cube 593
Unit matrix 392
Unit vector 99
Unitarily similar 385, 515
Unitary 183, 383, 499, 502, 678
Unitary space 99, 508
Unknowns 115
Upper limit 713

Vandermonde matrix 195

Vector 69
length of 99
lightlike 614
norm of 99
spacelike 614
timelike 614
Vector multiplication 227
Vector space 68
complex 69
dimension of 77, 83
generated by 578
infinite-dimensional 640
isometric 113
normed 101
ordinary Euclidean 613
pseudo-Euclidean 613
real 69
singular 613
Vector space homomorphism 79