2 Hilbert spaces

A complex Hilbert space H is a complete normed space over C whose norm is derived from an
inner product. That is, we assume that there is a sesquilinear form (·, ·) : H × H → C, linear in
the first variable and conjugate linear in the second, such that

(f , д) = (д, f ),

(f , f ) ≥ 0 ∀f ∈ H, and (f , f ) = 0 =⇒ f = 0.
The norm and inner product are related by

(f , f ) = ∥ f ∥ 2 .

We will always assume that H is separable (has a countable dense subset).

√
• As usual for a normed space, the distance on H is given by d(f , д) = ∥ f −д∥ = (f − д, f − д).
• The Cauchy-Schwarz and triangle inequalities,

|(f , д)| ≤ ∥ f ∥∥д∥, ∥ f + д∥ ≤ ∥ f ∥ + ∥д∥,

can be derived fairly easily from the inner product.

To prove the Cauchy-Schwarz inequality we use the fact that

z(t) = (f + tд, f + tд)

is a norm and hence positive, and in particular positive at its minimum value (as a function of
ℜ(f ,д)
t). The minimum occurs at t = − (д,д) , and if (f , д) happens to be real substituting this into
z(t) ≥ 0 immediately gives the desired inequality. For the general case, we can choose д′ to be
some unit complex number multiple of д so that (f , д′) is real, and see that the Cauchy-Schwarz
inequality for f and д is equivalent to the corresponding statement for f and д′.
The triangle inequality then comes from

f + д 2 = (f + д, f + д)
= (f , f ) + 2ℜ(f , д) + (д, д)
≤ (f , f ) + 2|(f , д)| + (д, д)
≤ f 2 + 2 f д + д 2
= ( f + д )2 .

1
 • You should have seen some examples last semester. The simplest (finite-dimensional) ex-
ample is Cn with its standard inner product. It’s worth recalling from linear algebra that if V is
an n-dimensional (complex) vector space, then from any set of n linearly independent vectors we
can manufacture an orthonormal basis e 1 , e 2 , . . . , en using the Gram-Schmidt process. In terms
of this basis we can write any v ∈ V in the form
∑
v= ai ei , ai = (v, ei )
∑
which can be derived by taking the inner product of the equation v = ai ei with ei . We also
have
∑n
∥v ∥ =
2
|ai | 2 .
i=1

• Standard infinite-dimensional examples are l 2 (N) or l 2 (Z), the space of square-summable

sequences, and L2 (Ω) where Ω is a measurable subset of Rn .

2.1 Orthogonality
We say that f , д ∈ H are orthogonal (perpendicular) if (f , д) = 0. An orthonormal set E is one
such that for all e 1 , e 2 ∈ E, we have ∥e 1 ∥ = ∥e 2 ∥ = 1 and (e 1 , e 2 ) = 0. Such a set is automatically
linearly independent. We say that E is an orthonormal basis if it is an orthonormal set, and in
addition, that the set of all finite linear combinations of elements of E is dense in H . Note that if
H is separable, then E is countable.
Exercise. Verify that a Hilbert space orthonormal basis in a finite dimensional Hilbert space is
exactly the same thing as a orthonormal basis in the sense of linear algebra.
How do we see that an orthonormal set in a separable Hilbert space has at most countably
many elements? Fix a countable dense set D. For each basis element ei pick some element
√
xi ∈ D with d(ei , xi ) < 1/2. We claim that if i , j, xi , x j ; this follows from d(ei , e j ) = 2, and
the triangle inequality. Thus we’ve found an injective function from our basis to a countable set.
(You may want to read about the Schroeder-Bernstein theorem. Which categories does it hold
in?)
Note that this argument used the Axiom of Choice — was that necessary?

Theorem 2.1 (Theorem 2.3 of Stein-Shakarchi). The following properties of an orthonormal set
E = {e 1 , e 2 , . . . } are equivalent:

2
 (i) E is an orthonormal basis.
(ii) If (f , e j ) = 0 for all j, then f = 0.
(iii) If f ∈ H and we define
∑
N
S N (f ) = (f , e j )e j ,
j=1

then S N (f ) → f as N → ∞.
∑
(iv) If a j = (f , e j ) then ∥ f ∥ 2 = j |a j | 2 .

Proof:

(i) =⇒ (ii) We’ll try to show that f = 0 by showing that it has arbitrarily small norm. Do the
∑N
only thing you can do, and begin by picking a finite linear combination д = i=1 ai ei within ϵ of
f . The trick is that we can replace (f , f ) with (f , f − д), and then by Cauchy-Schwarz we have

f 2 = (f , f − д) ≤ f f − д < ϵ f .

(ii) =⇒ (iii) Here we need

Lemma 2.2 (Bessel’s inequality). If {xi } is an orthonormal set (whether a basis or not),
∑
|(f , xi )| 2 ≤ f 2 .
i

Proof: Note that f − S N (f ) is perpendicular to S N (f ), so we have

∑
N
f 2 = f − S N (f ) 2 + S N (f ) 2 = f − S N (f ) 2 + |(f , xi )| 2
i=1

giving the result. □

Then we have
∑
∞
|(f , e j )| 2 ≤ f 2 < ∞
j=1
∑∞
and so in particular j=M |(f , e j )| 2 → 0 as M → ∞. Thus easily S N (f ) is a Cauchy sequence and
converges to something, say д. But then (ii) applied to f − д shows д = f .

(iii) =⇒ (iv) is trivial.

3
 ∑
(iv) =⇒ (i) We have f − S N (f ) 2 = f 2 − Nj=1 |(f , e j )| 2 → 0 as N → ∞, so finite linear
combinations of the ei are dense, as required. □

We see that when an orthonormal set is also a basis, Bessel’s inequality becomes an equality,
usually referred to as Parseval’s identity.
Theorem 2.3. Every (separable) Hilbert space has an orthonormal basis.
Proof: To prove this, we start with a countable dense subset { f 1 , f 2 , . . . }. In this list, let us elimi-
nate every f j that is not linearly independent from fi , i < j. Our new list may not be dense, but
it still has the property that its finite linear combinations are dense.
Next we apply the Gram-Schmidt process to the resulting list to generate an orthonormal set
e 1 , e 2 , . . . . Since the linear span of { f 1 , . . . fn } is the same as the linear span of {e 1 , . . . , en } for any
n, it is still the case that the finite linear combinations are dense, and so our orthonormal set is a
basis. □

2.2 Closed subspaces

Subspaces of a Hilbert space need not be closed. For example, the subspace of C ∞ functions inside
L2 (R) is not closed. Nor is the subspace of simple functions (those taking on only finitely many
values, except on a set of measure zero). Note that closed subspaces are Hilbert spaces in their
own right.
Closed subspaces are important due to the following two results. (These are proved in the
opposite order in the text, with a completely different proof. Either one follows quite easily from
the other.)
Given any subset E ⊂ H , the orthogonal complement E ⊥ is the set

{ f ∈ H | (f , e) = 0 for all e ∈ E}.

A fundamental property of E ⊥ is that it is always closed. This shows that H has lots of proper
closed subspaces, in contrast to many complete normed spaces.
Theorem 2.4. Let S be a closed subspace of a Hilbert space H , and let S ⊥ be its orthogonal comple-
ment. Then H = S ⊕ S ⊥ as inner product spaces.
Recall that H = S ⊕T means that every f ∈ H can be expressed uniquely as f = s+t with s ∈ S
and t ∈ T , with s and t depending continuously on f . The statement ‘as inner product spaces’
means that if f 1 = s 1 + t 1 , f 2 = s 2 + t 2 are decomposed as above, then (f 1 , f 2 ) = (s 1 , s 2 ) + (t 1 , t 2 ).
Continuity of the decomposition is a direct consequence of this.

4
Proof: To prove this we can observe first that S and S ⊥ are Hilbert spaces in their own right. So
we can choose orthonormal bases e 1 , e 2 , . . . for S and e 1′ , e 2′ , . . . for S ⊥ . Then using property (ii)
of Theorem 2.3 above, we show that the union of these two orthonormal bases is an orthonormal
basis for H . Expressing any f ∈ H in terms of this basis automatically gives a decomposition of
f as s + t, s ∈ S, t ∈ S ⊥ . Uniqueness follows easily since if f = s + t = s ′ + t ′, then s − s ′ = t ′ − t
then s − s ′ is both in S and orthogonal to it, hence s − s ′ = 0. □

The second reason that closed subspaces are important is that we have the following result:

Theorem 2.5. Let S be a closed subspace of a Hilbert space H , and let f ∈ H . Then there is a unique
point s in S closest to f , characterized by the property that s − f is orthogonal to S.

Proof: Using the decomposition H = S ⊕ S ⊥ , we write f = s + t with s ∈ S and t ∈ S ⊥ .

Notice that the distance from f to s is ∥t ∥. For any other s ′ ∈ S, say s ′ = s + x, we have
f − s ′ 2 = f − s − x 2 = f − s 2 + ∥x ∥ 2 ≥ ∥t ∥ 2 , with equality exactly when x = 0. □

2.3 Linear functionals

A continuous linear functional on H is a continuous linear map l from H to C. Continuity gives
that
∃δ > 0 such that ∥ f ∥ ≤ δ =⇒ |l(f )| ≤ 1.
Then by linearity, we have for all f
|l(f )| ≤ δ −1 ∥ f ∥.
So there exists M such that |l(f )| ≤ M ∥ f ∥, namely M = δ −1 . Such an M is called a bound on
l and l is called bounded. The least upper bound for l is written ∥l ∥. We thus see that for linear
functionals, continuity and boundedness are equivalent.
There are ‘obvious’ such functionals, given by the inner product with a fixed vector д, i.e. the
linear functional
l : f 7→ (f , д).
Notice that by Cauchy-Schwarz, we have

|l(f )| ≤ ∥ f ∥∥д∥ =⇒ ∥l ∥ ≤ ∥д∥.

Putting f = д we see that ∥l ∥ = ∥д∥.

5
Theorem 2.6 (Riesz representation theorem). The space of linear functionals is isomorphic, as a
normed space, to H itself, i.e. every continuous linear functional is given by the inner product with
a fixed vector.

Proof: Given a continuous linear functional l , 0, we look at the orthogonal complement of

its null space N , which is a closed subspace of H . This is one dimensional: Otherwise, suppose
x, y ∈ N ⊥ are linearly independent. We must have l(x) , 0 and l(y) , 0, but then l(x)
y
x
− l(y) ∈ N .
Next, choose д ∈ N ⊥ so that l(д) = ∥д∥ 2 . Using the decomposition H = N ⊕ N ⊥ , we see that
(f ,д)
any f ∈ H can be written as f = h + zд, for some h ∈ N and z ∈ C. In fact, z = (д,д) . Then
(f ,д)
l(f ) = l(h + (д,д)
д) = (f , д), as desired. □

2.4 Linear Transformations between Hilbert spaces

Let H 1 and H 2 be two Hilbert spaces. A continuous linear transformation is a linear map T : H 1 →
H 2 , continuous in the sense of metric spaces. As for linear functionals, continuity is equivalent
to boundedness (exercise!). We say that T is bounded with bound M if

∥T f ∥H2 ≤ M ∥ f ∥H1 for all f ∈ H 1 .

The infimum of these upper bounds is called the operatorn norm of T (or just the norm of T ) and
denoted ∥T ∥.
∥T f ∥
Exercise. Show that we can calculate ∥T ∥ as sup f ,0 f or as sup ∥ f ∥ =1 T f .
∥ ∥
Notice that we can also express the norm of T by

∥T ∥ = sup |(T f , д)| where f ∈ H 1 , д ∈ H 2 . (2.1)

∥ f ∥≤1,∥д∥≤1

Let’s write ∥T ∥ ′ for the supremum above. We’re going to prove ∥T ∥ = ∥T ∥ ′ by showing ∥T ∥ ′ ≤
∥T ∥ and ∥T ∥ ≤ ∥T ∥ ′.
First, if M is a bound for T , then by Cauchy-Schwarz |(T f , д)| ≤ M f д , and so ∥T ∥ ′ ≤
∥T ∥ .
Second, write T f 2 = (T f ,T f ) as f T f (T f , T f ) ≤ f T f ∥T ∥ ′. Cancelling a
f Tf
∥ ∥ ∥ ∥
factor of T f , we then have T f ≤ f ∥T ∥ ′ for all f , and so ∥T ∥ ′ is a bound for T and hence
at least ∥T ∥.
Exercise. Find a linear transformation T : H → H so ∥T ∥ > sup ∥ f ∥ ≤1 |(T f , f )|. (Hint: H = C2
will suffice.)

6
 Examples:
• A convolution operator. For example, if k ∈ L1 (R), then
∫ ∞
f (x) 7→ k(x − y)f (y) dy : L2 (R) → L2 (R).
−∞

• Integration on L2 ([0, 1]): ∫ x

f (x) 7→ f (s) ds.
0
• Let S be a closed subspace in H . We write H = S ⊕ S ⊥ and send f 7→ P f = s where
f = s + t, with s ∈ S, t ∈ S ⊥ . This is an example of a projection operator.
∑
• Let e 1 , e 2 , . . . be an orthonormal basis of H . Write f = a j e j . Then the map f 7→
(a 1 , a 2 , . . . ) is a linear transformation from H to l 2 (N).
Exercise. Prove that the first two examples really do give bounded linear transformations.
If T : H 1 → H 2 is a bounded linear transformation, then its adjoint is the operator T ∗ : H 2 →
H 1 defined by
(f ,T ∗д)1 = (T f , д)2 for all f ∈ H 1 , д ∈ H 2 .
Check that this really does define a unique operator. Notice that (2.1) shows that ∥T ∗ ∥ = ∥T ∥.
Show that the kernel of T ∗ is the orthogonal complement of the range of T , and that the adjoint
of T ∗ is T .

2.5 Projections and unitaries

A bounded linear transformation P : H → H is called a projection if P 2 = P and an orthogonal
projection if, in addition, P = P ∗ . Notice that any projection satisfies P f = f for f ∈ ran(P).
It follows that the range of P is closed, since if P fn → д, then P fn → Pд. If it is an orthogonal
projection, then f − P f is orthogonal to P f , since

(f − P f , P f ) = (f , P f ) − (f , P ∗P f )

= (f , P f ) − (f , P 2 f ) = 0.
Therefore, if R = ran(P), then P f is the first component of f in the decomposition H = R ⊕
R ⊥ . There is thus a one-to-one correspondence between closed subspaces of H and orthogonal
projections P.

7
 Exercise. Suppose that S is a closed subspace of H with orthogonal complement S ⊥ , and let
P, P ⊥ be the corresponding orthogonal projections. Show that

P + P ⊥ = Id .

Exercise. Suppose that P is an orthogonal projection with one dimensional range S. Write
down an expression for P in terms of (i) a nonzero vector s ∈ S and (ii) the inner product on H .

Theorem 2.7. Let S be a closed subspace of a Hilbert space H , and let f ∈ H . Then there is a
unique point s in S closest to f , and it is given by s = P f where P is the orthogonal projection onto
S. (Compare with Theorem 2.5.)

The following variation is often useful in the case that S ⊥ is finite dimensional.

Corollary 2.8. Let S, H , f and s be above. Then s = f − P ⊥ f where P ⊥ is the orthogonal projection
onto S ⊥ .

A BLT U : H 1 → H 2 is called unitary if it is invertible, with U −1 = U ∗ . It follows that

(U f , U д)2 = (U ∗U f , д)1 = (U −1U f , д)1 = (f , д)1 ,

so U preserves the inner product and therefore the norm: ∥U f ∥2 = ∥ f ∥1 . Two Hilbert spaces
are said to be unitarily equivalent if there is a unitary transformation mapping between them.

Theorem 2.9. Any two infinite dimensional, separable Hilbert spaces are unitarily equivalent.

The idea of the proof is to choose an orthonormal basis e 1 , e 2 , . . . for the first Hilbert space
and an orthonormal basis f 1 , f 2 , . . . for the second, and then map ei → fi . It is not hard to see
that there is a unique BLT with this property, and that it is unitary.