1 Banach spaces and Hilbert spaces

We will first discuss Banach spaces, since much of what we say applies to
Hilbert spaces, without change.
Let V be a normed linear space over either the real or complex numbers.
A sequence of vectors {vj }
j=1 is a map from the natural numbers to V . We
say that vj converges to v V if

lim kvj vk = 0.
j

A sequence {vj } is said to be Cauchy if for each  > 0, there exists

a natural number N such that kvj vk k <  for every j, k N . Every
convergent sequence is Cauchy, but there are many examples of normed linear
spaces V for which there exists non-convergent Cauchy sequences. One such
example is theset of rational numbers Q. The sequence (1.4, 1.41, 1.414, . . . )
converges to 2 which is not a rational number. We say a normed linear
space is complete if every Cauchy sequence is convergent in the space. The
real numbers are an example of a complete normed linear space.
We say that a normed linear space is a Banach space if it is complete. We
call a complete inner product space a Hilbert space. Consider the following
examples:
1. Every finite dimensional normed linear space is a Banach space. Like-
wise, every finite dimensional inner product space is a Hilbert space.
2. Let x = (x1 , x2 , ..., xn , ...) be a sequence. The following spaces of se-
quences are Banach spaces:

X
p
` = {x : |xj |p = kxk`p < }, 1 p < (1.1)
j=1

` = {x : sup |xj | < } (1.2)

j

c = {x : lim xj exists}, kxkc = kxk (1.3)

j

c0 = {x : lim xj = 0}, kxkc0 = kxk (1.4)

j

Except for ` . the spaces above are separable i.e., each has a countably
dense subset,

1
 3. The following function spaces are Banach spaces:

C[0, 1], kf kC[0,1] = max |f (x)| (1.5)

x[0,1]
k
X
(k)
C [a, b], kf kC (k) [a,b] = sup |f j (x)| (1.6)
j=0 x[0,1]
Z  p1
p p
L (I), kf k = |f (x)| (1.7)
I
L (I), kf kL (I) = ess-supxI |f (x)| (1.8)

There are two Hilbert spaces among the spaces listed: the sequence space
`2 and the function space L2 . In the result below, we will show that ` is
complete. After that, we will show that C[0, 1] is complete, relative to the
sup-norm, kf kC[0,1] = max |f (x)|. Of course, this means that both of them
are Banach spaces.

Proposition 1.1. The space ` is a Banach space.

Proof. The norm on ` is given by

kxk = sup |x(j)|.

j

Let {xn }n=1 `

denote a Cauchy sequence of elements in ` . We show
that this sequence converges to x ` . Since {xn } is Cauchy, for each  > 0,
there exists an N such that for all n, m N ,

kxn xm k < . (1.9)

This implies that |xn (j) xm (j)| <  for all j. Therefore, the sequence
{xn (j)} is a Cauchy sequence of real numbers, and hence converges to some
value x(j). That is, limn xn (j) = x(j) exists. From (1.9), if we choose
 = 1, then for all n, m N , we have

kxn xm k < 1.

In particular, it follows that

kxn k < 1 + kxm k, n, m N.

2
Fix m. Then, for all n N , |xn (j)| kxn k < 1 + kxm k . Letting n ,
we see that
|x(j)| 1 + kxm k
holds uniformly in j. Therefore, x ` . To complete the proof, we need to
show that xn converges to x in norm. We have

|xn (j) xm (j)| <  j N and n, m N.

Let n . Then, xn (j) x(j) so we have that

|x(j) xm (j)| <  j N, m N.

Since this holds for all j, it follows that kx xm k <  for all m N .
Therefore, the sequence xm converges to x ` .
Proposition 1.2. Relative to the sup norm, C[0, 1] is complete and is thus
a Banach space.
Proof. Let {fn (x)} n=1 be a Cauchy sequence in C[0, 1]. Then, for every  > 0,
there exists an N such that kfn fm k <  for all n, m N . For any fixed
t [0, 1], this implies that

|fn (t) fm (t)| <  m, n N. (1.10)

Thus, for t fixed, {fn (t)}

n=1 is a Cauchy sequence of real numbers, and so it
converges. Define f (t) by the pointwise limit of this sequence:

f (t) = lim fn (t), t [0, 1]. (1.11)

n

By taking the limit as m in (1.10), we see that

|fn (t) f (t)|  n N,

which holds uniformly for all t [0, 1]. Consequently,

kfn f k = sup |fn (t) f (t)| , n N,

t[0,1]

and so limn kfn f k = 0. What remains is to show that f C[0, 1]. To

do that, fix t and let  > 0. Because fn C[0, 1], there exists a > 0 such
that

|fn (t + h) fn (t)| < |h| < ,
3
3
assuming, of course, that t + h [0, 1]. Then applying the triangle inequality
gives

|f (t + h) f (t)| |f (t + h) fn (t + h)| + |fn (t + h) fn (t)| + |fn (t) f (t)|.

From (1.11), we may choose n so large that both |f (t + h) fn (t + h)| < 3

and |f (t) fn (t)| < 3 . Once we have found an n such that this holds, we
note that for these n
 
|f (t + h) f (t)| < + |fn (t + h) fn (t)| + .
3 3
Since the fn are continuous functions, we can find a such that |fn (t + h)
f (t)| < 3 whenever |h| < . It follows that, for this choice of , we have
  
|f (t + h) f (t)| < + + =
3 3 3
whenever |h| < , which implies that f C[0, 1].
Just because a space is complete relative to one norm does not mean
that the same space will also be complete in another. The example below
illustrates this for the space of continuous functions, C[0, 1].
Example 1.1. If we replace the sup norm on C[0, 1] with the L1 norm, then
C[0, 1] is not complete.
Proof. To simplify the discussion, we will work with [1, 1] rather than [0, 1].
Consider the sequence of continuous functions fn C[1, 1] defined piece-
wise by

1
1 x [1, n ]

fn (t) = nx x [ n1 , n1 ]

1 x [ n1 , 1].

Let n > m. We note that |fn (t) fm (t)| will be symmetric about t = 0.
Using this, we see that

1
nt mt t [0, n ]

|fn (t) fm (t)| = 1 mt t [ n1 , m1 ]

0 t [ m1 , 1]

4
Computing the integrals over [0, 1] yields
Z 1 Z 1 Z 1
n m
|fn (t) fm (t)| dt = (n m)t dx + 1 mt dx
1
0 0 n
1 1 1 m 1 m 1
= (n m) 2
+ 2
+
2n m n 2m 2 n2
1 1
 1 
=
2 m n
Making use of the symmetry in conjunction with the result above then gives
us kfn fm kL1 [1,1] = m1 n1 . Let N > 2 . Then, for m, n N , we find that
1 1  
kfn fm kL1 [1,1] < + < + = ,
n m 2 2
and so the sequence is Cauchy in the L1 [1, 1] norm. However, the limit is
not continuous. A computation shows that, for the step function f (t) defined
by

1 t [1, 0)

f (t) = 0 t=0

1 t (0, 1],

we have kf fn kL1 [,1] = n1 , and so

lim kf fn kL1 = 0.
n

Therefore, the sequence of functions fn C[1, 1] converges to a discontin-

uous function under the L1 [1, 1] norm, and, consequently, C[1, 1] is not
complete under the L1 [1, 1] norm.

2 Approximating Continuous Functions

Modulus of continuity Every f C[0, 1] is uniformly continuous: for
every  > 0, there is a > 0 such that

|f (t2 ) f (t1 )| <  (2.1)

as long as t1 , t2 [0, 1] satisfy |t2 t1 | < .

Using this fact, we are able to make the following definition:

5
Definition 2.1. Let f C[0, 1]. The modulus of continuity for f is defined
to be

(f, ) = sup |f (t1 ) f (t2 )|, t1 , t2 [0, 1]. (2.2)

|t1 t2 |

Example 2.1. Let f (x) = x, 0 x 1. Show that (f, ) C .

Proof. Let 0 < s < t 1. We note that

ts
0< t s=
t+ s

ts

= ts
t+ s
1 st
p

= ts
1 + st
p

ts=

Hence, (f, ) . To get equality, take s = 0 and t = .

Example 2.2. Suppose that f C (1) [0, 1]. Show that (f, ) ||f 0 || .

Proof. Because f C (1) , we can estimate f (t) f (s) this way:

Z t
|f (t) f (s)| |f 0 (x)| dx (t s)kf 0 k kf 0 k , (2.3)
s

which immediately gives (f, ) kf 0 k .

Linear splines One very effective way to approximate a continuous func-

tion f C[0, 1], given values of f at a finite set of points in t0 = 0 < t1 <
t2 < < tn = 1, is to use a connect-the-dots interpolant. The interpolant
is obtained by joining points (tj , f (tj ) by a straight. This procedure results
in a piecwise-linaer, continuous function with corners at the the tj s. More
formally, this is a called linear spline interpolant. Linear spline interpolants
are used for generating plots in many standard programs, such as Matlab or
Mathematica.
Defining a space of linear splines starts with sequence of points (or par-
tition of [0, 1]) = {t0 = 0 < t1 < t2 < < tn = 1}. called a knot

6
sequence. Linear splines on [0, 1] with knot sequence are the set of all
piecewise linear functions that are continuous on [0, 1] and (possibly) have
corners at the knots. As described above, we can interpolate continuous
functions using linear splines. Let f C[0, 1] and let yj = f (tj ). This is
linear spline sf (x) that is constructed by joining pairs of points (tj , yj ) and
(tj+1 , yj+1 ) with straight lines; the resulting spline is the unique. The result
below gives an estimate of the error made by replacing f by sf .
Proposition 2.1. Let f C[0, 1] and let = {t0 = 0 < t1 < < tn = 1}
be a knot sequence with norm kk = max |tj tj+1 |, j = 0, . . . , n 1. If sf
is the linear spline that interpolates f at the tj s, then,
kf sf k (f, kk) (2.4)
Proof. Consider the interval Ij = [tj , tj+1 ]. We have on Ij that sf (t) is a line
joining (tj , f (tj )) and (tj+1 , f (tj+1 )); it has the form
tj+1 t t tj
sf (t) = f (tj ) + f (tj+1 )
tj+1 tj tj+1 tj
Also, note that we have
tj+1 t t tj
+ = 1.
tj+1 tj tj+1 tj
Using these equations, we see that f (t) sf (t) for any t [tj , tj+1 ] can be
written as
tj+1 t t tj

f (t) sf (t) = f (t) + sf (t)
tj+1 tj tj+1 tj
tj+1 t t tj
= (f (t) f (tj )) + (f (t) f (tj+1 )) .
tj+1 tj tj+1 tj

By the definition of the modulus of continuity, |f (x) f (y)| (f, ) for

any x, y such that |x y| . If we set j = tj+1 tj , then we see that on
the interval Ij we have
  tj+1 t   t tj
|f (t) sf (t)| (f (t) f (tj )) + f (t) f (tj+1 )
tj+1 tj tj+1 tj
 
tj+1 t t tj
+ (f, j ) = (f, j ).
tj+1 tj tj+1 tj

7
Because the modulus of continuity is non decreasing (exercise 4.5(c)) and
j kk, we have (f, j ) (f, kk). Consequently, |f (t) sf (t)|
(f, kk), uniformly in t. Taking the supremum on the right side of this
inequality then yields (2.4).

3 Basis Splines B-Splines

For notation, we define
(
x x0
(x)+ =
0 x<0

and

N2 (x) = (x)+ 2(x 1)+ + (x 2)+ . (3.1)

Let be an equally spaced knot sequence tj = nj , j = 0, . . . , n.

Proposition 3.1. Let B = {N2 (nx j + 1) : j = 0, . . . , n}. Then, B is a

1
basis for S n (1, 0) provided x [0, 1].

Proof. Exercise.

Example 3.1. Consider n = 4. Recall that the values at the corners and
endpoints determine the linear spline. So, let yj be given at j = 0, 1, 2, 3, 4.
Then, the interpolating spline is
4
X
s(x) = yj N2 (4x j + 1), 0 x 1.
j=0

Theorem 3.1. There is a countably dense set of continuous functions.

Proof. Consider the functions of the form

n
X
qj N2 (nx j + 1), x [0, 1], qj Q. (3.2)
j=0

For each n, there are countably many such splines because the rationals
are a countable set. Since there are countably many n and the union of

8
countably many countable sets is countable, the collection of such functions
is countable. We now demonstrate that this is a dense subset of C[0, 1]. Let
f C[0, 1] and consider the interpolant
n
X j
sf (x) = f ( )N2 (nx j + 1
j=0
n

for x [0, 1]. Then, we have previously shown that kf sf k (f, n1 ).

Consider another spline
N
X
s(x) = qj N2 (nx j + 1) (3.3)
j=0

which is a member of the countable set of splines we defined previously. By

computation, it follows that
 j+1
x x nj
    
n j
j + 1

s(x) s(x) = 1 f qj + 1 f qj+1 .
n
n n
n

Therefore, we can bound the maximum pointwise error by

j
ks sk max |f ( ) qj |.
0jn n

Choose each qj to be a rational approximation of f ( nj ). That is, we choose

qj Q such that |qj f ( nj )| < 2 for each j = 0, . . . , n. Then, it follows that
ks sk < 2 . By choosing n large enough so that (f, n1 ) < 2 and the qj in
this manner, we find by applying the triangle inequality that

kf sk kf sk + ks sk
 
+ = .
2 2
Therefore, we have constructed a countable set of splines and demonstrated
that they are dense in C[0, 1].
Note that both ` and L are not separable. In ` , consider the un-
countable set A = {x ` : xj {0, 1}}. If x , x0 A differ, then
kx x0 k = 1, since they differ in at least one coordinate j, and the only
possibility is for |x (j) x0 (j)| = 1. Suppose that ` is separable. Then,

9
there exists a countable dense subset {vj } j=1 . In particular, for any x `

and for each , there exists at least one vj such that kvj xk < . Let  = 14 .
Since there are uncountably many x A, there exists a vj such that there
exists x 6= x0 A such that both |vj x | <  and |vj x0 | < . Then,
by the triangle inequality, we find
kx x0 k kx vj k + kx0 vj k
1
<+=
2
1
which implies that kx x0 k < 2 , which is a contradicts the computation
kx x0 k = 1.

4 Finite Element Spaces

Let t0 = 0 < t1 < t2 < < tn = 1. We refer to this as a knot sequence,
which we denote by = {tj }nj=0 . We define the subintervals Ij = [tj , tj+1 )
and let k denote the set of polynomials of degree less than or equal to k.
We define the space of splines as follows:
S (k, r) = { : [0, 1] R : |Ij k (Ij ) and C (r) ([0, 1])} (4.1)
1
When r = 1, is discontinuous. The finite element spaces S n (k, r) are
degree k polynomials on each interval and have r k 1 derivatives that
match at the interior points. We consider the following question: how many
1
parameters are required to describe a function in S n (k, r)? That is, what is
the dimension of this linear space?
There are n intervals and on each interval and on each interval there
are k + 1 free parameters since the function is a degree k polynomial on
each interval. Therefore, we have n(k + 1) free parameters. At each of the
n 1 knots, the polynomials most smoothly join, there are r + 1 equations
that must match (the polynomials across a knot must match and their r
derivatives must match). This yields (n 1)(r + 1) constraints. Therefore,
we have at least n(k + 1) (n 1)(r + 1) = n(k r) + r + 1 parameters. It
1
follows that the dimension of S n (k, r) = n(k r) + r + 1 provided that the
equations at the knots are independent (which can be shown).
1
For an example, consider k = 1, r = 0. This is the space S n (1, 0) which
has dimension n(1 0) + 0 + 1 = n + 1. If we consider k = m 1, r = m 2,
1
then the dimension S n (m1, m2) is n(m1m+2)+m2+1 = n+m1.

10