October 3, 2014

III. THE DUAL SPACE

BASED ON NOTES BY RODICA D. COSTIN

Contents
1. The dual of a vector space 1
1.1. Linear functionals 1
1.2. The dual space 2
1.3. Dual basis 3
1.4. Linear functionals as covectors; change of basis 4
1.5. Functionals and hyperplanes 5
1.6. Application: Spline interpolation of sampled data (Based on
Prof. Gerlach’s notes) 5
1.7. The bidual 8
1.8. Orthogonality 8
1.9. The transpose 9
1.10. Eigenvalues of the transpose 11

1. The dual of a vector space

1.1. Linear functionals. Let V be a vector space over the scalar field
F = R or C.
Recall that linear functionals are particular cases of linear transformation,
namely those whose values are in the scalar field (which is a one dimensional
vector space):
Definition 1. A linear functional on V is a linear transformation with
scalar values: φ : V → F .
A simple example is the first component functional: φ(x) = x1 .
We denote φ(x) ≡ (φ, x).

Example in finite dimensions. If M is a row matrix, M = [a1 , . . . , an ]

where aj ∈ F , then φ : F n → F defined by matrix multiplication, φ(x) =
M x, is a linear functional on F n . These are, essentially, all the linear func-
tionals on a finite dimensional vector space.
Indeed, the matrix associated to a linear functional φ : V → F in a basis
BV = {v1 , . . . , vn } of V , and the (standard) basis {1} of F is the row vector
(1) [φ1 , . . . , φn ], where φj = (φ, vj )
1
2 BASED ON NOTES BY RODICA D. COSTIN
P P P P
If x = j xj vj ∈ V then (φ, x) = (φ, j x j vj ) = j xj (φ, vj ) = j xj φj
hence
 
x1
(2) (φ, x) = [φ1 , . . . , φn ]  ... 
 

xn

Examples in infinite dimensions. In infinite dimensions, linear func-

tionals are also called linear forms. We mention a common notation in th
Hilbert spaces used in quantum mechanics: Dirac’s bracket notation
(φ, x) ≡ hφ|xi
In this notation, a functional φ is rather denoted hφ| and called bra-, while
a vector x ∈ V is rather denoted |xi and called -ket; they combine to hφ|xi,
a bra-ket.

1. The most often encountered linear functionals are integrals. For exam-
ple, on the linear space of continuous functions on [a, b] let I : C[a, b] → R
Rb
defined as (I, f ) = a f (t) dt.
More generally, given some w(t) > 0 on [a, b], the integral with respect to
Rb
the weight w, (Iw , f ) = a f (t) w(t)dt is a linear functional on C[a, b].
2. Evaluation at a point: if t1 ∈ [a, b] define, for V = C[a, b]
(3) Et1 : V → R, (Et1 , f ) = f (t1 )
It is easy to see that Et1 is linear.
One could think of Et1 as an integral with the weight the Dirac’s delta
function δ(t − t1 ). Except, δ is not a function! It is a distribution.
Linear combinations of evaluation functions: if t1 , . . . , tn ∈ [a, b] and
c1 , . . . , cn ∈ F then c1 Et1 + . . . + cn Etn defined by
(c1 Et1 + . . . + cn Etn , f ) = c1 f (t1 ) + . . . + cn f (tn )
is also a linear functional on C[a, b].
3. On the linear space C ∞ (a, b) of infinitely many times differentiable
functions on an interval (a, b), fix some t0 ∈ [a, b]; then (φ, f ) = f (k) (t0 ) is
a linear functional, and so is (φ, f ) = nk=0 ck f (k) (t0 ).
P

1.2. The dual space. Given two linear functionals φ, ψ on V and c ∈ F

we can define their addition φ + ψ and scalar multiplication cφ by
(φ + ψ, x) = (φ, x) + (ψ, x), (cφ, x) = c(φ, x)
and φ + ψ, cφ are also linear functionals on V (check!).
Moreover, it can be easily verified that the set of linear functional on V
form a vector space over F .
Definition 2. The vector space V 0 of all the linear functionals on V is called
the dual space of V .
 III. THE DUAL SPACE 3

Often, knowledge of all functionals on a space yields complete information

about the space. In finite dimensional cases, this is clear if we think of the
component functionals of a vector.
The operation (φ, x) is also called a pairing between V 0 and V .
Note that
(φ, cx + dy) = c(φ, x) + d(φ, y)
(cφ + dψ, x) = c(φ, x) + d(ψ, x)
More generally:
Definition 3. Let U, V be vector spaces over the same scalar field F . A
function B : U × V → F is called a bilinear functional if B(u, v) is linear
in u for each fixed v, and linear in v for each fixed u, in other words
B(c1 u1 + c2 u2 , v) = c1 B(u1 , v) + c2 B(u2 , v)
and
B(u, d1 v1 + d2 v2 ) = d1 B(u, v1 ) + d2 B(u, v2 )
for all u, u1,2 ∈ U and all v, v1,2 ∈ V and scalars cj , dj ∈ F .
The pairing between V 0 and V is a bilinear functional.

1.3. Dual basis. Given B = {v1 , . . . , vn } a basis in V , its dual is one special
associated basis for V 0 : consider the linear functionals v10 , . . . , vn0 defined by
(vj0 , vi ) = δi,j for all i, j = 1, . . . , n, where δi,j is the Kronecker symbol:

0 if i 6= j
δi,j =
1 if i = j
Note that the row vector representations of v10 , v20 , . . . , vn0 in the basis
{v1 , v2 , . . . , vn } are
[1, 0, . . . , 0], [0, 1, . . . , 0], . . . , [0, 0, . . . , 1]
In particular, it is clear that v10 , v20 , . . . , vn0 are linearly independent.
Consider any functional φ ∈ V 0 and its row vector representation (1) in
the basis B. Then
X n
(4) φ= φj vj0
j=1

In particular, v10 , v20 , . . . , vn0 span V 0 .

We have shown:
Theorem 4. v10 , . . . , vn0 is a basis for V 0 . Therefore dim V 0 =dim V .
Definition 5. B 0 = {v10 , . . . , vn0 } is called the basis dual to B = {v1 , . . . , vn }.
Notes.
1. Applying elements of B 0 to vectors produces their coordinates in B:
(5) (vk0 , x) = xk
4 BASED ON NOTES BY RODICA D. COSTIN

Applying functionals to vectors in B produces their coordinates in B 0 :

(φ, vj ) = φj
2. If x = j xj vj is the representation in B, and φ = k φk vk0 , is the
P P

representation in B 0 then
 
x1
(φ, x) = φ1 x1 + . . . + φn xn = [φ1 , . . . , φn ]  ... 
 

xn

1.4. Linear functionals as covectors; change of basis. Suppose BV =

{v1 , . . . , vn } is a basis of V . Consider a linear functional φ : V → F with
the row vector representation (2) in this basis.
Let B̃V = {ṽ1 , . . . ,Pṽn } be another basis in V linked to BV by the transi-
tion matrix S: ṽj = i Sij vi .
Recall that the coordinates of vectors change with S −1 : if x = nj=1 xj vj =
P
Pn
j=1 x̃j ṽj then
   
x̃1 x1
 ..  −1  . 
 .  = S  .. 
x̃n xn
while matrix representations of linear transformations changes with S:
[φ̃1 , . . . , φ̃n ] = [φ1 , . . . , φn ] S
(where φ̃j = (φ, ṽj )).
Functionals are covariant, while vectors are contravariant. This is one
reason functionals are also called covectors. Sometimes, covectors are sim-
ply written as rows, acting on columns by the usual matrix multiplication.
Note: if φ is a functional and x is a vector, we can use their representations
in different bases to calculate (φ, x):
   
x1 x̃1
(φ, x) = [φ1 , . . . , φn ]  ...  = [φ̃1 , . . . , φ̃n ]  ... 
   

xn x̃n

1.4.1. Change of the dual basis upon a change of coordinates. To find the
transition matrix from the basis dual to BV to the one dual to B̃V note that
X X X
−1 −1 0 −1 −1
(ṽj0 , vk ) = (ṽj0 , Slk ṽl ) = Slk (ṽj , ṽl ) = Slk δjl = Sjk
l l l
X X X
−1 −1 0 −1 0
= Sji δik = Sji (vi , vk ) = ( Sji vi , vk )
i i i
0
P −1 0
for all k, hence ṽj = i Sji vi therefore the transition matrix between the
dual bases is (S −1 )T .
 III. THE DUAL SPACE 5

Exercise. How does this compare with the matrix change of variables
S −1 M S? Why do we get something different?
1.5. Functionals and hyperplanes. Two-dimensional subspaces in R3 are
planes going through the origin; they can be specified as the set of all vectors
orthogonal to a given vector n ∈ R3 as {x ∈ R3 | n · x = (nT , x) = 0}. The
row vector nT is the transpose of n.
The plane in R3 passing through a point x0 and orthogonal to n ∈ R3
is {x ∈ R3 | (nT , x − x0 ) = 0}. Equivalently, this is {x ∈ R3 | (nT , x) =
k} where k = (nT , x0 ). Also, equivalently, this is a translation of a two-
dimensional subspace: x0 + {x ∈ R3 | (nT , x) = 0}.
This situation is generalized in arbitrary dimensions as follows.
Hyperspaces are subspaces on Rn of dimension n−1 (they are said to have
codimension 1). Hyperspaces H of a vector space V are maximal subspaces:
if U is a subspace so that H ⊂ U ⊂ V then U = H or U = V .
Hyperplanes are translations of hyperspaces.
Theorem 6. Let V be a vector space.
a) Every hyperspace in V is the null space of a linear functional.
b) Every hyperplane in V is the level set of a linear functional. In other
words, a hyperplane is the set Hφ,k = {x ∈ V | (φ, x) = k} where φ ∈ V 0 is
a nonzero linear functional and k ∈ F .
We may prove these facts a bit later.
1.6. Application: Spline interpolation of sampled data (Based on
Prof. Gerlach’s notes). Suppose we measure some data: at n + 1 data
points s0 , s1 , . . . , sn , assumed distinct, we measure the quantities y0 , y1 , . . . , yn .
The question we want to address is to fit them on a curve, namely to find a
function ψ(t) so that
(6) ψ(xj ) = yj for all j = 0, 1, . . . , n
Of course, there are infinitely many such functions ψ, and to have a
definite (unique) answer, we need to restrict our search to special classes of
functions. The choice of the type of functions used for interpolation is done
taking into account expected error, calculation time etc.
1.6.1. Polynomial interpolation. Polynomials in Pn , of degree at most n,
depend on n + 1 parameters, so it is reasonable to expect one can find a
unique polynomial ψ ∈ Pn satisfying (6). A polynomial of degree n passing
through the given points is called spline polynomial of degree n. Before
computers, elastic strips were used to produce the interpolation. Look up
on Wikipedia “flat spline”.
To find the spline polynomial, we will seek a general formula for a poly-
nomial taking the value pj at x = xj .
A general formula is derived as follows. Consider the evaluation linear
functionals (3) on V = Pn : Es0 , . . . , Esn which satisfy (Esj , p) = p(sj ) for
all j = 0, . . . , , n and all p ∈ Pn .
6 BASED ON NOTES BY RODICA D. COSTIN

The evaluation functional P Es0 , . . . , Esn are linearly independent: assum-

j
P
ing a linear combination k ck EP sk = 0, then in particular, ( k k sk , t ) =
c E
0 for all j = 0, 1, . . . , n therefore k ck sk j = 0 for all j = 0, 1, . . . , n. This is a
homogeneous system of n+1 equations with n+1 unknowns c0 , . . . , cn whose
determinant is a Vandermonde determinant, equal to Πi>j (si − sj ) 6= 0, and
therefore all cj = 0. Since Es0 , . . . , Esn are an n + 1 set of independent
elements in the n + 1 dimensional linear space Pn0 , then they form a basis.
Let us find the basis Q0 , . . . , Qn of Pn for which Es0 , . . . , Esn is the dual
basis. It means (Esj , Qk ) = δjk . OnceP we accomplish that, then any poly-
nomial p ∈ Pn can be writtenPas p = k αk Qk , on the one hand. P On the
other hand, (Esj , p) = (Esj , k αk Qk ) = αk and thus p = k (Esk , p)Qk
(by (5)), therefore
X n
X
p= (Esk , p)Qk = p(sk )Qk
k k=0

which gives any polynomial p ∈ Pn based on its sample values p(s0 ), . . . , p(sn ).
To find the basis Q0 , . . . , Qn , note that the polynomial Qk must satisfy
(Esj , Qk ) = Qk (sj ) = 0 for all j 6= k, therefore Qk = ck (t − s0 )(t − s1 ) . . . (t −
sj−1 )(t − sj+1 ) . . . (t − sn ) with ck determined so that Qk (sk ) = 1, therefore
(t − s0 )(t − s1 ) . . . (t − sj−1 )(t − sj+1 ) . . . (t − sn )
Qk =
(sk − s0 )(sk − s1 ) . . . (sk − sj−1 )(sk − sj+1 ) . . . (sk − sn )
In conclusion, there is a unique polynomial ψ ∈ Pn satisfying (6), namely
n
X
(7) ψ(t) = yk Qk (t)
k=0

which is called the Lagrange formula for polynomial interpolation.

Note: the fact that there exists a unique interpolating function in a certain
linear space is based on the fact that the evaluation functionals form a basis
for that linear space.

1.6.2. Linear interpolation. A simpler interpolating function can be obtained

by joining each consecutive sample points (sj , yj ) and (sj+1 , yj+1 ) by line
segments; the result is an interpolating function which is continuous, and
piecewise linear (but not differentiable).
Of course, it is easy to write formulas, and such an interpolating function
satisfying (6), is
yj+1 − yj
ψ(t) = yj + (t − sj ) for all t ∈ [sj , sj+1 ]
sj+1 − sj
Let us see how we can write ψ as a linear combination of basic functions,
an analogue of (7) from the polynomial interpolation case.
 III. THE DUAL SPACE 7

Given s0 < s1 < . . . < sn we look for an interpolating function in the

space

V = f : [s0 , sn ] → R f (t) = a line segment for t ∈ [sj , sj+1 ], j = 0, . . . , n
This is a linear space, of dimension n + 1 (you may wish to check!).
As in §1.6.1, we find the basis F0 , . . . , Fn of V for which Es0 , . . . , Esn is the
dual basis. (It is quite clear that these evaluations form a basis for V since
any function is completely determined by its values at the sample points.)
The piecewise linear function Fk must satisfy (Esj , Fk ) = Fk (sj ) = 0 for all
j 6= k, and Fk (sk ) = 1. Therefore Fk is a ”tent” function which is zero on
[s0 , sj−1 ] and on [sj+1 , sn ] and whose graph joins by a segment the points
(sj−1 , 0) and (sj , 1) and by a segment the points (sj , 1) and (sj+1 , 0).
Then
X n
ψ(t) = yk Fk (t)
k=0
is the interpolating function in V for the data (6).

1.6.3. Band-limited interpolation. Another type of interpolating function is

a superposition of oscillations. We assume here the samples are taken over
an interval which we take to be 2π (for simplicity of formulas).
We look for interpolating functions in the linear space spanned by the
linearly independent functions 1, sin(kt), cos(kt), k = 1, . . . , N :
N N
( )
X X
BN = f | f (x) = ak cos(kt) + bk sin(kt), ak , bk ∈ R
k=0 k=1
Since this an 2N + 1 dimensional space, we need an equal number of
samples: s0 , s1 , . . . , s2N ∈ [0, 2π].
As in the examples before, we look for a basis Sk , k = 0, . . . , 2N of BN
so that the evaluation functionals Esk from its dual basis.1 The function
Sk must satisfy Sk (xj ) = 0 for all j 6= k. By analogy with the Lagrange
formula for polynomial interpolation a first attempt may be to think of Sk
as a scalar multiple of the product of all sin(t − sj ) with j 6= k; however
this does not work because this product belongs to B2N rater than BN .
What works however is to take Sk as a scalar multiple of the product of all
sin 21 (t − sj ) with j 6= k.
Indeed, first note that this product belongs to BN . There is an even
number of factors in the product, and a product of any two of them is
s +s
sin 12 (t − sj ) sin 12 (t − si ) = cos 12 (si − sj ) − cos(t − i 2 j ) ∈ BN . Thus Sk is
the product of N terms of the form ak + bk sin t + ck cos t which is known
to have the form of functions in BN (e.g. 2 sin x cos x = sin(2x), 2 cos2 x =
1In this example we will not prove that the evaluations form a basis, rather attempt
to produce the basis for BN ; we will succeed, proving that the evaluations were a basis
indeed.
8 BASED ON NOTES BY RODICA D. COSTIN

1 + cos(2x), 4 cos3 x = cos(3x) + cos x etc.). Normalizing to ensure that

Sk (xk ) = 1 we obtain the formula
2N
Y sin 12 (t − sj )
Sk (t) =
j=0, j6=k
sin 12 (sk − sj )

and the interpolating function is

2N
X
ψ(t) = yk Sk (t)
k=0

1.7. The bidual. The dual V 0 is a vector space, therefore it has a dual
V 00 , called the bidual of V . It turns out that the bidual can be naturally
identified with V by the pairing (φ, x), which can be interpreted either as φ
acting on x, or as x acting on φ. First we show the following result, which
is sometimes phrased as “linear functionals separate points”.
Lemma 7. If x ∈ V is so that (φ, x) = 0 for all φ ∈ V 0 , then x = 0.
P
To prove the Lemma, let v1 , . . . , vn of a basis of V ; then x = j xj vj
and we have in particular, (vk0 , x) = xk = 0 for all k, therefore x = 0.
Theorem 8. The bidual V 00 of a finite dimensional space V is isomorphic
to V .

Why: formalizing the intuitive argument above, let

V 00 = {L : V 0 → F |, L linear}
and define T : V → V 00 as follows. For each x ∈ V let T x ∈ V 00 be defined
by (T x)φ = (φ, x). Clearly T is linear. To show T is 1-to-1, assume that
there is an x ∈ V such that T x = 0. This means that (T x)φ = (φ, x) = 0
for all φ ∈ V 0 . It follows that x = 0. Since T is injective and since dim V =
dim V 0 = dim V 00 , then T is also onto, hence it is an isomorphism. 2.
Note that the construction of the isomorphism of Theorem 8 relies on no
particular basis: this is called a natural isomorphism.

1.8. Orthogonality. The construction of hyperplanes suggests that the

dual of a vector space can be used to define an orthogonality-like relation.
For example,
Definition 9. If U is a subspace of a vector space V the annihilator of U
is the set
U ⊥ = {φ ∈ V 0 | (φ, u) = 0, for all u ∈ U }
It can be checked that the annihilator U ⊥ is a subspace of V 0 . In absence
of an inner product, the annihilator works quite well.
 III. THE DUAL SPACE 9

1.9. The transpose.

Warning: change of notation. Consistent notations are useful for helping
the thought process focus on the essential features rather than mere names,
and these notes have tried to use consistent notations. However...
Up to now we used consistently the letter T to denote linear transforma-
tions. At this point we are going to encounter the transpose of matrices,
denoted by a superscript T . To avoid collisions of notations, from now on
linear transformations will be denoted by the letter L.

Definition 10. If L : U → V is a linear transformation between two vector

spaces U, V over the scalar field F then the dual transformation, or the
transpose transformation is the linear transformation L0 : V 0 → U 0
defined by
(8) (L0 φ, x) = (φ, Lx) for all x ∈ U, φ ∈ V 0
As the words suggest:
Theorem 11. If M is the matrix of L : U → V in the bases BU , BV then
M T is the matrix of L0 in the dual bases BV0 , BU0 .
P
Formal proof. Since M is the matrix of L we have Luk = i Mik vi . To
calculate L0 vj0 we apply it to the vectors x:
X X X
(L0 vj0 , x) = (L0 vj0 , xk uk ) = xk (L0 vj0 , uk ) = xk (vj0 , Luk )
k k k
X X XX XX
= xk (vj0 , Mik vi ) = xk Mik (vj0 , vi ) = xk Mik δij
k i k i k i
X X X
= xk Mjk = Mjk (u0k , x) = ( Mjk u0k , x) for all x
k k k
L0 v 0 which proves the theorem. 2
P
which shows that j = k Mjk uk ,

1.9.1. The four fundamental spaces of a matrix. Consider an m × n matrix

M with entries in F (F = R or C) and its transpose M T . To rewrite
(8) in matrix notation it is sometimes preferred to replace the row vectors
φ by transposes of (usual) column vectors, yT . Relation (8) then reads
(M T y)T x = yT M x for all x ∈ Rn (or Cn ) and all y ∈ Rm (respectively CM ),
a relation which is clear bases on the basic property that (AB)T = B T AT
for any two matrices A, B for which the multiplication makes sense.
Recall that we defined:
R(M ) = {y ∈ F m | M x = y for some x ∈ F m } the column space of M
N (M ) = {x ∈ F n | M x = 0} the null space of M
Since the rows of M T are the columns of M , then clearly the row space
of M T equals the column space of M , and column space of M T equals the
row space of M :
R(M T ) = the row space of M .
10 BASED ON NOTES BY RODICA D. COSTIN

Define
N (M T ) = {y ∈ F m | M T y = 0} the left null space of M
Note that M T y = 0 is equivalent to yT M = 0, justifying the name of left
null space.
Recall that dim R(M ) + dim N (M ) = n and using this for M T , we obtain
dim R(M T ) + dim N (M T ) = m.
Recall that dim R(M ) = rank(M ) and the rank of a matrix is the order
of the largest nonzero minor; then rank(M ) = rank(M T ).
Summarizing:
Theorem 12. The fundamental theorem of linear algebra
Let M be an m × n matrix. Ler r = rank(M ). Then

dim R(M ) = r
dim R(M T ) = r
dim N (M ) = n − r
dim N (M T ) = m − r
Also
N (M ) = R(M T )⊥
N (M T ) = R(M )⊥
where ⊥ signifies the annihilator of the set.
Only the second to last line needs a proof. Here it is: consider first x ∈
N (M ) and show that x belongs to the annihilator of R(M T ), in other words,
that xT (M T y) = 0 for all y ∈ F m , which is obvious since xT (M T y) =
(M x)T y. Conversely, taking x ∈ R(M T )⊥ , this means that 0 = xT (M T y) =
(M x)T y for all y ∈ F m , which means that M x = 0, hence x ∈ N (M ). 2
 III. THE DUAL SPACE 11

1.10. Eigenvalues of the transpose. As a corollary of Theorem 11, L and

L0 have the same eigenvalues. The following theorem shows that eigenvectors
corresponding to different eigenvalues are orthogonal-like:
Theorem 13. Let V be a finite dimensional space over C, and L : V → V
be a linear transformation. Let L0 be its transpose, L0 : V 0 → L0 .
Let λ1 6= λ2 be two eigenvalues of L, with eigenvectors v1 of L and φ2 of
L0 :
L(v1 ) = λ1 v1 , L0 (φ2 ) = λ2 φ2
Then
(φ2 , v1 ) = 0
In particular, if λ1 6= λ2 are eigenvalues of the n × n matrix M (and
therefore of M T as well), and if M v = λ1 v and M T y = λ2 y then
(y, v) := y1 v1 + . . . + yn vn = 0
Proof.
By the definition of the transpose transformation (see (8)) we must have
(L0 φ2 , v1 ) = (φ2 , Lv1 )
therefore (λ2 φ2 , v1 ) = (φ2 , λ1 v1 ) so λ2 (φ2 , v1 ) = λ1 (φ2 , v1 ) and sinve λ1 6=
λ2 the theorem follows. 2