Math 3412: Advanced Linear Algebra

1. Real Vector Spaces

Chapter 1: Real Vector Spaces
In this chapter we present some of the fundamental aspects of real
vector spaces, especially those with finite dimension. Such spaces are
1.1 Real Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 more general than Rn but in many ways are essentially the same as Rn .
The central idea is that of a coordinate system. A coordinate system for
1.2 The Standard Basis in Rn . . . . . . . . . . . . . . . . . . . . . 4
a real vector space V is derived from a basis for V and allows one to
1.3 Spanning and Independent Sets . . . . . . . . . . . . . . . 7 represent vectors in V as ordinary columns of coordinates. Furthermore,
this representation is consistent with the two operations—vector addition
1.4 More on Spanning and Independent Sets . . . 12
and scalar multiplication—intrinsic to V. Thus for any two vectors in V
1.5 Basis and Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 the column for their sum is the sum of their columns, and the column
1.6 Changing Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 for a multiple of a vector is the multiple of its column. By representing
vectors as columns, we may work with vectors in any real n-dimensional
vector space V as if they were columns in Rn , at least as far as their
arithmetic is concerned.
R. McEachin
1.1. Real Vector Spaces
Department of Mathematics
University of the West Indies Let V be a collection of objects denoted by u, v, w, . . .. We say V
Mona Campus is a vector space over R to mean, first, that there is a vector addition
Kingston 7, Jamaica mapping V × V → V satisfying the properties:
1) (u + v) + w = u + (v + w) for all u, v, w ∈ V,
September, 2016
2) v + w = w + v for all v, w ∈ V,
3) there is some 0 ∈ V such that 0 + v = v + 0 = v for all v ∈ V, and
4) for any v ∈ V there is some −v ∈ V such that −v+v = v+(−v) = 0.
Second, there is a scalar multiplication mapping R × V → V satisfying
the properties:

1
 5) (α + β)v = αv + βv for all α, β ∈ R and v ∈ V, in Rn , we say u = v to mean that αi = βi for all i. Given any γ ∈ R as
6) α(v + w) = αv + αw for all α ∈ R and v, w ∈ V, well, we define u + v and γu by
7) (αβ)v = α(βv) for all α, β ∈ R and v ∈ V, and    
α 1 + β1 γα1
8) 1v = v for all v ∈ V.
 α 2 + β2   γα2 
u+v =
 .. ;
 γu =  
 ..  .
Thus we define a vector space in such a way that addition and scalar . .
multiplication are intrinsic. Furthermore, the properties listed here char- α n + βn γαn
acterize the fundamental aspects of these operations.
We will refer to a finite collection of vectors, such as {u, v, w}, many Using this arithmetic, one verifies that Rn satisfies Properties (1)–(8)
times in this book. We call such a collection a family to mean that it is an listed above. Thus Rn is a real vector space.
ordered k-tuple of vectors. A family differs from a set in two ways. First,
vectors may occur more than once in a family and we do not discard any Example 1.1.2. Let Zn consist of all polynomials with real coefficients
repetitions; otherwise we alter the family. Thus, for example, the family and having degree at most n, subject to the arithmetic described below.
{u, v, v} is not the same as {u, v}. Second, if we reorder the vectors in a Thus
family we consider the resulting family distinct. Thus, for example, the Zn = {α0 + α1 x + · · · + αn xn : α0 , α1 , . . . , αn ∈ R}
families {u, v, w} and {w, u, v} are distinct. We will also refer to a finite
subject to the arithmetic we presently describe. Note that Zn includes
collection of scalars many times in this book. We call such a collection a
the polynomial with αi = 0 for i = 0, 1, . . ., n. We call this the zero
family to mean that it is an ordered k-tuple of scalars. Pn
polynomial and usually denote it by 0. In Zn , given p(x) = i=0 αi xi
Pn
and q(x) = i=0 βi xi , we say that p = q to mean that αi = βi for i = 0,
Example 1.1.1. Let n be a positive integer. Then Rn denotes the set 1, . . ., n. Given any γ ∈ R as well, we define p + q and γp by
of all columns of real numbers of length n, subject to the arithmetic


described below. Thus p + q (x) = (α0 + β0 ) + (α1 + β1 )x + · · · + (αn + βn )xn ;


   γp (x) = γα0 + γα1 x + · · · + γαn xn
 α1 

 α2  

R = 
n
.  : α1 , α2 , . . . , αn ∈ R
 .  Using this arithmetic, one verifies that Zn satisfies Properties (1)–(8)

 

 .  listed above. Thus Zn is a real vector space.
αn

Example 1.1.3. Let Rm×n consist of all m×n matrices with real entries,
subject to the arithmetic we now describe. Given
subject to the arithmetic described below. Thus
   
α1 β1   
 α11 α12 · · · α1n 
 α2   β2   
u= 
 ...  , v= 
 ..  Rm×n
α
=  21
α22 · · · α2n 
 ij
: α ∈ R for all i and j
. 
 ···················· 

αn βn αm1 αm2 · · · αmn

2 3
subject to the arithmetic we now describe. Given Definition 1.2.1. Let A = {v1 , v2 , . . . , vk } be a family of vectors in a
    real vector space V. A linear combination of vectors in A is an expression
α11 α12 · · · α1n β11 β12 · · · β1n of the form α1 v1 + α2 v2 + · · · + αn vn , where α1 , α2 , . . . , αk ∈ R.
α α22 · · · α2n  β β22 · · · β2n 
A =  21 , B =  21 ,
···················· ···················· Thus linear combinations are produced by scaling each vi ∈ A by some αi
αm1 αm2 · · · αmn βm1 βm2 · · · βmn and summing the results. Note that some or even all of the αi can be 0.
For example, 0v1 + 0v2 + · · · + 0vk = 0 is a linear combination of vectors
we say A = B to mean that αij = βij for all i and j. Given any γ ∈ R
in A. Also, say, 0v1 + 1v2 + · · · + 0vk = v2 is such a linear combination.
as well, we define A + B and γA by
Furthermore, the sum of two linear combinations is a linear combination:
 
α11 + β11 α12 + β12 · · · α1n + β1n
(α1 v1 + α2 v2 + · · · + αk vk ) + (β1 v1 + β2 v2 + · · · + βk vk )
 α + β21 α22 + β22 · · · α2n + β2n 
A + B =  21 , = (α1 + β1 )v1 + (α2 + β2 )v2 + · · · + (αk + βk )vk
·········································
αm1 + βm1 αm2 + βm2 · · · αmn + βmn
  Similarly, a scalar multiple of a linear combination is a linear combination:
γα11 γα12 · · · γα1n
 γα21 γα22 · · · γα2n  γ(α1 v1 + α2 v2 + · · · + αk vk ) = (γα1 )v1 + (γα2 )v2 + · · · + (γαk )vk
γA =  
························
γαm1 γαm2 · · · γαmn
One particularly valuable use of linear combinations is to establish a
Using this arithmetic, one verifies that R m×n
satisfies Properties (1)–(8) coordinate system in a vector space V. To see how this may be done, we
listed above. Thus R m×n
is a real vector space. consider in some detail the system of standard coordinates in Rn . Let n
be a positive integer and define
Example 1.1.4. Let C[0, 1] consist of all continuous functions mapping      

1 0 0
[0, 1] → R. Given f, g ∈ C[0, 1], define f + g by f + g (x) = f (x) + g(x) 0 1 0



for all x. Given γ ∈ R as well, define γf by (γf (x) = γ f (x) . Using e1 =  
 ...  , e2 =  
 ...  , ..., en =  
 ...  ,
this arithmetic, one verifies that C[0, 1] satisfies Properties (1)–(8) listed
0 0 1
above. Thus C[0, 1] is a real vector space.
Thus each ej is the column of length n whose jth entry is 1 and whose
1.2. The Standard Basis in Rn other entries are 0. Define E = {e1 , e2 , . . . , en }. We call E the standard
basis for Rn . This set has two essential properties which allow it to be
In any real vector space, there are two basic arithmetical operations the basis for a coordinate system.
to be found: vector addition and scalar multiplication. The essential First, in Rn let
properties of these operations are given in Properties (1)-(8) listed at the  
α1
beginning of Section 1.1. These operations are fundamental to vector  α2 
x= ... 
 (1.1)
spaces and lead to the basic notion of a linear combination of a set of
vectors. αn

4 5
Then, using the vector arithmetic described in Example 1.1.1, Exercise Set 1.2
            1) Let A = {v1 , v2 , . . . , vk } and B = {w1 , w2 , . . . , wp } be families of
α1 0 0 1 0 0
 0   α2   0  0 1 0 vectors in a real vector space V. Let also v ∈ V and suppose v is a
x=    
 ...  +  ...  + · · · +  ..  = α1      
 ...  + α2  ...  + · · · + αn  ... 
.  linear combination of vectors in A. If each vi ∈ A is itself a linear
0 0 αn 0 0 1 combination of vectors in B, show that v is a linear combination of
vectors in B.
That is, x = α1 e1 + α2 e2 + · · · + αn en . Thus each x ∈ Rn can be written
as a linear combination of vectors in E in at least one way.
1.3. Spanning and Independent Sets
On the other hand, suppose x ∈ Rn and
Let B be a family of vectors in a real vector space V. We consider in
x = α1 e1 + α2 e2 + · · · + αn en , x = β1 e1 + β2 e2 + · · · + βn en more detail the properties B must have in order to serve as the basis for
a coordinate system in V. We also give examples of families of vectors
for some scalars α1 , α2 , . . ., αn , and β1 , β2 , . . ., βn . Rewriting each ej in a real vector space V which do or do not have some of the requisite
as a column of length n and using the arithmetic in Rn , one obtains properties. We begin with a definition.
  

α1 β1
Definition 1.3.1. Let B = {v1 , v2 , . . . , vk } be a family of vectors in a
 α2   β2 
x= 
 ...  , x= 
 ..  real vector space V. The span of B, denoted by span B, is the set of all
. linear combinations of vectors in B. Thus
αn βn

Thus αi = βi for all i. It follows that each x ∈ Rn can be written as a span B = {α1 v1 + α2 v2 + · · · + αk vk : α1 α2 , . . . , αk ∈ R}
linear combination of vectors in E in at most one way
In summary, each x as in (1.1) has a unique representation as a linear
Example 1.3.2. In R3 , let
combination of vectors in E. This is x = α1 e1 + α2 e2 + · · · + αn en . We
call the scalars α1 , α2 , . . ., αn the coordinates of x relative to E. Clearly,        
1 −1 −1 2
these n numbers (in the order listed) completely characterize x. v1 =  2  , v2 =  0  , v3 =  4  , v =  −6 
In general, given a real vector space V we may seek a family of vectors 0 1 3 −5
B = {v1 , v2 , . . . , vn } for use in establishing a coordinate system in V. It
is natural to require of B that each v ∈ V have a unique expression as a Setting B = {v1 , v2 , v3 }, we determine whether v ∈ span B. Thus we
linear combination of its vectors. In this case, if v = α1 v1 + α2 v2 + · · · + seek α1 , α2 , α3 such that α1 v1 + α2 v2 + α3 v3 = v. That is, we seek α1 ,
αn vn , we call α1 , α2 , . . ., αn the coordinates of v relative to B. We also α2 , α3 such that
say that B is the basis for this coordinate system.        
1 −1 −1 2
α1  2  + α2  0  + α3  4  =  −6 
0 1 3 −5

6 7
Using the arithmetic of vectors in R3 , this holds if and only if for a coordinate system in V we must first be able to write each v ∈ V
 as a linear combination of the vectors in B in at least one way. That is,
 α1 − α2 − α3 = 2
we must have v ∈ span B for all v ∈ V. In this case, we say that B is a
2α1 + 4α3 = −6
 spanning set for V or simply that B spans V.
α2 + 3α3 = −5
For B to be suitable as a basis for V, we’ve also seen that each v ∈ V
Using Gauss-Jordan elimination, one obtains the solutions must be expressible as a linear combination of vectors in B in at most
      one way. Thus if v ∈ span B and, say,
α1 −3 −2
 α2  =  −5  + t  −3  ,
α3 0 1 v = α 1 v1 + α 2 v2 + · · · + α k vk , v = β1 v1 + β2 v2 + · · · + βk vk , (1.2)

where t ∈ R is an arbitrary parameter. Thus v ∈ span B. In fact, v we must have αi = βi for all i. We now consider ways of reformulating
can be represented as a linear combination of vectors in B in infinitely this property.
many ways, one for each choice of t. For example (choosing t = 0), Suppose first that at least one v ∈ span B can be written as a linear
v = −3v1 − 5v2 + 0v3 . Also (choosing t = 1), v = −5v1 − 8v2 + v3 . combination as in (1.2), and furthermore αi 6= βi for at least one index i.
Then, subtracting equations,
Example 1.3.3. In Z2 , let
0 = (α1 − β1 )v1 + (α2 − β2 )v2 + · · · + (αk − βk )vk
p1 (x) = 2 − 3x + x2 , p2 (x) = −1 + x + 2x2 , q(x) = −8 + 11x + 8x2 .

Setting B = {p1 , p2 }, we determine whether q ∈ span B. Thus we seek In this case at least one coefficient, namely αi − βi , is nonzero. However,
scalars α1 and α2 such that α1 p1 + α2 p2 = q. That is, we seek α1 and α2 it is obvious that 0 = 0v1 +0v2 +· · ·+0vk as well. Thus 0 can be written
such that as a linear combination in more than one way. In other words, if some
v ∈ span B has more than one representation as a linear combination of
α1 (2 − 3x + x2 ) + α2 (−1 + x + 2x2 ) = −8 + 11x + 8x2 vectors in B then 0 has more than one such representation as well. It
(2α1 − α2 ) + (−3α1 + α2 )x + (α1 + 2α2 )x2 = −8 + 11x + 8x2 follows that if the representation for 0 is unique then the representation
for each v ∈ span B is unique. This motivates the following definition.
Equating coefficients, this last equation holds if and only if

 2α1 − α2 = −8 Definition 1.3.4. Let B = {v1 , v2 , . . . , vk } be a family of vectors in a
−3α1 + α2 = 11 real vector space V. We say B is linearly independent to mean that the

α1 + 2α2 = 8 only solution to the equation α1 v1 + α2 v2 + · · · + αk vk = 0 is the trivial
solution α1 = 0, α2 = 0, . . ., αk = 0.
Using Gauss-Jordan elimination, one determines that this system has no
solution. Thus q ∈
/ span{p1 , p2 }.
Proposition 1.3.5. Let B be a linearly independent family of vectors in a
Suppose now that V is a real vector space and B = {v1 , v2 , . . . , vk } real vector space V. Then each v ∈ span B can be written as a linear
is a family of vectors in B. We’ve seen that for B to be suitable as a basis combination of vectors in B in only one way.

8 9
This follows immediately from Definition 1.3.4 and the discussion preced- We determine whether A is linearly independent. To do this we seek all
ing it. solutions to the equation β1 B1 + β2 B2 + β3 B3 + β4 B4 = O, where O
Given a family B = {v1 , v2 , . . . , vk } in a real vector space V, we denotes the 2 × 2 zero matrix. Thus we seek β1 , β2 , β3 , β4 such that
say that B is linearly dependent to mean that it is not independent. In          
1 0 1 1 1 1 1 1 0 0
this case, the equation α1 v1 + α2 v2 + · · · + αk vk = 0 has at least one β1 + β2 + β3 + β4 =
0 0 0 0 1 0 1 1 0 0
nontrivial solution. In fact, it has infinitely many in this case.
Using the standard arithmetic of matrices, one obtains the linear system
Example 1.3.6. In R3 , let A = {v1 , v2 , v3 }, where of scalar equations
      
1 4 7  β1 + β2 + β3 + β4 = 0

v1 =  2  , v2 =  5  , v3 =  8  β2 + β3 + β4 = 0
3 6 9 
 β3 + β4 = 0
β4 = 0
We determine whether A is linearly independent. To do this we seek all
solutions to the equation α1 v1 + α2 v2 + α3 v3 = 0. That is, we seek α1 , Since this has unique solution β1 = β2 = β3 = β4 = 0, A is linearly
α2 , α3 such that independent.
        Exercise Set 1.3
1 4 7 0
α1  2  + α2  5  + α3  8  =  0  4
1) In R , let B = {b1 , b2 , . . . , b5 }, where b1 , b2 , b3 , b4 , and b5 are
3 6 9 0 the respective vectors
         
This leads to the system of equations 1 0 2 −3 1
  −2  1  0   4   5 
 ,  ,  ,  , and  .
 α1 + 4α2 + 7α3 = 0 3 2 14 −4 −1
2α1 + 5α2 + 8α3 = 0 2 0 4 −1 −8

3α1 + 6α2 + 9α3 = 0
a) Determine whether B is linearly independent.
The solution set is     b) Determine whether x1 = [1, 4, −3, −8]T is in span B. If so, de-
α1 1
 α2  = t  −2  termine whether its representation as a linear combination of
α3 1 the vectors in B is unique.
c) Determine whether x2 = [4, −1, 6, 2]T is in span B. If so, deter-
Since there are infinitely many solutions, A is linearly dependent.
mine whether its representation as a linear combination of the
Example 1.3.7. In R2×2 let A = {B1 , B2 , B3 , B4 }, where vectors in B is unique.

        2) In Z2 , let B = {p0 , p1 , p2 }, where

1 0 1 1 1 1 1 1
B1 = , B2 = , B3 = , B4 =
0 0 0 0 1 0 1 1 p0 (x) = x2 + 2x + 1, p1 (x) = x2 − 1, p2 (x) = x2 + x + 1

10 11
 Determine whether B is linearly independent. Proof. (⇒) Suppose that A is linearly dependent. Then the equation
3) Let A = {v1 , v2 , v3 } be the set vectors given in Example 1.3.6.
α 1 v1 + α 2 v2 + · · · + α k vk = 0
a) Write 0 as a linear combination of the vectors in A in two dis-
tinct ways. has at least one nontrivial solution. That is, there is some solution with
b) Define v = v1 + 2v2 − 3v3 . Write v as a linear combination at least one coefficient, say αi , nonzero. For this solution,
of the vectors in A in another way. What are all other ways of
−αi vi = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αk vk
writing v as a linear combination of the vectors in A?
α1 αi−1 αi+1 αk
4) Let B = {v1 , v2 , . . . , vk } be a family of vectors in a real vector vi = − v1 − · · · − vi−1 − vi+1 − · · · − vk
αi αi αi αi
space V.
a) If x, y ∈ span B, show that x + y ∈ span B. Thus vi ∈ span A0 , where A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vk }.
(⇐) Suppose there is an index i such that vi is in the span of the
b) If x ∈ span B and γ ∈ R, show that γx ∈ span B.
remaining vectors in A. Thus there are scalars α1 , . . ., αi−1 , αi+1 , . . ., αk
5) Let {v1 , v2 } be a family in a real vector space V, and define w1 = such that
v1 + v2 and w2 = v1 − v2 . If {v1 , v2 } is linearly independent, show
that {w1 , w2 } is linearly independent. vi = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αn vn
6) Let V be a real vector space and let A = {v1 , v2 , . . . , vk } be a
Subtracting vi from both sides,
family of vectors in V. If vj = 0 for some j show that A is linearly
dependent. 0 = α1 v1 + · · · + αi−1 vi−1 − vi + αi+1 vi+1 + · · · + αn vn .

Since not all coefficients on the right side are 0 (the coefficient of vi is −1),
1.4. More on Spanning and Independent Sets
A is linearly dependent.

Let A = {v1 , v2 , . . . , vk } be a family of vectors in a real vector Corollary 1.4.2. Let A = {v1 , v2 , . . . , vk } be a linearly independent
space V. We’ve seen that the span of A is the set of all v ∈ V that can be family of vectors in a real vector space V. Let also v ∈ V and suppose
written as a linear combination of vectors of A in at least one way; A is v∈/ span A. Then B = A ∪ {v} is linearly independent.
linearly independent if each v ∈ span A can be written so in at most one
way. Since the notions of span and independence both arise from that of Proof. Suppose α1 , α2 , . . ., αk , β are scalars such that
a linear combination, they are related to each other in subtle ways. In
this section, we give results establishing some of these relationships. α1 v1 + α2 v2 + · · · + αk vk + βv = 0. (1.3)

Proposition 1.4.1. Let A = {v1 , v2 , . . . , vk } be a family of vectors in a If β 6= 0 then it follows that

real vector space V. Then A is linearly dependent if and only if at least α1 α2 αk
one vector in A is in the span of the others. v=− v1 − v2 − · · · − vk
β β β

12 13
In particular, v ∈ span A. Since this is impossible, β = 0. Thus (1.3) Adding (1.4) and (1.5), one obtains
reduces to α1 v1 + α2 v2 + · · ·+ αk vk = 0. Since A is linearly independent,
the only solution here is α1 = 0, α2 = 0, . . ., αk = 0. Therefore all v = (α1 + β1 )v1 + · · · + (αi−1 + βi−1 )vi−1
coefficients in (1.3) must be 0, and B is linearly independent. + (αi+1 + βi+1 )vi+1 + · · · + (αk + βk )vk

Theorem 1.4.3. Let A = {v1 , v2 , . . . , vk } be a family of vectors in a real Thus v ∈ span A0 , and hence span A ⊆ span A0 . It follows that span A0 =
vector space V. If A is linearly dependent then A contains some subset span A.
A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vk } such that span A0 = span A.
Before giving our last results on independent and spanning sets, we
Proof. Suppose A is linearly dependent. By Proposition 1.4.1, some introduce a result we will use presently.
vector, say vi , is in the span of the others. Define
Proposition 1.4.4. Let A ∈ Rm×n . If n > m then the homogeneous
0
A = {v1 , . . . , vi−1 , vi+1 , . . . , vk }, system Ax = 0 has at least one nontrivial solution. That is, there is
some x0 ∈ Rn with x0 6= 0 such that Ax0 = 0.
so that vi ∈ span A0 . We show that span A0 = span A.
This result follows from the row-reduction algorithm for solving linear
First, let v ∈ span A0 . Then there are scalars α1 , . . ., αi−1 , αi+1 ,
systems of equations. An alternative, somewhat more formal proof can
. . ., αk such that
be given by induction on m, the number of rows of A. We omit the
v = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αk vk . details.

Then, obviously, Theorem 1.4.5. Let A = {w1 , w2 , . . . , wp } be a spanning set for a real
vector space V. If B = {v1 , v2 , . . . , vk } is a family in V and k > p then
v = α1 v1 + · · · + αi−1 vi−1 + 0vi + αi+1 vi+1 + · · · + αk vk B is linearly dependent.

as well. Thus v ∈ span A, and hence span A0 ⊆ span A. Proof. Since A spans V and each vj is a vector in V, each vj is a linear
Suppose now that v ∈ span A. Then there are scalars α1 , α2 , . . ., αk combination of the vectors in A. Thus there are scalars αij with i = 1, 2,
such that v = α1 v1 + α2 v2 + · · · + αk vk . Thus . . . p and j = 1, 2, . . ., k such that

v − αi vi = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αk vk (1.4) 
 v1 = α11 w1 + α21 w2 + · · · + αp1 wp


v2 = α12 w1 + α22 w2 + · · · + αp2 wp
Furthermore, since vi ∈ span A0 one has αi vi ∈ span A0 as well. (See also 
 ·································


Problem 2 in Exercise Set 1.3.) Thus there are scalars β1 , . . ., βi−1 , βi+1 , vk = α1k w1 + α2k w2 + · · · + αpk wp
. . ., βk such that
(Note that the indices on the right sides above do not follow the usual
αi vi = β1 v1 + · · · + βi−1 vi−1 + βi+1 vi+1 + · · · + βk vk . (1.5) pattern for matrices.) Using these coefficients (but in the “transposed”

14 15
order), produce the homogeneous system of equations Corollary 1.4.6. Let {w1 , w2 , . . . , wp } be a spanning set for a real vector
 space V. If {v1 , v2 , . . . , vk } is a linearly independent family in V then

 α11 x1 + α12 x2 + · · · + α1k xk = 0 k ≤ p.


α21 x1 + α22 x2 + · · · + α2k xk = 0
 ······························
 Exercise Set 1.4


αp1 x1 + αp2 x2 + · · · + αpk xk = 0 1) Let A = {v1 , v2 , . . . , vk } be a linearly independent family of vectors
in a real vector space V. Let also v ∈ V and define B = A ∪ {v}. If
Since k > p, our last proposition implies that this system has at least B is linearly dependent, show that v ∈ span A.
one nontrivial solution. Let β1 , β2 , . . ., βk be such a solution. We show
that β1 v1 + β2 v2 + · · · + βk vk = 0. Since at least one βj is nonzero, this 2) Let A = {v1 , v2 , . . . , vk } be a family of vectors in a real vector
proves that B is linearly dependent. space V, let α1 , α2 , . . ., αk be nonzero scalars, define wi = αi vi for
Note that all i, and set B = {w1 , w2 , . . . , wk }.
a) Show that A is linearly independent if and only if B is linearly
β1 v1 + β2 v2 + · · · + βk vk = β1 (α11 w1 + α21 w2 + · · · + αp1 wp )
independent.
+ β2 (α12 w1 + α22 w2 + · · · + αp2 wp )
b) Show that span A = span B.
+ ·······························
+ βk (α1k w1 + α2k w2 + · · · + αpk wp ) 3) Let A = {v1 , v2 , . . . , vm } be a family of vectors in a real vector
space V, let k < m, and set A0 = {v1 , v2 , . . . , vk }.
Regrouping terms on the right side above, a) If A is linearly independent, show that A0 is linearly indepen-
dent.
β1 v1 + β2 v2 + · · · + βk vk = (α11 β1 + α12 β2 + · · · + α1k βk )w1
+ (α21 β1 + α22 β2 + · · · + α2k βk )w2 b) If A0 is linearly dependent, show that A is linearly dependent.

+ ······························ 4) Let A = {v1 , v2 , v3 } be a linearly independent family of vectors in

+ (αp1 β1 + αp2 β2 + · · · + αpk βk )wp a real vector space V. Define w1 = 2v1 + v2 , w2 = 2v2 + v3 , and
w3 = 2v3 + v1 , and set B = {w1 , w2 , w3 }. Show that B is linearly
Since αi1 β1 + αi2 β2 + · · · + αik βk = 0 for i = 1, 2, . . ., p, our last equation independent.
reduces to 5) Let A = {v1 , v2 , . . . , vk } be a linearly independent family of vectors
in a real vector space V. Let also αij with i, j = 1, 2, . . . , k be real
β1 v1 + β2 v2 + · · · + βk vk = 0w1 + 0w2 + · · · + 0wp = 0 scalars, define wj = α1j v1 + α2j v2 + · · · + αkj vk for all j, and set
B = {w1 , w2 , . . . , wk }. Assuming that A = [αij ]ni,j=1 is an invertible
Since at least one βj here is nonzero, B is linearly dependent.
matrix, show that B is linearly independent.
The following corollary is just a restatement of the last result in a
slightly more natural form.

16 17
 1.5. Basis and Dimension B = {v1 , v2 , . . . , vn } it has infinitely many such bases. (We will prove
this in Section 1.6.) Thus there are infinitely many coordinate systems
Let V be a real vector space. Clearly, any family which is linearly
that may be produced in V. Despite this abundance of bases (and hence
independent and spans V has particularly desirable properties. In partic-
of coordinate systems), however, there is one feature common to all of
ular, it may serve as a basis for a coordinate system in V. In this section,
them: if any basis contains, say, n vectors then all of them contain n
we consider such sets in more detail. We also formalize our notion of
vectors.
coordinates.

Definition 1.5.1. Let B be a family of vectors in a real vector space V. Theorem 1.5.4. Let V be a real vector space and let B = {v1 , v2 , . . . , vn }
We say B is a basis for V to mean that B is linearly independent and and D = {w1 , w2 , . . . , wm } be bases for V. Then n = m.
spans V.
Proof. By definition, B and D are each linearly independent families
Theorem 1.5.2. Let B = {v1 , v2 , . . . , vn } be a basis for a real vector
that span V. Since B is linearly independent and D spans V, n ≤ m by
space V. For every v ∈ V, there are unique scalars α1 , α2 , . . ., αn such
Corollary 1.4.6. Since at the same time D is linearly independent and B
that
spans V, we also have m ≤ n. Therefore, n = m.
v = α 1 v1 + α 2 v2 + · · · + α n vn . (1.6)

Definition 1.5.5. Let V be a vector space over R. We say that V is

Proof. Let v ∈ V. Since B is a spanning set for V, every vector in V can
n-dimensional, or V has dimension n, to mean that there is some basis
be written as a linear combination of vectors in B in at least one way.
for V containing n vectors. In this case, we also write dim V = n.
(See also Definition 1.3.1.) Since B is linearly independent as well, each
vector can be written as such a linear combination in at most one way In the special case that V = {0}, it is a common error to assume
(see Proposition 1.3.5 and the discussion preceding it.) Thus there are that B = {0} is a basis for V. However, since any set containing the
unique scalars α1 , α2 , . . ., αn such that (1.6) holds. zero vector is linearly dependent (see Problem 6 of Exercise Set 1.3), B
is not a basis for V. In fact, V has no basis (unless, as some argue, it
Definition 1.5.3. Let B = {v1 , v2 , . . . , vn } be a basis for a real vector is ∅). At any rate, we say that V is 0-dimensional by convention. We
space V. For each v ∈ V, the unique scalars α1 , α2 , . . ., αn of The- note in particular that V is not a 1-dimensional vector space as some first
orem 1.5.2 corresponding to v are called the coordinates of v relative assume.
to B. We also denote the relation (1.6) by writing The definition of dimensionality given above is unambiguous. If one
  basis for V contains n vectors then every basis for V contains n vectors.
α1
 α2  We may also observe the following facts about the dimension of a vector
v=  ... 

space. Suppose V has dimension n and let B = {v1 , v2 , . . . , vn } be a
αn basis for V. If A is a linearly independent family of vectors in V then A
B
contains at most n vectors; if C is a spanning set for V then C contains at
Clearly, a basis is an extremely important set of vectors in a real least n vectors. Indeed, let A = {u1 , u2 , . . . , uk } be linearly independent.
vector space V. Less clearly, perhaps, whenever V has a basis of the form By Corollary 1.4.6, since B is a spanning set for V we have k ≤ n.

18 19
Similarly, let C = {w1 , w2 , . . . , wp } be a spanning set for V. Since B is fewer vectors than the linearly independent set B. Since this is impossible
linearly independent, p ≥ n by Corollary 1.4.6. (see Corollary 1.4.6), A is linearly independent.
We now give two more results concerning properties of bases. [(2) and (3) ⇒ (1)] Suppose A is linearly independent and spans V.
Then A is a basis for V. Since B is also a basis for V and all bases contain
Theorem 1.5.6. Let V be a real n-dimensional vector space, let A = the same number of vectors, k = n.
{w1 , w2 , . . . , wk } be a family of vectors in V, and consider the following
three statements: This result is useful for verifying that some family of vectors forms a
basis for a vector space V. Suppose, for example, that V is n-dimensional
1) k = n;
and A = {v1 , v2 , . . . , vn } is a family of vectors in V. Thus, in particular,
2) A is linearly independent;
A contains n vectors. If A is shown to be linearly independent then by
3) A spans V.
the foregoing result A also spans V. On the other hand, if A spans V
If any two of these statements are true then so is the third. then by the foregoing result A is also linearly independent. In either case,
A forms a basis for V.
Proof. Note first that since V is n-dimensional it has some basis con- Suppose now that V is an n-dimensional vector space and A =
taining n vectors. Let B = {v1 , v2 , . . . , vn } be such a basis. {v1 , v2 , . . . , vk } is a family in V. If A is linearly independent then k ≤ n.
[(1) and (2) ⇒ (3)] Suppose A contains n vectors and is linearly If k = n then A spans V and hence is a basis for V. Otherwise, k < n
independent. Suppose also that v ∈ V. Since B is a spanning set for V and it is impossible for A to span V. On the other hand, suppose A is a
containing n vectors and A∪{v} contains n+1 vectors, A∪{v} is linearly spanning set for V. Then k ≥ n. If k = n then A is linearly independent
dependent by Theorem 1.4.5. Thus there are scalars α1 , α2 , . . ., αn , β and hence a basis for V. Otherwise, k > n and it is impossible for A to
with at least one of them nonzero such that be linearly independent. Our last result elaborates further on these facts.

α1 v1 + α2 v2 + · · · + αn vn + βv = 0. Theorem 1.5.7. Let V be an n-dimensional vector space and let A =

{v1 , v2 , . . . , vk } be a family in V.
If β = 0, this equation reduces to α1 v1 + α2 v2 + · · · + αn vn = 0. Since
1) If A is linearly independent but not a basis for V then it is possible
not all coefficients here are 0, this implies that A is linearly dependent.
to add n − k vectors to A in such a way that the resulting family is
Since this is impossible, β 6= 0. It follows that
a basis for V.
α1 α2 αn 2) If A spans V but is not a basis for V, then it is possible to discard
v=− v1 − v2 − · · · − vn
β β β k − n vectors from A in such a way that the resulting family is a
basis for V.
In particular, v ∈ span A. It follows that V ⊂ span A.
[(1) and (3) ⇒ (2)] Suppose A contains n vectors and spans V. Proof. (1) Since A is linearly independent, k ≤ n; since A is not a basis
Suppose also, for the sake of contradiction, that A is linearly dependent. for V, k < n and A does not span V. Then we may choose vk+1 , vk+2 ,
By Theorem 1.4.3, it contains a subset A0 = {v1 , . . . , vi−1 , vi+1 . . . , vn } . . ., vn in such a way that vk+i ∈ / span{v1 , v2 , . . . , vk+i−1 } for i = 1, 2,
such that span A0 = span A = V. But then the spanning set A0 contains . . ., n − k. To see this, note that all spanning sets for V contain at least

20 21
n vectors, but when i ≤ n − k the set {v1 , v2 , . . . , vk+i−1 } contains fewer also v, w ∈ V and suppose
than n vectors. Thus a vector vk+i ∈ / span{v1 , v2 , . . . , vk+i−1 } may be    
chosen as indicated. Furthermore, since vk+1 ∈ / span A and A is linearly β1 δ1
 β2   δ2 
independent, the set {v1 , v2 , . . . , vk+1 } is linearly independent as well. v= 
 ..  , w= 
 ...  .
Similarly, the sets {v1 , v2 , . . . , vk+2 } and so on, up to {v1 , v2 , . . . , vn }, .
are all linearly independent. Since this last set is linearly independent βn B δn B
and contains n vectors, it is a basis for V.
Show that, if γ ∈ R as well,
(2) We prove by induction on k − n. If k − n = 1 then, in particular,
k > n. Thus A is linearly dependent. By Theorem 1.4.3, there is some   

β1 + δ1 γβ1
set A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vk } such that span A0 = span A = V.  β2 + δ2   γβ2 
Furthermore, A0 contains n vectors. Thus, by Theorem 1.5.6, A0 is a v+w =
 ..  ,
 γv =  
 ..  .
. .
basis for V. βn + δn γβn B
B
Suppose now that m is a natural number and the conclusion holds
whenever k − n = m. If k − n = m + 1 then, as before, k > n and hence
A is linearly dependent. By Theorem 1.4.3, there is some set 1.6. Changing Basis

A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vn+m+1 }. Let V be a real n-dimensional vector space. Thus V has some basis,
say B = {b1 , b2 , . . . , bn }, containing n vectors. We have mentioned in
such that span A0 = span A = V. By our induction hypothesis, we may Section 1.5 that in this case V has infinitely many bases. Thus there are
discard m more vectors from A0 in such a way that the remaining vec- infinitely many coordinate systems for V as well. In this section, given a
tors form a basis for V. Since, in this case, the total number of vectors basis B for V, we characterize all other bases for V. We also show how
discarded is m + 1, this proves our result. . to change from one coordinate system to another.
We begin by showing that V has infinitely many bases.
Exercise Set 1.5
Theorem 1.6.1. Let B = {v1 , v2 , . . . , vn } be a basis for a real vector
1) What is the dimension of Rn ? [Justify your answer.] space V and let D = {w1 , w2 , . . . , wm } be a family of vectors in V. Then
2) Let n ∈ N and let Zn be as in Example 1.1.2. Find a basis for Zn and D is a basis for V if and only if m = n and there is an invertible matrix
show that your answer is correct. Hence determine the dimension A = [αij ]ni,j=1 such that wj = α1j v1 + α2j v2 + · · · + αnj vn for all j.
of Zn . Before proving this result, we recall the following fact usually proved in
an introductory lineary algebra course. A square matrix A is invertible if
3) Let m, n ∈ N and let Rm×n be as in Example 1.1.3. Find a basis
and only if the matrix-vector equation Ax = 0 has unique solution x = 0.
for Rm×n and show that your answer is correct. Hence determine
the dimension of Rm×n .
Proof. (⇒) Suppose D is a basis for V. Then m = n by Theorem 1.5.4.
4) Let B = {b1 , b2 , . . . , bn } be a basis for a real vector space V. Let Also, since B is a basis for V and wj ∈ V for all j, there are scalars αij

22 23
with i, j = 1, 2, . . ., n such that wj = α1j v1 + α2j v2 + · · · + αnj vn for Regrouping on the right side above,
all j. Define A = [αij ]ni,j=1 . To show that A is invertible we show that
the equation Ax = 0 has unique solution x = 0. 0 = (α11 β1 + α12 β2 + · · · + α1n βn )v1
Clearly, x = 0 is a solution. Suppose now that x = [β1 , β2 , . . . , βn ]T + (α21 β1 + α22 β2 + · · · + α2n βn )v2
is also a solution. Thus + ·······························
αi1 β1 + αi2 β2 + · · · + αin βn = 0 + (αn1 β1 + αn2 β2 + · · · + αnn βn )vn

for all i. Then Since B is linearly independent, all coefficients on the right side above
0 = 0v1 + 0v2 + · · · + 0vn are 0. That is, αi1 β1 + αi2 β2 + · · · + αin βn = 0 for i = 1, 2, . . ., n.
Equivalently,
= (α11 β1 + α12 β2 + · · · + α1n βn )v1    
β1 0
+ (α21 β1 + α22 β2 + · · · + α2n βn )v2  β2   0 
A   
 ..  =  ... 
+ ······························· .
+ (αn1 β1 + αn2 β2 + · · · + αnn βn )vn βn 0

Regrouping terms on the right side above, Since A is invertible, βi = 0 for all i. Thus D is linearly independent.
Since D also contains n vectors, D spans V by Theorem 1.5.6. Thus D
0 = β1 (α11 v1 + α21 v2 + · · · + αn1 vn ) is a basis for V.
+ β2 (α12 v1 + α22 v2 + · · · + αn2 vn )
Suppose now that V is a real n-dimensional vector space and B =
+ ······························· {v1 , v2 , . . . , vn } and D = {w1 , w2 , . . . , wn } are bases for V. Then for
+ βn (α1n v1 + α2n v2 + · · · + αnn vn ) each v ∈ V, there are scalars β1 , β2 , . . ., βn and δ1 , δ2 , . . ., δn such that
= β1 w1 + β2 w2 + · · · + βn wn
v = β1 v 1 + β2 v 2 + · · · + βn v n , v = δ1 w1 + δ2 w2 + · · · + δn wn .
Since D is a basis for V, it is linearly independent. Thus βi = 0 for all i.
Since the equation Ax = 0 has unique solution x = 0, A is invertible.
Alternatively, we may write
(⇒) Suppose now that m = n, there are scalars αij such that wj =
α1j v1 + α2j v2 + · · · + αnj vn for all j, and A = [αij ] is invertible. We   
β1 δ1
show first that D = {w1 , w2 , . . . , wn } is linearly independent. Suppose  β2   δ2 
β1 , β2 , . . ., βn are scalars such that β1 w1 + β2 w2 + · · · + βn wn = 0. Then v= 
 ..  , v= 
 ...  (1.7)
.
0 = β1 (α11 v1 + α21 v2 + · · · + αn1 vn ) βn B δn D

+ β2 (α12 v1 + α22 v2 + · · · + αn2 vn ) We now show how these columns of coordinates are related.
+ ······························· Since B is a basis for V it is, among other things, a spanning set
+ βn (α1n v1 + α2n v2 + · · · + αnn vn ) for V. Since wj ∈ V for all j, there are scalars αij with i, j = 1, 2, . . ., n

24 25
such that so that x and y are ordinary columns in Rn . Then the columns of
      coordinates for v relative to B and D, respectively, satisfy the relation
α11 α12 α1n
 α21   α22   α2n  S −1 y = x. Alternatively, Sx = y. We call S the change-of-basis matrix
w1 =  
 ...  , w2 =  
 ...  , ..., wn =  
 ...  (1.8) from the B- to the D-system of coordinates, while S −1 is the change-of-
αn1 αn2 αnn basis matrix from the D- to the B-system.
B B B
To summarize the foregoing discussion, we have the following al-
Let v ∈ V be a vector with B- and D-system coordinates as in (1.7). As
gorithm for changing from one system of coordinates to another. Let
noted above, the latter relation means that
B = {v1 , v2 , . . . , vn } be a given basis for V and let D = {w1 , w2 , . . . , wn }
δ1 w1 + δ2 w2 + · · · + δn wn = v. be another basis. To produce the change-of-basis matrix from the B- to
the D-system of coordinates:
Representing w1 , w2 , . . ., wn , and v in their B-system coordinates,
        1) For j = 1, 2, . . . , n, produce the family of coordinates α1j , α2j ,
α11 α12 α1n β1
 α21   α22   α2n   β2  . . ., αnj for wj relative to B. Thus each wj may be represented as
δ1      
 ...  + δ2  ...  + · · · + δn  ...  =  .. 
  in (1.8);
.
αn1 αn2 αnn βn 2) Define S −1 = [αij ]n×n . (This is the change-of-basis matrix for pass-
B B B B
ing from the D- to the B-system of coordinates.)
Following the arithmetic established in Problem 1 of Exercise Set 1.5, we
obtain 3) Compute S = (S −1 )−1 . This is the desired change-of-basis matrix.
   
α11 δ1 + α12 δ2 + · · · + α1n δn β1
 α21 δ1 + α22 δ2 + · · · + α2n δn   β2  Exercise Set 1.6
   
 · · · · · · · · · · · · · · · · · · · · · · · · · · · ·  =  .. 
. 2
1) In R let B = {v1 , v2 }, where
αn1 δ1 + αn2 δ2 + · · · + αnn δn B βn B    
Since the coordinates of v relative to B are unique, this yields the simul- −2 4
v1 = , v2 =
3 −3
taneous scalar equations

 α11 δ1 + α12 δ2 + · · · + α1n δn = β1 a) Find S, the change-of-basis matrix from the E- to the B-system


 of coordinates.
α21 δ1 + α22 δ2 + · · · + α2n δn = β2
 ······························ b) Find the B-coordinates for x and y, where


    
αn1 δ1 + αn2 δ2 + · · · + αnn δn = βn 5 7
x= , y=
Define S −1 = [αij ]n×n . Note that S −1 is, indeed, invertible by our last 3 −4
result. Define also
    2) In R3 let B = {v1 , v2 , v3 }, where
β1 δ1
 β2   δ2       
x=  y=  −2 −1 2
 ..  ,  ...  , v1 =  1  , v2 =  0  , v3 =  1 
.
βn δn 0 1 1

26 27
 a) Find S, the change-of-basis matrix from the E- to the B-system
of coordinates.
b) Find the B-coordinates for x and y, where
   
1 2
x = 2, y = 1
3 1

3) In Z2 , let p0 , p1 , and p2 denote the respective polynomials 1, x,

and x2 , and set B = {p0 , p1 , p2 }. Let also q0 , q1 , and q2 denote the
respective polynomials

1 + x − x2 , 1 − x + x2 , −1 + x + x2 .

and set D = {q0 , q1 , q2 }. Then B and D are each bases for Z2 , and
you may assume so here.
a) Find S, the change-of-basis matrix from the B- to the D-system
of coordinates.
b) Let r(x) = 3 + 4x + 5x2 . Hence find the coordinates of r relative
to D.

28 29