i i “main”

2007/2/16
page 301

i i

4.8 Row Space and Column Space 301

25. V = M2 (R); B = {E12 , E22 , E21 , E11 }; 33. v = (−1, 2, 0); V , B, and C from Problem 19.
C = {E22 , E11 , E21 , E12 }.
34. p(x) = 6 − 4x; V , B, and C from Problem 20.
For Problems 26–31, find the change-of-basis matrix PB←C
from the given basis C to the given basis B of the vector 35. p(x) = 5 − x + 3x 2 ; V , B, and C from Problem 21.
space V .  
−1 −1
36. A = ; V , B, and C from Problem 24.
26. V , B, and C from Problem 16. −4 5
27. V , B, and C from Problem 17. 37. Prove part (b) of Lemma 4.7.5.
28. V , B, and C from Problem 18. 38. Prove that if every vector v in a vector space V can be
29. V , B, and C from Problem 20. written uniquely as a linear combination of the vectors
in {v1 , v2 , . . . , vn }, then {v1 , v2 , . . . , vn } is a basis for
30. V , B, and C from Problem 22. V.
31. V , B, and C from Problem 25. 39. Show that if B is a basis for a finite-dimensional vec-
For Problems 32–36, verify Equation (4.7.6) for the given tor space V , and C is a basis obtained by reordering
vector. the vectors in B, then the matrices PC←B and PB←C
each contain exactly one 1 in each row and column,
32. v = (−5, 3); V , B, and C from Problem 16. and zeros elsewhere.

4.8 Row Space and Column Space

In this section, we consider two vector spaces that can be associated with any m × n
matrix. For simplicity, we will assume that the matrices have real entries, although the
results that we establish can easily be extended to matrices with complex entries.

Row Space
Let A = [aij ] be an m × n real matrix. The row vectors of this matrix are row n-vectors,
and therefore they can be associated with vectors in Rn . The subspace of Rn spanned by
these vectors is called the row space of A and denoted rowspace(A). For example, if
 
2 −1 3
A= ,
5 9 −7
then
rowspace(A) = span{(2, −1, 3), (5, 9, −7)}.
For a general m × n matrix A, how can we obtain a basis for rowspace(A)? By its
very definition, the row space of A is spanned by the row vectors of A, but these may
not be linearly independent, hence the row vectors of A do not necessarily form a basis
for rowspace(A). We wish to determine a systematic and efficient method for obtaining
a basis for the row space. Perhaps not surprisingly, it involves the use of elementary row
operations.
If we perform elementary row operations on A, then we are merely taking linear
combinations of vectors in rowspace(A), and we therefore might suspect that the row
space of the resulting matrix coincides with the row space of A. This is the content of
the following theorem.

Theorem 4.8.1 If A and B are row-equivalent matrices, then

rowspace(A) = rowspace(B).

i i

i i
 i i “main”
2007/2/16
page 302

i i

302 CHAPTER 4 Vector Spaces

Proof We establish that the matrix that results from performing any of the three ele-
mentary row operations on a matrix A has the same row space as the row space of A. If
we interchange two rows of A, then clearly we have not altered the row space, since we
still have the same set of row vectors (listed in a different order).
Now let a1 , a2 , . . . , am denote the row vectors of A. We combine the remaining
two types of elementary row operations by considering the result of replacing ai by the
vector rai + saj , where r (= 0) and s are real numbers. If s = 0, then this corresponds
to scaling ai by a factor of r, whereas if r = 1 and s = 0, this corresponds to adding a
multiple of row j to row i. If B denotes the resulting matrix, then

rowspace(B) = {c1 a1 + c2 a2 + · · · + ci (rai + saj ) + · · · + cm am }

= {c1 a1 + c2 a2 + · · · + (rci )ai + · · · + (cj + sci )aj + · · · + cm am }
= {c1 a1 + c2 a2 + · · · + di ai + · · · + dj aj + · · · + cm am },

where di = rci and dj = cj + sci . Note that di and dj can take on arbitrary values,
hence the vectors in rowspace(B) consist precisely of arbitrary linear combinations of
a1 , a2 , . . . , am . That is,

rowspace(B) = span{a1 , a2 , . . . , am } = rowspace(A).

The previous theorem is the key to determining a basis for rowspace(A). The idea
we use is to reduce A to row-echelon form. If d1 , d2 , . . . , dk denote the nonzero row
vectors in this row-echelon form, then from the previous theorem,

rowspace(A) = span{d1 , d2 , . . . , dk }.

We now establish that {d1 , d2 , . . . , dk } is linearly independent. Consider

c1 d1 + c2 d2 + · · · + ck dk = 0. (4.8.1)

Owing to the positioning of the leading ones in a row-echelon matrix, each of the row
vectors d1 , d2 , . . . , dk−1 will have a leading one in a position where each succeeding
row vector in the row-echelon form has a zero. Hence, Equation (4.8.1) is satisfied only if

c1 = c2 = · · · = ck−1 = 0,

and therefore, it reduces to

ck dk = 0.
However, dk is a nonzero vector, and so we must have ck = 0. Consequently, all of
the constants in Equation (4.8.1) must be zero, and therefore {d1 , d2 , . . . , dk } not only
spans rowspace(A), but also is linearly independent. Hence, {d1 , d2 , . . . , dk } is a basis
for rowspace(A). We have therefore established the next theorem.

Theorem 4.8.2 The set of nonzero row vectors in any row-echelon form of an m × n matrix A is a basis
for rowspace(A).
As a consequence of the preceding theorem, we can conclude that all row-echelon
forms of A have the same number of nonzero rows. For if this were not the case, then we
could find two bases for rowspace(A) containing a different number of vectors, which
would contradict Corollary 4.6.5. We can therefore consider Theorem 2.4.10 as a direct
consequence of Theorem 4.8.2.

i i

i i
 i i “main”
2007/2/16
page 303

i i

4.8 Row Space and Column Space 303

Example 4.8.3 Determine a basis for the row space of

 
1 −1 1 3 2
 2 −1 1 5 1 
A= 
 3 −1 1 7 0  .
0 1 −1 −1 −3

Solution: We first reduce A to row-echelon form:

   
1 −1 1 3 2 1 −1 1 3 2
1 0 1 −1 −1 −3   
 2  0 1 −1 −1 −3 
A∼  0 2 −2 −2 −6  ∼  0 0 0 0 0  .
0 1 −1 −1 −3 0 0 0 0 0

1. A12 (−2), A13 (−3) 2. A23 (−2), A24 (−1)

Consequently, a basis for rowspace(A) is {(1, −1, 1, 3, 2), (0, 1, −1, −1, −3)}, and
therefore rowspace(A) is a two-dimensional subspace of R5 .

Theorem 4.8.2 also gives an efficient method for determining a basis for the subspace
of Rn spanned by a given set of vectors. If we let A be the matrix whose row vectors are
the given vectors from Rn , then rowspace(A) coincides with the subspace of Rn spanned
by those vectors. Consequently, the nonzero row vectors in any row-echelon form of A
will be a basis for the subspace spanned by the given set of vectors.

Example 4.8.4 Determine a basis for the subspace of R4 spanned by {(1,2,3,4),(4,5,6,7),(7,8,9,10)}.

Solution: We first let A denote the matrix that has the given vectors as row vectors.
Thus,
 
1 2 3 4
A = 4 5 6 7 .
7 8 9 10
We now reduce A to row-echelon form:
     
1 2 3 4 1 2 3 4 1 2 3 4
A ∼  0 −3 −6 −9  ∼  0 1 3 ∼ 0 1 2 3.
1 2 3
2
0 −6 −12 −18 0 −6 −12 −18 0 0 0 0

1. A12 (−4), A13 (−7) 2. M2 (− 13 ) 3. A23 (6)

Consequently, a basis for the subspace of R4 spanned by the given vectors is {(1, 2, 3, 4),
(0, 1, 2, 3)}. We see that the given vectors span a two-dimensional subspace of R4 .

Column Space
If A is an m × n matrix, the column vectors of A are column m-vectors and therefore
can be associated with vectors in Rm . The subspace of Rm spanned by these vectors is
called the column space of A and denoted colspace(A).

i i

i i
 i i “main”
2007/2/16
page 304

i i

304 CHAPTER 4 Vector Spaces

Example 4.8.5 For the matrix

 
2 −1 3
A= ,
5 9 −7
we have colspace(A) = span{(2, 5), (−1, 9), (3, −7)}.

We now consider the problem of determining a basis for the column space of an
m × n matrix A. Since the column vectors of A coincide with the row vectors of AT , it
follows that
colspace(A) = rowspace(AT ).
Hence one way to obtain a basis for colspace(A) would be to reduce AT to row-echelon
form, and then the nonzero row vectors in the resulting matrix would form a basis for
colspace(A).
There is, however, a better method for determining a basis for colspace(A) directly
from any row-echelon form of A. The derivation of this technique is somewhat involved
and will require full attention.
We begin by determining the column space of an m×n reduced row-echelon matrix.
In order to introduce the basic ideas, consider the particular reduced row-echelon matrix
 
1 2 0 3 0
0 0 1 5 0
E= 0 0 0 0 1.


0 0 0 0 0
In this case, we see that the first, third, and fifth column vectors, which are the column
vectors containing the leading ones, coincide with the first three standard basis vectors
in R4 (written as column vectors):
     
1 0 0
0 1 0
e1 = 0,
 e2 = 0,
 e3 = 1.


0 0 0
Consequently, these column vectors are linearly independent. Furthermore, the remain-
ing column vectors in E (those that do not contain leading ones) are both linear combina-
tions of e1 and e2 , columns that do contain leading ones. Therefore {e1 , e2 , e3 } is a linearly
independent set of vectors that spans colspace(E), and so a basis for colspace(E) is

{(1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0)}.

Clearly, the same arguments apply to any reduced row-echelon matrix E. Thus, if
E contains k (necessarily ≤ n) leading ones, a basis for colspace(E) is {e1 , e2 , . . . , ek }.
Now consider an arbitrary m×n matrix A, and let E denote the reduced row-echelon
form of A. Recall from Chapter 2 that performing elementary row operations on a linear
system does not alter its solution set. Hence, the two homogeneous systems of equations

Ac = 0 and Ec = 0 (4.8.2)

have the same solution sets. If we write A and E in column-vector form as A =

[a1 , a2 , . . . , an ] and E = [d1 , d2 , . . . , dn ], respectively, then the two systems in (4.8.2)
can be written as
c1 a1 + c2 a2 + · · · + cn an = 0,
c1 d1 + c2 d2 + · · · + cn dn = 0,
respectively. Thus, the fact that these two systems have the same solution set means that
a linear dependence relationship will hold between the column vectors of E if and only if

i i

i i
 i i “main”
2007/2/16
page 305

i i

4.8 Row Space and Column Space 305

precisely the same linear dependence relation holds between the corresponding column
vectors of A. In particular, since our previous work shows that the column vectors in
E that contain leading ones give a basis for colspace(E), they give a maximal linearly
independent set in colspace(E). Therefore, the corresponding column vectors in A will
also be a maximal linearly independent set in colspace(A). Consequently, this set of
vectors from A will be a basis for colspace(A).
We have therefore shown that the set of column vectors of A corresponding to those
column vectors containing leading ones in the reduced row-echelon form of A is a basis
for colspace(A). But do we have to reduce A to reduced row-echelon form? The answer
is no. We need only reduce A to row-echelon form. The reason is that going further
to reduce a matrix from row-echelon form to reduced row-echelon form does not alter
the position or number of leading ones in a matrix, and therefore the column vectors
containing leading ones in any row-echelon form of A will correspond to the column
vectors containing leading ones in the reduced row-echelon form of A. Consequently,
we have established the following result.

Theorem 4.8.6 Let A be an m × n matrix. The set of column vectors of A corresponding to those column
vectors containing leading ones in any row-echelon form of A is a basis for colspace(A).

Example 4.8.7 Determine a basis for colspace(A) if

 
1 2 −1 −2 −1
 2 4 −2 −3 −1 
A= 
 5 10 −5 −3 −1  .
−3 −6 3 2 1

Solution: We first reduce A to row-echelon form:

   
1 2 −1 −2 −1 1 2 −1 −2 −1
1 0 0 0 1 1 
 2 0 0 0 1 1

A∼  0 0 0 7 4  ∼  0 0 0 0 −3 
0 0 0 −4 −2 0 0 0 0 2
   
1 2 −1 −2 −1 1 2 −1 −2 −1
3 0 0 0 1 1 
 4 0 0 0 1 1

∼ 0 0 0 0 1 ∼ 0 0 0 0 1.
0 0 0 0 2 0 0 0 0 0

1. A12 (−2), A13 (−5), A14 (3) 2. A23 (−7), A24 (4) 3. M3 (− 13 ) 4. A34 (−2)

Since the first, fourth, and fifth column vectors in this row-echelon form of A contain
the leading ones, it follows from Theorem 4.8.6 that the set of corresponding column
vectors in A is a basis for colspace(A). Consequently, a basis for colspace(A) is

{(1, 2, 5, −3), (−2, −3, −3, 2), (−1, −1, −1, 1)}.

Hence, colspace(A) is a three-dimensional subspace of R4 . Notice from the row-echelon

form of A that a basis for rowspace(A) is {(1, 2, −1, −2, −1), (0, 0, 0, 1, 1), (0, 0, 0,
0, 1)} so that rowspace(A) is a three-dimensional subspace of R5 .

We now summarize the discussion of row space and column space.

i i

i i
 i i “main”
2007/2/16
page 306

i i

306 CHAPTER 4 Vector Spaces

Summary: Let A be an m × n matrix. In order to determine a basis for rowspace(A)

and a basis for colspace(A), we reduce A to row-echelon form.

1. The row vectors containing the leading ones in the row-echelon form
give a basis for rowspace(A) (a subspace of Rn ).
2. The column vectors of A corresponding to the column vectors containing
the leading ones in the row-echelon form give a basis for colspace(A)
(a subspace of Rm ).

Since the number of vectors in a basis for rowspace(A) or in a basis for colspace(A)
is equal to the number of leading ones in any row-echelon form of A, it follows that
dim[rowspace(A)] = dim[colspace(A)] = rank(A).
However, we emphasize that rowspace(A) and colspace(A) are, in general, subspaces
of different vector spaces. In Example 4.8.7, for instance, rowspace(A) is a subspace
of R5 , while colspace(A) is a subspace of R4 . For an m × n matrix, rowspace(A) is a
subspace of Rn , whereas colspace(A) is a subspace of Rm .

Exercises for 4.8

Key Terms 6. If A is an n × n invertible matrix, then colspace(A) =
Row space, Column space. Rn .

Skills Problems
For Problems 1–6, determine a basis for rowspace(A) and a
• Be able to compute a basis for the row space of a basis for colspace(A).
matrix.
 
• Be able to compute a basis for the column space of a 1 −2
1. A = .
matrix. −3 6
 
True-False Review 1 1 −3 2
2. A = .
3 4 −11 7
For Questions 1–6, decide if the given statement is true or
false, and give a brief justification for your answer. If true,  
1 2 3
you can quote a relevant definition or theorem from the text.
3. A =  5 6 7 .
If false, provide an example, illustration, or brief explanation
9 10 11
of why the statement is false.
 
1. If A is an m × n matrix such that rowspace(A) = 0 3 1
colspace(A), then m = n. 4. A =  0 −6 −2 .
0 12 4
2. A basis for the row space of a matrix A consists of the
row vectors of any row-echelon form of A.  
1 2 −1 3
3. The nonzero column vectors of a row-echelon form of 3 6 −3 5 
5. A = 
1
.
a matrix A form a basis for colspace(A). 2 −1 −1 
5 10 −5 7
4. The sets rowspace(A) and colspace(A) have the same
dimension.  
1 −1 2 3
5. If A is an n × n invertible matrix, then rowspace(A) = 6. A =  1 1 −2 6 .
Rn . 3 1 4 2

i i

i i
 i i “main”
2007/2/16
page 307

i i

4.9 The Rank-Nullity Theorem 307

 
For Problems 7–10, use the ideas in this section to determine 1 2 4
a basis for the subspace of Rn spanned by the given set of 12. Let A =  5 11 21 .
vectors. 3 7 13

7. {(1, −1, 2), (5, −4, 1), (7, −5, −4)}. (a) Find a basis for rowspace(A) and colspace(A).

8. {(1, 3, 3), (1, 5, −1), (2, 7, 4), (1, 4, 1)}. (b) Show that rowspace(A) corresponds to the plane
with Cartesian equation 2x + y − z = 0, whereas
9. {(1, 1, −1, 2), (2, 1, 3, −4), (1, 2, −6, 10)}. colspace(A) corresponds to the plane with Carte-
sian equation 2x − y + z = 0.
10. {(1, 4, 1, 3), (2, 8, 3, 5), (1, 4, 0, 4), (2, 8, 2, 6)}.
13. Give examples to show how each type of elementary
11. Let   row operation applied to a matrix can change the col-
−3 9
A= . umn space of the matrix.
1 −3
Find a basis for rowspace(A) and colspace(A). Make 14. Give an example of a square matrix A whose row space
a sketch to show each subspace in the xy-plane. and column space have no nonzero vectors in common.

4.9 The Rank-Nullity Theorem

In Section 4.3, we defined the null space of a real m × n matrix A to be the set of all real
solutions to the associated homogeneous linear system Ax = 0. Thus,

nullspace(A) = {x ∈ Rn : Ax = 0}.

The dimension of nullspace(A) is referred to as the nullity of A and is denoted nullity(A).

In order to find nullity(A), we need to determine a basis for nullspace(A). Recall that if
rank(A) = r, then any row-echelon form of A contains r leading ones, which correspond
to the bound variables in the linear system. Thus, there are n−r columns without leading
ones, which correspond to free variables in the solution of the system Ax = 0. Hence,
there are n − r free variables in the solution of the system Ax = 0. We might therefore
suspect that nullity(A) = n − r. Our next theorem, often referred to as the Rank-Nullity
Theorem, establishes that this is indeed the case.

Theorem 4.9.1 (Rank-Nullity Theorem)

For any m × n matrix A,

rank(A) + nullity(A) = n. (4.9.1)

Proof If rank(A) = n, then by the Invertible Matrix Theorem, the only solution to
Ax = 0 is the trivial solution x = 0. Hence, in this case, nullspace(A) = {0}, so
nullity(A) = 0 and Equation (4.9.1) holds.
Now suppose rank(A) = r < n. In this case, there are n − r > 0 free variables
in the solution to Ax = 0. Let t1 , t2 , . . . , tn−r denote these free variables (chosen as
those variables not attached to a leading one in any row-echelon form of A), and let
x1 , x2 , . . . , xn−r denote the solutions obtained by sequentially setting each free variable
to 1 and the remaining free variables to zero. Note that {x1 , x2 , . . . , xn−r } is linearly inde-
pendent. Moreover, every solution to Ax = 0 is a linear combination of x1 , x2 , . . . , xn−r :

x = t1 x1 + t2 x2 + · · · + tn−r xn−r ,

which shows that {x1 , x2 , . . . , xn−r } spans nullspace(A). Thus, {x1 , x2 , . . . , xn−r } is a
basis for nullspace(A), and nullity(A) = n − r.

i i

i i