4.

8 Row space, column space, and null space of a matrix

Definition 1. Let A ∈ Rm×n .
1. The space spanned by the rows of A is called the row space of A.
It is a subspace of Rn , and it is denoted by ROW(A).
2. The space spanned by the columns of A is called the column space of A.
It is a subspace of Rm , and it is denoted by COL(A).
3. The space of solutions of the homogeneous system Ax = 0 is called the null space of A.
It is a subspace of Rn , and it is denoted by NUL(A).
 
1 2 3
Example: For the matrix A = ∈ R2×3 , we have:
2 4 6

1. ROW(A) = sp((1, 2, 3), (2, 4, 6)) ⊆ R3 ,

     
1 2 3
2. COL(A) = sp , , ⊆ R2 .
2 4 6
Since the columns of an m × n matrix are elements of Rm , we can also write them as row vectors.
In this example: COL(A) = sp((1, 2), (2, 4), (3, 6)) ⊆ R2 .

3. NULL(A) = (x, y, z) ∈ R3 : x + 2y + 3z = 0, 2x + 4y + 6z = 0 .
Definition 2.
1. The dimension of ROW(A) is called the row rank of A, it is denoted by rankr (A).
2. The dimension of COL(A) is called the column rank of A, it is denoted by rankc (A).
3. The dimension of NUL(A) is called the nullity of A, it is denoted by nullity(A) .

Observation. We obviously have ROW(A) = COL(A⊤ ) and (hence) rankr (A) = rankc (A⊤ ).
In the next proposition we will see that elementary row operations do not change rankr and rankc .
Thus, if a matrix B is obtained form A by an elementary row operation, then rankr (A) = rankr (B) and
rankc (A) = rankc (B). For the row rank, the result is quite expected and the proof is easy: in fact we will prove
that A and B have the same row space (NB: rankr is the dimension of this space). In contrast,the column

0 0 1 1
spaces of A and of B are not necessarily the same. (A simple example: A = −→ B = . We
1 1 0 0
have COL(A) = sp((0, 1)), COL(B) = sp((1, 0)).) Therefore it is somewhat unexpected that their column
spaces have necessarily the same dimension.
Proposition. Let A, B ∈ Rm×n , and assume that B is obtained form A by an elementary row operation.
Then we have:
1. ROW(A) = ROW(B).
2. rankr (A) = rankr (B).
3. rankc (A) = rankc (B).

Proof.
1. If B is obtained from A by an elementary row operation, then every row of B is a linear combination
of the rows of A, and (since elementary row operations are “invertible”) vice versa. Hence ROW(A) =
ROW(B).
 2. We have rankc (A) = rankc (B) just because these are the dimensions of ROW(A) and of ROW(B).
3. For every v ∈ Rn we have Av = 0 ⇔ Bv = 0: this was a crucial claim for the Gauss-Jordan method.
This means that columns of A and the corresponding columns of B satisfy precisely the same linear
dependencies. More precisely: For every way to write 0 (the zero vector of Rm ) as a linear combination
of some columns of A, we can write 0 as a linear combination of the corresponding columns of B with
the same coefficients.
   
1 2 3 1 2 3
Example. Let A = and B = . (B is obtained form A by R2 → R2 − 3R1 .)
4 5 6 0 −3 −6
Then we have, for example:
               
1 2 3 0 1 2 3 0
1· −2· +1· = and 1 · −2· +1· =
4 5 6 0 0 −3 −6 0
       
1 3 1 3
The set , is linearly independent, and the set , is linearly independent.
4 6 0 −6
In particular, the maximal linearly independent subset of columns of A corresponds to the maximal
linearly independent subset of columns of B. Therefore we have bases of COL(A) and of COL(B) that
have the same size. Hence, we have dim(COL(A)) = dim(COL(B)), that is, rankc (A) = rankc (B).

We proved that elementary row operations do not change rankr and rankc . Similarly to elementary row
operations, one can define elementary column operations: Ci ↔ Cj , Ci → kCi , Ci → Ci + kCj . Then a
similar proof (or using the transpose) shows that elementary column operations also do not change rankr
and rankc .
The next theorem is non-trivial and fundamental.
Theorem. For every matrix A we have rankr (A) = rankc (A).
Proof. Using elementary row operations, we can bring the given matrix A to the canonical form:

1 ∗ ∗ 0 0 ∗ ∗ ∗ 0 ∗ ∗

r 1 0 ∗ ∗ ∗ 0 ∗ ∗
1 ∗ ∗ ∗ 0 ∗ ∗
C=
0 1 ∗ ∗

Then, using elementary column operations, we can bring this matrix to the form:
r
1
r 1
D= 1
1

(all other entries are 0-s). According to the last proposition, we have rankr (A) = rankr (D) and rankc (A) =
rankc (D). And we clearly have rankr (D) = rankc (D) = r. Hence, rankr (A) = r = rankc (A).
Remarks.
1. Due to this theorem, we will just say “the rank of A” and denote it by rank(A).
 2. The theorem can be also stated as rank(A) = rank(A⊤ ).
3. The proof also yields a practical method for finding rank(A): it is the number of non-zero rows in any
staircase matrix that corresponds to A.

The next results are needed in order to prove another fundamental theorem and to develop general
practical methods for finding bases of vector spaces.
Observation.
(a) Non-zero rows of a staircase matrix C are linearly independent. Hence, they form a basis of ROW(C).
(b) Let A ∈ Rm×n , and consider NUL(A) (the space of solutions of the homogeneous system Ax = 0).
Then the spanning set of NUL(A) found by the Gauss-Jordan method is linearly independent, and,
hence it is a basis of NUL(A).

Proof.
(a) There are many ways for proving this. For example, let R1 , R2 , . . . , Rr be non-zero rows of a staircase
matrix, and denote by b1 , b2 , . . . , br their leading elements. Then if we write 0 = α1 R1 + α2 R2 +
. . . + αr Rr and compare the values in the columns that contain the leading elements, we obtain
α1 b1 = 0, α2 b2 = 0, . . . , αr br = 0 and, hence, α1 = α2 = . . . = αr = 0.
Alternatively, we can modify C as in the proof of the last Theorem. Then we obtain rank(C) = r, which
means that {R1 , R2 , . . . , Rr } is a set of size r that spans a vector space (ROW(C)) of dimension r.
Therefore, by one of the propositions about bases and dimensions, {R1 , R2 , . . . , Rr } is a basis of
ROW(C).
(b) If we construct a matrix whose rows are the elements of this spanning set, then its restriction to the
columns that correspond to the free variables, is a copy of an identity matrix. Then we can perform
elementary column operations to make these columns to be the first columns, thus obtaining a staircase
matrix. Then the result follows from (a).

The next theorem is also a fundamental one.

Theorem. For every A ∈ Rm×n we have

nullity(A) = n − rank(A).

Proof. When we write the general solution of Ax = 0, we partition the variables into leading and free. The
number of leading variables is equal to the number of non-zero rows in a staircase matrix that corresponds
to A: that is, to rank(A). Hence, the number of free variables is n − rank(A). On the other hand, the number
of free variables is nullity(A) by the last Observation (b).

Methods for finding basis / dimension of a subspace of Rn . There are two main ways to specify
a subspace W of Rn , and the standard methods of finding a basis are as follows:
• W is given by a spanning set: W = sp(v1 , v2 , . . . , vm ). We construct a matrix A whose rows are
these vectors. This means W = ROW(A). Then we bring A to a staircase matrix C. By the last
Observation (a), the non-zero rows of C are a basis of W = ROW(A) = ROW(C). Finally, dim(W ) =
rank(A) = the number of non-zero rows in C.
• W is given by a homogeneous system of linear equations Ax = 0. That is, W = NUL(A) for some
matrix A. In this case one solves this system by Gauss-Jordan method, writes the general solution,
and extracts a spanning set of W . As mentioned in the Observation (b), this spanning set is always
linearly independent, and, hence, a basis of W .
 The following two examples illustrate these methods. Note that in this examples we work with the same
matrix A, but the final step is different: in the first example we just collect the rows, in the second example
we still need to solve the system.
Example 1. Find a basis and determine the dimension of W = sp((1, 1, 1, 1, 1), (1, 2, 3, 4, 5), (1, 3, 5, 7, 9))
(a subspace of R5 ).
Solution. Let A be the matrix whose rows are the given vectors. We bring it to a staircase form:
     
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 2 3 4 5 → −  0 1 2 3 4 → − 0 1 2 3 4 .
1 3 5 7 9 0 2 4 6 8 0 0 0 0 0

Therefore, {(1, 1, 1, 1, 1), (0, 1, 2, 3, 4)} is a basis of W , and dim(W ) = 2.

Example 2. Find a basis and determine the dimension of U , the solution space of the system

 x + y + z + t + w = 0
x + 2y + 3z + 4t + 5w = 0

x + 3y + 5z + 7t + 9w = 0

In order to find the general solution of the system, we use the Gauss-Jordan method. The first step is writing
the system using a matrix — this is the same matrix as in Example 1 — and bringing it to a staircase form.
This is also done above, but we can make one more step and bring it to the canonical form (this will save
some calculations at the following steps):
 
1 0 −1 −2 −3
− 0 1 2
··· → 3 4 .
0 0 0 0 0

Now we have the system 

x − z − 2t − 3w = 0
y + 2z + 3t + 4w = 0
The free variables are z, t, w, and the general solution is (z + 2t + 3w, −2z − 3t − 4w, z, t, w), which yields
the spanning set of U : {(1, −2, 1, 0, 0), (2, −3, 0, 1, 0), (3, −4, 0, 0, 1)}. By Observation (b) above, this set is
linearly independent, hence it is a basis of W ; and we have dim(U ) = 3.
Note that we obtained dim(ROW(A)) = 2 and dim(NUL(A)) = 3, in accordance to the formula nullity(A) =
n − rank(A).

Finally, we prove one more theorem which will be of use later.

Theorem. Let A ∈ Rk×m , B ∈ Rm×n . Then we have rank(AB) ≤ rank(A) and rank(AB) ≤ rank(B).
Proof. Since every column of AB is a linear combination of the columns of A, we have COL(AB) ⊆ COL(A).
Hence, dim(COL(AB)) ≤ dim(COL(A)), which means rank(AB) ≤ rank(A). For the second claim one can
similarly use row spaces.

4.9 Coordinate vectors

Proposition + Definition. Let V be a vector space over F , let B = {v1 , . . . , vn } be an ordered basis of V ,
and let v ∈ V . Then there exist uniquely determined scalars α1 , . . . , αn ∈ F such that v = α1 v1 + · · · + αn vn .
The vector [v]B := (α1 , . . . , αn ) ∈ F n is called the coordinate vector of v with respect to the basis B.
Proof. The existence of α1 , . . . , αn follows from the fact that B is a spanning set. To prove the uniqueness,
assume v = α1 v1 + · · · + αn vn = β1 v1 + · · · + βn vn . Then we have (α1 − β1 )v1 + · · · + (αn − βn )vn = 0. Since
B is linearly independent, it follows α1 − β1 = · · · = αn − βn = 0, and hence we have α1 = β1 , . . . , αn = βn .
This means: there is just one way to write v as a linear combination of the vectors from B.
 As usual, finding the coefficients for writing v as a linear combination of given vectors v1 , . . . , vn leads
to a system of linear equations. And in this case we know that this system has a unique solution! For the
standard bases of Rn , Rm×n , Rn [x] (and for some other simple bases) these coefficients are seen directly.
Examples.
1. Let V = R2 , v = (3, −1).

• Consider the standard basis E = {(1, 0), (0, 1)}.

We see directly (3, −1) = 3 · (1, 0) + (−1) · (0, 1). Hence, [(3, −1)]E = (3, −1).
In general, if V = Rn and E is its standard basis, then we have [v]E = v for every v ∈ V .
• Consider the basis B = {(2, 1), (1, 3)}.
In order to determine [(3, −1)]B , we need to find α, β such that (3, −1) = α(2, 1) + β(1, 3). This
leads to the system 
2α + β = 3
α + 3β = −1
Its unique solution is α = 2, β = −1. Hence (3, −1) = 2 · (2, 1) + (−1) · (1, 3), and, therefore,
[(3, −1)]B = (2, −1).
 
9 7
2. Let V = R2×2 , v = .
5 2
       
1 0 0 1 0 0 0 0
• Consider the standard basis E = , , , .
0 0 0 0 1 0 0 1
         
9 7 1 0 0 1 0 0 0 0
We see directly that =9 +7 +5 +2 .
5 2 0 0 0 0 1 0 0 1
 
9 7
Therefore, = (9, 7, 5, 2).
5 2 E
   
a b a b
Similarly, we have = (a, b, c, d) for every ∈ R2 .
c d E c d
In general, if V = Rm×n and E is its standard basis, then, for every v ∈ V , the coordinate vector
[v]E is the row vector that consists of the entries of v, written row by row.
       
1 1 1 2 1 0 1 2
• Consider the basis E = , , , .
1 1 1 2 −1 −2 3 3
 
9 7
In order to determine , we need to find α, β γ, δ such that
5 2 B
         
9 7 1 1 1 2 1 0 1 2
=α +β +γ +δ . This leads to the system
5 2 1 1 1 2 −1 −2 3 3


 α + β + γ + δ = 9

α + 2β + 2δ = 7

 α + β − γ + 3δ = 5

α + 2β − 2γ + 3δ = 2

Its unique
 solution
 is α = 
5, β =0, γ =3, δ  = 1.   
9 7 1 1 1 2 1 0 1 2
Hence, =5 +0 +3 +1
5 2 1 1 1 2 −1 −2 3 3
 
9 7
and, therefore, = (5, 0, 3, 1).
5 2 B

3. Let V = R2 [x], v = 5x2 − 7x + 4.

 • Consider the standard basis E = {1, x, x2 }. Then [v]E = (4, −7, 5) is easily seen directly (however,
note the order of the components); and in general the coordinate vector of v ∈ Rn [x] with respect
to the standard basis consists just of the coefficients of v.
• Consider the basis B = {1, x − 1, (x − 1)2 }. To find [v]B , we need to find α, β, γ such that
5x2 − 7x + 4 = α · 1 + β · (x − 1) + γ · (x − 1)2 . This leads to the system

 α − β + γ = 4
β − 2γ = −7

γ = 5

which has a unique solution α = 2, β = 3, γ = 5. Therefore [v]B = (2, 3, 5).

Proposition. Let V be a vector space over F , and let B be its basis. Then we have

• [u + v]B = [u]B + [v]B for all u, v ∈ V .

• [γv]B = γ[v]B for every v ∈ V and γ ∈ F .
The proof is straightforward, as well as that of the following corollaries.
Corollary. Let V be a vector space over F , B be its basis, and v1 , . . . , vk , v vectors in V . Then we have
1. v = β1 v1 + · · · + βm vm (in V) ⇐⇒ [v]B = β1 [v1 ]B + · · · + βm [vm ]B (in F n ).

2. {v1 , . . . vk } is independent in V ⇐⇒ {[v1 ]B , . . . [vk ]B } is independent in F n .

These results are frequently used when one wants to find a basis for some space of matrices or polynomials.
If a spanning set S is given, then, in order to eliminate possible linear dependence, one constructs a matrix
whose lines contain coefficients of the elements of S, and brings it to a staircase form to obtain a basis.
Formally, the rows of this matrix are coordinate vectors of the elements of S with respect to some basis
(typically, the standard basis).


Example. Find a basis of W = span (x − 1)(x − 2), (x − 1)(x − 3), (x − 1)(x − 4) , a subspace of R2 [x].
Solution. First we write the given vectors in the expanded form: x2 − 3x + 2, x2 − 4x + 3, x2 − 5x + 4.
Then we consider the matrix  
1 −3 2
1 −4 3 .
1 −5 4
The rows of this matrix are the coordinate vectors of the elements of the given spanning set, with respect
to the basis {x2 , x, 1}.
We bring this matrix to a staircase form:
     
1 −3 2 1 −3 2 1 −3 2
1 −4 3 → 0 −1 1 → 0 1 −1 .
1 −5 4 0 −2 2 0 0 0
Once a staircase matrix is obtained, we collect its non-zero rows, convert them back to the polyno-
mials, using the same basis (for example: (1, −3, 2) → 1 · x2 + (−3) · x + 2 · 1), and obtain the answer:
{x2 − 3x + 2, x − 1} is a basis of W , and the dimension of W is 2.