CHAPTER 3

Linear Equations and

Matrices

In this chapter we introduce matrices via the theory of simultaneous linear

equations. This method has the advantage of leading in a natural way to the
concept of the reduced row-echelon form of a matrix. In addition, we will for-
mulate some of the basic results dealing with the existence and uniqueness of
systems of linear equations. In Chapter 5 we will arrive at the same matrix
algebra from the viewpoint of linear transformations.

3.1 SYSTEMS OF LINEAR EQUATIONS

Let a, . . . , a, y be elements of a field F, and let x, . . . , x be unknowns

(also called variables or indeterminates). Then an equation of the form

a x + ~ ~ ~ + a x = y

is called a linear equation in n unknowns (over F). The scalars a are called
the coefficients of the unknowns, and y is called the constant term of the
equation. A vector (c, . . . , c) Fn is called a solution vector of this equa-
tion if and only if
a1 c1 + ~ ~ ~ + an cn = y

115
116 LINEAR EQUATIONS AND MATRICES

in which case we say that (c, . . . , c) satisfies the equation. The set of all
such solutions is called the solution set (or the general solution).
Now consider the following system of m linear equations in n
unknowns:
a11 x1 +!+ a1n xn = y1
a21 x1 +!+ a2n xn = y2
!!!!"!!!!!!!!!!!!!!!!!"!!!!!!!"
am1 x1 +!+ amn xn = ym

We abbreviate this system by

n
! aij x j = yi ,!!!!!!!!!!!!i = 1,!!,!m!!.
j=1

If we let Si denote the solution set of the equation ax = y for each i, then
the solution set S of the system is given by the intersection S = S. In other
words, if (c, . . . , c) Fn is a solution of the system of equations, then it is a
solution of each of the m equations in the system.

Example 3.1 Consider this system of two equations in three unknowns over
the real field :
2x1 ! 3x2 +!!!x3 = 6
!!x1 + 5x2 ! 2x3 = 12

The vector (3, 1, 3) 3 is not a solution of this system because

2(3) - 3(1) + 3 = 6
while
3 + 5(1) - 2(3) = 2 12 .

However, the vector (5, 1, -1) 3 is a solution since

2(5) - 3(1) + (-1) = 6

and
5 + 5(1) - 2(-1) = 12 .

Associated with a system of linear equations are two rectangular arrays of

elements of F that turn out to be of great theoretical as well as practical
significance. For the system ax = y, we define the matrix of coefficients
A as the array
 3.1 SYSTEMS OF LINEAR EQUATIONS 117

! a11 a12 ! a1n $

# &
# a21 a22 ! a2n &
A=
# " " " &
# &
" am1 am2 ! amn %

and the augmented matrix as the array aug A given by

! a11 a12 ! a1n y1 $

# &
a a22 ! a2n y2 &
aug!A = # 21 !!.
# " " " "&
# &
" am1 am2 ! amn yn %

In general, we will use the term matrix to denote any array such as the
array A shown above. This matrix has m rows and n columns, and hence is
referred to as an m x n matrix, or a matrix of size m x n. By convention, an
element a F of A is labeled with the first index referring to the row and the
second index referring to the column. The scalar a is usually called the i, jth
entry (or element) of the matrix A. We will frequently denote the matrix A
by the symbol (a).
Another rather general way to define a matrix is as a mapping from a sub-
set of all ordered pairs of positive integers into the field F. In other words, we
define the mapping A by A(i, j) = a for every 1 i m and 1 j n. In this
sense, a matrix is actually a mapping, and the m x n array written above is just
a representation of this mapping.
Before proceeding with the general theory, let us give a specific example
demonstrating how to solve a system of linear equations.

Example 3.2 Let us attempt to solve the following system of linear equa-
tions:
2x1 +!!x2 ! 2x3 = !3
!!x1 ! 3x2 +!!x3 =!!!8
4x1 !!!x2 ! 2x3 =!!3

That our approach is valid in general will be proved in our first theorem
below.
Multiply the first equation by 1/2 to get the coefficient of x equal to 1:
118 LINEAR EQUATIONS AND MATRICES

!!x1 +!!(1 / 2)x2 !!!!x3 = !3 / 2

!!x1 !!!!!!!!!!3x2 +!!!x3 =!!!!!!!8
4x1 !!!!!!!!!!!!x2 ! 2x3 =!!!!!!!3

Multiply the first equation by -1 and add it to the second to obtain a new sec-
ond equation, then multiply the first by -4 and add it to the third to obtain a
new third equation:
x1 +!!(1 / 2)x2 !!!x3 = !3 / 2
!!!!!!(7 / 2)x2 + 2x3 =!19 / 2
!!!!!!!!!!!!!!3x2 ! 2x3 =!!!!!!!9

Multiply the second by -2/7 to get the coefficient of x equal to 1, then mul-
tiply this new second equation by 3 and add to the third:

x1 +!!(1 / 2)x2 !!!!!!!!!!!x3 =!!!!3 / 2

!!!!!!!!!!!!!!!!!x2 ! (4 / 7)x3 =!!19 / 7
!!!!!!!!!!!!!!!!!!!!!!!!!(2 / 7)x3 =!!!!6 / 7

Multiply the third by 7/2, then add 4/7 times this new equation to the second:

x1 +!!(1 / 2)x2 ! x3 =!!3 / 2

!!!!!!!!!!!!!!!!!x2 !!!!!!!!=!!!!!!1
!!!!!!!!!!!!!!!!!!!!!!!!!x3 =!!!!!!3

Add the third equation to the first, then add -1/2 times the second equation to
the new first to obtain
x1 =!!2
x2 = !1
x3 =!!3

This is now a solution of our system of equations. While this system could
have been solved in a more direct manner, we wanted to illustrate the system-
atic approach that will be needed below.

Two systems of linear equations are said to be equivalent if they have

equal solution sets. That each successive system of equations in Example 3.2
is indeed equivalent to the previous system is guaranteed by the following
theorem.

Theorem 3.1 The system of two equations in n unknowns over a field F

 3.1 SYSTEMS OF LINEAR EQUATIONS 119

a11 x1 + a12 x2 +!+ a1n xn = b1

(1)
a21 x1 + a22 x2 +!+ a2n xn = b2

with a 0 is equivalent to the system

a11 x1 + a12 x2 +!+ a1n xn = b1

(2)
a22
! x2 +!+ a2n
! xn = b2!

in which
a2i = a21 a1i - a11 a2i

for each i = 1, . . . , n and

b2 = a21 b1 - a11 b2 .

Proof Let us define

n
Li = ! aij x j
j=1

so that (1) may be written as the system

L1 = b1
(1)
L2 = b2
while (2) is just
L1 = b1
(2)
a21 L1 ! a11 L2 = a21b1 ! a11b2

If (x, . . . , xn) Fn is a solution of (1), then the two equations

a21 L1 = a21b1
a11 L2 = a11b2
and hence also
a L - a L = a b - a b

are all true equations. Therefore every solution of (1) also satisfies (2).
Conversely, suppose that we have a solution (x, . . . , x) to the system
(2). Then clearly
a L = a b

is a true equation. Hence, subtracting the second of (2) from this gives us
120 LINEAR EQUATIONS AND MATRICES

a L - (a L - a L) = a b - (a b - a b)
or
a L = a b .

Thus L = b is also a true equation. This shows that any solution of (2) is a
solution of (1) also.

We point out that in the proof of Theorem 3.1 (as well as in Example 3.2),
it was only the coefficients themselves that were of any direct use to us. The
unknowns x were never actually used in any of the manipulations. This is the
reason that we defined the matrix of coefficients (a). What we now proceed
to do is to generalize the above method of solving systems of equations in a
manner that utilizes this matrix explicitly.

Exercises

1. For each of the following systems of equations, find a solution if it exists:

(a)!!!x +!2y ! 3z = !1 (b)!!2x +!!y ! 2z = 10

3x !!!!y + 2z =!!7 3x + 2y + 2z =!!1
5x + 3y ! 4z =!!2 5x + 4y + 3z =!!4

(c)!!!!x +!2y ! 3z =!!!6

2x !!!!y + 4z =!!!2
4x + 3y ! 2z =!14

2. Determine whether or not the each of the following two systems is equiva-
lent (over the complex field):

(a)!!!!!x ! y = 0!!!!!!and!!!!!!3x + y = 0
2x + y = 0!!!!!!!!!!!!!!!!!!!!x + y = 0

(b)!!!!!!!!!!x +!!y +!!!!!!!!4z = 0!!!!!!and!!!!!!x !!!!!!!!!z = 0

!!x + 3y +!!!!!!!!8z = 0!!!!!!!!!!!!!!!!!!!!!!!!y + 3z = 0
(1 / 2)x +!!y + (5 / 2)z = 0

(c)!!2x + (!1+ i)y +!!!!!!!!!!!t = 0

3y ! 2iz + 5t = 0

and
 3.1 SYSTEMS OF LINEAR EQUATIONS 121

(1+ i / 2)x +!!!!!!!8y ! iz !!!t = 0

(2 / 3)x ! (1 / 2)y + z + 7t = 0

3.2 ELEMENTARY ROW OPERATIONS

The important point to realize in Example 3.2 is that we solved a system of

linear equations by performing some combination of the following operations:

(a) Change the order in which the equations are written.

(b) Multiply each term in an equation by a nonzero scalar.
(c) Multiply one equation by a nonzero scalar and then add this new
equation to another equation in the system.

Note that (a) was not used in Example 3.2, but it would have been necessary if
the coefficient of x in the first equation had been 0. The reason for this is that
we want the equations put into echelon form as defined below.
We now see how to use the matrix aug A as a tool in solving a system of
linear equations. In particular, we define the following so-called elementary
row operations (or transformations) as applied to the augmented matrix:

() Interchange two rows.

() Multiply one row by a nonzero scalar.
() Add a scalar multiple of one row to another.

It should be clear that operations () and () have no effect on the solution set
of the system and, in view of Theorem 3.1, that operation () also has no
effect.
The next two examples show what happens both in the case where there is
no solution to a system of linear equations, and in the case of an infinite
number of solutions. In performing these operations on a matrix, we will let R
denote the ith row. We leave it to the reader to repeat Example 3.2 using this
notation.

Example 3.3 Consider this system of linear equations over the field :

x + 3y + 2z = 7
2x +!!y !!!!z = 5
!x + 2y + 3z = 4

The augmented matrix is

122 LINEAR EQUATIONS AND MATRICES

" 1 3 2 7%
$ '
$ 2 1 !1 5 '
$ '
#!1 2 3 4 &

and the reduction proceeds as follows. We first perform the following elemen-
tary row operations:
#1 !!3 !2 !7 &
% (
R2 ! 2R1 " % 0 !5 !5 !9 (
% (
!!R3 + R1 " $ 0 !!5 !5 11 '

Now, using this matrix, we obtain

#1 3 2 7&
% (
!!!!!!R2 " % 0 5 5 9 (
% (
R3 + R2 " $ 0 0 0 2 '

It is clear that the equation 0z = 2 has no solution, and hence this system has
no solution.

Example 3.4 Let us solve the following system over the field :

x1 ! 2x2 + 2x3 !!!!x4 = !14

3x1 + 2x2 !!!!x3 + 2x4 =!!!17
2x1 + 3x2 !!!!x3 !!!x4 =!!!18
!2x1 + 5x2 ! 3x3 ! 3x4 =!!!26

We have the matrix aug A given by

" !1 !2 !!2 !1 !14 %

$ '
$ !3 !2 !1 !!2 !!17 '
$ !2 !3 !1 !1 !!18 '
$ '
#!2 !5 !3 !3 !!26 &

and hence we obtain the sequence

# 1 !2 !!2 !1 !14 &

% (
R2 ! 3R1 " % 0 !!8 !7 !!5 !!59 (
R3 ! 2R1 " % 0 !!7 !5 !!1 !!46 (
% (
R4 + 2R1 " $ 0 !!1 !!1 !5 !!!2 '
 3.2 ELEMENTARY ROW OPERATIONS 123

# 1 "2 !!!2 "1 "14 &

% (
!!!!!!!!!!R4 ! % 0 !!1 !!!!1 "5 !!"2 (
R2 " 8R4 ! % 0 !!0 "15 45 !!75 (
% (
R3 " 7R4 ! $ 0 !!0 "12 36 !!60 '

# 1 !2 !2 !1 !14 &
% (
% 0 !!1 !1 !5 !!!2 (
(!1 / 15)R3 " % 0 !!0 !1 !3 !!!5 (
% (
(!1 / 12)R4 " $ 0 !!0 !1 !3 !!!5 '

We see that the third and fourth equations are identical, and hence we have
three equations in four unknowns:

x1 ! 2x2 + 2x3 !!!!x4 = !14

x2 +!!!x3 ! 5x4 =!!!!2
x3 ! 3x4 =!!!!5

It is now apparent that there are an infinite number of solutions because, if

we let c be any real number, then our solution set is given by x4 = c, x3 =
3c - 5, x = 2c + 3 and x = -c + 2.

Two m x n matrices are said to be row equivalent if one can be trans-

formed into the other by a finite number of elementary row operations. We
leave it to the reader to show that this defines an equivalence relation on the
set of all m x n matrices (see Exercise 3.2.1).
Our next theorem is nothing more than a formalization of an earlier
remark.

Theorem 3.2 Let A and B be the augmented matrices of two systems of m

linear equations in n unknowns. If A is row equivalent to B, then both systems
have the same solution set.

Proof If A is row equivalent to B, then we can go from the system repre-

sented by A to the system represented by B by a succession of the operations
(a), (b) and (c) described above. It is clear that operations (a) and (b) have no
effect on the solutions, and the method of Theorem 3.1 shows that operation
(c) also has no effect.

A matrix is said to be in row-echelon form if successive rows of the

matrix start out (from the left) with more and more zeros. In particular, a
124 LINEAR EQUATIONS AND MATRICES

matrix is said to be in reduced row-echelon form if it has the following

properties (which are more difficult to state precisely than they are to under-
stand):

(1) All zero rows (if any) occur below all nonzero rows.
(2) The first nonzero entry (reading from the left) in each row is equal to
1.
(3) If the first nonzero entry in the ith row is in the jth column, then
every other entry in the jth column is 0.
(4) If the first nonzero entry in the ith row is in the jth column, then j <
j < ~ ~ ~ .

We will call the first (or leading) nonzero entries in each row of a row-
echelon matrix the distinguished elements of the matrix. Thus, a matrix is in
reduced row-echelon form if the distinguished elements are each equal to 1,
and they are the only nonzero entries in their respective columns.

Example 3.5 The matrix

"1 2 !3 0 1 %
$ '
$0 0 5 2 !4 '
$0 0 0 7 3'
$ '
#0 0 0 0 0&

is in row-echelon form but not in reduced row-echelon form. However, the

matrix
!1 0 5 0 2$
# &
#0 1 2 0 4&
#0 0 0 1 7&
# &
"0 0 0 0 0%

is in reduced row-echelon form. Note that the distinguished elements of the

first matrix are the numbers 1, 5 and 7, and the distinguished elements of the
second matrix are the numbers 1, 1 and 1.

The algorithm detailed in the proof of our next theorem introduces a tech-
nique generally known as Gaussian elimination.

Theorem 3.3 Every m x n matrix A is row equivalent to a reduced row-

echelon matrix.
 3.2 ELEMENTARY ROW OPERATIONS 125

Proof This is essentially obvious from Example 3.4. The detailed description
which follows is an algorithm for determining the reduced row-echelon form
of a matrix.
Suppose that we first put A into the form where the leading entry in each
nonzero row is equal to 1, and where every other entry in the column contain-
ing this first nonzero entry is equal to 0. (This is called simply the row-
reduced form of A.) If this can be done, then all that remains is to perform a
finite number of row interchanges to achieve the final desired reduced row-
echelon form.
To obtain the row-reduced form we proceed as follows. First consider row
1. If every entry in row 1 is equal to 0, then we do nothing with this row. If
row 1 is nonzero, then let j be the smallest positive integer for which aj 0
and multiply row 1 by (aj). Next, for each i 1 we add -aj times row 1 to
row i. This leaves us with the leading entry aj of row 1 equal to 1, and every
other entry in the jth column equal to 0.
Now consider row 2 of the matrix we are left with. Again, if row 2 is equal
to 0 there is nothing to do. If row 2 is nonzero, assume that the first nonzero
entry occurs in column j (where j j by the results of the previous para-
graph). Multiply row 2 by (aj) so that the leading entry in row 2 is equal to
1, and then add -aj times row 2 to row i for each i 2. Note that these opera-
tions have no effect on either column j, or on columns 1, . . . , j of row 1.
It should now be clear that we can continue this process a finite number of
times to achieve the final row-reduced form. We leave it to the reader to take
an arbitrary matrix (a) and apply successive elementary row transformations
to achieve the desired final form.

While we have shown that every matrix is row equivalent to at least one
reduced row-echelon matrix, it is not obvious that this equivalence is unique.
However, we shall show in the next section that this reduced row-echelon
matrix is in fact unique. Because of this, the reduced row-echelon form of a
matrix is often called the row canonical form.

Exercises

1. Show that row equivalence defines an equivalence relation on the set of all
matrices.

2. For each of the following matrices, first reduce to row-echelon form, and
then to row canonical form:
126 LINEAR EQUATIONS AND MATRICES

" 1 !2 !3 !1 % " 1 !2 !1 !!2 !1 %

$ ' $ '
(a)!!$ 2 !1 !2 !!2 '!!!!!!!!!!!!!!!!!!!(b)!!$ 2 !4 !!1 !2 !3'
$ ' $ '
# 3 !!1 !2 !!3& # 3 !6 !!2 !6 !5 &

" 1 !3 !1 !2 %
$ '
$ 0 !1 !5 !3'
(c)!!
$ 2 !5 !!3 !1 '
$ '
# 4 !1 !!1 !5 &

3. For each of the following systems, find a solution or show that no solution
exists:

(a)!!!!!x +!!y +!!!z = 1 (b)!!!!!x ! y + 2z = 1

2x ! 3y + 7z = 0 x +!y +!!z = 2
!!3x ! 2y + 8z = 4 2x !!y +!!z = 5

(c)!!!!!x ! y + 2z = 4 (d)!!!!!x + 3y +!!!z = 2

3x + y + 4z = 6 2x + 7y + 4z = 6
x + y +!!!z = 1 x +!!y ! 4z = 1

(e)!!!!!x +!3y +!!!z = 0 ( f )!!!!2x !!!y + 5z = 19

2x + 7y + 4z = 0 x + 5y ! 3z = 4
x +!!!y ! 4z = 0 3x + 2y + 4z = 5

(g)!!!!2x !!!y + 5z = 19
x + 5y ! 3z = 4
3x + 2y + 4z = 25

4. Let f, f and f3 be elements of F[] (i.e., the space of all real-valued func-
tions defined on ).
(a) Given a set {x, x, x3} of real numbers, define the 3 x 3 matrix F(x) =
(f(x)) where the rows are labelled by i and the columns are labelled by j.
Prove that the set {f} is linearly independent if the rows of the matrix F(x)
are linearly independent.
(b) Now assume that each f has first and second derivatives defined on
some interval (a, b) , and let f(j) denote the jth derivative of f (where
f(0) is just f). Define the matrix W(x) = (f(j-1 )(x)) where 1 i, j 3.
Prove that {f} is linearly independent if the rows of W(x) are independent
 3.2 ELEMENTARY ROW OPERATIONS 127

for some x (a, b). (The determinant of W(x) is called the Wronskian of
the set of functions {fi}.)

Show that each of the following sets of functions is linearly independent:

(c) f(x) = -x2 + x + 1, f(x) = x2 + 2x, f3(x) = x2 - 1.
(d) f(x) = exp(-x), f(x) = x, f3(x) = exp(2x).
(e) f(x) = exp(x), f(x) = sin x, f3(x) = cos x.

5. Let
" 3 !1 !2 %
$ '
A = $ 2 !!1 !1 '!!.
$ '
# 1 !3 !0 &

Determine the values of Y = (y, y, y3) for which the system ax = y

has a solution.

6. Repeat the previous problem with the matrix

" !3 !6 2 !1%
$ '
$ !2 !4 1 !3 '
A= !!.
$ !0 !0 1 !1 '
$ '
# !1 !2 1 !0 &

3.3 ROW AND COLUMN SPACES

We will find it extremely useful to consider the rows and columns of an arbi-
trary m x n matrix as vectors in their own right. In particular, the rows of A
are to be viewed as vector n-tuples A, . . . , Am where each A = (ai1, . . . ,
ain) Fn. Similarly, the columns of A are to be viewed as vector m-tuples
A1, . . . , An where each Aj = (a1j, . . . , amj) Fm. For notational clarity, we
should write Aj as the column vector
! a1 j $
# &
# ! &
#a &
" mj %
but it is typographically easier to write this horizontally whenever possible.
Note that we label the row vectors of A by subscripts, and the columns of A
by superscripts.
128 LINEAR EQUATIONS AND MATRICES

Since each row A is an element of Fn, the set of all rows of a matrix can
be used to generate a new vector space V over F. In other words, V is the
space spanned by the rows A, and hence any v V may be written as
m
v = ! ci Ai
i=1

where each c F. The space V (which is apparently a subspace of Fn) is

called the row space of A. The dimension of V is called the row rank of A,
and will be denoted by rr(A). Since V is a subspace of Fn and dim Fn = n, it
follows that rr(A) = dim V n. On the other hand, V is spanned by the m
vectors A, so that we must have dim V m. It then follows that rr(A)
min{m, n}.
In an exactly analogous manner, we define the column space W of a
matrix A as that subspace of Fm spanned by the n column vectors Aj. Thus
any w W is given by
n
w = !bj A j
j=1

The column rank of A, denoted by cr(A), is given by cr(A) = dim W and, as

above, we must have cr(A) min{m, n}.
An obvious question is whether a sequence of elementary row operations
changes either the row space or the column space of a matrix. A moments
thought should convince you that the row space should not change, but it may
not be clear exactly what happens to the column space. These questions are
answered in our next theorem. While the following proof appears to be rather
long, it is actually quite simple to understand.

Theorem 3.4 Let A and A be row equivalent m x n matrices. Then the row
space of A is equal to the row space of A, and hence rr(A) = rr(A).
Furthermore, we also have cr(A) = cr(A). (However, note that the column
space of A is not necessarily the same as the column space of A.)

Proof Let V be the row space of A and V the row space of A. Since A and A
are row equivalent, A may be obtained from A by applying successive ele-
mentary row operations. But then each row of A is a linear combination of
rows of A, and hence V V. On the other hand, A may be obtained from A
in a similar manner so that V V. Therefore V = V and hence rr(A) = rr(A).
Now let W be the column space of A and W the column space of A.
Under elementary row operations, it will not be true in general that W = W,
but we will show it is still always true that dim W = dim W. Let us define the
mapping f: W W by
 3.3 ROW AND COLUMN SPACES 129

"n % n
f $$! ci Ai '' = ! ci A! i !!.
# i=1 & i=1
In other words, if we are given any linear combination of the columns of A,
then we look at the same linear combination of columns of A. In order to
show that this is well-defined, we must show that if aAi = bAi, then
f(aAi) = f(bAi). This equivalent to showing that if cAi = 0 then
f(cAi) = 0 because if (a - b)Ai = 0 and f((a - b)Ai) = 0, then

0 = f ( ! ( ai " bi ) Ai ) = ! ( ai " bi ) A! i = !ai A! i " !bi A! i

=!! f ( !ai Ai ) " f ( !bi Ai )
and therefore
f(aAi) = f(bAi) .
Now note that

f ( ! (bi Ai + ci Ai )) = f ( ! (bi + ci ) Ai ) = ! (bi + ci ) A! i

= !bi A! i + !ci A! i = f ( !bi Ai ) + f ( !ci Ai )
and
f(k(cAi)) = f((kc)Ai) = (kc) Ai = kf(cAi)

so that f is a vector space homomorphism. If we can show that W and W are

isomorphic, then we will have cr(A) = dim W = dim W = cr(A). Since f is
clearly surjective, we need only show that Ker f = {0} for each of the three
elementary row transformations.
In the calculations to follow, it must be remembered that

A = (a, . . . , a)
and
! a1i $
# &
A i = # ! &!!.
# &
" a mi %
Since
cA = (ca, . . . , ca)

we see that cA = 0 if and only if ca = 0 for every j = 1, . . . , n.

Similarly,
cAi = (ca, . . . , cam)

so that cAi = 0 if and only if ca = 0 for every j = 1, . . . , m (remember

that we usually write a column vector in the form of a row vector).
130 LINEAR EQUATIONS AND MATRICES

We first consider a transformation of type . For definiteness, we inter-

change rows 1 and 2, although it will be obvious that any pair of rows will
work. In other words, we define A = A, A = A and A = A for j = 3, . . . ,
n. Therefore

f(cAi) = cAi = (ca, ca, ca3, . . . , cam) .

If
cAi = 0
then
ca = 0

for every j = 1, . . . , m and hence we see that f(cAi) = 0. This shows that f is
well-defined for type transformations. Conversely, if

f(cAi) = 0
then we see that again
ca = 0

for every j = 1, . . . , m since each component in the expression cA i = 0

must equal 0. Hence cAi = 0 if and only if f(cAi) = 0, and hence Ker f =
{0} for type transformations (which also shows that f is well-defined).
We leave it to the reader (see Exercise 3.3.1) to show that f is well-defined
and Ker f = {0} for transformations of type , and we go on to those of type .
Again for definiteness, we consider the particular transformation A = A +
kA and A = A for j = 2, . . . , m. Then

f ( !ci Ai ) = !ci A! i = !ci ( a1i + ka2i ,!a2i ,!!,!ami )

= ( !ci a1i + !kci a2i ,! !ci a2i ,!!,! !ci ami )
If
cAi = 0
then
ca = 0

for every j = 1, . . . , m so that cAi = 0 and f is well-defined for type

transformations. Conversely, if

cAi = 0
then
ca = 0
 3.3 ROW AND COLUMN SPACES 131

for j = 2, . . . , m, and this then shows that cia1i = 0 also. Thus cAi = 0
implies that cAi = 0, and hence cAi = 0 if and only if f(cAi) = 0. This
shows that Ker f = {0} for type transformations also, and f is well-defined.
In summary, by constructing an explicit isomorphism in each case, we
have shown that the column spaces W and W are isomorphic under all three
types of elementary row operations, and hence it follows that the column
spaces of row equivalent matrices must have the same dimension.

Corollary If A is the row-echelon form of A, then cAi = 0 if and only if

cAi = 0.

Proof This was shown explicitly in the proof of Theorem 3.4 for each type of
elementary row operation.

In Theorem 3.3 we showed that every matrix is row equivalent to a

reduced row-echelon matrix, and hence (by Theorem 3.4) any matrix and its
row canonical form have the same row space. Note though, that if the original
matrix has more rows than the dimension of its row space, then the rows
obviously can not all be linearly independent. However, we now show that the
nonzero rows of the row canonical form are in fact linearly independent, and
hence form a basis for the row space.

Theorem 3.5 The nonzero row vectors of an m x n reduced row-echelon

matrix R form a basis for the row space of R.

Proof From the four properties of a reduced row-echelon matrix, we see that
if R has r nonzero rows, then there exist integers j, . . . , j r with each j n
and j < ~ ~ ~ < jr such that R has a 1 in the ith row and jth column, and every
other entry in the jth column is 0 (it may help to refer to Example 3.5 for
visualization). If we denote these nonzero row vectors by R, . . . , Rr then any
arbitrary vector
r
v = ! ci Ri
i=1
has c as its jth coordinate (note that v may have more than r coordinates if r <
n). Therefore, if v = 0 we must have each coordinate of v equal to 0, and
hence c = 0 for each i = 1, . . . , r. But this means that the R are linearly
independent, and since {R} spans the row space by definition, we see that
they must in fact form a basis.
132 LINEAR EQUATIONS AND MATRICES

Corollary If A is any matrix and R is a reduced row-echelon matrix row

equivalent to A, then the nonzero row vectors of R form a basis for the row
space of A.

Proof In Theorem 3.4 we showed that A and R have the same row space.
The corollary now follows from Theorem 3.5.

Example 3.6 Let us determine whether or not the following matrices have
the same row space:

" 1 !2 !1 !3 %
$ ' " 1 2 !4 11%
A = $ 2 !4 !!1 !2 '!!!!!!!!!!!!!!!!!B = $ '!!.
$ ' # 2 4 !5 14 &
# 3 !6 !3 !7 &

We leave it to the reader to show (and you really should do it) that the reduced
row-echelon form of these matrices is

" 1 !2 !0 !!1 / 3 %
$ ' " 1 !2 !0 !1 / 3 %
A = $ 0 !0 !1 !8 / 3'!!!!!!!!!!!!!!!!!B = $ '!!.
$ ' # 0 !0 !1 !8 / 3&
# 0 !0 !0 !!0 &

Since the the nonzero rows of the reduced row-echelon form of A and B are
identical, the row spaces must be the same.

Now that we have a better understanding of the row space of a matrix, let
us go back and show that the reduced row-echelon form of a given matrix is
unique. We first prove a preliminary result dealing with the row-echelon form
of two matrices having the same row space.

Theorem 3.6 Let A = (a) be a row-echelon matrix with distinguished ele-

ments a1j , a2j , . . . , arj and let B = (b) be another row-echelon matrix with
distinguished elements b1k , b2k , . . . , bsk . Assume that A and B have the
same row space (and therefore the same number of columns). Then the dis-
tinguished elements of A are in the same position as those of B, i.e., r = s and
j = k, j = k, . . . , jr = kr .

Proof Since A = 0 if and only if B = 0, we need only consider the nontrivial

case where r 1 and s 1. First suppose that j < k. This means that the jth
column of B is equal to (0, . . . , 0). Since A and B have the same row space,
the first row of A is just a linear combination of the rows of B. In particular,
we then have a1j = cbij for some set of scalars c (see the proof of
 3.3 ROW AND COLUMN SPACES 133

Theorem 3.4). But bij = 0 for every i, and hence a1j = 0 which contradicts the
assumption that a1j is a distinguished element of A (and must be nonzero by
definition). We are thus forced to conclude that j k. However, we could
clearly have started with the assumption that k < j, in which case we would
have been led to conclude that k j. This shows that we must actually have
j = k.
Now let A and B be the matrices which result from deleting the first row
of A and B respectively. If we can show that A and B have the same row
space, then they will also satisfy the hypotheses of the theorem, and our con-
clusion follows at once by induction.
Let R = (a, a, . . . , a) be any row of A (and hence a row of A), and let
B, . . . , Bm be the rows of B. Since A and B have the same row space, we
again have
m
R = ! d i Bi
i=1
for some set of scalars d. Since R is not the first row of A and A is in row-
echelon form, it follows that a = 0 for i = j = k. In addition, the fact that B is
in row-echelon form means that every entry in the kth column of B must be 0
except for the first, i.e., b1k 0, b2k = ~ ~ ~ = bmk = 0. But then

0 = ak = d b1k + d b2k + ~ ~ ~ + dm bmk = d b1k

which implies that d = 0 since b1k 0. This shows that R is actually a linear
combination of the rows of B, and hence (since R was arbitrary) the row
space of A must be a subspace of the row space of B. This argument can
clearly be repeated to show that the row space of B is a subspace of the row
space of A, and hence we have shown that A and B have the same row
space.

Theorem 3.7 Let A = (a) and B = (b) be reduced row-echelon matrices.

Then A and B have the same row space if and only if they have the same
nonzero rows.

Proof Since it is obvious that A and B have the same row space if they have
the same nonzero rows, we need only prove the converse. So, suppose that A
and B have the same row space. Then if A is an arbitrary nonzero row of A,
we may write
A i = !r c r Br (1)

where the Br are the nonzero rows of B. The proof will be finished if we can
show that cr = 0 for r i and c = 1.
134 LINEAR EQUATIONS AND MATRICES

To show that c = 1, let aij be the first nonzero entry in A, i.e., aij is the
distinguished element of the ith row of A. Looking at the jth component of
(1) we see that
a iji = !r c r b rji (2)

(see the proof of Theorem 3.4). From Theorem 3.6 we know that bij is the
distinguished element of the ith row of B, and hence it is the only nonzero
entry in the jth column of B (by definition of a reduced row-echelon matrix).
This means that (2) implies aij = c bij . In fact, it must be true that aij = bij =
1 since A and B are reduced row-echelon matrices, and therefore c = 1.
Now let bkj be the first nonzero entry of B (where k i). From (1) again
we have
a ijk = !r c r b rjk !!. (3)

Since B is a reduced row-echelon matrix, bkj = 1 is the only nonzero entry in

the jth column of B, and hence (3) shows us that aij = cbkj . But from
Theorem 3.6, akj is a distinguished element of A, and hence the fact that A is
row-reduced means that aij = 0 for i k. This forces us to conclude that c =
0 for k i as claimed.

Suppose that two people are given the same matrix A and asked to trans-
form it to reduced row-echelon form R. The chances are quite good that they
will each perform a different sequence of elementary row operations to
achieve the desired result. Let R and R be the reduced row-echelon matrices
that our two students obtain. We claim that R = R. Indeed, since row equiva-
lence defines an equivalence relation, we see from Theorem 3.4 that the row
spaces of R and R will be the same. Therefore Theorem 3.7 shows us that the
rows of R must equal the rows of R. Hence we are justified in calling the
reduced row-echelon form of a matrix the row canonical form as mentioned
earlier.

Exercises

1. In the proof of Theorem 3.4, show that Ker f = {0} for a type operation.

2. Determine whether or not the following matrices have the same row
space:
" 1 !1 3 %
"1 !2 !1% "1 !1 2 % $ '
A=$ '!!!!!!!!B = $ '!!!!!!!!C = $ 2 !1 10 '!!.
# 3 !4 5 & # 3 3 !1& $ '
# 3 !5 1 &
 3.3 ROW AND COLUMN SPACES 135

3. Show that the subspace of 3 spanned by the vectors u = (1, 1, -1), u =

(2, 3, -1) and u3 = (3, 1, -5) is the same as the subspace spanned by the
vectors v = (1, -1, -3), v = (3, -2, -8) and v3 = (2, 1, -3).

4. Determine whether or not each of the following sets of vectors is linearly

independent:
(a) u = (1, -2, 1), u = (2, 1, -1) and u3 = (7, -4, 1).
(b) u = (1, 2, -3), u = (1, -3, 2) and u3 = (2, -1, 5).

5. (a) Suppose we are given an m x n matrix A = (a), and suppose that one
of the columns of A, say Ai, is a linear combination of the others. Show
that under any elementary row operation resulting in a new matrix A, the
column Ai is the same linear combination of the columns of A that Ai is of
the columns of A. In other words, show that all linear relations between
columns are preserved by elementary row operations.
(b) Use this result to give another proof of Theorem 3.4.
(c) Use this result to give another proof of Theorem 3.7.

3.4 THE RANK OF A MATRIX

It is important for the reader to realize that there is nothing special about the
rows of a matrix. Everything that we have done up to this point in discussing
elementary row operations could just as easily have been done with columns
instead. In particular, this means that Theorems 3.4 and 3.5 are equally valid
for column spaces if we apply our elementary transformations to columns
instead of rows. This observation leads us to our next fundamental result.

Theorem 3.8 If A = (a) is any m x n matrix over a field F, then rr(A) =

cr(A).

Proof Let A be the reduced row-echelon form of A. By Theorem 3.4 it is

sufficient to show that rr(A) = cr(A). If j < ~ ~ ~ < jr are the columns containing
the distinguished elements of A, then {Aj, . . . , Aj} is a basis for the column
space of A, and hence cr(A) = r. (In fact, these columns are just the first r
standard basis vectors in Fn.) But from the corollary to Theorem 3.5, we see
that rows A, . . . , Ar form a basis for the row space of A, and thus rr(A) = r
also.

In view of this result, we define the rank r(A) of a matrix A as

136 LINEAR EQUATIONS AND MATRICES

r(A) = rr(A) = cr(A) .

Our next theorem forms the basis for a practical method of finding the rank of
a matrix.

Theorem 3.9 If A is any matrix, then r(A) is equal to the number of nonzero
rows in the (reduced) row-echelon matrix row equivalent to A. (Alternatively,
r(A) is the number of nonzero columns in the (reduced) column-echelon
matrix column equivalent to A.)

Proof Noting that the number of nonzero rows in the row-echelon form is the
same as the number of nonzero rows in the reduced row-echelon form, we see
that this theorem follows directly from the corollary to Theorem 3.5.

If A is an n x n matrix such that a = 0 for i j and a = 1, then we say that

A is the identity matrix of size n and write this matrix as I. Since the size is
usually understood, we will generally simply write I. If I = (I), then another
useful way of writing this is in terms of the Kronecker delta as I = .
Written out, I has the form

!1 0 0 ! 0$
# &
0 1 0 ! 0&
I =# !!.
#" " " "&
# &
"0 0 0 ! 1%

Theorem 3.10 If A is an n x n matrix of rank n, then the reduced row-

echelon matrix row equivalent to A is the identity matrix I.

Proof This follows from the definition of a reduced row-echelon matrix and
Theorem 3.9.

Example 3.7 Let us find the rank of the matrix A given by

" !!1 !!2 !3 %

$ '
$ !!2 !!1 !!0 '
A= !!.
$ !2 !1 !!3'
$ '
# !1 !!4 !2 &
 3.4 THE RANK OF A MATRIX 137

To do this, we will apply Theorem 3.9 to columns instead of rows (just for
variety). Proceeding with the elementary transformations, we obtain the fol-
lowing sequence of matrices:
" !!1 !!0 !!0 %
$ '
$ !!2 !3 !!6 '
$ !2 !!3 !3 '
$ '
# !1 !!6 !5 &

A2-2A1 A3 + 3A1

" !!1 !!0 0 %

$ '
$ !!2 !1 0 '
$ !2 !!1 1 '
$ '
# !1 !!2 7 / 3&

(1/3)A2 (1/3)(A3 + 2A2)

! 1 !!0 0 $
# &
#0 !!1 0 &
# 0 !!0 1 &
# &
" 3 !!1 / 3 7 / 3%

A1 + 2A2 -(A2 - A3)

Thus the reduced column-echelon form of A has three nonzero columns,

so that r(A) = cr(A) = 3. We leave it to the reader (see Exercise 3.4.1) to show
that the row canonical form of A is

!1 0 0$
# &
#0 1 0&
#0 0 1&
# &
"0 0 0%

and hence r(A) = cr(A) = rr(A) as it should.

138 LINEAR EQUATIONS AND MATRICES

Exercises

1. Verify the row-canonical form of the matrix in Example 3.7.

2. Let A and B be arbitrary m x n matrices. Show that r(A + B) r(A) + r(B).

3. Using elementary row operations, find the rank of each of the following
matrices:

"1 !3 !!1 !2 !3 % " !!1 !2 !3 %

$ ' $ '
1 !4 !!3 !1 !4 ' !!2 !!1 !!0 '
(a)!!$ (b)!! $
$2 !3 !4 !7 !3 ' $ !2 !1 !!3'
$ ' $ '
#3 !8 !!1 !7 !8 & # !1 !4 !2 &

" !!1 !3 %
$ ' " 5 !1 !!1%
$ !!0 !2 ' $ '
(c)!! (d)!!$ 2 !!1 !2 '
$ !!5 !1' $ '
$ ' # 0 !7 12 &
# !2 !!3&

4. Repeat the previous problem using elementary column operations.

3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS

We now apply the results of the previous section to the determination of some
general characteristics of the solution set to systems of linear equations. We
will have more to say on this subject after we have discussed determinants in
the next chapter.
To begin with, a system of linear equations of the form
n
! aij x j = 0,!!!!!!!!!!i = 1,!!,!m
j=1

is called a homogeneous system of m linear equations in n unknowns. It is

obvious that choosing x = x = ~ ~ ~ = x = 0 will satisfy this system, but this
is not a very interesting solution. It is called the trivial (or zero) solution.
Any other solution, if it exists, is referred to as a nontrivial solution.
A more general type of system of linear equations is of the form
n
! aij x j = yi ,!!!!!!!!!!i = 1,!!,!m
j=1

where each y is a given scalar. This is then called a nonhomogeneous system

of linear equations. Let us define the column vector
 3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS 139

Y = (y, . . . , ym) Fm .

From our discussion in the proof of Theorem 3.4, we see that ax is just x
times the ith component of the jth column Aj Fm. Thus our system of non-
homogeneous equations may be written in the form

! a11 $ ! a12 $ ! a1n $

n # & # & # &
a a a
' A j x j = x1 # ! & + x2 # ! & +!"!+xn ## !2n && = Y
# 21 & # 22 &
j=1
# & # & # &
" am1 % " am2 % " amn %

where this vector equation is to be interpreted in terms of its components. (In

the next section, we shall see how to write this as a product of matrices.) It
should also be obvious that a homogeneous system may be written in this
notation as
n
! A j x j = 0!!.
j=1

Let us now look at some elementary properties of the solution set of a

homogeneous system of equations.

Theorem 3.11 The solution set S of a homogeneous system of m equations

in n unknowns is a subspace of Fn.

Proof Let us write our system as jax = 0. We first note that S since
(0, . . . , 0) Fn is the trivial solution of our system. If u = (u, . . . , u) Fn
and v = (v, . . . , v) Fn are both solutions, then

a(u + v) = au + av = 0

so that u + v S. Finally, if c F then we also have

a(cu) = cau = 0
so that cu S.

If we look back at Example 3.4, we see that a system of m equations in n >

m unknowns will necessarily result in a nonunique, and hence nontrivial, solu-
tion. The formal statement of this fact is contained in our next theorem.
140 LINEAR EQUATIONS AND MATRICES

Theorem 3.12 Let a homogeneous system of m equations in n unknowns

have the m x n matrix of coefficients A. Then the system has a nontrivial
solution if and only if r(A) < n.

Proof By writing the system in the form xAj = 0, it is clear that a non-
trivial solution exists if and only if the n column vectors Aj Fm are linearly
dependent. Since the rank of A is equal to the dimension of its column space,
we must therefore have r(A) < n.

It should now be clear that if an n x n (i.e., square) matrix of coefficients

A (of a homogeneous system) has rank equal to n, then the only solution will
be the trivial solution since reducing the augmented matrix (which then has
the last column equal to the zero vector) to reduced row-echelon form will
result in each variable being set equal to zero (see Theorem 3.10).

Theorem 3.13 Let a homogeneous system of linear equations in n unknowns

have a matrix of coefficients A. Then the solution set S of this system is a sub-
space of Fn with dimension n - r(A).

Proof Assume that S is a nontrivial solution set, so that by Theorem 3.12 we

have r(A) < n. Assume also that the unknowns x, . . . , x have been ordered
in such a way that the first k = r(A) columns of A span the column space (this
is guaranteed by Theorem 3.4). Then the remaining columns Ak+1, . . . , An
may be written as
k
Ai = ! bij A j ,!!!!!!!!!!i = k +1,!!,!n
j=1

and where each b F. If we define b = -1 and b = 0 for j i and j > k,

then we may write this as
n
! bij A j = 0,!!!!!!!!!!i = k +1,!!,!n
j=1

(note the upper limit on this sum differs from the previous equation). Next we
observe that the solution set S consists of all vectors x Fn such that
n
!xjAj = 0
j=1

and hence in particular, the n - k vectors

b(i) = (b, . . . , b)
 3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS 141

for each i = k + 1, . . . , n must belong to S. We show that they in fact form a

basis for S, which is then of dimension n - k.
To see this, we first write out each of the b(i):

b (k+1) = (bk+1!1,!!,!bk+1! k ,!!1,!0,!0,!!,!0)

b (k+2) = (bk+2!1,!!,!bk+2! k ,!0,!!1,!0,!!,!0)
!
b (n) = (bn1,!!,!bnk ,!0,!0,!!,!0,!!1)!!.

Hence for any set {c} of n - k scalars we have

n # n n &
! i c b % ! i i1
(i) = % c b ,!!,! ! cibin ,!"ck+1,!!,!"cn ((
i=k+1 $ i=k+1 i=k+1 '
and therefore
n
! cib(i) = 0
i=k+1

if and only if ck+1 = ~ ~ ~ = c = 0. This shows that the b(i) are linearly inde-
pendent. (This should have been obvious from their form shown above.)
Now suppose that d = (d, . . . , d) is any solution of
n
! x j A j = 0!!.
j=1

Since S is a vector space (Theorem 3.11), any linear combination of solutions

is a solution, and hence the vector
n
y=d+ ! dib(i)
i=k+1
must also be a solution. In particular, writing out each component of this
expression shows that
n
yj = d j + ! dibij
i=k+1
and hence the definition of the b shows that y = (y, . . . , y, 0, . . . , 0) for
some set of scalars y. Therefore, we have
n k
0 = ! yj A j = ! yj A j
j=1 j=1

and since {A1, . . . , Ak} is linearly independent, this implies that y = 0 for
each j = 1, . . . , k. Hence y = 0 so that
n
d = ! " di b (i)
i=k+1
142 LINEAR EQUATIONS AND MATRICES

and we see that any solution may be expressed as a linear combination of the
b(i).
Since the b(i) are linearly independent and we just showed that they span
S, they must form a basis for S.

Suppose that we have a homogeneous system of m equations in n > m

unknowns, and suppose that the coefficient matrix A is in row-echelon form
and has rank m. Then each of the m successive equations contains fewer and
fewer unknowns, and since there are more unknowns than equations, there
will be n - m = n - r(A) unknowns that do not appear as the first entry in any
of the rows of A. These n - r(A) unknowns are called free variables. We may
arbitrarily assign any value we please to the free variables to obtain a solution
of the system.
Let the free variables of our system be xi , . . . , xi where k = n - m = n -
r(A), and let vs be the solution vector obtained by setting xi equal to 1 and
each of the remaining free variables equal to 0. (This is essentially what was
done in the proof of Theorem 3.13.) We claim that v, . . . , v are linearly
independent and hence form a basis for the solution space of the (homoge-
neous) system (which is of dimension n - r(A) by Theorem 3.13).
To see this, we basically follow the proof of Theorem 3.13 and let B be
the k x n matrix whose rows consist of the solution vectors vs . For each s, our
construction is such that we have xi = 1 and xi = 0 for r s (and the remain-
ing m = n - k unknowns are in general nonzero). In other words, the solution
vector vs has a 1 in the position of xi, while for r s the vector vr has a 0 in
this same position. This means that each of the k columns corresponding to
the free variables in the matrix B contains a single 1 and the rest zeros. We
now interchange column 1 and column i, then column 2 and column i, . . . ,
and finally column k and column i. This yields the matrix

! 1 0 0 ! 0 0 b1! k+1 ! b1n $

# &
# 0 1 0 ! 0 0 b2! k+1 ! b2n &
C=
#" " " " " " " &
# &
"0 0 0 0 1 bk ! k+1 ! bkn %

where the entries bi k+1 , . . . , bin are the values of the remaining m unknowns
in the solution vector v. Since the matrix C is in row-echelon form, its rows
are independent and hence r(C) = k. However, C is column-equivalent to B,
and therefore r(B) = k also (by Theorem 3.4 applied to columns). But the rows
of B consist precisely of the k solution vectors vs, and thus these solution vec-
tors must be independent as claimed.
 3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS 143

Example 3.8 Consider the homogeneous system of linear equations

x + 2y ! 4z + 3w !!!!t = 0
x + 2y ! 2z + 2w +!!!t = 0
2x + 4y ! 2z + 3w + 4t = 0

If we reduce this system to row-echelon form, we obtain

x + 2y ! 4z + 3w !!!t = 0
(*)
2z !!!w + 2t = 0

It is obvious that the rank of the matrix of coefficients is 2, and hence the
dimension of the solution space is 5 - 2 = 3. The free variables are clearly y,
w and t. The solution vectors vs are obtained by choosing (y = 1, w = 0, t = 0),
(y = 0, w = 1, t = 0) and (y = 0, w = 0, t = 1). Using each of the these in the
system (*), we obtain the solutions

v1 = (!2,!1,!0,!0,!0)
v2 = (!1,!0,!1 / 2,!1,!0)
v3 = (!3,!0,!!1,!0,!1)

Thus the vectors v, v and v3 form a basis for the solution space of the homo-
geneous system.

We emphasize that the corollary to Theorem 3.4 shows us that the solution
set of a homogeneous system of equations is unchanged by elementary row
operations. It is this fact that allows us to proceed as we did in Example 3.8.
We now turn our attention to the solutions of a nonhomogeneous system
of equations ax = y .

Theorem 3.14 Let a nonhomogeneous system of linear equations have

matrix of coefficients A. Then the system has a solution if and only if r(A) =
r(aug A).

Proof Let c = (c, . . . , c) be a solution of ax = y. Then writing this as

cAj = Y

shows us that Y is in the column space of A, and hence

r(aug A) = cr(aug A) = cr(A) = r(A) .

144 LINEAR EQUATIONS AND MATRICES

Conversely, if cr(aug A) = r(aug A) = r(A) = cr(A), then Y is in the column

space of A, and hence Y = cAj for some set of scalars c . But then the
vector c = (c, . . . , c) is a solution since it obviously satisfies ax = y.

Using Theorem 3.13, it is easy to describe the general solution to a non-

homogeneous system of equations.

Theorem 3.15 Let

n
! aij x j = y j
j=1

be a system of nonhomogeneous linear equations. If u = (u, . . . , u) Fn is a

solution of this system, and if S is the solution space of the associated homo-
geneous system, then the set

u + S = {u + v: v S}

is the solution set of the nonhomogeneous system.

Proof If w = (w, . . . , w) Fn is any other solution of ax = y, then

a(w - u) = aw - au = y - y = 0

so that w - u S, and hence w = u + v for some v S. Conversely, if v S

then
a(u + v) = au + av = y + 0 = y

so that u + v is a solution of the nonhomogeneous system.

Theorem 3.16 Let A be an n x n matrix of rank n. Then the system

n
!Ajxj = Y
j=1

has a unique solution for arbitrary vectors Y Fn.

Proof Since Y = Ajx, we see that Y Fn is just a linear combination of

the columns of A. Since r(A) = n, it follows that the columns of A are linearly
independent and hence form a basis for Fn. But then any Y Fn has a unique
expansion in terms of this basis (Theorem 2.4, Corollary 2) so that the vector
X with components x must be unique.
 3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS 145

Example 3.9 Let us find the complete solution set over the real numbers of
the nonhomogeneous system

3x1 +!!x2 +!!2x3 +!!4x4 =!!!!1

x1 !!!x2 +!!3x3 !!!!!x4 =!!!!3
x1 + 7x2 !11x3 +13x4 = !13
11x1 +!!x2 +12x3 +10x4 =!!!!9

We assume that we somehow found a particular solution u = (2, 5, 1, -3)

4, and hence we seek the solution set S of the associated homogeneous sys-
tem. The matrix of coefficients A of the homogeneous system is given by

" !3 !!1 !!!2 !!4 %

$ '
!!1 !1 !!!3 !1 '
A=$ !!.
$ !!1 !!7 !11 13 '
$ '
#11 !!1 !!12 10 &

The first thing we must do is determine r(A). Since the proof of Theorem 3.13
dealt with columns, we choose to construct a new matrix B by applying ele-
mentary column operations to A. Thus we define

" !!1 !!!0 !!!0 !!!0 %

$ '
$ !1 !!!4 !!!5 !!!3 '
B=
$ !!7 !20 !25 !15 '
$ '
# !!1 !!!!8 !!10 !!!6 &

where the columns of B are given in terms of those of A by B1 = A2, B2 =

A1 - 3A2, B3 = A3 - 2A2 and B4 = A4 - 4A2 . It is obvious that B1 and B2 are
independent, and we also note that B3 = (5/4)B2 and B4 = (3/4)B2. Then
r(A) = r(B) = 2, and hence we have dim S = 4 - 2 = 2.
(An alternative method of finding r(A) is as follows. If we interchange the
first two rows of A and then add a suitable multiple the new first row to elimi-
nate the first entry in each of the remaining three rows, we obtain

" 1 !1 !!!3 !1 %
$ '
$ 0 !4 !!7 !!7 '!!.
$ 0 !8 !14 14 '
$ '
# 0 12 !21 21 &
146 LINEAR EQUATIONS AND MATRICES

It is now clear that the first two rows of this matrix are independent, and that
the third and fourth rows are each multiples of the second. Therefore r(A) = 2
as above.)
We now follow the first part of the proof of Theorem 3.13. Observe that
since r(A) = 2 and the first two columns of A are independent, we may write

A3 = (5/4)A1 - (7/4)A2
and
A4 = (3/4)A1 + (7/4)A2 .

We therefore define the vectors

b(3) = (5/4, -7/4, -1, 0)

and
b(4) = (3/4, 7/4, 0, -1)

which are independent solutions of the homogeneous system and span the
solution space S. Therefore the general solution set to the nonhomogeneous
system is given by

u + S = {u + !b (3) + "b (4) }

= {(2,!5,!1,!#3) + ! (5 / 4,!#7 / 4,!#1,!0) + " (3 / 4,!7 / 4,!0,!1)}

where , are arbitrary.

Exercises

1. Find the dimension and a basis for the solution space of each of the fol-
lowing systems of linear equations over :

(a)!!!!x + 4y + 2z = 0 (b)!!!!x + 3y +!2z = 0

2x +!!!y + 5z = 0 x + 5y +!!!z = 0
! 3x + 5y + 8z = 0

(c)!!!!x + 2y + 2z ! w + 3t = 0 (d)!!!!x + 2y ! 2z ! 2w !!!t = 0

x + 2y + 3z + w +!!t = 0 x + 2y !!!z + 3w ! 2t = 0
3x + 6y + 8z + w +!!t = 0 2x + 4y ! 7z +!!w +!!!t = 0
 3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS 147

2. Consider the subspaces U and V of 4 given by

U = {(a,!b,!c,!d) ! ! 4 :!b + c + d = 0}
V = {(a,!b,!c,!d) ! ! 4 :!a + b = 0!and c = 2d}!!.

(a) Find the dimension and a basis for U.

(b) Find the dimension and a basis for V.
(c) Find the dimension and a basis for U V.

3. Find the complete solution set of each of the following systems of linear
equations over :

(a)!!3x ! y = 7 (b)!!2x !!!!y + 3z =!!5

2x + y =!1 3x + 2y ! 2z =!!1
! 7x +!!!!!!!!4z = 11

(c)!!5x + 2y !!!z = 0 (d)!!!!x !!!y +!2z +!!w = 3

3x + 5y + 3z = 0 2x +!!y !!!!z !!!w = 1
x + 8y + 7z = 0 3x +!!y +!!!z ! 3w = 2
! 3x ! 2y + 6z!!!!!!!!!= 7

3.6 MATRIX ALGEBRA

We now introduce the elementary algebraic operations on matrices. These

operations will be of the utmost importance throughout the remainder of this
text. In Chapter 5 we will see how these definitions arise in a natural way
from the algebra of linear transformations.
Given two m x n matrices A = (a) and B = (b), we define their sum A +
B to be the matrix with entries

(A + B) = a + b

obtained by adding the corresponding entries of each matrix. Note that both A
and B must be of the same size. We also say that A equals B if a = b for all
i and j. It is obvious that
A+B = B+A
and that
A + (B + C) = (A + B) + C
148 LINEAR EQUATIONS AND MATRICES

for any other m x n matrix C. We also define the zero matrix 0 as that matrix
for which A + 0 = A. In other words, (0) = 0 for every i and j. Given a matrix
A = (a), we define its negative (or additive inverse)

-A = (-a)

such that A + (-A) = 0. Finally, for any scalar c we define the product of c and
A to be the matrix
cA = (ca) .

Since in general the entries a in a matrix A = (a) are independent of each

other, it should now be clear that the set of all m x n matrices forms a vector
space of dimension mn over a field F of scalars. In other words, any m x n
matrix A with entries a can be written in the form
m n
A = ! ! aij Eij
i=1 j=1

where the m x n matrix E is defined as having a 1 in the (i, j)th position and
0s elsewhere, and there are clearly mn such matrices. We denote the space of
all m x n matrices over the field F by Mmxn(F). The particular case of m = n
defines the space Mn(F) of all square matrices of size n. We will often refer
to a matrix in Mn(F) as an n-square matrix.
Now let A Mmxn(F) be an m x n matrix, B Mrxm(F) be an r x m
matrix, and consider the two systems of linear equations
n
! aij x j = yi ,!!!!!!!!!!i = 1,!!,!m
j=1

and
m
! bij y j = zi ,!!!!!!!!!!i = 1,!!,!r
j=1

where X = (x, . . . , x) Fn, Y = (y, . . . , ym) Fm and Z = (z, . . . , zr)

Fr. Substituting the first of these equations into the second yields

z = by = bax = cx

where we defined the product of the r x m matrix B and the m x n matrix A to

be the r x n matrix C = BA whose entries are given by
m
cik = ! bij a jk !!.
j=1

Thus the (i, k)th entry of C = BA is given by the standard scalar product
 3.6 MATRIX ALGEBRA 149

(BA) = B Ak

of the ith row of B with the kth column of A (where both are considered as
vectors in Fm). Note that matrix multiplication is generally not commutative,
i.e., AB BA. Indeed, the product AB may not even be defined.

Example 3.10 Let A and B be given by

" 1 6 !2 % " 2 !9 %
$ ' $ '
A =!$ 3 4 !5 '!!!!!!!!!!!!!!!!!!!!B =!$ 6 !!1'!!.
$ ' $ '
# 7 0 !8 & # 1 !3&

Then the product of A and B is given by

" 1 6 !2 % " 2 !9 % " 1( 2 + 6 ( 6 ! 2 (1 !1( 9 + 6 (1+ 2 ( 3 %

$ '$ ' $ '
C = AB =!$ 3 4 5 '!$ 6 1 ' =!$ 3 ( 2 + 4 ( 6 + 5 (1 !3 ( 9 + 4 (1 ! 5 ( 3'
$ '$ ' $ '
# 7 0 8 & # 1 !3& # 7 ( 2 + 0 ( 6 + 8 (1 !7 ( 9 + 0 (1 ! 8 ( 3&

" 36 !!!3 %
$ '
=!$ 35 !38 '!!.
$ '
# 22 !87 &

Note that it makes no sense to evaluate the product BA.

It is also easy to see that if we have the matrices

!1 2 $ !0 1$
A=# & and B=# &
"3 4% "1 0%

then
!1 2 $! 0 1 $ ! 2 1 $
AB = # &# &=# &
" 3 4 % " 1 0 % " 4 3%

while
! 0 1 $!1 2 $ ! 3 4 $
BA = # &# &=# & ' AB!!.
" 1 0 %" 3 4 % "1 2 %

Example 3.11 Two other special cases of matrix multiplication are worth ex-
plicitly mentioning. Let X Fn be the column vector
150 LINEAR EQUATIONS AND MATRICES

! x1 $
# &
X = # ! &!!.
# &
" xn %

If A is an m x n matrix, we may consider X to be an n x 1 matrix and form the

product AX:

! a11 ! a1n $ ! x1 $ ! a11 x1 +!!!+a1n xn $ ! A1 X $

# &# & # & # &
AX =!# " " &# " & = # " & = # " &!!.
# &# & # & # &
" am1 ! amn % " xn % " am1 x1 +!!!+amn xn % " Am X %

As expected, the product AX is an m x 1 matrix with entries given by the stan-

dard scalar product AX in Fn of the ith row of A with the vector X. Note
that this may also be written in the form

! a11 $ ! a1n $
# & # &
AX = # ! & x1 +!"!+ # ! & xn
# & # &
" am1 % " amn %

which clearly shows that AX is just a linear combination of the columns of A.

Now let Y Fm be the row vector Y = (y, . . . , ym). If we view this as a
1 x m matrix, then we may form the 1 x n matrix product YA given by

! a11 ! a1n $
# &
YA = (y1,!!,!ym )!# " " &
# &
" am1 ! amn %
= (y1a11 +!!!+!ym am1,!!,!y1a1n +!!!+ym amn )
= (Y A1,!!,!Y A n )!!.

This again yields the expected form of the product with entries YAi.

This example suggests the following commonly used notation for systems
of linear equations. Consider the system
n
! aij x j = yi
j=1

where A = (a) is an m x n matrix. Suppose that we define the column vectors

 3.6 MATRIX ALGEBRA 151

! x1 $ ! y1 $
# & # &
X = # ! & ' Fn and Y = # ! & ' Fm .
# & # &
" xn % " ym %

If we consider X to be an n x 1 matrix and Y to be an m x 1 matrix, then we

may write this system in matrix notation as

AX = Y .

Note that the ith row vector of A is A = (a, . . . , a) so that the expression
ax = y may be written as the standard scalar product

A X = y .

We leave it to the reader to show that if A is an n x n matrix, then

A I = IA = A .

Even if A and B are both square matrices (i.e., matrices of the form m x m),
the product AB will not generally be the same as BA unless A and B are
diagonal matrices (see Exercise 3.6.4). However, we do have the following.

Theorem 3.17 For matrices of proper size (so that these operations are
defined), we have:
(a) (AB)C = A(BC) (associative law).
(b) A(B + C) = AB + AC (left distributive law).
(c) (B + C)A = BA + CA (right distributive law).
(d) k(AB) = (kA)B = A(kB) for any scalar k.

Proof (a)!![(AB)C]ij = !k (AB)ik ckj = !r,! k (air brk )ckj = !r,! k air (brk ckj )
= !r air (BC)rj = [A(BC)]ij !!.

(b)!![A(B + C)]ij = !k aik (B + C)kj = !k aik (bkj + ckj )

= !k aik bkj + !k aik ckj = (AB)ij + (AC)ij
= [(AB) + (AC)]ij !!.

(c) Left to the reader (Exercise 3.6.1).

(d) Left to the reader (Exercise 3.6.1).
152 LINEAR EQUATIONS AND MATRICES

Given a matrix A = (a), we define the transpose of A, denoted by AT =

(aTij) to be the matrix with entries given by aT = a. In other words, if A is an
m x n matrix, then AT is an n x m matrix whose columns are just the rows of
A. Note in particular that a column vector is just the transpose of a row vector.

Example 3.12 If A is given by

! 1 2 3$
!# &
"4 5 6%

then AT is given by
!1 4 $
# &
!# 2 5 &!!.
# &
"3 6%

Theorem 3.18 The transpose has the following properties:

(a) (A + B)T = AT + BT.
(b) (AT)T = A.
(c) (cA)T = cAT for any scalar c.
(d) (AB)T = BT AT.

Proof (a) [(A + B)T] = [(A + B)] = a + b = aT + bT = (AT + BT).

(b) (AT)T = (AT) = a = (A).
(c) (cA)T = (cA) = ca = c(AT).
(d) (AB)T = (AB) = k ab = k bTaT = (BT AT).

We now wish to relate this algebra to our previous results dealing with the
rank of a matrix. Before doing so, let us first make some elementary observa-
tions dealing with the rows and columns of a matrix product. Assume that
A Mmxn(F) and B Mnxr(F) so that the product AB is defined. Since the
(i, j)th entry of AB is given by (AB) = ab, we see that the ith row of AB
is given by a linear combination of the rows of B:

(AB) = (ab, . . . , abkr) = a(b, . . . , bkr) = kaB .

Another way to write this is to observe that

 3.6 MATRIX ALGEBRA 153

(AB)i = ( !k aik bk1,!!,! !k aik bkr )

" b11 ! b1r %
$ '
= (ai1,!!,!ain )!$ " " ' = Ai B!!.
$ '
#bn1 ! bnr &

Similarly, for the columns of a product we find that the jth column of AB is a
linear combination of the columns of A:

" !k a1k bkj % " a1k %

n n
$ ' $ '
(AB) j = $ ! = !
' ( $ ' kj ( A bkj
b = k

$ ! a b ' k =1 $ a ' k =1
# k mk kj & # mk &
and
" !k a1k bkj % " a11 " a1n % "b1 j %
$ ' $ '$ '
(AB) j = $ ! ' =!$ ! ! ' $ ! ' = AB j !!.
$ ! a b ' $a '$ '
# k mk kj & # m1 " amn & # bnj &

These formulas will be quite useful to us in several of the following theorems.

Theorem 3.19 For any matrix A we have r(AT) = r(A).

Proof This is Exercise 3.6.2.

Theorem 3.20 If A and B are any matrices for which the product AB is
defined, then the row space of AB is a subspace of the row space of B, and the
column space of AB is a subspace of the column space of A.

Proof Using (AB) = aB, we see that the ith row of AB is in the space
spanned by the rows of B, and hence the row space of AB is a subspace of the
row space of B.
Now note that the column space of AB is just the row space of (AB)T =
BTAT, which is a subspace of the row space of AT by the first part of the
theorem. But the row space of AT is just the column space of A.

Corollary r(AB) min{r(A), r(B)}.

Proof Let VA be the row space of A, and let WA be the column space of A.
Then
r(AB) = dim VAB dim VB = r(B)
154 LINEAR EQUATIONS AND MATRICES

while
r(AB) = dim WAB dim WA = r(A) .

Exercises

1. Complete the proof of Theorem 3.17.

2. Prove Theorem 3.19.

3. Let A be any m x n matrix and let X be any n x 1 matrix, both with

entries in F. Define the mapping f : Fn Fm by f(X) = AX.
(a) Show that f is a linear transformation (i.e., a vector space homomor-
phism).
(b) Define Im f = {AX: X Fn}. Show that Im f is a subspace of Fm.
(c) Let U be the column space of A. Show that Im f = U. [Hint: Use
Example 3.11 to show that Im f U. Next use the equation (AI)j = AIj to
show that U Im f.]
(d) Let N denote the solution space to the system AX = 0. In other
words, N = {X Fn: AX = 0}. (N is usually called the null space of A.)
Show that
dim N + dim U = n .

[Hint: Suppose dim N = r, and extend a basis {x, . . . , xr} for N to a

basis {x} for Fn. Show that U is spanned by the vectors Axr+1 , . . . ,
Axn , and then that these vectors are linearly independent. Note that this
exercise is really just another proof of Theorem 3.13.]

4. A matrix of the form

! a11 0 0 ! 0 $
# &
# 0 a22 0 ! 0 &
# " " " " &
# &
"0 0 0 ! ann %

is called a diagonal matrix. In other words, A = (a) is diagonal if a = 0

for i j. If A and B are both square matrices, we may define the commu-
tator [A, B] of A and B to be the matrix [A, B] = AB - BA. If [A, B] = 0,
we say that A and B commute.
(a) Show that any diagonal matrices A and B commute.
 3.6 MATRIX ALGEBRA 155

(b) Prove that the only n x n matrices which commute with every n x n
diagonal matrix are diagonal matrices.

5. Given the matrices

6.
" !!2 !1 %
$ ' "1 !2 !5 %
A =!$ !!1 !!0 '!!!!!!!!!!B =!$ '
$ ' # 3 !!4 !!0 &
# !3 !!4 &

compute the following:

(a) AB.
(b) BA.
(c) AAT.
(d) ATA.
(e) Verify that (AB)T = BTAT.

6. Consider the matrix A Mn(F) given by

!0 1 0 0 ! 0 0$
# &
#0 0 1 0 ! 0 0&
#0 0 0 1 ! 0 0&
A =!# &!!.
#" " " " " "&
#0 0 0 0 ! 0 1&
# &
"0 0 0 0 ! 0 0%

Thus A has zero entries everywhere except on the superdiagonal where

the entries are 1s. Let A2 = AA, A3 = AAA, and so on. Show that An = 0
but An-1 0.

7. Given a matrix A = (a) Mn(F), the sum of the diagonal elements of A

is called the trace of A, and is denoted by Tr A. Thus
n
Tr A = ! aii !!.
i=1
(a) Prove that Tr(A + B) = Tr A + Tr B and that Tr(A) = (Tr A) for
any scalar .
(b) Prove that Tr(AB) = Tr(BA).

8. (a) Prove that it is impossible to find matrices A, B Mn() such that

their commutator [A, B] = AB - BA is equal to 1.
156 LINEAR EQUATIONS AND MATRICES

(b) Let F be a field of characteristic 2 (i.e., a field in which 1 + 1 = 0; see

Exercise 1.5.17). Prove that it is possible to find matrices A, B M2(F)
such that [A, B] = 1.

9. A matrix A = (a) is said to be upper-triangular if a = 0 for i > j. In

other words, every entry of A below the main diagonal is zero. Similarly,
A is said to be lower-triangular if a = 0 for i < j. Prove that the product
of upper (lower) triangular matrices is an upper (lower) triangular matrix.

10. Consider the so-called Pauli spin matrices

"0 1% " 0 (i % "1 0 %

!1 = $ '!!!!!!!!!!! 2 = $ '!!!!!!!!!!! 3 = $ '
#1 0& #i 0 & # 0 (1&

and define the permutation symbol ijk by

# !!1 !!if (i,! j,!k) is an even permutation of (1,!2,!3)

%
!ijk = $"1 !!!if (i,! j,!k) is an odd permutation of (1,!2,!3) !!.
% 0 if any two indices are the same!!!!!!!!!!!!!!!!!!!
&

The commutator of two matrices A, B Mn(F) is defined by [A, B] =

AB - BA, and the anticommutator is given by [A, B]+ = AB + BA.

(a) Show that [, ] = 2i ijk . In other words, show that = i

where (i, j, k) is an even permutation of (1, 2, 3).
(b) Show that [, ]+ = 2I .
(c) Using part (a), show that Tr = 0.
(d) For notational simplicity, define = I. Show that {0, 1, 2, 3}
forms a basis for M2(). [Hint: Show that Tr( ) = 2 where 0 ,
3. Use this to show that {} is linearly independent.]
(e) According to part (d), any X M2() may be written in the form X =
x. How would you find the coefficients x?
(f) Show that , = (1/2)Tr() defines an inner product on
M2().
(g) Show that any matrix X M2() that commutes with all of the
(i.e., [X, ] = 0 for each i = 1, 2, 3) must be a multiple of the identity
matrix.
 3.6 MATRIX ALGEBRA 157

11. A square matrix S is said to be symmetric if ST = S, and a square matrix

A is said to be skewsymmetric (or antisymmetric) if AT = -A. (We
continue to assume as usual that F is not of characteristic 2.)
(a) Show that S 0 and A are linearly independent in Mn(F).
(b) What is the dimension of the space of all n x n symmetric matrices?
(c) What is the dimension of the space of all n x n antisymmetric
matrices?

12. Find a basis {A} for the space Mn(F) that consists only of matrices with
the property that A2 = A (such matrices are called idempotent or
projections). [Hint: The matrices

!1 0$ !1 1$ !0 0$ !0 0$
# &!!!!!!# &!!!!!!# &!!!!!!# &
"0 0% "0 0% "1 0% "1 1%

will work in the particular case of M2(F).]

13. Show that it is impossible to find a basis for Mn(F) such that every pair
of matrices in the basis commute with each other.

14. (a) Show that the set of all nonsingular n x n matrices forms a spanning
set for Mn(F). Exhibit a basis of such matrices.
(b) Repeat part (a) with the set of all singular matrices.

15. Show that the set of all matrices of the form AB - BA do not span
Mn(F). [Hint: Use the trace.]

16. Is it possible to span Mn(F) using powers of a single matrix A? In other

words, can {I , A, A2, . . . , An, . . .} span Mn(F)? [Hint: Consider
Exercise 4 above.]

3.7 INVERTIBLE MATRICES

We say that a matrix A M(F) is nonsingular if r(A) = n, and singular if

r(A) < n. Given a matrix A M(F), if there exists a matrix B M(F) such
that AB = BA = I, then B is called an inverse of A, and A is said to be
invertible.
Technically, a matrix B is called a left inverse of A if BA = I, and a
matrix B is a right inverse of A if AB = I. Then, if AB = BA = I, we say that
B is a two-sided inverse of A, and A is then said to be invertible.
158 LINEAR EQUATIONS AND MATRICES

Furthermore, if A has a left inverse B and a right inverse B, then it is easy to

see that B = B since B = BI = B(AB) = (BA)B = IB = B. We shall now
show that if B is either a left or a right inverse of A, then A is invertible.

Theorem 3.21 A matrix A M(F) has a right (left) inverse if and only if A
is nonsingular. This right (left) inverse is also a left (right) inverse, and hence
is an inverse of A.

Proof Suppose A has a right inverse B. Then AB = I so that r(AB) = r(I).

Since r(I) is clearly equal to n (Theorem 3.9), we see that r(AB) = n. But then
from the corollary to Theorem 3.20 and the fact that both A and B are n x n
matrices (so that r(A) n and r(B) n), it follows that r(A) = r(B) = n, and
hence A is nonsingular.
Now suppose that A is nonsingular so that r(A) = n. If we let Ej be the jth
column of the identity matrix I, then for each j = 1, . . . , n the system of
equations
n
! Ai xi = AX = E j
i=1

has a unique solution which we denote by X = Bj (Theorem 3.16). Now let B

be the matrix with columns Bj. Then the jth column of AB is given by

(AB)j = ABj = Ej

and hence AB = I. It remains to be shown that BA = I. To see this, note that

r(AT) = r(A) = n (Theorem 3.19) so that AT is nonsingular also. Hence apply-
ing the same argument shows there exists a unique n x n matrix CT such that
AT CT = I. Since (CA)T = ATCT and IT = I, this is the same as CA = I.
We now recall that it was shown prior to the theorem that if A has both a left
and a right inverse, then they are the same. Therefore B = C so that BA = AB
= I, and hence B is an inverse of A. Clearly, the proof remains valid if right
is replaced by left throughout.

Corollary 1 A matrix A Mn(F) is nonsingular if and only if it has an

inverse. Furthermore, this inverse is unique.

Proof As we saw above, if B and C are both inverses of A, then B = BI =

B(AC) = (BA)C = IC = C.

In view of this corollary, the unique inverse to a matrix A will be denoted

by A from now on.
 3.7 INVERTIBLE MATRICES 159

Corollary 2 If A is an n x n nonsingular matrix, then A is nonsingular and

(A) = A.

Proof If A is nonsingular, then (by Theorem 3.21) A exists so that AA =

AA = I. But this means that (A) exists and is equal to A, and hence A is
also nonsingular.

Corollary 3 If A and B are nonsingular then so is AB, and (AB) = B A.

Proof The fact that A and B are nonsingular means that A and B exist.
We therefore see that

(BA)(AB) = BIB = BB = I

and similarly (AB)(BA) = I. It then follows that B A = (AB). Since we

have now shown that AB has an inverse, Theorem 3.21 tells us that AB must
be nonsingular.

Corollary 4 If A is nonsingular then so is AT, and (AT) = (A)T.

Proof That AT is nonsingular is a direct consequence of Theorem 3.19. Next

we observe that
(A)TAT = (AA)T = IT = I

so that the uniqueness of the inverse tells us that (AT) = (A)T. Note this
also shows that AT is nonsingular.

Corollary 5 A system of n linear equations in n unknowns has a unique

solution if and only if its matrix of coefficients is nonsingular.

Proof Consider the system AX = Y. If A is nonsingular, then a unique A

exists, and therefore we have X = AY as the unique solution. (Note that this
is essentially the content of Theorem 3.16.)
Conversely, if this system has a unique solution, then the solution space of
the associated homogeneous system must have dimension 0 (Theorem 3.15).
Then Theorem 3.13 shows that we must have r(A) = n, and hence A is non-
singular.

A major problem that we have not yet discussed is how to actually find the
inverse of a matrix. One method involves the use of determinants as we will
see in the next chapter. However, let us show another approach based on the
fact that a nonsingular matrix is row-equivalent to the identity matrix
160 LINEAR EQUATIONS AND MATRICES

(Theorem 3.10). This method has the advantage that it is algorithmic, and
hence is easily implemented on a computer.
Since the jth column of a product AB is ABj, we see that considering the
particular case of AA = I leads to

(AA)j = A(A)j = Ej

where Ej is the jth column of I. What we now have is the nonhomogeneous

system
AX = Y

(or ax = y) where X = (A)j and Y = Ej. As we saw in Section 3.2, we

may solve for the vector X by reducing the augmented matrix to reduced row-
echelon form. For the particular case of j = 1 we have

! a11 ! a1n 1$
# &
a ! a2n 0&
aug A = # 21
# " " "&
# &
" an1 ! ann 0%

and hence the reduced form will be

!1 0 0 ! 0 c11 $
# &
#0 1 0 ! 0 c21 &
#" " " " " &
# &
"0 0 0 ! 1 cn1 %

for some set of scalars c. This means that the solution to the system is x =
c, x = c, . . . , x = c. But X = (A)1 = the first column of A, and
therefore this last matrix may be written as

" 1 ! 0 a !111 %
$ '
$" " " '!!.
$ 0 ! 1 a !1 '
# n1 &

Now, for each j = 1, . . . , n the system AX = A(A)j = Ej always has the

same matrix of coefficients, and only the last column of the augmented matrix
depends on j. Since finding the reduced row-echelon form of the matrix of
 3.7 INVERTIBLE MATRICES 161

coefficients is independent of this last column, it follows that we may solve all
n systems simultaneously by reducing the single matrix

! a11 ! a1n 1 ! 0$
# &
# " " " " &!!.
#a 0 ! 1 &%
" n1 ! ann

In other words, the reduced form will be

"1 ! 0 a !111 ! a !11n %

$ '
$" " " " '
$0 ! 1 a !1n1 ! a !1nn '&
#

where the matrix A = (a) satisfies AA = I since (AA)j = A(A)j = Ej is

satisfied for each j = 1, . . . , n.

Example 3.13 Let us find the inverse of the matrix A given by

" !1 !2 !1 %
$ '
!$ !!0 !3 !2 '
$ '
# !!2 !1 !!0 &

We leave it as an exercise for the reader to show that the reduced row-echelon
form of
" !1 !2 !1 1 0 0%
$ '
$ !!0 !3 !2 0 1 0'
$ !!2 !1 !0 0 0 1 '&
#
is

"1 0 0 1 / 6 1 / 12 7 / 12 %
$ '
$0 1 0 1/ 3 1/ 6 1/ 6 '
$0 0 1 1 / 2 !1 / 4 1 / 4 '&
#

and hence A is given by

"1 / 6 !1 / 12 7 / 12 %
$ '
!$1 / 3 !!1 / 6 1 / 6 '!!.
$ '
#1 / 2 !1 / 4 1 / 4 &
162 LINEAR EQUATIONS AND MATRICES

Exercises

1. Verify the reduced row-echelon form of the matrix given in Example

3.13.

2. Find the inverse of a general 2 x 2 matrix. What constraints are there on

the entries of the matrix?

3. Show that a matrix is not invertible if it has any zero row or column.

4. Find the inverse of each of the following matrices:

"1 0 2% " 1 3 4% "1 2 1%

$ ' $ ' $ '
(a)!!!$ 2 !1 3'!!!!!!!!!!!!(b)!!!$ 3 !1 6 '!!!!!!!!!!!!(c)!!!$ 2 5 2 '
$ ' $ ' $ '
#4 1 8& #!1 5 1 & # 1 3 3&

5. Use the inverse of the matrix in Exercise 4(c) above to find the solutions
of each of the following systems:

(a)!!!!x + 2y +!!!z = 10 (b)!!!!x + 2y +!!!z =!!2

2x + 5y + 2z = 14 2x + 5y + 2z = !1
x + 3y + 3z = 30 x + 3y + 3z =!!6

6. What is the inverse of a diagonal matrix?

7. (a) Prove that an upper-triangular matrix is invertible if and only if every

entry on the main diagonal is nonzero (see Exercise 3.6.9 for the defini-
tion of an upper-triangular matrix).
(b) Prove that the inverse of a lower (upper) triangular matrix is lower
(upper) triangular.

8. Find the inverse of the following matrix:

!1 2 3 4$
# &
#0 2 3 4&
!!.
#0 0 3 4&
# &
"0 0 0 4%

9. (a) Let A be any 2 x 1 matrix, and let B be any 1 x 2 matrix. Prove that
AB is not invertible.
 3.7 INVERTIBLE MATRICES 163

(b) Repeat part (a) where A is any m x n matrix and B is any n x m

matrix with n < m.

10. Summarize several of our results by proving the equivalence of the fol-
lowing statements for any n x n matrix A:
(a) A is invertible.
(b) The homogeneous system AX = 0 has only the zero solution.
(c) The system AX = Y has a solution X for every n x 1 matrix Y.

11. Let A and B be square matrices of size n, and assume that A is

nonsingular. Prove that r(AB) = r(B) = r(BA).

12. A matrix A is called a left zero divisor if there exists a nonzero matrix B
such that AB = 0, and A is called a right zero divisor if there exists a
nonzero matrix C such that CA = 0. If A is an m x n matrix, prove that:
(a) If m < n, then A is a left zero divisor.
(b) If m > n, then A is a right zero divisor.
(c) If m = n, then A is both a left and a right zero divisor if and only if A
is singular.

13. Let A and B be nonsingular symmetric matrices for which AB - BA = 0.

Show that AB, AB, AB and AB are all symmetric.

3.8 ELEMENTARY MATRICES

Recall the elementary row operations , , described in Section 3.2. We now

let e denote any one of these three operations, and for any matrix A we define
e(A) to be the result of applying the operation e to the matrix A. In particular,
we define an elementary matrix to be any matrix of the form e(I). The great
utility of elementary matrices arises from the following theorem.

Theorem 3.22 If A is any m x n matrix and e is any elementary row opera-

tion, then
e(A) = e(Im)A .

Proof We must verify this equation for each of the three types of elementary
row operations. First consider an operation of type . In particular, let be
the interchange of rows i and j. Then

[e(A)] = A for k i, j
164 LINEAR EQUATIONS AND MATRICES

while
[e(A)] = A and [e(A)] = A .

On the other hand, using (AB) = A B we also have

[e(I)A] = [e(I)]A .

If k i, j then [e(I)] = I so that

[e(I)]A = IA = A .

If k = i, then [e(I)] = I and

[e(I)]A = IA = A .

Similarly, we see that

[e(I)]A = IA = A .

This verifies the theorem for transformations of type . (It may be helpful for
the reader to write out e(I) and e(I)A to see exactly what is going on.)
There is essentially nothing to prove for type transformations, so we go
on to those of type . Hence, let e be the addition of c times row j to row i.
Then

[e(I)] = I for k i

and
[e(I)] = I + cI .

Therefore
[e(I)]A = (I + cI)A = A + cA = [e(A)]

and for k i we have

[e(I)]A = IA = A = [e(A)] .

If e is of type , then rows i and j are interchanged. But this is readily

undone by interchanging the same rows again, and hence e is defined and is
another elementary row operation. For type operations, some row is multi-
plied by a scalar c, so in this case e is simply multiplication by 1/c. Finally, a
type operation adds c times row j to row i, and hence e adds -c times row j
 3.8 ELEMENTARY MATRICES 165

to row i. Thus all three types of elementary row operations have inverses
which are also elementary row operations.
By way of nomenclature, a square matrix A = (a) is said to be diagonal if
a = 0 for i j. The most common example of a diagonal matrix is the identity
matrix.

Theorem 3.23 Every elementary matrix is nonsingular, and

[e(I)] = e(I) .

Furthermore, the transpose of an elementary matrix is an elementary matrix.

Proof By definition, e(I) is row equivalent to I and hence has the same rank
as I (Theorem 3.4). Thus e(I) is nonsingular since r(I) = n, and hence e(I)
exists. Since it was shown above that e is an elementary row operation, we
apply Theorem 3.22 to the matrix e(I) to obtain

e(I)e(I) = e(e(I)) = I .

Similarly, applying Theorem 3.22 to e(I) yields

e(I)e(I) = e(e(I)) = I .

This shows that e(I) = [e(I)].

Now let e be a type transformation that interchanges rows i and j (with
i < j). Then the ith row of e(I) has a 1 in the jth column, and the jth row has a
1 in the ith column. In other words,

[e(I)] = 1 = [e(I)]

while for r, s i, j we have

[e(I)]rs = 0 if r s

and
[e(I)]rr = 1 .

Taking the transpose shows that

[e(I)]T = [e(I)] = 1 = [e(I)]

and
[e(I)]Trs = [e(I)]sr = 0 = [e(I)]rs .
166 LINEAR EQUATIONS AND MATRICES

Thus [e(I)]T = e(I) for type operations.

Since I is a diagonal matrix, it is clear that for a type operation which
simply multiplies one row by a nonzero scalar, we have [e(I)]T = e(I).
Finally, let e be a type operation that adds c times row j to row i. Then
e(I) is just I with the additional entry [e(I)] = c, and hence [e(I)]T is just I
with the additional entry [e(I)] = c. But this is the same as c times row i
added to row j in the matrix I. In other words, [e(I)]T is just another elemen-
tary matrix.

We now come to the main result dealing with elementary matrices. For
ease of notation, we denote an elementary matrix by E rather than by e(I). In
other words, the result of applying the elementary row operation e to I will be
denoted by the matrix E = e(I).

Theorem 3.24 Every nonsingular n x n matrix may be written as a product

of elementary n x n matrices.

Proof It follows from Theorem 3.10 that any nonsingular n x n matrix A is

row equivalent to I. This means that I may be obtained by applying r suc-
cessive elementary row operations to A. Hence applying Theorem 3.22 r times
yields
Er ~ ~ ~ EA = I

so that
A = E ~ ~ ~ ErI = E ~ ~ ~ Er .

The theorem now follows if we note that each E is an elementary matrix ac-
cording to Theorem 3.23 (since E = [e(I)] = e(I) and e is an elementary
row operation).

Corollary If A is an invertible n x n matrix, and if some sequence of ele-

mentary row operations reduces A to the identity matrix, then the same
sequence of row operations reduces the identity matrix to A.

Proof By hypothesis we may write Er ~ ~ ~ EA = I. But then multiplying from

the right by A shows that A = Er ~ ~ ~ EI.

Note this corollary provides another proof that the method given in the
previous section for finding A is valid.
There is one final important property of elementary matrices that we will
need in a later chapter. Let E be an n x n elementary matrix representing any
 3.8 ELEMENTARY MATRICES 167

of the three types of elementary row operations, and let A be an n x n matrix.

As we have seen, multiplying A from the left by E results in a new matrix
with the same rows that would result from applying the elementary row
operation to A directly. We claim that multiplying A from the right by ET
results in a new matrix whose columns have the same relationship as the rows
of EA. We will prove this for a type operation, leaving the easier type and
operations to the reader (see Exercise 3.8.1).
Let be the addition of c times row j to row i. Then the rows of E are
given by E = I for k i, and E = I + cI. Therefore the columns of ET are
given by
(ET)k = Ik for k i
and
(ET)i = Ii + cIj .

Now recall that the kth column of AB is given by (AB)k = ABk. We then have

(AET)k = A(ET)k = AIk = Ak for k i

and
(AET)i = A(ET)i = A(Ii + cIj) = AIi + cAIj = Ai + cAj .

This is the same relationship as that found between the rows of EA where
(EA) = A and (EA) = A + cA (see the proof of Theorem 3.22).

Exercises

1. Let A be an n x n matrix, and let E be an n x n elementary matrix repre-

senting a type or operation. Show that the columns of AET have the
same relationship as the rows of EA.

2. Write down 4 x 4 elementary matrices that will induce the following ele-
mentary operations in a 4 x 4 matrix when used as left multipliers. Verify
that your answers are correct.
(a) Interchange the 2nd and 4th rows of A.
(b) Interchange the 2nd and 3rd rows of A.
(c) Multiply the 4th row of A by 5.
(d) Add k times the 4th row of A to the 1st row of A.
(e) Add k times the 1st row of A to the 4th row of A.

3. Show that any e(A) may be written as a product of e(A)s and e(A)s.
(The notation should be obvious.)
168 LINEAR EQUATIONS AND MATRICES

4. Pick any 4 x 4 matrix A and multiply it from the right by each of the ele-
mentary matrices found in the previous problem. What is the effect on A?

5. Prove that a matrix A is row equivalent to a matrix B if and only if there

exists a nonsingular matrix P such that B = PA.

6. Reduce the matrix

" 1 !0 !!2 %
$ '
A =!$ 0 !3 !1 '
$ '
# 2 !3 !!3&

to the reduced row-echelon form R, and write the elementary matrix cor-
responding to each of the elementary row operations required. Find a
nonsingular matrix P such that PA = R by taking the product of these ele-
mentary matrices.

7. Let A be an n x n matrix. Summarize several of our results by proving

that the following are equivalent:
(a) A is invertible.
(b) A is row equivalent to I .
(c) A is a product of elementary matrices.

8. Using the results of the previous problem, prove that if A = A A ~ ~ ~ A

where each A is a square matrix, then A is invertible if and only if each
of the A is invertible.

The remaining problems are all connected, and should be worked in the given
order.

9. Suppose that we define elementary column operations exactly as we did

for rows. Prove that every elementary column operation on A can be
achieved by multiplying A on the right by an elementary matrix. [Hint:
You can either do this directly as we did for rows, or by taking transposes
and using Theorem 3.23.]

10. Show that an m x n reduced row-echelon matrix R of rank k can be

reduced by elementary column operations to an m x n matrix C of the
form
 3.8 ELEMENTARY MATRICES 169

!1 0 ! 0 ! 0 0$
# &
#0 1 ! 0 ! 0 0&
#" " " " "&
# &
C = #0 0 ! 1 ! 0 0&
#0 0 ! 0 ! 0 0&
# &
#" " " " "&
# &
"0 0 ! 0 ! 0 0%

where the first k entries on the main diagonal are 1s, and the rest are 0s.

11. From the previous problem and Theorem 3.3, show that every m x n
matrix A of rank k can be reduced by elementary row and column opera-
tions to the form C. We call the matrix C the canonical form of A.

12. We say that a matrix A is row-column-equivalent (abbreviated by r.c.e.)

to a matrix B if A can be transformed into B by a finite number of ele-
mentary row and column operations. Prove:
(a) If A is a matrix, e is an elementary row operation, and e is an
elementary column operation, then (eA)e = e(Ae).
(b) r.c.e. is an equivalence relation.
(c) Two m x n matrices A and B are r.c.e. if and only if they have the
same canonical form, and hence if and only if they have the same rank.

13. If A is any m x n matrix of rank k, prove that there exists a nonsingular

m x m matrix P and a nonsingular n x n matrix Q such that PAQ = C (the
canonical form of A).

14. Prove that two m x n matrices A and B are r.c.e. if and only if there exists
a nonsingular m x m matrix P and a nonsingular n x n matrix Q such that
PAQ = B.