9 A P R I L 2 0 13

L I N E A R A L G & M U LT I VA R I A B L E C A L C §3
3.1 Linear Systems

A linear system may be expressed as:

a11 x1 + a12 x2 + ⋯ + a1n x n = b1

a21 x1 + a22 x2 + ⋯ + a2n x n = b2
⋮ ⋮
a m1 x1 + a m2 x2 + ⋯ + a mn x n = b m

or in matrix form as:

⎡⎢ a a12 ⋯ a1n ⎤⎥ ⎡⎢ x1 ⎤⎥ ⎡⎢ b1 ⎤⎥
⎢⎢ 11 ⎥⎢ ⎥ ⎢ ⎥
⎢⎢ a21 a22 ⋯ a2n ⎥⎥⎥ ⎢⎢⎢ x2 ⎥⎥⎥ = ⎢⎢⎢ b2 ⎥⎥⎥
⎢⎢ ⋮ ⋮ ⋱ ⋮ ⎥⎢ ⋮ ⎥ ⎢ ⋮ ⎥
⎢⎣ a m1 a m2 ⋯ a mn ⎥⎥⎦ ⎢⎢⎣ x n ⎥⎥⎦ ⎢⎢⎣b m ⎥⎥⎦

or by the augmented matrix:

⎡⎢ a a12 ⋯ a1n b1 ⎤⎥
⎢⎢ 11 ⎥
⎢⎢ a 21 a 22 ⋯ a2n b2 ⎥⎥
⎢⎢ ⋮ ⋮ ⋱ ⋮ ⋮ ⎥⎥
⎢⎣ a m1 a m2 ⋯ a mn ⎥
b m ⎥⎦

The matrix [a i j ] is called the coefficient matrix.

3.1.1 Solving Linear Systems

Definition 1 (RREF). Each nonzero row of a matrix has a leftmost nonzero

entry, which we call a pivot entry. A matrix [a i j ] is in reduced row echelon
form (RREF) if the following are true:
• Each pivot has value 1.

• Each pivot is strictly to the right of pivots above it.

• The column of a pivot consists only of zeros except for the 1 at

the pivot.

• Each row without a pivot occurs below each row with a pivot.
(The zero rows are below the nonzero rows.) •
2

Note 1. The definition says that the pivots resemble a diagonal staircase
pattern (downward and rightward). The entries below (and to the left)
of the staircase must be zero. The entries above (and to the right) of the
staircase may be nonzero unless directly above a pivot. •

Definition 2 (Free/Pivot). If a system

a11 x1 + a12 x2 + ⋯ + a1n x n = b1

a21 x1 + a22 x2 + ⋯ + a2n x n = b2
⋮ ⋮
a m1 x1 + a m2 x2 + ⋯ + a mn x n = b m

is in reduced row echelon form, then a variable x i is a pivot variable if the

corresponding column, the ith column, has a pivot and is a free variable
otherwise. The corresponding columns may also be called pivot columns
or free columns, respectively. •

Example 1. Letting ∗ represent a potentially nonzero entry and leaving

blank any entry that must be zero, the following are examples of reduced
row echelon forms:
⎡⎢ 1 ∗ ∗ ∗⎤⎥
⎢⎢ ⎥ ⎡⎢1 ⎤⎥
1 ∗ ∗⎥⎥ 1 ∗ ∗ ∗ ∗ ∗ ⎢
⎢⎢ ⎥ [ ] ⎢⎢ 1 ⎥⎥⎥
⎢⎢ 1 ∗⎥ 1 ∗ ∗ ∗ ⎢⎣
⎢⎣ ⎥⎥ 1⎥⎦
⎦
and, for certain choices of ∗, the following are non-examples of reduced
row echelon forms:
⎡⎢ 1⎤
[
1 ∗ ∗ ∗ ∗
] and ⎢⎢ 1 ∗ ⎥⎥⎥
−2 ⎢⎢ ⎥⎥
⎣ ⎦
and
⎡⎢ 1 ∗ ⎤⎥
1 ∗ ∗ ⎢⎢ ⎥⎥
[ ] and ⎢⎢ ⎥
1
⎣ 1⎥⎦
The first matrix has a pivot with entry other than 1. The second matrix
has a pivot to the left of a pivot in a higher row. The third matrix has
a nonzero entry in the column of a pivot. The fourth matrix has a row
without pivots occurring above a row with a pivot. •

Keep reduced row echelon form in mind during the following ex-
ample.
 3

Example 2 (Compare with Example 5.2 in l a). Find all solutions of the
system:

x1 + x2 + x3 = 6
x1 + 2x2 + 3x3 = 7

and describe the set of solutions geometrically. •

Solution. Eliminate x1 from the second equation by subtracting the first

equation from the second equation:

x1 + x2 + x3 = 6
0x1 + x2 + 2x3 = 1

Then eliminate x2 from the first equation by subtracting the second

equation from the first equation:

x1 + 0x2 − x3 = 5
0x1 + x2 + 2x3 = 1

Express the solution in terms of x3 :

⎡⎢ x1 ⎤⎥ ⎡⎢ 5 + x3 ⎤⎥ ⎡⎢5⎤⎥ ⎡⎢ x3 ⎤⎥ ⎡⎢5⎤⎥ ⎡1⎤
⎢⎢ x ⎥⎥ = ⎢⎢1 − 2x ⎥⎥ = ⎢⎢1⎥⎥ + ⎢⎢−2x ⎥⎥ = ⎢⎢1⎥⎥ + x ⎢⎢⎢−2⎥⎥⎥
⎢⎢ 2 ⎥⎥ ⎢⎢ 3⎥ ⎢ ⎥ ⎢ 3⎥ ⎢ ⎥ 3⎢ ⎥
x x ⎥ ⎢
⎣ 3⎦ ⎣ 3 ⎦ ⎣ ⎦ ⎣ 3 ⎦ ⎣ ⎦ 0 ⎥ ⎢ x ⎥ ⎢ 0⎥ ⎢⎣ 1 ⎥⎦

The above expression parametrizes solutions to the given system as x3

varies in R. Geometrically, this is a line in R3 . Specifically, the line is the
intersection of the two planes that have equations:

x1 + x2 + x3 = 6 and x1 + 2x2 + 3x3 = 7 ∎

Note 2. The manipulation of equations was directed at transforming

the coefficient matrix to reduced row echelon form:
1 1 1 1 0 −1
[ ] Ô⇒ [ ]
1 2 3 0 1 2

In the end, we solved for the pivot variables x1 and x2 in terms of the
free variable x3 . •

Example 3. Solve the system of equations:

x1 + 0x2 + 0x3 = 1
x1 + x2 + x3 = 2
0x1 + x2 + x3 = 3 •
4

Solution. The augmented matrix is:

⎡⎢1 0 0 1⎤⎥
⎢⎢1 1 1 2⎥⎥
⎢⎢ ⎥
⎣0 1 1 3⎥⎦
Proceeding as in Example 2, row reduce the left side (before the augmen-
tation) to obtain:
⎡⎢1 0 0 1⎤⎥
⎢⎢0 1 1 1⎥⎥
⎢⎢ ⎥
⎣0 0 0 2⎥⎦
Consequently there are no solutions. ∎

Example 4. Solve the system of equations:

x1 + 0x2 + 0x3 = 1
x1 + 2x2 + 4x3 = 3
0x1 + 4x2 + 5x3 = 3
6x1 + 7x2 − 13x3 = 4 •

Solution. Row reduce to obtain:

⎡⎢1 0 0 1 ⎤⎥
⎢⎢ ⎥
⎢⎢0 1 0 1⁄3 ⎥⎥
⎢⎢0 0 1 1⁄3 ⎥⎥
⎢⎣0 ⎥
0 0 0 ⎥⎦

from which it follows that the only solution is x1 = 1, x2 = 1⁄3 , x3 = 1⁄3 .∎

The possibilities for the number of solutions to a linear system of

equations are 0, 1 or infinitely many. The geometric explanation is that
the possible solution sets are intersections of (hyper)planes, and hence
are empty or a point or a line or a plane or ...(not necessarily containing
the origin).
Next we will study solutions to linear systems of equations Ax = b
where the constant vector b is the zero vector 0.

3.2 Null Space

Definition 3 (Null Space). The null space of an m × n matrix A is the

subset of Rn given by:

N(A) = {x ∈ Rn ∣ Ax = 0} •
 5

The null space is an example of a subspace, which is either the set

consisting of the origin, a line through the origin, a plane through the
origin, .. ..
Definition 4 (Subspace). A subspace of Rn is a subset E of Rn satisfying
the following 3 properties:
• the zero vector 0 is in E
• if x and y are in E, then x + y is in E
• if x is in E and c is in R, then cx is in E •

Note 3. One can show that a subspace of Rn is the same as a span of a

sets of vectors in Rn . •

In particular, a null space N(A) has the following properties:

• the zero vector 0 is in N(A):
Solution. The identity A0 = 0 shows that 0 is in N(A). ∎

• if x and y are in N(A), then x + y is in N(A)

Solution. Assume x and y are in N(A) so that by definition Ax =
0 and A y = 0. Then A(x + y) = Ax + A y = 0 + 0 = 0 so x + y is
in N(A). ∎

• if x is in N(A) and c is in R, then cx is in N(A)

Solution. Assume x is in N(A) so that by definition Ax = 0, and
let c be an element of R. Then A(cx) = cAx = c0 = 0 so cx is in
N(A). ∎

Note that for proving the above assertions, linearity was the essential
property of matrix multiplication.

3.2.1 Computing the Null Space

To find the null space of

1 1 1
[ ]
1 2 3
one would solve the system of equations:
x1 + x2 + x3 = 0
x1 + 2x2 + 3x3 = 0
As described in the next section, a better way is first to find the
reduced row echelon form.
6

3.2.2 Computing the Null Space with RREF

Very important fact:

N(A) = N(rref (A))

where rref (A) is the reduced row echelon form of A. Thus to find the
null space of A we should:

• compute rref (A)

• solve Ax = 0 by determining the pivot variables in terms of the

free variables

Example 5. Find the null space of

1 1 1
A=[ ] •
1 2 3

Solution. In Example 2 we found the reduced row echelon form of A to

be:
1 0 −1
rref (A) = [ ]
0 1 2
We now solve

x1 + 0x2 − x3 = 0
0x1 + x2 + 2x3 = 0

for the pivot variables x1 and x2 in terms of the free variable x3 :

x1 = x3
x2 = −2x3

Then the set of

⎡⎢ x1 ⎤⎥ ⎡⎢ x3 ⎤⎥ ⎡1⎤
⎢⎢ x ⎥⎥ = ⎢⎢−2x ⎥⎥ = x ⎢⎢⎢−2⎥⎥⎥
⎢⎢ 2 ⎥⎥ ⎢⎢ 3⎥ 3⎢ ⎥
⎣ x3 ⎦ ⎣ x3 ⎥⎦ ⎢⎣ 1 ⎥⎦

for varying x3 in R is N(rref (A)) = N(A). ∎

Note 4. Notice that the direction vector

⎡⎢ 1 ⎤⎥
⎢⎢−2⎥⎥
⎢⎢ ⎥⎥
⎣1⎦
 7

occurred also in the solution to Example 2, which was just an inhomo-

geneous version of this homogeneous equation. (The right side is zero
here, but was nonzero for Example 2.) In general, the solution to a linear
system is a translation of the solution to the associated homogeneous
system. •

Example 6. Find the null spaces of the matrices

⎡⎢0 1 0⎤⎥ ⎡⎢1 3 −1 9 ⎤⎥
⎢⎢0 0 0⎥⎥ and ⎢⎢1 1 3 1 ⎥⎥
⎢⎢ ⎥ ⎢⎢ ⎥
⎣0 0 0⎥⎦ ⎣2 7 −4 22⎥⎦
which are 8.2 and 8.3 from Levandosky. •