1

Math S-21b Lecture #1 Notes

The primary focus of this lecture is a systematic way of solving and understanding systems of linear equations
algebraically, geometrically, and logically.
Example #1: Solve the system
4 11
5 3 9
x y
x y
=
`
+ =
)
.
This is easy to solve. We would like to explore the many ways of casting this problem geometrically,
algebraically, and otherwise.
Substitution: We can rewrite the first equation as 11 4 x y = + and then substitute this into the second equation to
get 5(11 4 ) 3 9 y y + + = or 55 23 9 y + = , so 23 46 y = and therefore 2 y = . We then substitute this into
11 4 x y = + to get 11 4( 2) x = + or 3 x = . This approach should be familiar to all.
Geometry: Each of these linear equations represents a line, and
the simultaneous solution represents the intersection of these two
lines. The geometric perspective may not produce an explicit
solution, but its helpful in understanding the nature of the
solution. In this case, we see that the lines intersect in a point,
namely (3, 2) . It could have been the case that the lines were
parallel (with no solution) or coincident (with an entire line of
solutions). It should be clear that these are the only three
possibilities in this case.
Gauss-J ordan elimination: It is possible to work much more systematically to arrive at an algebraic solution by
using just three basic rules:
1) Its OK to scale any equation by a nonzero constant (this will not change the solutions).
2) Its OK to interchange any pair of equations (this clearly will not affect the intersection).
3) Its OK to add any scalar multiple of one equation to any other equation. [This will not affect the solutions,
though this observation needs some justification (and you are encouraged to do so).]
In our example, we can then produce the following sequence of equivalent linear systems:
1
23
copy 1st eqn. copy 1st eqn. 1st eqn. +4(2nd eqn.)
2nd eqn. 5(1st eqn.) (2nd eqn.) copy 2nd eqn.
4 11 4 11 4 11 3
5 3 9 23 46 2 2
x y x y x y x
x y y y y
= = = =

` ` ` `
+ = = = =
) ) ) )

This may not at first seem simpler than our earlier method, but it is systematic and generalizable. In fact, if we
agree to maintain the order of the variables and keep the left and right sides of the equations distinct, all of the
essential information can be captured in the following augmented matrix:
1 4 11
5 3 9
(
(

. We see the matrix of
coefficients to the left of the divider bar and the constants of the right-hand sides to the right of the divider bar.
Once we know how to interpret these numbers, we can easily reproduce the equations. The algebraic rules that
we used above can now be translated into valid rules for manipulating the rows of this array.
Elementary Row Operations:
1) Its OK to scale any row by a nonzero constant.
2) Its OK to interchange any pair of rows.
3) Its OK to add any scalar multiple of one row to any other row.
The previous manipulations of equations then become manipulation of rows:
1 1 1 2
1
2 1 2 2 23
4
5
1 4 11 1 4 11 1 4 11 1 0 3
5 3 9 0 23 46 0 1 2 0 1 2
R R R R
R R R R
+

( ( ( (

( ( ( (



2
The solution of this system thus becomes a strategy for getting from the starting augmented matrix to the final,
simplified augmented matrix that allows us to directly read off the solution. This final matrix (which is actually
uniquely determined) is called the reduced row-echelon form (RREF) of the given matrix.
Well see as we look at other examples that theres a simple strategy for getting to the RREF for any given
system. Basically, its this:
Strategy for getting to the reduced row-echelon form (RREF):
A) Scale or interchange to produce a Leading 1 in the upper left position (if possible). A Leading 1 refers to
where the first nonzero entry on a row (as read from left to right) is equal to 1.
B) Use the first row as a pivot row, leaving it unchanged but adding (or subtracting) appropriate multiples of it
to the other rows to produce 0s elsewhere in the column contain the Leading 1 (the first column in this
case). We call this cleaning the column.
C) Scale or interchange to produce a Leading 1 in the second row shifted one column to the right (possibly
more).
D) Using the second row as the pivot row to clean this next column leaving only the Leading 1 in the pivot row
and 0s elsewhere in that column.
E) Continue this process to get Leading 1s in each of the rows, shifting to the right as you descend through the
rows.
F) Any all-zero rows should appear at the bottom of the array.
Its possible to use different steps to get from the initial matrix to its RREF equivalent. It can be proved (try it!)
that no matter what steps are taken, the RREF will be uniquely determined.
Properties of a matrix in reduced row-echelon form (RREF):
1) Every nonzero row has a Leading 1.
2) The Leading 1s shift one or more columns to the right as you descend the rows.
3) Any column containing a Leading 1 has 0s elsewhere in that column (clean).
4) Any all-zero rows are located at the bottom of the matrix.
Definition: The rank of a matrix is the number of Leading 1s in the RREF of the matrix.
As well see in the subsequent examples, the rank for a consistent system is the number of variables that we can
solve for either for a specific value or in terms of one or more parameters. In our initial example with two
equations in two variables, the rank was 2 and we were able to solve for both variables to get specific values.
Example #2: Solve the system
3 2 2 12
4 4 4
x y z
x y z
+ =
`
+ + =
)
.
Geometrically, each equation represents a plane in R
3
, and the simultaneous solution will correspond to the
intersection of these two planes. Three possible things could happen: (a) the planes intersect in a line, (b) the
planes are parallel and dont intersect at all, or (c) the planes are coincident and intersect everywhere to give an
entire plane of solutions. Its not possible to get a unique solution, and its really not sufficient to say that there
are infinitely many solutions (though there may be) without making clear whether these are on a line or a plane
and explicitly representing these solutions (in terms of one or more parameters).
Algebraically, we can proceed using augmented matrices and row reduction:
2 1 1 1
1
2 2 1 2 20
1 2
2
4 1
4
1 1 1
6
2 2 2
1 1 1
4 4 4
1 6 1 16
3 2 2 12 1 6 1 16
1 6 1 16
4 4 1 4 0 20 5 60
0 1 3 4 4 1 4
1 0 2 2 2
0 1 3 3 3
R R R R
R R R R
R R
R
x z x z
y z y z

( ( ( (

( (
( (




+ = = (

` `
(
= = +
) )

This system has rank 2 and we were able to solve for x and y in terms of the other variable z. In order to express
the solutions in a more evenhanded way, we introduce a parameter t and express all of the variables in terms of
3
this parameter. To minimize fractions, lets take 4 z t = . Then
2 2
3
4
t
x t
y t
z t
e
=

= +
`

=
)
R
. If you are familiar with the
parameterization of lines, you might rewrite this in vector form as
2 2
3 1
0 4
x
y t
z
( ( (
( ( (
= +
( ( (

. This represents a line
passing through the point (2, 3,0) in the direction of the vector 2,1,4 = v . You dont need to know this to
be able to produce solutions, but it does make explicit the earlier observation that we expected the two planes to
intersect in a line.
Example #3: Solve the system
3 2 5
7
2 1
x y
x y
x y
+ =

+ =
`

+ =
)
.
Geometrically, we are seeking the intersection of three lines in R
2
. The likelihood is that there is no common
intersection of all three lines. For an overdetermined system such as this (more conditions than there are
variables) we would typically expect to get no solutions. However, it is possible that we could have a unique
solution or even that the three lines might coincide to yield an entire line of solutions.
Algebraically, we proceed as before using augmented matrices and a row reduction strategy:
1 1 7
3 2 5 1 1 7 1 1 7 1 0 2
1 1 7 3 2 5 0 5 26 0 1 5 0 1 5
2 1 1 2 1 1 0 3 15 0 5 26 0 0 1
( (
( ( (
( (
( ( (

( (
( ( (
( (

, where some steps have been merged.
This final array contains a clear contradiction. The bottom row says that 0 0 1 x y + = . The only logical
conclusion is that the premise that a common solution existed must have been false. There are no solutions. Its
important to note that the final array above is actually not in reduced row-echelon form. Technically we should
still clean the last column:
1 0 2 1 0 0
0 1 5 0 1 0
0 0 1
0 0 1
(
(
(
(

(
(
(

. This is worth noting because if a calculator is used to
find the RREF, it will carry out this process to completion.
Note: If a Leading 1 appears to the right of the divider bar in the RREF for a given augmented matrix, this
indicates that the system must be inconsistent, i.e. there are no solutions to the linear system.
Example #4: Solve the system
2 3 1
3 7
2 3 4 5
x y z
x y z
x y z
+ =

+ + =
`

+ =
)
.
Here we have a system of 3 equations in 3 unknowns. Each equation represents a plane in R
3
. The intersection
of these planes could give either (a) a single point (unique solution), (b) no solutions (either due to parallel
planes or the intersection of two planes giving a line parallel to the third plane), (c) a line of solutions if the
three planes intersect in that way, or (d) an entire plane of solutions (if all three planes coincide).
Algebraically, proceed with augmented matrices and row reduction:
1 2 3 1 1 2 3 1 1 2 3 1 1 0 1 2.6 1 0 1 2.6
3 1 1 7 0 5 10 4 0 1 2 .8 0 1 2 .8 0 1 2 .8
2 3 4 5 0 7 10 7 0 7 10 7 0 0 4 12.6 0 0 1 3.15
1 0 0 .55 .55
0 1 0 5.5 5.5 ( , , )
0 0 1 3.15 3.15
x
y x y z
z
( ( ( ( (
( ( ( ( (

( ( ( ( (


= (

(
= =
`
(

=
)
( .55,5.5,3.15)

4
Note that the rank in this case is 3 and we were able to solve for specific values for all 3 variables. There was no
need for parameters and the solution is unique.
Example #5: Solve the system
1 2 3 4
1 2 3 4
2 3 4 10
2 7 3 4
x x x x
x x x x
+ + + =
`
+ + =
)
.
Geometrically, we are a little beyond the comfort zone. There are now 4 variables, so we are looking at the
intersection of two objects in R
4
, but what are these objects? We can proceed by analogy. If we were in R
2
with
coordinates
1 2
( , ) x x and had the relation
1 2
2 10 x x + = , we would recognize this as a line (one degree of
freedom). If we were in R
3
with coordinates
1 2 3
( , , ) x x x and had the relation
1 2 3
2 3 10 x x x + + = , we would
recognize this as a plane (two degrees of freedom). So, by analogy, if we are in R
4
with coordinates
1 2 3 4
( , , , ) x x x x and had the relation
1 2 3 4
2 3 4 10 x x x x + + + = , we might intuitively think of this as a
hyperplane (three degrees of freedom) in this four-dimensional space.
This visualization may not get us to an explicit solution, but its helpful for understanding the possibilities. The
intersection of two hyperplanes could yield no solutions (if they are parallel), but they will most likely yield a
set of solutions with two degrees of freedom (thus requiring two independent parameters). Typically, the
imposition of each addition constraint (equation) reduces by one the number of degrees of freedom of the
intersection, though this is definitely not always the case. It should be emphasized that we can formally solve
this system and parametrically represent all solutions without any geometric visualization.
Algebraically, we proceed with augmented matrices and row reduction:
16 1 11
5 5 5
17 18 2 17 18 2
5 5 5 5 5 5
1 0 1 2 7 3 4 2 1 3 4 10 1 2 7 3 4 1 2 7 3 4
0 1 1 2 7 3 4 2 1 3 4 10 0 5 17 2 18 0 1
( ( ( ( (

( ( ( ( (



The corresponding equations are
16 16 1 11 1 11
1 3 4 1 3 4 5 5 5 5 5 5
17 18 18 17 2 2
2 3 4 2 3 4 5 5 5 5 5 5
x x x x x x
x x x x x x
+ = = +

` `
+ = = +
) )
.
If we now introduce two independent parameters s and t and, in order to minimize fractions, let
3
5 x s = and
4
5 x t = , we can express all solutions parametrically as
16
1 5
18
2 5
3
4
,
11
17 2
5
5
s t
x s t
x s t
x s
x t
e
= +

= +
`
=

=

)
R
. In terms of vectors in R
4
, this
can be expressed in the vector form
16
5 1
18
2 5
3
4
1 11
17 2
5 0 0
0 5 0
x
x
s t
x
x
( ( ( (
( ( ( (

= + + ( ( ( (
( ( ( (
( ( ( (

. This agrees with the geometric discussion
above in the sense that we have an initial vector to get to a point in the solution space, and we can then move in
two independent directions from there to span the expected two degrees of freedom.
Rank vs. parameters
In each of the previous examples where solutions existed (consistent systems), there was a simple relationship
between the number of variables, the rank of the system, and the number of parameters necessary to express all
solutions. The rank represents the number of variables we can solve for in terms of the other variables, and
well need to introduce an independent parameter for each of the variables that we cannot otherwise solve for.
Therefore, [Rank] + [Number of parameters] = [Number of Variables]. That is, if the system is consistent
with rank k and if there are n variables, we will require n k parameters to express all solutions. The greater
the rank, the fewer the number of parameters. If the rank is n (full rank), then no parameters are needed and the
system will have a unique solution. Keep in mind that a given system may also be inconsistent in which case
there are no solutions.
5
The situation of an inconsistent system can also be characterized by the fact that the rank of its augmented
matrix (
| |
rank | A b ) is greater that the rank of its coefficient matrix (
| |
rank A ) because of the Leading 1
appearing to the right of the divider bar. [See Example #3.]
Observations about rank
For any m n matrix A (m rows, n columns):
1) ( ) rank n s A because each column can have no more than one Leading 1 in the RREF of the matrix.
2) ( ) rank m s A because each row can have no more than one Leading 1 in the RREF of the matrix.
3) ( ) min( , ) rank m n s A follows from the observations above.
4) If
| |
| A b is an augmented matrix for a linear system and if
| |
RREF | A b has a Leading 1 in its last column
(to the right of the divider bar), then the system is inconsistent. [This is equivalent to having
| | | | RREF | 1 RREF = + A b A .]
5) If a system with augmented matrix
| |
| A b is consistent and if ( ) rank n = A , then the system will have a
unique solution.
6) If a system with augmented matrix
| |
| A b is consistent and if ( ) rank k n = < A , then the system will require
n k parameters to express all solutions.
Vector form of a linear system
Given a system of m linear equations in n variables
11 1 1 1
1 1
n n
m mn n m
a x a x b
a x a x b
+ + =

`

+ + =
)

, we can use vector addition and

scalar multiplication to express this in the vector form:
11 1 1
1
1
n
n
m mn m
a a b
x x
a a b
( ( (
( ( (
+ + =
( ( (

. If we let
11 1
1
1
, ,
n
n
m mn
a a
a a
( (
( (
= =
( (

v v , and let
1
m
b
b
(
(
=
(

b , this can be more succinctly expressed as
1 1 n n
x x + + = v v b .
In this perspective, the existence of solutions can be understood in terms of whether the given vector b on the
right-hand-side can be built out of the collection of vectors { }
1
, ,
n
v v . We will soon re-express this by
saying that { }
1
span , ,
n
e b v v . If this is not the case, the system will be inconsistent. If it is the case, there
may be only one way to do this (unique solution) or many ways to do this (parameterized family of solutions).
Looking back at Example #1 with
4 11
5 3 9
x y
x y
=
`
+ =
)
, we could write this in vector form as
1 4 11
5 3 9
x y
( ( (
+ =
( ( (

. If
you draw the vectors
1
1
5
(
=
(

v and
2
4
3
(
=
(

v as well as the vector
11
9
(
=
(

b , youll see that youll need to take
3 x = and 2 y = to get
1 4 11
3 2
5 3 9
( ( (
=
( ( (

and this is unique in this case.
Matrix form of a linear system
If we take the vector form above and assemble the vectors { }
1
, ,
n
v v side-by-side to form an m n matrix
11 1
1
1
n
n
m mn
a a
a a
( | | (
(
(
= =
(
(
+ +
(

A v v

, and if we write
1
n
x
x
(
(
=
(

x , we can define the product of this matrix and the
6
vector as
1
1 1 1 n n n
n
x
x x
x
( | | (
(
(
= + +
(
(
+ +
(

Ax v v v v . Using this definition, we can express the linear system
11 1 1 1
1 1
n n
m mn n m
a x a x b
a x a x b
+ + =

`

+ + =
)

simply as = Ax b. This is called the matrix form of the linear system.

This can also be understood in terms of (linear) functions. Note that if we write ( ) L = x Ax , we have the input
vector
n
e x R and the output vector ( )
m
L = = e x Ax b R . We can therefore understand such a system of linear
equations in terms of the (linear) function :
n m
L R R . We also sometimes represent this by writing either:
L n m
R R or
n m

A
R R or
n m
e = e
A
x R Ax b R
We will explore this perspective in detail in the next lecture.
Homogeneous systems
A system of linear equations of the form
11 1 1
1 1
0
0
n n
m mn n
a x a x
a x a x
+ + =

`

+ + =
)

is called a homogeneous system (the right-

hand-sides are all equal to 0). It can easily be seen that every homogeneous system is consistent because a
solution is
1
( , , ) (0, ,0)
n
x x = . If you look back at all of the consistent examples we previously considered
and replace all the right-hand-sides by 0, youll also find that the solutions will be quite similar and require the
same number of parameters, but the constant terms will all be 0. The geometric interpretation of this is that the
solutions for a given non-homogeneous system and the corresponding homogeneous system will simply be
parallel translates with the homogeneous solutions passing through the origin (well soon call this a subspace)
and the non-homogeneous solutions parallel to the homogeneous solutions (known as an affine space).


Notes by Robert Winters