Lecture 1

Lecture 1

Systems of Linear Equations

In this lecture, we will introduce linear systems and the method of row reduction to solve
them. We will introduce matrices as a convenient structure to represent and solve linear
systems. Lastly, we will discuss geometric interpretations of the solution set of a linear
system in 2- and 3-dimensions.

1.1 What is a system of linear equations?

Definition 1.1: A system of m linear equations in n unknown variables x1 , x2 , . . . , xn
is a collection of m equations of the form

a11 x1 + a12 x2 + a13 x3 + · · · + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + · · · + a2n xn = b2
a31 x1 + a32 x2 + a33 x3 + · · · + a3n xn = b3 (1.1)
.. .. .. .. ..
. . . . .
am1 x1 + am2 x2 + am3 x3 + · · · + amn xn = bm

The numbers aij are called the coefficients of the linear system; because there are m equa-
tions and n unknown variables there are thefore m × n coefficients. The main problem with
a linear system is of course to solve it:

Problem: Find a list of n numbers (s1 , s2 , . . . , sn ) that satisfy the system of linear equa-
tions (1.1).

In other words, if we substitute the list of numbers (s1 , s2 , . . . , sn ) for the unknown
variables (x1 , x2 , . . . , xn ) in equation (1.1) then the left-hand side of the ith equation will
equal bi . We call such a list (s1 , s2 , . . . , sn ) a solution to the system of equations. Notice
that we say “a solution” because there may be more than one. The set of all solutions to a
linear system is called its solution set. As an example of a linear system, below is a linear

1
Systems of Linear Equations

system consisting of m = 2 equations and n = 3 unknowns:

x1 − 5x2 − 7x3 = 0
5x2 + 11x3 = 1

Here is a linear system consisting of m = 3 equations and n = 2 unknowns:

−5x1 + x2 = −1
πx1 − 5x2 = 0
√
63x1 − 2x2 = −7

And finally, below is a linear system consisting of m = 4 equations and n = 6 unknowns:

−5x1 + x3 − 44x4 − 55x6 = −1

√
πx1 − 5x2 − x3 + 4x4 − 5x5 + 5x6 =0
√ 1
63x1 − 2x2 − 15 x3 + ln(3)x4 + 4x5 − 33 x6 =0
√ 1
63x1 − 2x2 − x3 − 18 x4 − 5x6 =5
5

Example 1.2. Verify that (1, 2, −4) is a solution to the system of equations

2x1 + 2x2 + x3 = 2
x1 + 3x2 − x3 = 11.

Is (1, −1, 2) a solution to the system?

Solution. The number of equations is m = 2 and the number of unknowns is n = 3. There
are m × n = 6 coefficients: a11 = 2, a12 = 1, a13 = 1, a21 = 1, a22 = 3, and a23 = −1. And
b1 = 0 and b2 = 11. The list of numbers (1, 2, −4) is a solution because

2 · (1) + 2(2) + (−4) = 2

(1) + 3 · (2) − (−4) = 11

On the other hand, for (1, −1, 2) we have that

2(1) + 2(−1) + (2) = 2

but
1 + 3(−1) − 2 = −4 6= 11.
Thus, (1, −1, 2) is not a solution to the system.

A linear system may not have a solution at all. If this is the case, we say that the linear
system is inconsistent:

2
 Lecture 1

INCONSISTENT ⇔ NO SOLUTION

A linear system is called consistent if it has at least one solution:

CONSISTENT ⇔ AT LEAST ONE SOLUTION

We will see shortly that a consistent linear system will have either just one solution or
infinitely many solutions. For example, a linear system cannot have just 4 or 5 solutions. If
it has multiple solutions, then it will have infinitely many solutions.

Example 1.3. Show that the linear system does not have a solution.

−x1 + x2 = 3
x1 − x2 = 1.

Solution. If we add the two equations we get

0=4

which is a contradiction. Therefore, there does not exist a list (s1 , s2 ) that satisfies the
system because this would lead to the contradiction 0 = 4.

Example 1.4. Let t be an arbitrary real number and let

s1 = − 32 − 2t
s2 = 23 + t
s3 = t.

Show that for any choice of the parameter t, the list (s1 , s2 , s3 ) is a solution to the linear
system

x1 + x2 + x3 = 0
x1 + 3x2 − x3 = 3.

Solution. Substitute the list (s1 , s2 , s3 ) into the left-hand-side of the first equation



− 32 − 2t + 32 + t + t = 0

and in the second equation

− 23 − 2t + 3( 23 + t) − t = − 23 + 9
2
=3

Both equations are satisfied for any value of t. Because we can vary t arbitrarily, we get an
infinite number of solutions parameterized by t. For example, compute the list (s1 , s2 , s3 )
for t = 3 and confirm that the resulting list is a solution to the linear system.

3
Systems of Linear Equations

1.2 Matrices
We will use matrices to develop systematic methods to solve linear systems and to study
the properties of the solution set of a linear system. Informally speaking, a matrix is an
array or table consisting of rows and columns. For example,
 
1 −2 1 0
A= 0 2 −8 8 
−4 7 11 −5
is a matrix having m = 3 rows and n = 4 columns. In general, a matrix with m rows and
n columns is a m × n matrix and the set of all such matrices will be denoted by Mm×n .
Hence, A above is a 3 × 4 matrix. The entry of A in the ith row and jth column will be
denoted by aij . A matrix containing only one column is called a column vector and a
matrix containing only one row is called a row vector. For example, here is a row vector
 
u = 1 −3 4
and here is a column vector  
3
v= .
−1

We can associate to a linear system three matrices: (1) the coefficient matrix, (2) the
output column vector, and (3) the augmented matrix. For example, for the linear system
5x1 − 3x2 + 8x3 = −1
x1 + 4x2 − 6x3 = 0
2x2 + 4x3 = 3
the coefficient matrix A, the output vector b, and the augmented matrix [A b] are:
     
5 −3 8 −1 5 −3 8 −1
A= 1 4 −6 , b =  0  , [A b] = 1 4 −6 0  .
0 2 4 3 0 2 4 3
If a linear system has m equations and n unknowns then the coefficient matrix A must be a
m × n matrix, that is, A has m rows and n columns. Using our previously defined notation,
we can write this as A ∈ Mm×n .
If we are given an augmented matrix, we can write down the associated linear system in
an obvious way. For example, the linear system associated to the augmented matrix
 
1 4 −2 8 12
0 1 −7 2 −4
0 0 5 −1 7
is
x1 + 4x2 − 2x3 + 8x4 = 12
x2 − 7x3 + 2x4 = −4
5x3 − x4 = 7.

4
 Lecture 1

We can study matrices without interpreting them as coefficient matrices or augmented ma-
trices associated to a linear system. Matrix algebra is a fascinating subject with numerous
applications in every branch of engineering, medicine, statistics, mathematics, finance, biol-
ogy, chemistry, etc.

1.3 Solving linear systems

In algebra, you learned to solve equations by first “simplifying” them using operations that
do not alter the solution set. For example, to solve 2x = 8 − 2x we can add to both sides
2x and obtain 4x = 8 and then multiply both sides by 14 yielding x = 2. We can do
similar operations on a linear system. There are three basic operations, called elementary
operations, that can be performed:

1. Interchange two equations.

2. Multiply an equation by a nonzero constant.

3. Add a multiple of one equation to another.

These operations do not alter the solution set. The idea is to apply these operations itera-
tively to simplify the linear system to a point where one can easily write down the solution
set. It is convenient to apply elementary operations on the augmented matrix [A b] repre-
senting the linear system. In this case, we call the operations elementary row operations,
and the process of simplifying the linear system using these operations is called row reduc-
tion. The goal with row reducing is to transform the original linear system into one having
a triangular structure and then perform back substitution to solve the system. This is
best explained via an example.

Example 1.5. Use back substitution on the augmented matrix

 
1 0 −2 −4
0 1 −1 0 
0 0 1 1
to solve the associated linear system.
Solution. Notice that the augmented matrix has a triangular structure. The third row
corresponds to the equation x3 = 1. The second row corresponds to the equation
x2 − x3 = 0
and therefore x2 = x3 = 1. The first row corresponds to the equation
x1 − 2x3 = −4
and therefore
x1 = −4 + 2x3 = −4 + 2 = −2.
Therefore, the solution is (−2, 1, 1).

5
Systems of Linear Equations

Example 1.6. Solve the linear system using elementary row operations.

−3x1 + 2x2 + 4x3 = 12

x1 − 2x3 = −4
2x1 − 3x2 + 4x3 = −3

Solution. Our goal is to perform elementary row operations to obtain a triangular structure
and then use back substitution to solve. The augmented matrix is
 
−3 2 4 12
1 0 −2 −4 .
2 −3 4 −3

Interchange Row 1 (R1 ) and Row 2 (R2 ):

   
−3 2 4 12 1 0 −2 −4
R ↔R2
1 0 −2 −4 −−1−−→ −3 2 4 12 
2 −3 4 −3 2 −3 4 −3

As you will see, this first operation will simplify the next step. Add 3R1 to R2 :
   
1 0 −2 −4 1 0 −2 −4
3R +R2
−3 2 4 12  −−1−−→ 0 2 −2 0 
2 −3 4 −3 2 −3 4 −3

Add −2R1 to R3 :
   
1 0 −2 −4 1 0 −2 −4
−2R1 +R3
0 2 −2 0  − −−−−→ 0 2 −2 0 
2 −3 4 −3 0 −3 8 5

Multiply R2 by 12 :
   
1 0 −2 −4 1
R2
1 0 −2 −4
0 2 −2 0  −2−→ 0 1 −1 0 
0 −3 8 5 0 −3 8 5
Add 3R2 to R3 :    
1 0 −2 −4 1 0 −2 −4
3R2 +R3
0 1 −1 0  − −−−→ 0 1 −1 0 
0 −3 8 5 0 0 5 5
Multiply R3 by 15 :
   
1 0 −2 −4 1
R3
1 0 −2 −4
0 1 −1 0  −5−→ 0 1 −1 0 
0 0 5 5 0 0 1 1
We can continue row reducing but the row reduced augmented matrix is in triangular form.
So now use back substitution to solve. The linear system associated to the row reduced

6
 Lecture 1

augmented matrix is
x1 − 2x3 = −4
x2 − x3 = 0
x3 = 1
The last equation gives that x3 = 1. From the second equation we obtain that x2 − x3 = 0,
and thus x2 = 1. The first equation then gives that x1 = −4 + 2(1) = −2. Thus, the solution
to the original system is (−2, 1, 1). You should verify that (−2, 1, 1) is a solution to the
original system.

The original augmented matrix of the previous example is

 
−3 2 4 12 −3x1 + 2x2 + 4x3 = 12
M= 1 0 −2 −4 →
x1 − 2x3 = −4
2 −3 4 −3
2x1 − 3x2 + 4x3 = −3.
After row reducing we obtained the row reduced matrix
 
1 0 −2 −4 x1 − 2x3 = −4
N = 0 1 −1 0  →
x2 − x3 = 0
0 0 1 1
x3 = 1.
Although the two augmented matrices M and N are clearly distinct, it is a fact that they
have the same solution set.

Example 1.7. Using elementary row operations, show that the linear system is inconsistent.
x1 + 2x3 = 1
x2 + x3 = 0
2x1 + 4x3 = 1
Solution. The augmented matrix is
 
1 0 2 1
 0 1 1 0
2 0 4 1
Perform the operation −2R1 + R3 :
   
1 0 2 1 1 0 2 1
−2R1 +R3
 0 1 1 0 − −−−−→ 0 1 1 0 
2 0 4 1 0 0 0 −1
The last row of the simplified augmented matrix
 
1 0 2 1
0 1 1 0
0 0 0 −1

7
Systems of Linear Equations

corresponds to the equation

0x1 + 0x2 + 0x3 = −1
Obviously, there are no numbers x1 , x2 , x3 that satisfy this equation, and therefore, the
linear system is inconsistent, i.e., it has no solution. In general, if we obtain a row in an
augmented matrix of the form
 
0 0 0 ··· 0 c

where c is a nonzero number, then the linear system is inconsistent. We will call this type
of row an inconsistent row. However, a row of the form
 
0 1 0 0 0

corresponds to the equation x2 = 0 which is perfectly valid.

1.4 Geometric interpretation of the solution set

The set of points (x1 , x2 ) that satisfy the linear system

x1 − 2x2 = −1
(1.2)
−x1 + 3x2 = 3

is the intersection of the two lines determined by the equations of the system. The solution
for this system is (3, 2). The two lines intersect at the point (x1 , x2 ) = (3, 2), see Figure 1.1.

Figure 1.1: The intersection point of the two lines is the solution of the linear system (1.2)

Similarly, the solution of the linear system

x1 − 2x2 + x3 = 0
2x2 − 8x3 = 8 (1.3)
−4x1 + 5x2 + 9x3 = −9

8
 Lecture 1

is the intersection of the three planes determined by the equations of the system. In this case,
there is only one solution: (29, 16, 3). In the case of a consistent system of two equations,
the solution set is the line of intersection of the two planes determined by the equations of
the system, see Figure 1.2.

x1 − 2x2 + x3 = 0
−4x1 + 5x2 + 9x3 = −9

the solution set is this line

Figure 1.2: The intersection of the two planes is the solution set of the linear system (1.3)

After this lecture you should know the following:

• what a linear system is
• what it means for a linear system to be consistent and inconsistent
• what matrices are
• what are the matrices associated to a linear system
• what the elementary row operations are and how to apply them to simplify a linear
system
• what it means for two matrices to be row equivalent
• how to use the method of back substitution to solve a linear system
• what an inconsistent row is
• how to identify using elementary row operations when a linear system is inconsistent
• the geometric interpretation of the solution set of a linear system

9
Systems of Linear Equations

10
 Lecture 2

Lecture 2

Row Reduction and Echelon Forms

In this lecture, we will get more practice with row reduction and in the process introduce
two important types of matrix forms. We will also discuss when a linear system has a unique
solution, infinitely many solutions, or no solution. Lastly, we will introduce a convenient
parameter called the rank of a matrix.

2.1 Row echelon form (REF)

Consider the linear system

x1 + 5x2 − 2x4 − x5 + 7x6 = −4

2x2 − 2x3 + 3x6 = 0
−9x4 − x5 + x6 = −1
5x5 + x6 = 5
0=0

having augmented matrix

 
1 5 0 −2 −1 7 −4
0 2 −2 0 0 3 0
 
0 0 0 −9 −1 1 −1
 .
0 0 0 0 5 1 5
0 0 0 0 0 0 0

The above augmented matrix has the following properties:

P1. All nonzero rows are above any rows of all zeros.

P2. The leftmost nonzero entry of a row is to the right of the leftmost nonzero entry of
the row above it.

11
Row Reduction and Echelon Forms

Any matrix satisfying properties P1 and P2 is said to be in row echelon form (REF). In
REF, the leftmost nonzero entry in a row is called a leading entry:
 
1 5 0 −2 −1 7 −4
 0 2 −2 0 0 3 0
 
 0 0 0 −9 −1 1 −1
 
0 0 0 0 5 1 5
0 0 0 0 0 0 0
A consequence of property P2 is that every entry below a leading entry is zero:
 
1 5 0 −2 −4 −1 −7
0 2 −2 0 0 3 0
 
0 0 0 −9 −1 1 −1
 
0 0 0 0 5 1 5
0 0 0 0 0 0 0
We can perform elementary row operations, or row reduction, to transform a matrix into
REF.

Example 2.1. Explain why the following matrices are not in REF. Use elementary row
operations to put them in REF.
   
3 −1 0 3 7 5 0 −3
M = 0 0 0 0 N = 0 3 −1 1 
0 1 3 0 0 6 −5 2
Solution. Matrix M fails property P1. To put M in REF we interchange R2 with R3 :
   
3 −1 0 3 3 −1 0 3
R ↔R3
M = 0 0 0 0 −−2−−→ 0 1 3 0
0 1 3 0 0 0 0 0
The matrix N fails property P2. To put N in REF we perform the operation −2R2 + R3 →
R3 :    
7 5 0 −3 7 5 0 −3
−2R2 +R3
0 3 −1 1  − −−−−→ 0 3 −1 1 
0 6 −5 2 0 0 −3 0

Why is REF useful? Certain properties of a matrix can be easily deduced if it is in REF.
For now, REF is useful to us for solving a linear system of equations. If an augmented matrix
is in REF, we can use back substitution to solve the system, just as we did in Lecture 1.
For example, consider the system

8x1 − 2x2 + x3 = 4
3x2 − x3 = 7
2x3 = 4

12
 Lecture 2

whose augmented matrix is already in REF:

 
8 −2 1 4
0 3 −1 7
0 0 2 4
From the last equation we obtain that 2x3 = 4, and thus x3 = 2. Substituting x3 = 2 into
the second equation we obtain that x2 = 3. Substituting x3 = 2 and x2 = 3 into the first
equation we obtain that x1 = 1.

2.2 Reduced row echelon form (RREF)

Although REF simplifies the problem of solving a linear system, later on in the course we
will need to completely row reduce matrices into what is called reduced row echelon form
(RREF). A matrix is in RREF if it is in REF (so it satisfies properties P1 and P2) and in
addition satisfies the following properties:

P3. The leading entry in each nonzero row is a 1.

P4. All the entries above (and below) a leading 1 are all zero.

A leading 1 in the RREF of a matrix is called a pivot. For example, the following matrix
in RREF:  
1 6 0 3 0 0
0 0 1 −4 0 5
0 0 0 0 1 7
has three pivots:  
1 6 0 3 0 0
0 0 1 −4 0 5
0 0 0 0 1 7

Example 2.2. Use row reduction to transform the matrix into RREF.
 
0 3 −6 6 4 −5
 3 −7 8 −5 8 9 
3 −9 12 −9 6 15
Solution. The first step is to make the top leftmost entry nonzero:
   
0 3 −6 6 4 −5 3 −9 12 −9 6 15
R3 ↔R1
3 −7 8 −5 8 9  − −−−→ 3 −7 8 −5 8 9 
3 −9 12 −9 6 15 0 3 −6 6 4 −5
Now create a leading 1 in the first row:
   
3 −9 12 −9 6 15 1
R1
1 −3 4 −3 2 5
3 −7 8 −5 8 9  −3−→ 3 −7 8 −5 8 9 
0 3 −6 6 4 −5 0 3 −6 6 4 −5

13
Row Reduction and Echelon Forms

Create zeros under the newly created leading 1:

   
1 −3 4 −3 2 5 1 −3 4 −3 2 5
−3R1 +R2
3 −7 8 −5 8 9  − −−−−→ 0 2 −4 4 2 −6
0 3 −6 6 4 −5 0 3 −6 6 4 −5

Create a leading 1 in the second row:

   
1 −3 4 −3 2 5 1
R2
1 −3 4 −3 2 5
0 2 −4 4 2 −6 −2−→ 0 1 −2 2 1 −3
0 3 −6 6 4 −5 0 3 −6 6 4 −5

Create zeros under the newly created leading 1:

   
1 −3 4 −3 2 5 1 −3 4 −3 2 5
−3R2 +R3
0 1 −2 2 1 −3 − −−−−→ 0 1 −2 2 1 −3
0 3 −6 6 4 −5 0 0 0 0 1 4

We have now completed the top-to-bottom phase of the row reduction algorithm. In the
next phase, we work bottom-to-top and create zeros above the leading 1’s. Create zeros
above the leading 1 in the third row:
   
1 −3 4 −3 2 5 1 −3 4 −3 2 5
−R3 +R2
0 1 −2 2 1 −3 − −−−−→ 0 1 −2 2 0 −7
0 0 0 0 1 4 0 0 0 0 1 4
   
1 −3 4 −3 2 5 1 −3 4 −3 0 −3
−2R3 +R1
0 1 −2 2 0 −7 − −−−−→ 0 1 −2 2 0 −7
0 0 0 0 1 4 0 0 0 0 1 4
Create zeros above the leading 1 in the second row:
   
1 −3 4 −3 0 −3 1 0 −2 3 0 −24
3R +R1
0 1 −2 2 0 −7 −−2−−→ 0 1 −2 2 0 −7 
0 0 0 0 1 4 0 0 0 0 1 4

This completes the row reduction algorithm and the matrix is in RREF.

Example 2.3. Use row reduction to solve the linear system.

2x1 + 4x2 + 6x3 = 8

x1 + 2x2 + 4x3 = 8
3x1 + 6x2 + 9x3 = 12

Solution. The augmented matrix is

 
2 4 6 8
1 2 4 8 
3 6 9 12

14
 Lecture 2

Create a leading 1 in the first row:

   
2 4 6 8 1
R1
1 2 3 4
1 2 4 8  −2−→ 1 2 4 8 
3 6 9 12 3 6 9 12
Create zeros under the first leading 1:
   
1 2 3 4 1 2 3 4
−R1 +R2
1 2 4 8 −−−−−→ 0
  0 1 4
3 6 9 12 3 6 9 12
   
1 2 3 4 1 2 3 4
−3R +R3
0 0 1 4  −−−1−−→ 0 0 1 4
3 6 9 12 0 0 0 0
The system is consistent, however, there are only 2 nonzero rows but 3 unknown variables.
This means that the solution set will contain 3 − 2 = 1 free parameter. The second row
in the augmented matrix is equivalent to the equation:

x3 = 4.

The first row is equivalent to the equation:

x1 + 2x2 + 3x3 = 4

and after substituting x3 = 4 we obtain

x1 + 2x2 = −8.

We now must choose one of the variables x1 or x2 to be a parameter, say t, and solve for the
remaining variable. If we set x2 = t then from x1 + 2x2 = −8 we obtain that

x1 = −8 − 2t.

We can therefore write the solution set for the linear system as
x1 = −8 − 2t
x2 = t (2.1)
x3 = 4
where t can be any real number. If we had chosen x1 to be the parameter, say x1 = t,
then the solution set can be written as
x1 = t
x2 = −4 − 12 t (2.2)
x3 = 4

Although (2.1) and (2.2) are two different parameterizations, they both give the same solution
set.

15
Row Reduction and Echelon Forms

In general, if a linear system has n unknown variables and the row reduced augmented
matrix has r leading entries, then the number of free parameters d in the solution set is

d = n − r.

Thus, when performing back substitution, we will have to set d of the unknown variables
to arbitrary parameters. In the previous example, there are n = 3 unknown variables and
the row reduced augmented matrix contained r = 2 leading entries. The number of free
parameters was therefore
d = n − r = 3 − 2 = 1.
Because the number of leading entries r in the row reduced coefficient matrix determine the
number of free parameters, we will refer to r as the rank of the coefficient matrix:

r = rank(A).

Later in the course, we will give a more geometric interpretation to rank(A).

Example 2.4. Solve the linear system represented by the augmented matrix
 
1 −7 2 −5 8 10
0 1 −3 3 1 −5
0 0 0 1 −1 4

Solution. The number of unknowns is n = 5 and the augmented matrix has rank r = 3
(leading entries). Thus, the solution set is parameterized by d = 5 − 3 = 2 free variables,
call them t and s. The last equation of the augmented matrix is x4 − x5 = 4. We choose x5
to be the first parameter so we set x5 = t. Therefore, x4 = 4 + t. The second equation of
the augmented matrix is
x2 − 3x3 + 3x4 + x5 = −5

and the unassigned variables are x2 and x3 . We choose x3 to be the second parameter, say
x3 = s. Then

x2 = −5 + 3x3 − 3x4 − x5
= −5 + 3s − 3(4 + t) − t
= −17 − 4t + 3s.

We now use the first equation of the augmented matrix to write x1 in terms of the other
variables:

x1 = 10 + 7x2 − 2x3 + 5x4 − 8x5

= 10 + 7(−17 − 4t + 3s) − 2s + 5(4 + t) − 8t
= −89 − 31t + 19s

16
 Lecture 2

Thus, the solution set is

x1 = −89 − 31t + 19s

x2 = −17 − 4t + 3s
x3 =s
x4 =4+t
x5 =t

where t and s are arbitrary real numbers.. Choose arbitrary numbers for t and s and
substitute the corresponding list (x1 , x2 , . . . , x5 ) into the system of equations to verify that
it is a solution.

2.3 Existence and uniqueness of solutions

The REF or RREF of an augmented matrix leads to three distinct possibilities for the
solution set of a linear system.

Theorem 2.5: Let [A b] be the augmented matrix of a linear system. One of the following
distinct possibilities will occur:

1. The augmented matrix will contain an inconsistent row.

2. All the rows of the augmented matrix are consistent and there are no free parameters.

3. All the rows of the augmented matrix are consistent and there are d ≥ 1 variables
that must be set to arbitrary parameters

In Case 1., the linear system is inconsistent and thus has no solution. In Case 2., the linear
system is consistent and has only one (and thus unique) solution. This case occurs when
r = rank(A) = n since then the number of free parameters is d = n − r = 0. In Case 3., the
linear system is consistent and has infinitely many solutions. This case occurs when r < n
and thus d = n − r > 0 is the number of free parameters.

After this lecture you should know the following:

• what the REF is and how to compute it
• what the RREF is and how to compute it
• how to solve linear systems using row reduction (Practice!!!)
• how to identify when a linear system is inconsistent
• how to identify when a linear system is consistent
• what is the rank of a matrix
• how to compute the number of free parameters in a solution set
• what are the three possible cases for the solution set of a linear system (Theorem 2.5)

17
Row Reduction and Echelon Forms

18
 Lecture 3

Lecture 3

Vector Equations

In this lecture, we introduce vectors and vector equations. Specifically, we introduce the
linear combination problem which simply asks whether it is possible to express one vector
in terms of other vectors; we will be more precise in what follows. As we will see, solving
the linear combination problem reduces to solving a linear system of equations.

3.1 Vectors in Rn
Recall that a column vector in Rn is a n × 1 matrix. From now on, we will drop the
“column” descriptor and simply use the word vectors. It is important to emphasize that a
vector in Rn is simply a list of n numbers; you are safe (and highly encouraged!) to forget
the idea that a vector is an object with an arrow. Here is a vector in R2 :
 
3
v= .
−1

Here is a vector in R3 :  
−3
v =  0 .
11

Here is a vector in R6 : 
9
0
 
−3
 6 .
v= 
 
0
3
To indicate that v is a vector in Rn , we will use the notation v ∈ Rn . The mathematical
symbol ∈ means “is an element of”. When we write vectors within a paragraph, we willwrite

−1
them using list notation instead of column notation, e.g., v = (−1, 4) instead of v = .
4

19
Vector Equations

We can add/subtract vectors, and multiply vectors by numbers or scalars. For example,
here is the addition of two vectors:
     
0 4 4
−5 −3 −8
  +   =  .
9 0 9
2 1 3
And the multiplication of a scalar with a vector:
   
1 3
3 −3 = −9 .
5 15
And here are both operations combined:
         
4 −2 −8 −6 −14
−2 −8 + 3  9  =  16  +  27  =  43  .
3 4 −6 12 6
These operations constitute “the algebra” of vectors. As the following example illustrates,
vectors can be used in a natural way to represent the solution of a linear system.

Example 3.1. Write the general solution in vector form of the linear system represented
by the augmented matrix
 
  1 −7 2 −5 8 10
A b = 0 1 −3 3 1 −5
0 0 0 1 −1 4
Solution. The number of unknowns is n = 5 and the associated coefficient matrix A has
rank r = 3. Thus, the solution set is parametrized by d = n − r = 2 parameters. This
system was considered in Example 2.4 and the general solution was found to be

x1 = −89 − 31t1 + 19t2

x2 = −17 − 4t1 + 3t2
x3 = t2
x4 = 4 + t1
x5 = t1

where t1 and t2 are arbitrary real numbers. The solution in vector form therefore takes the
form          
x1 −89 − 31t1 + 19t2 −89 −31 19
x2   −17 − 4t1 + 3t2  −17  −4  3
         
x3  =  t2
x=    =  0  + t1  0  + t2  1 
      
x4   4 + t1   4   1  0
x5 t1 0 1 0

20
 Lecture 3

A fundamental problem in linear algebra is solving vector equations for an unknown

vector. As an example, suppose that you are given the vectors
     
4 −2 −14
v1 = −8 , v2 =  9  , b =  43  ,
3 4 6

and asked to find numbers x1 and x2 such that x1 v1 + x2 v2 = b, that is,

     
4 −2 −14
x1 −8 + x2  9  =  43  .
3 4 6

Here the unknowns are the scalars x1 and x2 . After some guess and check, we find that
x1 = −2 and x2 = 3 is a solution to the problem since
     
4 −2 −14
−2 −8 + 3  9  =  43  .
3 4 6

In some sense, the vector b is a combination of the vectors v1 and v2 . This motivates the
following definition.

Definition 3.2: Let v1 , v2 , . . . , vp be vectors in Rn . A vector b is said to be a linear

combination of the vectors v1 , v2 , . . . , vp if there exists scalars x1 , x2 , . . . , xp such that
x1 v1 + x2 v2 + · · · + xp vp = b.

The scalars in a linear combination are called the coefficients of the linear combination.
As an example, given the vectors
       
1 −2 −1 −3
v1 = −2 , v2 =  4  , v3 =  5  , b =  0 
3 −6 6 −27

you can verify (and you should!) that

3v1 + 4v2 − 2v3 = b.

Therefore, we can say that b is a linear combination of v1 , v2 , v3 with coefficients x1 = 3,

x2 = 4, and x3 = −2.

3.2 The linear combination problem

The linear combination problem is the following:

21
Vector Equations

Problem: Given vectors v1 , . . . , vp and b, is b a linear combination of v1 , v2 , . . . , vp ?

For example, say you are given the vectors

     
1 1 2
v1 = 2 , v2 = 1 , v3 = 1
1 0 2
and also  
0
b =  1 .
−2
Does there exist scalars x1 , x2 , x3 such that

x1 v1 + x2 v2 + x3 v3 = b? (3.1)

For obvious reasons, equation (3.1) is called a vector equation and the unknowns are x1 ,
x2 , and x3 . To gain some intuition with the linear combination problem, let’s do an example
by inspection.

Example 3.3. Let v1 = (1, 0, 0), let v2 = (0, 0, 1), let b1 = (0, 2, 0), and let b2 = (−3, 0, 7).
Are b1 and b2 linear combinations of v1 , v2 ?
Solution. For any scalars x1 and x2
       
x1 0 x1 0
x1 v1 + x2 v2 = 0 + 0 = 0 6= 2
      
0 x2 x2 0
and thus no, b1 is not a linear combination of v1 , v2 , v3 . On the other hand, by inspection
we have that      
−3 0 −3
−3v1 + 7v2 = 0 + 0 = 0  = b2
    
0 7 7
and thus yes, b2 is a linear combination of v1 , v2 , v3 . These examples, of low dimension,
were more-or-less obvious. Going forward, we are going to need a systematic way to solve
the linear combination problem that does not rely on pure inspection.
We now describe how the linear combination problem is connected to the problem of
solving a system of linear equations. Consider again the vectors
       
1 1 2 0
v1 = 2 , v2 = 1 , v3 = 1 , b = 1  .
      
1 0 2 −2
Does there exist scalars x1 , x2 , x3 such that

x1 v1 + x2 v2 + x3 v3 = b? (3.2)

22
 Lecture 3

First, let’s expand the left-hand side of equation (3.2):

       
x1 x2 2x3 x1 + x2 + 2x3
x1 v1 + x2 v2 + x3 v3 = 2x1  + x2  +  x3  = 2x1 + x2 + x3  .
x1 0 2x3 x1 + 2x3

We want equation (3.2) to hold so let’s equate the expansion x1 v1 + x2 v2 + x3 v3 with b. In

other words, set    
x1 + x2 + 2x3 0
2x1 + x2 + x3  =  1  .
x1 + 2x3 −2
Comparing component-by-component in the above relationship, we seek scalars x1 , x2 , x3
satisfying the equations
x1 + x2 + 2x3 = 0
2x1 + x2 + x3 = 1 (3.3)
x1 + 2x3 = −2.
This is just a linear system consisting of m = 3 equations and n = 3 unknowns! Thus, the
linear combination problem can be solved by solving a system of linear equations for the
unknown scalars x1 , x2 , x3 . We know how to do this. In this case, the augmented matrix of
the linear system (3.3) is  
1 1 2 0
[A b] = 2 1 1 1 
1 0 2 −2
Notice that the 1st column of A is just v1 , the second column is v2 , and the third column
is v3 , in other words, the augment matrix is
 
[A b] = v1 v2 v3 b

Applying the row reduction algorithm, the solution is

x1 = 0, x2 = 2, x3 = −1

and thus these coefficients solve the linear combination problem. In other words,

0v1 + 2v2 − v3 = b

In this case, there is only one solution to the linear system, so b can be written as a
linear combination of v1 , v2 , . . . , vp in only one (or unique) way. You should verify these
computations.
We summarize the previous discussion with the following:

The problem of determining if a given vector b is a linear combination of the vectors

v1 , v2 , . . . , vp is equivalent to solving the linear system of equations with augmented matrix
   
A b = v1 v2 · · · vp b .

23
Vector Equations

Applying the existence and uniqueness Theorem 2.5, the only three possibilities to the linear
combination problem are:

1. If the linear system is inconsistent then b is not a linear combination of v1 , v2 , . . . , vp ,

i.e., there does not exist scalars x1 , x2 , . . . , xp such that x1 v1 + x2 v2 + · · · + xp vp = b.

2. If the linear system is consistent and the solution is unique then b can be written as a
linear combination of v1 , v2 , . . . , vp in only one way.

3. If the the linear system is consistent and the solution set has free parameters, then b
can be written as a linear combination of v1 , v2 , . . . , vp in infinitely many ways.

Example 3.4. Is the vector b = (7, 4, −3) a linear combination of the vectors
   
1 2
v1 = −2 , v2 = 5?

−5 6

Solution. Form the augmented matrix:

 
  1 2 7
v1 v2 b = −2 5 4 
−5 6 −3

The RREF of the augmented matrix is

 
1 0 3
 0 1 2
0 0 0

and therefore the solution is x1 = 3 and x2 = 2. Therefore, yes, b is a linear combination of

v1 , v2 :      
1 2 7
3v1 + 2v2 = 3 −2 + 2 5 = 4  = b
    
−5 6 −3
Notice that the solution set does not contain any free parameters because n = 2 (unknowns)
and r = 2 (rank) and so d = 0. Therefore, the above linear combination is the only way to
write b as a linear combination of v1 and v2 .

Example 3.5. Is the vector b = (1, 0, 1) a linear combination of the vectors

     
1 0 2
v1 = 0 ,
 v2 = 1 ,
 v3 = 1?

2 0 4

24
 Lecture 3

Solution. The augmented matrix of the corresponding linear system is

 
1 0 2 1
0 1 1 0 .
2 0 4 1
After row reducing we obtain that
 
1 0 2 1
0 1 1 0  .
0 0 0 −1
The last row is inconsistent, and therefore the linear system does not have a solution. There-
fore, no, b is not a linear combination of v1 , v2 , v3 .

Example 3.6. Is the vector b = (8, 8, 12) a linear combination of the vectors
     
2 4 6
v1 = 1 , v2 = 2 , v3 = 4?
    
3 6 9
Solution. The augmented matrix is
   
2 4 6 8 1 2 3 4
REF
1 2 4 8  −−→ 0 0 1 4 .
3 6 9 12 0 0 0 0
The system is consistent and therefore b is a linear combination of v1 , v2 , v3 . In this case,
the solution set contains d = 1 free parameters and therefore, it is possible to write b as a
linear combination of v1 , v2 , v3 in infinitely many ways. In terms of the parameter t, the
solution set is

x1 = −8 − 2t
x2 = t
x3 = 4

Choosing any t gives scalars that can be used to write b as a linear combination of v1 , v2 , v3 .
For example, choosing t = 1 we obtain x1 = −10, x2 = 1, and x3 = 4, and you can verify
that        
2 4 6 8
−10v1 + v2 + 4v3 = −10 1 + 2 + 4 4 =  8  = b
3 6 9 12
Or, choosing t = −2 we obtain x1 = −4, x2 = −2, and x3 = 4, and you can verify that
       
2 4 6 8
−4v1 − 2v2 + 4v3 = −4 1 − 2 2 + 4 4 = 8  = b
      
3 6 9 12

25
Vector Equations

We make a few important observations on linear combinations of vectors. Given vectors

v1 , v2 , . . . , vp , there are certain vectors b that can be written as a linear combination of
v1 , v2 , . . . , vp in an obvious way. The zero vector b = 0 can always be written as a linear
combination of v1 , v2 , . . . , vp :

0 = 0v1 + 0v2 + · · · + 0vp .

Each vi itself can be written as a linear combination of v1 , v2 , . . . , vp , for example,

v2 = 0v1 + (1)v2 + 0v3 + · · · + 0vp .

More generally, any scalar multiple of vi can be written as a linear combination of v1 , v2 , . . . , vp ,

for example,
xv2 = 0v1 + xv2 + 0v3 + · · · + 0vp .
By varying the coefficients x1 , x2 , . . . , xp , we see that there are infinitely many vectors b
that can be written as a linear combination of v1 , v2 , . . . , vp . The “space” of all the possible
linear combinations of v1 , v2 , . . . , vp has a name, which we introduce next.

3.3 The span of a set of vectors

Given a set of vectors {v1 , v2 , . . . , vp }, we have been considering the problem of whether
or not a given vector b is a linear combination of {v1 , v2 , . . . , vp }. We now take another
point of view and instead consider the idea of generating all vectors that are a linear
combination of {v1 , v2 , . . . , vp }. So how do we generate a vector that is guaranteed to be
a linear combination of {v1 , v2 , . . . , vp }? For example, if v1 = (2, 1, 3), v2 = (4, 2, 6) and
v3 = (6, 4, 9) then
       
2 4 6 8
−10v1 + v2 + 4v3 = −10 1 + 2 + 4 4 = 8  .
      
3 6 9 12

Thus, by construction, the vector b = (8, 8, 12) is a linear combination of {v1 , v2 , v3 }. This
discussion leads us to the following definition.

Definition 3.7: Let v1 , v2 , . . . , vp be vectors. The set of all vectors that are a linear
combination of v1 , v2 , . . . , vp is called the span of v1 , v2 , . . . , vp , and we denote it by

S = span{v1 , v2 , . . . , vp }.

By definition, the span of a set of vectors is a collection of vectors, or a set of vectors. If b is

a linear combination of v1 , v2 , . . . , vp then b is an element of the set span{v1 , v2 , . . . , vp },
and we write this as
b ∈ span{v1 , v2 , . . . , vp }.

26
 Lecture 3

By definition, writing that b ∈ span{v1 , v2 , . . . , vp } implies that there exists scalars x1 , x2 , . . . , xp

such that
x1 v1 + x2 v2 + · · · + xp vp = b.
Even though span{v1 , v2 , . . . , vp } is an infinite set of vectors, it is not necessarily true that
it is the whole space Rn .
The set span{v1 , v2 , . . . , vp } is just a collection of infinitely many vectors but it has some
geometric structure. In R2 and R3 we can visualize span{v1 , v2 , . . . , vp }. In R2 , the span of
a single nonzero vector, say v ∈ R2 , is a line through the origin in the direction of v, see
Figure 3.1.

Figure 3.1: The span of a single non-zero vector in R2 .

In R2 , the span of two vectors v1 , v2 ∈ R2 that are not multiples of each other is all of
R2 . That is, span{v1 , v2 } = R2 . For example, with v1 = (1, 0) and v2 = (0, 1), it is true
that span{v1 , v2 } = R2 . In R3 , the span of two vectors v1 , v2 ∈ R3 that are not multiples
of each other is a plane through the origin containing v1 and v2 , see Figure 3.2. In R3 , the

3
span{v,w}

z 0

−1

−2

−3

−4

−4
−3
−2
−1 −4
−3
0 −2
y 1 −1
0
2 1 x
3 2
3
4

Figure 3.2: The span of two vectors, not multiples of each other, in R3 .

span of a single vector is a line through the origin, and the span of three vectors that do not
depend on each other (we will make this precise soon) is all of R3 .

Example 3.8. Is the vector b = (7, 4, −3) in the span of the vectors v1 = (1, −2, −5), v2 =
(2, 5, 6)? In other words, is b ∈ span{v1 , v2 }?

27
Vector Equations

Solution. By definition, b is in the span of v1 and v2 if there exists scalars x1 and x2 such
that
x1 v1 + x2 v2 = b,
that is, if b can be written as a linear combination of v1 and v2 . From our previous
 discussion

on the linear combination problem, we must consider the augmented matrix v1 v2 b .
Using row reduction, the augmented matrix is consistent and there is only one solution (see
Example 3.4). Therefore, yes, b ∈ span{v1 , v2 } and the linear combination is unique.

Example 3.9. Is the vector b = (1, 0, 1) in the span of the vectors v1 = (1, 0, 2), v2 =
(0, 1, 0), v3 = (2, 1, 4)?

Solution. From Example 3.5, we have that

 
  REF 1 0 2 1
v1 v2 v3 b −−→ 0 1 1 0 
0 0 0 −1

The last row is inconsistent and therefore b is not in span{v1 , v2 , v3 }.

Example 3.10. Is the vector b = (8, 8, 12) in the span of the vectors v1 = (2, 1, 3), v2 =
(4, 2, 6), v3 = (6, 4, 9)?

Solution. From Example 3.6, we have that

 
  REF 1 2 3 4
v1 v2 v3 b −−→ 0 0 1 4 .
0 0 0 0

The system is consistent and therefore b ∈ span{v1 , v2 , v3 }. In this case, the solution set
contains d = 1 free parameters and therefore, it is possible to write b as a linear combination
of v1 , v2 , v3 in infinitely many ways.

Example 3.11. Answer the following with True or False, and explain your answer.
(a) The vector b = (1, 2, 3) is in the span of the set of vectors
     
 −1 2 4 
 3  , −7 , −5 .
 
0 0 0
 
(b) The solution set of the linear system whose augmented matrix is v1 v2 v3 b is the
same as the solution set of the vector equation
 x1 v1 + x2 v2 + x3 v3 = b.
(c) Suppose that the augmented matrix v1 v2 v3 b has an inconsistent row. Then
either b can be written as a linear combination of v1 , v2 , v3 or b ∈ span{v1 , v2 , v3 }.
(d) The span of the vectors {v1 , v2 , v3 } (at least one of which is nonzero) contains only the
vectors v1 , v2 , v3 and the zero vector 0.

28
 Lecture 3

After this lecture you should know the following:

• what a vector is
• what a linear combination of vectors is
• what the linear combination problem is
• the relationship between the linear combination problem and the problem of solving
linear systems of equations
• how to solve the linear combination problem
• what the span of a set of vectors is
• the relationship between what it means for a vector b to be in the span of v1 , v2 , . . . , vp
and the problem of writing b as a linear combination of v1 , v2 , . . . , vp
• the geometric interpretation of the span of a set of vectors

29
Vector Equations

30
 Lecture 4

Lecture 4

The Matrix Equation Ax = b

In this lecture, we introduce the operation of matrix-vector multiplication and how it relates
to the linear combination problem.

4.1 Matrix-vector multiplication

We begin with the definition of matrix-vector multiplication.

Definition 4.1: Given a matrix A ∈ Mm×n and a vector x ∈ Rn ,

   
a11 a12 a13 · · · a1n x1
 a21 a22 a23 · · · a2n   x2 
   
A =  .. .. .. .. ..  , x =  ..  ,
 . . . . .  .
am1 am2 am3 · · · amn xn

we define the product of A and x as the vector Ax in Rm given by

    
a11 a12 a13 · · · a1n x1 a11 x1 + a12 x2 + · · · + a1n xn
 a21 a22 a23 · · · a2n 
  x2   a21 x1 + a22 x2 + · · · + a2n xn 
   

Ax =  .. .. .. .. =
..   ..   .. .
 . . . . .  .   . 
am1 am2 am3 · · · amn xn am1 x1 + am2 x2 + · · · + amn xn
| {z } | {z }
A x

For the product Ax to be well-defined, the number of columns of A must equal the number
of components of x. Another way of saying this is that the outer dimension of A must equal
the inner dimension of x:

(m × n) · (n × 1) → m × 1

Example 4.2. Compute Ax.

31
The Matrix Equation Ax = b

(a)
 
2
  −4
A = 1 −1 3 0 , x=
−3


(b)
 
  1
3 3 −2
A= , x= 0 
4 −4 −1
−1

(c)
 
−1 1 0  
4 −1
1 −2
 3 −3 3  ,
A=  x= 2 
−2
0 −2 −3

Solution. We compute:
(a)

 
2
  −4
Ax = 1 −1 3 0  −3


8
   
= (1)(2) + (−1)(−4) + (3)(−3) + (0)(8) = −3

(b)

 
  1
3 3 −2  
Ax = 0
4 −4 −1
−1
 
(3)(1) + (3)(0) + (−2)(−1)
=
(4)(1) + (−4)(0) + (−1)(−1)
 
5
=
5

32
 Lecture 4

(c)
 
−1 1 0  
4 −1
1 −2
Ax = 
   2
3 −3 3 
−2
0 −2 −3
 
(−1)(−1) + (1)(2) + (0)(−2)
 (4)(−1) + (1)(2) + (−2)(−2) 
= 
 (3)(−1) + (−3)(2) + (3)(−2) 
(0)(−1) + (−2)(2) + (−3)(−2)
 
3
 2 
=−15


We now list two important properties of matrix-vector multiplication.

Theorem 4.3: Let A be an m × n a matrix.

(a) For any vectors u, v in Rn it holds that

A(u + v) = Au + Av.

(b) For any vector u and scalar c it holds that

A(cu) = c(Au).

Example 4.4. For the given data, verify that the properties of Theorem 4.3 hold:
     
3 −3 −1 2
A= , u= , v= , c = −2.
2 1 3 −1

4.2 Matrix-vector multiplication and linear combina-

tions
Recall the general definition of matrix-vector multiplication Ax is
    
a11 a12 a13 · · · a1n x1 a11 x1 + a12 x2 + · · · + a1n xn
 a21 a22 a23 · · · a2n   x2   a21 x1 + a22 x2 + · · · + a2n xn
    
 
 .. .. .. .. ..   ..  =  ..  (4.1)
 . . . . .  .   . 
am1 am2 am3 · · · amn xn am1 x1 + am2 x2 + · · · + amn xn

33
The Matrix Equation Ax = b

There is an important way to decompose matrix-vector multiplication involving a linear

combination. To see how, let v1 , v2 , . . . , vn denote the columns of A and consider the
following linear combination:
     
x1 a11 x2 a12 xn a1n
 x1 a21   x2 a22   xn a2n 
     
x1 v1 + x2 v2 + · · · + xn vn =  ..  +  ..  + · · · +  .. 
 .   .   . 
x1 am1 x2 am2 xn amn
 
x1 a11 + x2 a12 + · · · + xn a1n

 x1 a21 + x2 a22 + · · · + xn a2n


= .. . (4.2)
 . 
x1 am1 + x2 am2 + · · · + xn amn
 
We observe that expressions (4.1) and (4.2) are equal! Therefore, if A = v1 v2 · · · vn
and x = (x1 , x2 , . . . , xn ) then

Ax = x1 v1 + x2 v2 + · · · + xn vn .

In summary, the vector Ax is a linear combination of the columns of A where the scalar
in the linear combination are the components of x! This (important) observation gives an
alternative way to compute Ax.

Example 4.5. Given  

−1 1 0  
4 −1
1 −2
 3 −3 3  ,
A=  x =  2 ,
−2
0 −2 −3
compute Ax in two ways: (1) using the original Definition 4.1, and (2) as a linear combination
of the columns of A.

4.3 The matrix equation problem

As we have seen, with a matrix A and any vector x, we can produce a new output vector
via the multiplication Ax. If A is a m × n matrix then we must have x ∈ Rn and the output
vector Ax is in Rm . We now introduce the following problem:

Problem: Given a matrix A ∈ Mm×n and a vector b ∈ Rm , find, if possible, a vector

x ∈ Rn such that
Ax = b. (⋆)

Equation (⋆) is a matrix equation where the unknown variable is x. If u is a vector such
that Au = b, then we say that u is a solution to the equation Ax = b. For example,

34
 Lecture 4

suppose that    
1 0 −3
A= , b= .
1 0 7
 
x
Does the equation Ax = b have a solution? Well, for any x = 1 we have that
x2
    
1 0 x1 x
Ax = = 1
1 0 x2 x1

and thus any output vector Ax has equal entries. Since b does not have equal entries then
the equation Ax = b has no solution.
We now describe a systematic way to solve matrix equations. As we have seen, the vector
Ax is a linear combination of the columns of A with the coefficients given by the components
of x. Therefore, the matrix equation problem is equivalent to the linear combination problem.
In Lecture 2, we showed that the linear combination problem can be  solved by solving
 a
system of linear equations. Putting all this together then, if A = v1 v2 · · · vn and
b ∈ Rm then:

To find a vector x ∈ Rn that solves the matrix equation

Ax = b

we solve the linear system whose augmented matrix is

   
A b = v1 v2 · · · vn b .

From now on, a system of linear equations such as

a11 x1 + a12 x2 + a13 x3 + · · · + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + · · · + a2n xn = b2
a31 x1 + a32 x2 + a33 x3 + · · · + a3n xn = b3
.. .. .. .. ..
. . . . .
am1 x1 + am2 x2 + am3 x3 + · · · + amn xn = bm

will be written in the compact form

Ax = b
where A is the coefficient matrix of the linear system, b is the output vector, and x is the
unknown vector to be solved for. We summarize our findings with the following theorem.

Theorem 4.6: Let A ∈ Mm×n and b ∈ Rm . The following statements are equivalent:
(a) The equation Ax = b has a solution.
(b) The vector b is a linear combination of the columns of A.
 
(c) The linear system represented by the augmented matrix A b is consistent.

35
The Matrix Equation Ax = b

Example 4.7. Solve, if possible, the matrix equation Ax = b if

   
1 3 −4 −2
A= 1
 5 2 , b = 4 .
 
−3 −7 −6 12

Solution. First form the augmented matrix:

 
1 3 −4 −2
[A b] =  1 5 2 4
−3 −7 −6 12

Performing the row reduction algorithm we obtain that

   
1 3 −4 −2 1 3 −4 −2
1 5 2 4  ∼ 0 1 3 3 .
−3 −7 −6 12 0 0 −12 0

Here r = rank(A) = 3 and therefore d = 0, i.e., no free parameters. Peforming back

substitution we obtain that x1 = −11, x2 = 3, and x3 = 0. Thus, the solution to the matrix
equation is unique (no free parameters) and is given by

−11
x= 3 
0

Let’s verify that Ax = b:

      
1 3 −4 −11 −11 + 9 + 0 −2
Ax =  1 5 2   3  = −11 + 15 + 0 =  4  = b
−3 −7 −6 0 33 − 21 12

In other words, b is a linear combination of the columns of A:

       
1 3 −4 −2
−11 1 + 3 5 + 0 2 = 4 
      
−3 7 −6 12

36
 Lecture 4

Example 4.8. Solve, if possible, the matrix equation Ax = b if

   
1 2 3
A= , b= .
2 4 −4
 
Solution. Row reducing the augmented matrix A b we get
   
1 2 3 −2R1 +R2 1 2 3
−−−−−→ .
2 4 −4 0 0 −10

The last row is inconsistent and therefore there is no solution to the matrix equation Ax = b.
In other words, b is not a linear combination of the columns of A.

Example 4.9. Solve, if possible, the matrix equation Ax = b if

   
1 −1 2 2
A= , b= .
0 3 6 −1

Solution. First note that the unknown vector x is in R3 because A has n = 3 columns. The
linear system Ax = b has m = 2 equations and n = 3 unknowns. The coefficient matrix A
has rank r = 2, and therefore
 the solution set will contain d = n − r = 1 parameter. The
augmented matrix A b is
 
  1 −1 2 2
A b = .
0 3 6 −1

Let x3 = t be the parameter and use the last row to solve for x2 :

x2 = − 13 − 2t

Now use the first row to solve for x1 :

x1 = 2 + x2 − 2x3 = 2 + (− 31 − 2t) − 2t = 5
3
− 4t.

Thus, the solution set to the linear system is

x1 = 53 − 4t
x2 = − 13 − 2t
x3 = t

where t is an arbitrary number. Therefore, the matrix equation Ax = b has an infinite

number of solutions and they can all be written as
 5 
3
− 4t
x = − 31 − 2t
t

37
The Matrix Equation Ax = b

where t is an arbitrary number. Equivalently, b can be written as a linear combination of

the columns of A in infinitely many ways. For example, choosing t = −1 gives the particular
solution  
17/3
x = −7/3
−1
and you can verify that  
17/3
A −7/3 = b.
−1

Recall from Definition 3.7 that the span of a set of vectors v1 , v2 , . . . , vp , which we denoted
by span{v1 , v2 , . . . , vp }, is the space of vectors that can be written as a linear combination
of the vectors v1 , v2 , . . . , vp .
Example 4.10. Is the vector b in the span of the vectors v1 , v2 ?
     
0 3 −5
b = 4 , v1 = −2 , v2 = 6 
    
4 1 1

Solution. The vector b is in span{v1 , v2 } if we can find scalars x1 , x2 such that

x1 v1 + x2 v2 = b.

If we let A ∈ R3×2 be the matrix

 
3 −5
A = [v1 v2 ] = −2 6 
1 1
 
x1
then we need to solve the matrix equation Ax = b. Note that here x = ∈ R2 .
x2
Performing row reduction on the augmented matrix [A b] we get that
   
3 −5 0 1 0 2.5
−2 6 4 ∼ 0 1 1.5
1 1 4 0 0 0
Therefore, the linear system is consistent and has solution
 
2.5
x=
1.5

Therefore, b is in span{v1 , v2 }, and b can be written in terms of v1 and v2 as

2.5v1 + 1.5v2 = b

38
 Lecture 4

If v1 , v2 , . . . , vp are vectors in Rn and it happens to be true that span{v1 , v2 , . . . , vp } = Rn

then we would say that the set of vectors {v1 , v2 , . . . , vp } spans all of Rn . From Theorem 4.6,
we have the following.

Theorem 4.11: Let A ∈ Mm×n be a matrix with columns v1 , v2 , . . . , vn , that is, A =

v1 v2 · · · vn . The following are equivalent:
(a) span{v1 , v2 , . . . , vn } = Rm
(b) Every b ∈ Rm can be written as a linear combination of v1 , v2 , . . . , vn .
(c) The matrix equation Ax = b has a solution for any b ∈ Rm .
(d) The rank of A is m.

Example 4.12. Do the vectors v1 , v2 , v3 span R3 ?

     
1 2 −1
v1 = −3 , v2 = −4 , v3 = 2 
    
5 2 3
 
Solution. From Theorem 4.11, the vectors v1 , v2 , v3 span R3 if the matrix A = v1 v2 v3
has rank r = 3 (leading entries in its REF/RREF). The RREF of A is
   
1 2 −1 1 0 0
−3 −4 2  ∼ 0 1 0
5 2 3 0 0 1

which does indeed have r = 3 leading entries. Therefore, regardless of the choice of b ∈ R3 ,
the augmented matrix [A b] will be consistent. Therefore, the vectors v1 , v2 , v3 span R3 :

span{v1 , v2 , v3 } = R3 .

In other words, every vector b ∈ R3 can be written as a linear combination of v1 , v2 , v3 .

After this lecture you should know the following:

• how to multiply a matrix A with a vector x
• that the product Ax is a linear combination of the columns of A
• how to solve the matrix equation Ax = b if A and b are known
• how to determine if a set of vectors {v1 , v2 , . . . , vp } in Rm spans all of Rm
• the relationship between the equation Ax = b, when b can be written  as a linear
combination of the columns of A, and when the augmented matrix A b is consistent
(Theorem 4.6)
• when the columns of a matrix A ∈ Mm×n span all of Rm (Theorem 4.11)
• the basic properties of matrix-vector multiplication Theorem 4.3

39
The Matrix Equation Ax = b

40
 Lecture 5

Lecture 5

Homogeneous and Nonhomogeneous

Systems

5.1 Homogeneous linear systems

We begin with a definition.

Definition 5.1: A linear system of the form Ax = 0 is called a homogeneous linear

system.

A homogeneous system Ax = 0 always has at least one solution, namely, the zero solution
because A0 = 0. A homogeneous system is therefore always consistent. The zero solution
x = 0 is called the trivial solution and any non-zero solution is called a nontrivial
solution. From the existence and uniqueness theorem (Theorem 2.5), we know that a
consistent linear system will have either one solution or infinitely many solutions. Therefore,
a homogeneous linear system has nontrivial solutions if and only if its solution set has at
least one parameter.
Recall that the number of parameters in the solution set is d = n − r, where r is the rank
of the coefficient matrix A and n is the number of unknowns.

Example 5.2. Does the linear homogeneous system have any nontrivial solutions?

3x1 + x2 − 9x3 = 0
x1 + x2 − 5x3 = 0
2x1 + x2 − 7x3 = 0

Solution. The linear system will have a nontrivial solution if the solution set has at least one
free parameter. Form the augmented matrix:
 
3 1 −9 0
1 1 −5 0
2 1 −7 0

41
Homogeneous and Nonhomogeneous Systems

The RREF is:    

3 1 −9 0 1 0 −2 0
1 1 −5 0 ∼ 0 1 −3 0
2 1 −7 0 0 0 0 0
The system is consistent. The rank of the coefficient matrix is r = 2 and thus there will be
d = 3 − 2 = 1 free parameter in the solution set. If we let x3 be the free parameter, say
x3 = t, then from the row equivalent augmented matrix
 
1 0 −2 0
0 1 −3 0
0 0 0 0
we obtain that x2 = 3x3 = 3t and x1 = 2x3 = 2t. Therefore, the general solution of the
linear system is

x1 = 2t
x2 = 3t
x3 = t

The general solution can be written in vector notation as

 
2
x = 3 t

1
 
2
Or more compactly if we let v = 3 then x = vt. Hence, any solution x to the linear
1
 
2
system can be written as a linear combination of the vector v = 3. In other words, the
1
solution set of the linear system is the span of the vector v:

span{v}.

Notice that in the previous example, when solving a homogeneous

  system Ax = 0 using
row reduction, the last column of the augmented matrix A 0 remains unchanged (always
0) after every elementary row operation. Hence, to solve a homogeneous system, we can row
reduce the coefficient matrix A only and then set all rows equal to zero when performing
back substitution.

Example 5.3. Find the general solution of the homogenous system Ax = 0 where
 
1 2 2 1 4
A = 3 7 7 3 13 .
2 5 5 2 9

42
 Lecture 5

Solution. After row reducing we obtain

   
1 2 2 1 4 1 0 0 1 2
A = 3 7 7 3 13 ∼ 0 1 1 0 1
2 5 5 2 9 0 0 0 0 0
Here n = 5, and r = 2, and therefore the number of parameters in the solution set is
d = n − r = 3. The second row of rref(A) gives the equation
x2 + x3 + x5 = 0.
Setting x5 = t1 and x3 = t2 as free parameters we obtain that
x2 = −x3 − x5 = −t2 − t1 .
From the first row we obtain the equation
x1 + x4 + 2x5 = 0
The unknown x5 has already been assigned, so we must now choose either x1 or x4 to be a
parameter. Choosing x4 = t3 we obtain that
x1 = −x4 − 2x5 = −t3 − 2t1
In summary, the general solution can be written as
       
−t3 − 2t1 −2 0 −1
 −t2 − t1  −1 −1 0
       
x=  t2  = t1  0  +t2  1  +t3  0  = t1 v1 + t2 v2 + t3 v3
      
 t3  0 0 1
t1 1 0 0
| {z } | {z } | {z }
v1 v2 v3

where t1 , t2 , t3 are arbitrary parameters. In other words, any solution x is in the span of
v1 , v2 , v3 :
x ∈ span{v1 , v2 , v3 }.

The form of the general solution in Example 5.3 holds in general and is summarized in
the following theorem.

Theorem 5.4: Consider the homogenous linear system Ax = 0, where A ∈ Mm×n and
0 ∈ Rm . Let r be the rank of A.

1. If r = n then the only solution to the system is the trivial solution x = 0.

2. Otherwise, if r < n and we set d = n − r, then there exist vectors v1 , v2 , . . . , vd such

that any solution x of the linear system can be written as

x = t1 v1 + t2 v2 + · · · + tp vd .

43
Homogeneous and Nonhomogeneous Systems

In other words, any solution x is in the span of v1 , v2 , . . . , vd :

x ∈ span{v1 , v2 , . . . , vd }.

A solution x to a homogeneous system written in the form

x = t1 v1 + t2 v2 + · · · + tp vd

is said to be in parametric vector form.

5.2 Nonhomogeneous systems

As we have seen, a homogeneous system Ax = 0 is always consistent. However, if b is non-
zero, then the nonhomogeneous linear system Ax = b may or may not have a solution. A
natural question arises: What is the relationship between the solution set of the homogeneous
system Ax = 0 and that of the nonhomogeneous system Ax = b when it is consistent? To
answer this question, suppose that p is a solution to the nonhomogeneous system Ax = b,
that is, Ap = b. And suppose that v is a solution to the homogeneous system Ax = 0, that
is, Av = 0. Now let q = p + v. Then

Aq = A(p + v)
= Ap + Av
=b+0
= b.

Therefore, Aq = b. In other words, q = p + v is also a solution of Ax = b. We have

therefore proved the following theorem.

Theorem 5.5: Suppose that the linear system Ax = b is consistent and let p be a
solution. Then any other solution q of the system Ax = b can be written in the form
q = p + v, for some vector v that is a solution to the homogeneous system Ax = 0.

Another way of stating Theorem 5.5 is the following: If the linear system Ax = b is consistent
and has solutions p and q, then the vector v = q−p is a solution to the homogeneous system
Ax = 0. The proof is a simple computation:

Av = A(q − p) = Aq − Ap = b − b = 0.

More generally, any solution of Ax = b can be written in the form

q = p + t1 v1 + t2 v2 + · · · + tp vd

where p is one particular solution of Ax = b and the vectors v1 , v2 , . . . , vd span the solution
set of the homogeneous system Ax = 0.

44
 Lecture 5

There is a useful geometric interpretation of the solution set of a general linear system.
We saw in Lecture 3 that we can interpret the span of a set of vectors as a plane containing
the zero vector 0. Now, the general solution of Ax = b can be written as
x = p + t1 v1 + t2 v2 + · · · + tp vd .
Therefore, the solution set of Ax = b is a shift of the span{v1 , v2 , . . . , vd } by the vector p.
This is illustrated in Figure 5.1.

p + span{v}
p + tv
b

p span{v}
b
b
tv
b
v
b

Figure 5.1: The solution sets of a homogeneous and nonhomogeneous system.

Example 5.6. Write the general solution, in parametric vector form, of the linear system
3x1 + x2 − 9x3 = 2
x1 + x2 − 5x3 = 0
2x1 + x2 − 7x3 = 1.
Solution. The RREF of the augmented matrix is:
   
3 1 −9 2 1 0 −2 1
1 1 −5 0 ∼ 0 1 −3 −1
2 1 −7 1 0 0 0 0
The system is consistent and the rank of the coefficient matrix is r = 2. Therefore, there
are d = 3 − 2 = 1 parameters in the solution set. Letting x3 = t be the parameter, from the
second row of the RREF we have
x2 = 3t − 1
And from the first row of the RREF we have
x1 = 2t + 1
Therefore, the general solution of the system in parametric vector form is
     
2t + 1 1 2
x = 3t − 1 = −1 +t 3
    
t 0 1
| {z } |{z}
p v

45
Homogeneous and Nonhomogeneous Systems

You should check that p = (1, −1, 0) solves the linear system Ax = b, and that v = (2, 3, 1)
solves the homogeneous system Ax = 0.

Example 5.7. Write the general solution, in parametric vector form, of the linear system
represented by the augmented matrix
 
3 −3 6 3
−1 1 −2 −1 .
2 −2 4 2

Solution. Write the general solution, in parametric vector form, of the linear system repre-
sented by the augmented matrix
 
3 −3 6 3
−1 1 −2 −1
2 −2 4 2

The RREF of the augmented matrix is

   
3 −3 6 3 1 −1 2 1
−1 1 −2 −1 ∼ 0 0 0 0
2 −2 4 2 0 0 0 0

Here n = 3, r = 1 and therefore the solution set will have d = 2 parameters. Let x3 = t1
and x2 = t2 . Then from the first row we obtain

x1 = 1 + x2 − 2x3 = 1 + t2 − 2t1

The general solution in parametric vector form is therefore

     
1 −2 1
x = 0 +t1  0  +t2 1
0 1 0
|{z} | {z } |{z}
p v1 v2

You should verify that p is a solution to the linear system Ax = b:

Ap = b

And that v1 and v2 are solutions to the homogeneous linear system Ax = 0:

Av1 = Av2 = 0

46
 Lecture 5

5.3 Summary
The material in this lecture is so important that we will summarize the main results. The
solution set of a linear system Ax = b can be written in the form

x = p + t1 v1 + t2 v2 + · · · + td vd

where Ap = b and where each of the vectors v1 , v2 , . . . , vd satisfies Avi = 0. Loosely

speaking,

{Solution set of Ax = b} = p + {Solution set of Ax = 0}

or

{Solution set of Ax = b} = p + span{v1 , v2 , . . . , vd }

where p satisfies Ap = b and Avi = 0.

After this lecture you should know the following:

• what a homogeneous/nonhomogenous linear system is
• when a homogeneous linear system has nontrivial solutions
• how to write the general solution set of a homogeneous system in parametric vector
form Theorem 5.4)
• how to write the solution set of a nonhomogeneous system in parametric vector form
Theorem 5.5)
• the relationship between the solution sets of the nonhomogeneous equation Ax = b
and the homogeneous equation Ax = 0

47
Homogeneous and Nonhomogeneous Systems

48
 Lecture 6

Lecture 6

Linear Independence

6.1 Linear independence

In Lecture 3, we defined the span of a set of vectors {v1 , v2 , . . . , vn } as the collection of all
possible linear combinations
t1 v1 + t2 v2 + · · · + tn vn
and we denoted this set as span{v1 , v2 , . . . , vn }. Thus, if x ∈ span{v1 , v2 , . . . , vn } then by
definition there exists scalars t1 , t2 , . . . , tn such that

x = t1 v1 + t2 v2 + · · · + tn vn .

A natural question that arises is whether or not there are multiple ways to express x as a
linear combination of the vectors v1 , v2 , . . . , vn . For example, if v1 = (1, 2), v2 = (0, 1),
v3 = (−1, −1), and x = (3, −1) then you can verify that x ∈ span{v1 , v2 , v3 } and x can be
written in infinitely many ways using v1 , v2 , v3 . Here are three ways:

x = 3v1 − 7v2 + 0v3

x = −4v1 + 0v2 − 7v3
x = 0v1 − 4v2 − 3v3 .

The fact that x can be written in more than one way in terms of v1 , v2 , v3 suggests that there
might be a redundancy in the set {v1 , v2 , v3 }. In fact, it is not hard to see that v3 = −v1 +v2 ,
and thus v3 ∈ span{v1 , v2 }. The preceding discussion motivates the following definition.

Definition 6.1: A set of vectors {v1 , v2 , . . . , vn } is said to be linearly dependent if

some vj can be written as a linear combination of the other vectors, that is, if

vj ∈ span{v1 , . . . , vj−1 , vj+1, . . . , vn }.

If {v1 , v2 , . . . , vn } is not linearly dependent then we say that {v1 , v2 , . . . , vn } is linearly

independent.

49
Linear Independence

Example 6.2. Consider the vectors

     
1 4 2
v1 = 2 , v2 = 5 , v3 = 1 .
    
3 6 0

Show that they are linearly dependent.

Solution. By inspection, we have
     
2 2 4
2v1 + v3 = 4 + 1 = 5 = v2
    
6 0 6

Thus, v2 ∈ span{v1 , v3 } and therefore {v1 , v2 , v3 } is linearly dependent.

Notice that in the previous example, the equation 2v1 + v3 = v2 is equivalent to

2v1 − v2 + v3 = 0.

Hence, because {v1 , v2 v3 } is a linearly dependent set, it is possible to write the zero vector
0 as a linear combination of {v1 , v2 v3 } where not all the coefficients in the linear
combination are zero. This leads to the following characterization of linear independence.

Theorem 6.3: The set of vectors {v1 , v2 , . . . , vn } is linearly independent if and only if 0
can be written in only one way as a linear combination of {v1 , v2 , . . . , vn }. In other words,
if
t1 v1 + t2 v2 + · · · + tn vn = 0
then necessarily the coefficients t1 , t2 , . . . , tn are all zero.

Proof. If {v1 , v2 , . . . , vn } is linearly independent then every vector x ∈ span{v1 , v2 , . . . , vn }

can be written uniquely as a linear combination of {v1 , v2 , . . . , vn }, and this applies to the
particular case of the zero vector x = 0.
Now assume that 0 can be written uniquely as a linear combination of {v1 , v2 , . . . , vn }.
In other words, assume that if

t1 v1 + t2 v2 + · · · + tn vn = 0

then t1 = t2 = · · · = tn = 0. Now take any x ∈ span{v1 , v2 , . . . , vn } and suppose that there

are two ways to write x in terms of {v1 , v2 , . . . , vn }:

r1 v1 + r2 v2 + · · · + rn vn = x
s1 v1 + s2 v2 + · · · + sn vn = x.

Subtracting the second equation from the first we obtain that

(r1 − s1 )v1 + (r2 − s2 )v2 + · · · + (rn − sn )vn = x − x = 0.

50
 Lecture 6

The above equation is a linear combination of v1 , v2 , . . . , vn resulting in the zero vector 0.

But we are assuming that the only way to write 0 in terms of {v1 , v2 , . . . , vn } is if all the
coefficients are zero. Therefore, we must have r1 − s1 = 0, r2 − s2 = 0, . . . , rn − sn = 0, or
equivalently that r1 = s1 , r2 = s2 , . . . , rn = sn . Therefore, the linear combinations
r1 v1 + r2 v2 + · · · + rn vn = x
s1 v1 + s2 v2 + · · · + sn vn = x
are actually the same. Therefore, each x ∈ span{v1 , v2 , . . . , vn } can be written uniquely in
terms of {v1 , v2 , . . . , vn }, and thus {v1 , v2 , . . . , vn } is a linearly independent set.
Because of Theorem 6.3, an alternative definition of linear independence of a set of vectors
{v1 , v2 , . . . , vn } is that the vector equation
x1 v1 + x2 v2 + · · · + xn vn = 0
has only the trivial solution, i.e., the solution x1 = x2 = · · · = xn = 0. Thus, if {v1 , v2 , . . . , vn }
is linearly dependent, then there exist scalars x1 , x2 , . . . , xn not all zero such that
x1 v1 + x2 v2 + · · · + xn vn = 0.
Hence, if we suppose for instance that xn 6= 0 then we can write vn in terms of the vectors
v1 , . . . , vn−1 as follows:
xn−1
vn = − xxn1 v1 − x2
v
xn 2
−···− xn
vn−1 .
In other words, vn ∈ span{v1 , v2 , . . . , vn−1 }.
According to Theorem 6.3, the set of vectors {v1 , v2 , . . . , vn } is linearly independent if
the equation
x1 v1 + x2 v2 + · · · + xn vn = 0 (6.1)
has only the trivial solution. Now, the vector equation (6.1) is a homogeneous linear system
of equations with coefficient matrix
 
A = v1 v2 · · · vn .
Therefore, the set {v1 , v2 , . . . , vn } is linearly independent if and only if the the homogeneous
system Ax = 0 has only the trivial solution. But the homogeneous system Ax = 0 has only
the trivial solution if there are no free parameters in its solution set. We therefore have the
following.

Theorem 6.4: The set {v1 , v2 , . . . , vn } is linearly independent if and only if the the rank
of A is r = n, that is, if the number of leading entries r in the REF (or RREF) of A is
exactly n.

Example 6.5. Are the vectors below linearly independent?

     
0 1 4
v1 = 1 , v2 = 2 , v3 = −1
5 8 0

51
Linear Independence

Solution. Let A be the matrix

 
  0 1 4
A = v1 v2 v3 = 1 2 −1
5 8 0
Performing elementary row operations we obtain
 
1 2 −1
A ∼ 0 1 4 
0 0 13
Clearly, r = rank(A) = 3, which is equal to the number of vectors n = 3. Therefore,
{v1 , v2 , v3 } is linearly independent.
Example 6.6. Are the vectors below linearly independent?
     
1 4 2
v1 = 2 , v2 = 5 , v3 = 1
3 6 0
Solution. Let A be the matrix
 
  1 4 2
A = v1 v2 v3 = 2 5 1
3 6 0
Performing elementary row operations we obtain
 
1 4 2
A ∼ 0 −3 −3
0 0 0
Clearly, r = rank(A) = 2, which is not equal to the number of vectors, n = 3. Therefore,
{v1 , v2 , v3 } is linearly dependent. We will find a nontrivial linear combination of the vectors
v1 , v2 , v3 that gives the zero vector 0. The REF of A = [v1 v2 v3 ] is
 
1 4 2
A ∼ 0 −3 −3
0 0 0
Since r = 2, the solution set of the linear system Ax = 0 has d = n − r = 1 free parameter.
Using back substitution on the REF above, we find that the general solution of Ax = 0
written in parametric form is  
2
x = t −1
1
The vector  
2
v = −1
1

52
 Lecture 6

spans the solution set of the system Ax = 0. Choosing for instance t = 2 we obtain the
solution    
2 4
x = t −1 = −2 .
  
1 2
Therefore,
4v1 − 2v2 + 2v3 = 0
is a non-trivial linear combination of v1 , v2 , v3 that gives the zero vector 0. And, for instance,

v3 = −2v1 + v2

that is, v3 ∈ span{v1 , v2 }.

Below we record some simple observations on the linear independence of simple sets:

• A set consisting of a single non-zero vector {v1 } is linearly independent. Indeed, if v1

is non-zero then
tv1 = 0
is true if and only if t = 0.

• A set consisting of two non-zero vectors {v1 , v2 } is linearly independent if and only if
neither of the vectors is a multiple of the other. For example, if v2 = tv1 then

tv1 − v2 = 0

is a non-trivial linear combination of v1 , v2 giving the zero vector 0.

• Any set {v1 , v2 , . . . , vp } containing the zero vector, say that vp = 0, is linearly depen-
dent. For example, the linear combination

0v1 + 0v2 + · · · + 0vp−1 + 2vp = 0

is a non-trivial linear combination giving the zero vector 0.

6.2 The maximum size of a linearly independent set

The next theorem puts a constraint on the maximum size of a linearly independent set in
Rn .

Theorem 6.7: Let {v1 , v2 , . . . , vp } be a set of vectors in Rn . If p > n then v1 , v2 , . . . , vp

are linearly dependent. Equivalently, if the vectors v1 , v2 , . . . , vp in Rn are linearly inde-
pendent then p ≤ n.

53
Linear Independence

 
Proof. Let A = v1 v2 · · · vp . Thus, A is a n × p matrix. Since A has n rows, the
maximum rank of A is n, that is r ≤ n. Therefore, the number of free parameters d = p − r
is always positive because p > n ≥ r. Thus, the homogeneous system Ax = 0 has non-trivial
solutions. In other words, there is some non-zero vector x ∈ Rp such that

Ax = x1 v1 + x2 v2 + · · · + xp vp = 0

and therefore {v1 , v2 , . . . , vp } is linearly dependent.

Theorem 6.7 will be used when we discuss the notion of the dimension of a space.
Although we have not discussed the meaning of dimension, the above theorem says that in
n-dimensional space Rn , a set of vectors {v1 , v2 , . . . , vp } consisting of more than n vectors
is automatically linearly dependent.

Example 6.8. Are the vectors below linearly independent?

         
8 4 2 3 0
3  11  0 −9 −2
 0  , v2 = −4 , v3 = 1 , v4 = −5 , v5 = −7 .
v1 =          

−2 6 1 3 7

Solution. The vectors v1 , v2 , v3 , v4 , v5 are in R4 . Therefore,

 by Theorem 6.7,
 the set {v1 , . . . , v5 }
is linearly dependent. To see this explicitly, let A = v1 v2 v3 v4 v5 . Then
 
1 0 0 0 −1
0 1 0 0 1 
A∼ 0 0 1 0 0 


0 0 0 1 −2
One solution to the linear system Ax = 0 is x = (−1, 1, 0, −2, −1) and therefore

(−1)v1 + (1)v2 + (0)v3 + (−2)v4 + (−1)v5 = 0

Example 6.9. Suppose that the set {v1 , v2 , v3 , v4 } is linearly independent. Show that the
set {v1 , v2 , v3 } is also linearly independent.
Solution. We must argue that if there exists scalars x1 , x2 , x3 such that

x1 v1 + x2 v2 + x3 v3 = 0

then necessarily x1 , x2 , x3 are all zero. Suppose then that there exists scalars x1 , x2 , x3 such
that
x1 v1 + x2 v2 + x3 v3 = 0.
Then clearly it holds that
x1 v1 + x2 v2 + x3 v3 + 0v4 = 0.
But the set {v1 , v2 , v3 , v4 } is linearly independent, and therefore, it is necessary that x1 , x2 , x3
are all zero. This proves that v1 , v2 , v3 are also linearly independent.

54
 Lecture 6

The previous example can be generalized as follows: If {v1 , v2 , . . . , vd } is linearly inde-

pendent then any (non-empty) subset of the set {v1 , v2 , . . . , vd } is also linearly independent.

After this lecture you should know the following:

• the definition of linear independence and be able to explain it to a colleague
• how to test if a given set of vectors are linearly independent (Theorem 6.4)
• the relationship between the linear independence of {v1 , v2 , . . . , vp } and the solution
set of the homogeneous system Ax = 0, where A = v1 v2 · · · vp
• that in Rn , any set of vectors consisting of more than n vectors is automatically linearly
dependent (Theorem 6.7)

55
Linear Independence

56