MA1522: Linear Algebra for Computing

Chapter 1: Linear Systems

1.1 Introduction to Linear Systems
Introduction to Linear Systems
Consider the following

x + y = 2
x − y = 0

which is (formerly and fondly) know as

simultaneous equations.

(Only) Solutions: x = 1, y = 1.

Geometrically, the solution is the intersection of 2 lines.

The above is an example of a linear system. In most

applications, a linear system involves a large number of
variables and equations. We will learn what they are and
how to solve them in this chapter.
Definition of Linear Equations

Definition
A linear equation with n variables in standard form has the form

a1 x1 + a2 x2 + · · · + an xn = b.

Here a1 , a2 , ..., an are known constants, called the coefficients, b is called the constant, and x1 , x2 , ..., xn are variables.

The linear equation is homogeneous if b = 0, i.e.

a1 x1 + a2 x2 + · · · + an xn = 0.
Examples

Which of the following equations are linear in x and y ?

1. 2x + y = 3 2. x = 2 3. y = x sin( π6 ) 4. x = ky , for some k ∈ R.

5. xy = 3 6. y = x 2 7. y = sin(x) 8. 3 cos(x) + 4 sin(y ) = 2

Which of the linear equations are in standard form? Which of the linear equations are homogeneous?
Definition of Linear Systems

Definition
A system of linear equations, or a linear system consists of a finite number of linear equations. In general, a linear
system with n variables and m equations in standard form is written as


 a11 x1 + a12 x2 + · · · + a1n xn = b1
 a21 x1 + a22 x2 + · · · + a2n xn = b2

..


 .
am1 x1 + am2 x2 + · · · + amn xn = bm


The linear system is homogeneous if b1 = b2 = ... = bm = 0, that is, all the linear equations are homogeneous.
Solutions to a Linear System

Definition
Given a linear system 

 a11 x1 + a12 x2 + ··· + a1n xn = b1
 a21 x1 + a22 x2 + ··· + a2n xn = b2

..


 .
am1 x1 + am2 x2 + ··· + amn xn = bm


we say that
x1 = c1 , x2 = c2 , ..., xn = cn
is a solution to the linear system if the equations are simultaneously satisfied after making the substitution.
Example

x = 1, y = 1 is the (only) solution to the following linear system.


x − y = 0
x + y = 2

▶ x = 2, y = 2 is a solution to the first equation, but not the second.

▶ x = 2, y = 0 is a solution to the second equation, but not the first.
Question

Give a solution to the following linear system.


x + 2y = 5
2x + 4y = 10

Some possible solutions are

▶ x = 1, y = 2
▶ x = 3, y = 1
▶ x = 4, y = 21
In fact, we may choose y to be any real number, then x = 5 − 2y will be a solution to the system. Alternatively, we
may choose x to be any real number, then y = 5−x2 .
General Solutions

Definition
The general solution to a linear system captures all possible solutions to the linear system.

When there are infinitely many solutions to a linear system, we can assign some variables as parameters, and express
the other variables in terms of the parameters.

The general solution of the example above will be x = 5 − 2s, y = s, s ∈ R. Here the symbol s ∈ R means that s
can be any real number, which we call a parameter of the general solution. Alternatively, the general solution can be
expressed as y = 5−t
2 , x = t, t ∈ R.

Caution
Do not use the variables as parameters. That is, do not write x = 5 − 2x, as it is begging the question.
Inconsistent Linear System

Consider now another linear system. 

x + y = 2
x − y = 0
2x + y = 1


Are you able to provide any solution to the system?

Observe that x = 1 = y is the only solution to the first 2 equations, but not a solution to the third equations.

Hence, the system has not solution. In this case, we say that the system is inconsistent.
Inconsistent Linear Systems

Definition
A linear system is said to be inconsistent if it does not have any solutions. It is consistent otherwise, that is, a linear
system is consistent if it has at least one solution.

We have seen examples of linear system that are inconsistent, consistent with a unique solution, or consistent with
infinitely many solutions. Are there examples of linear systems that have more than 1 but only finitely many
solutions?
Linear Systems with 2 Variables

Consider a linear system with 2 variables 

a1 x + b1 y = c1
a2 x + b2 y = c2
for some real numbers ai , bi , ci , i = 1, 2.

Each equation is a line in the 2-dimensional graph . The solution of this system corresponds to a point of
intersection of the lines, and there are three possibilities.
(i) The lines are parallel and distinct. System has no solution.

(ii) The lines intersect at only one point. System has a unique solution.

(iii) The lines coincide. System has infinitely many solutions.

Head over to the following link to visualize the linear system.

https://www.geogebra.org/m/ahvsz6j9
Example

Go to www.geogebra.org/classic and type in the following equations.


2x + 3y = 1
x −y =3

(i) Is the system consistent?

(ii) How many solutions are there?

The system is consistent with only 1 solution, x = 2, y = −1.

Example

Refresh the page www.geogebra.org/classic and type in the following equations.


x − 2y = 1
3x − 6y = 5

(i) Is the system consistent?

(ii) How many solutions are there?

The system is inconsistent.

Example

Refresh the page www.geogebra.org/classic and type in the following equations.


x − 2y = 1
3x − 6y = 3

(i) Is the system consistent?

(ii) How many solutions are there?

The system is consistent with infinitely many solution. A general solution will be x = 1 + 2s, y = s, s ∈ R.
Linear Systems with 3 Variables
Consider a linear system with 3 variables

a1 x + b1 y + c1 z = d1
a2 x + b2 y + c2 z = d2
a3 x + b3 y + c3 z = d3


for some real numbers ai , bi , ci , di , i = 1, 2, 3.

Each equation is a plane in the 3d-space. First note that 2 planes intersect along a line.
(i) The system has no solution if either 2 planes are parallel but distinct, or the planes intersects, but the lines of
intersections do not intersect.
(ii) The system has a unique solution if all 3 planes intersects at a point.
(iii) The system has infinitely many solutions with 1 parameter in the general solution if the 3 planes intersects along
a line.
(iv) The system has infinitely many solutions with 2 parameters in the general solution if the 3 planes coincide.

Head over to the following link to visualize the linear system.

https://www.geogebra.org/m/wxneuj7k
Example

Go to \color{blue}www.geogebra.org/3d?lang=en. Type in the following equations.


 x +y +z =1
x +y −z =1
x −y +z =1


(i) Is the system consistent?

(ii) If the system is consistent, is the solution unique?

(iii) If the solution is not unique, how many parameters are needed in the general solution?

The system is consistent with a unique solution x = 1, y = z = 0.

Example

Fresh the page \color{blue}www.geogebra.org/3d?lang=en. Type in the following equations.


 x +y +z =1
x −y +z =1
x +z =1


(i) Is the system consistent?

(ii) If the system is consistent, is the solution unique?

(iii) If the solution is not unique, how many parameters are needed?

All 3 planes intersection along a line.

Example

Fresh the page www.geogebra.org/3d?lang=en. Type in the following equations.


 x +y +z =1
x +y +z =2
x +z =1


(i) Is the system consistent?

(ii) If the system is consistent, is the solution unique?

(iii) If the solution is not unique, how many parameters are needed?

The system is inconsistent. Any 2 planes intersects along a line, but the lines of intersections do not intersect.
Discussions

1. Give an example of a linear system with 3 variables such that the general solution has 2 parameters.

2. Is it possible to have a linear system with 3 variables, 3 equations, with the general solution having 3 parameters?

3. In the next section we will see that it is true that a linear system will always only have either no solution, a
unique solution, or infinitely many solutions.
1.2 Solving a Linear System and Row-Echelon Form
Example

Solve the following linear systems.

1. 
 x = 1 x = 1
y = 2 ⇒ y = 2
z = 3 z = 3


2. 
 x + y + 2z = 9 x = 1
7
y − 2z = − 17
2 ⇒ y = 2
z = 3 z = 3


3. 
 x + y + 2z = 9 x = 1
2x + 4y − 3z = 1 ⇒ y = 2
3x + 6y − 5z = 0 z = 3

Augmented Matrix
A linear system 

 a11 x1 + a12 x2 + ··· + a1n xn = b1
 a21 x1 + a22 x2 + ··· + a2n xn = b2

..


 .
am1 x1 + am2 x2 + ··· + amn xn = bm


can be expressed uniquely as an augmented matrix

 
a11 a12 ··· a1n b1
 a21 a22 ··· a2n b2 
.
 
 .. .. .. .. ..
 . . . . . 
am1 am2 ··· amn bm

That is, we first express the linear system in standard form, then put the coefficients in the entries of the left hand
side of the bar in the augmented matrix, and the constant on the right hand side of the bar in the augmented matrix.
Remark
The augmented matrix is an example of a matrix, which we will discuss in further details in Chapter 2.
Example

Linear system: 
 3x + 2y − z = 1
5y + z = 3
x + z = 2


Augmented matrix:  
3 2 −1 1
 0 5 1 3 
1 0 1 2
Example

Linear system: 
2x − 1 = 3y
3 − 9y = 6x
Augmented matrix:
We must first write the system in standard form.

2x − 3y = 1
6x + 9y = 3

Now we can write down the augmented matrix.

 
2 −3 1
6 9 3
Example

Linear system: 
 x1 − 4x2 + ax3 − 6x4 = 2
3x1 + 2x2 = a
6x1 + 2x2 − x3 + (a − 1)x4 = −1


for some a ∈ R.

Augmented matrix:
Here a is a fixed real number. Hence, the a in the first and equation is the coefficient of x3 and the constant,
respectively. Similarly, the a − 1 is the coefficient of x4 in the third equation. Therefore, the augmented matrix is
 
1 −4 a −6 2
 3 2 0 0 a 
6 2 −1 a − 1 −1

which includes the terms involving a.

Row-Echelon Form
Definition
In an (augmented) matrix, a zero row is a row with all entries 0. A row is called a nonzero row otherwise. The first
nonzero entry from the left of a nonzero row is called a leading entry.
An (augmented) matrix is in row-echelon form (REF) if
1. If zero rows exists, they are at the bottom of the matrix.
2. The leading entries are further to the right as we move down the rows.
An augmented matrix in REF has the form
 
∗ ∗
 0 ··· 0 ∗ ∗ 
 
 0 ··· 0 0 ··· 0 ∗ ∗ .


 .. .. .. 
 . . . 
0 ··· ··· 0 0

Definition
In row-echelon form, a column containing a leading entry is called a pivot column. It is called a non-pivot column
otherwise.
Reduced Row-Echelon Form

The (augmented) matrix is in reduced row-echelon form (RREF) if further

3. The leading entries are 1.
4. In each pivot column, all entries except the leading entry is 0.
An augmented matrix in RREF has the form
 
0 ··· 1 ∗ 0 ∗0 ∗ ∗
 0 ··· 0 ··· 0 1 ∗0 ∗ ∗ 
 
 0 ··· 0 ··· 0 0 ··· 01 ∗ ∗ 
.
 
 0 ··· 0 0 0 0 0
 
 .. .. .. .. .. 
 . . . . . 
0 ··· 0 ··· 0 ··· 0 0 0

Remark
Note that a (an augmented) matrix in reduced row-echelon form is also in row-echelon form. The reduced
row-echelon form is a special case of row-echelon form.
Examples

The following are examples of augmented matrices not in row-echelon form.

 
1 0 2 1
1.  0 1 0 2  is not in REF since the leading entry in the third row is in the same column as the leading
0 1 1 3
entry in the second row.
 
0 2 0 −5
2. is not in REF as the leading entry in row 2 is on the left of the leading entry in row 1.
1 0 1 3
 
1 −1 2 0
3.  0 0 0 0  is not in REF since the second row is a zero row while the third is a non zero row.
0 0 1 3
Examples

The following are examples of augmented matrices in row-echelon form but not in reduced row-echelon form.

 
−1 2 3 4
1.  0 1 1 2  is not in RREF as the leading entries in the first and third row are not 1.
0 0 2 3
 
1 −1 1
2. is not in RREF as the third column is a pivot column, but the entry in the first row is a
0 0 1
nonzero non leading entry.
 
1 −1 0
3. is not in RREF as the leading entry in the second row is not 1.
0 0 3
Question

Consider the following augmented matrix  

a b c d
 0 e f 1 
0 g h i
for some real numbers a, b, c, d, e, f , g , h, i. Suppose the augmented matrix is in row-echelon form.

1. What are the possible values of g ?

2. If h = 0, what are the possible values of i?

3. If the augmented matrix is in reduced row-echelon form, and f = −1, what are the possible values of e?
Solutions from REF and RREF

When the augmented matrix is in row-echelon or reduced row-echelon form, it is easy to extract the solutions of the
linear system.

▶ If the augmented matrix is in row-echelon form, we perform back substitution to obtain the solutions.

▶ If the augmented matrix is in reduced row-echelon form, we will read off the solutions directly.
Example

1. The augmented matrix  

1 −1 0 3 −2
0 0 1 2 5
is in reduced row-echelon form. it corresponds to the linear system

x1 − x2 + 3x4 = −2
x3 + 2x4 = 5

First, observe that if we parameterize x4 = t, then the second equation tells us that x3 = 5 − 2t. If we
parameterize x2 = s, then the first equation tells us that x1 = −2 + s − 3t. Hence, the general solution is

x1 = −2 + s − 3t, x2 = s, x3 = 5 − 2t, x4 = t, s, t ∈ R.

Observe that we can obtain the general solution directly by assigning parameters to the variables in the
non-pivot columns in the LHS of the augmented matrix, here x2 = s and x4 = t. Then from row 1, we get
x1 = −2 + s − 3t, and from row 2, we get x3 = 5 − 2t.
Example

2 Consider another example  

1 0 1 0 2
 0 1 1 −1 3 
0 0 0 0 0

This augmented matrix is in reduced row-echelon form. The non-pivot columns in the LHS of the augmented
matrix are the second and fourth. We let x3 = s and x4 = t. Then from the first row, we have x1 = 2 − s, and
from the second, x2 = 3 − s + t. Hence, a general solution is

x1 = 2 − s, x2 = 3 − s + t, x3 = s, x4 = t, s, t ∈ R.
Example

3 Consider the augmented matrix  

1 1 −2 11
 0 1 −1 7 .
0 0 1 −1

It is in row-echelon form only. So, we need to perform back-substitution. Since there are no non-pivot columns
in the LHS of the augmented matrix, we do not need to assign any parameters. We start from the last row; it
tells us that x3 = −1. Then from the second row, we have x2 = 7 + x3 = 7 − 1 = 6. Finally, the first row tells
us that x1 = 11 − x2 + 2x3 = 11 − 6 − 2 = 3. So, the system has a unique solution

x1 = 3, x2 = 6, x3 = −1.
Example

4 Consider the augmented matrix  

0 1 0 0 −1 2
 0 0 1 −1 0 −1 
0 0 0 0 1 4

Note that it is in row-echelon form only. So, we will perform back-substitution. The non-pivot columns in the
LHS of the augmented matrix are columns 1 and 4. Hence, we let x1 = s and x4 = t. From row 3, we get
x5 = 4. From row 2, we have x3 = −1 + x4 = −1 + t. Finally, row 1 tells us that x2 = 2 + x5 = 2 + 4 = 6. So,
the general solution is

x1 = s, x2 = 6, x3 = t − 1, x4 = t, x5 = 4, s, t, ∈ R.
Example

5 Consider the last example  

3 1 4 1
 0 1 −1 1 .
0 0 0 1

Observe that the third row tells us that 0x1 + 0x2 + 0x3 = 1. There exists no solution x1 , x2 , x3 such that
0x1 + 0x2 + 0x3 = 1. Hence, we can conclude that the system is inconsistent. Observe that we get such an
equation if and only if RHS of the augmented column is a pivot column. We will state it as a result.

The linear system is inconsistent if and only if the RHS (last column) of the augmented matrix in row-echelon form is
a pivot column.
Remarks

1. It is easy to obtain the solutions when the augmented matrix is in row-echelon form (by performing
back-substitution) or reduced row-echelon form (reading off the solutions directly).

2. The linear system is inconsistent if and only if the RHS (last column) of the augmented matrix in row-echelon
form is a pivot column.

3. Assign parameters to the variables corresponding to the non-pivot columns in the LHS of the augmented matrix.

4. The number of parameters needed is equal to the number of non-pivot columns in the LHS of the augmented
matrix.

5. We can convert/reduce the augmented matrix of a linear system to a row-echelon form or its reduced
row-echelon form to find the solutions (if exists). This is achieved using elementary row operations.
Challenge

Let R be a n × m matrix in reduced row-echelon form. Which of the following statements are true?
1. The number of pivot columns of R is equal to the number of nonzero rows of R.

2. The number of nonpivot columns of R is equal to the number of zero rows in R.

For each statement that is false, what restrictions can we impose on R such that the statement is true?
1.3 Elementary Row Operations
Motivation

Consider the following linear system 

x + 2y = 3
4x + 5y = 6
To solve it, we perform the following operations

(equation 2) − 4 × (equation 1) → −3y = −6 (equation 3)

− 13 (equation 3) → y = 2 (equation 4)

Substitute equation 4 into equation 1, or

(equation 1) − 2 × (equation 4) → x = −1 (equation 5)

we conclude that x = −1, y = 2 is the (unique) solution to the system.

Observe that in all the operations above, the solution(s) to the system is preserved. This motivated the introduction
of elementary row operations, that is, a fundamental set of operations that do not alternative the solution set when
performed on the augmented matrix of a linear system.
Elementary Row Operations

There are 3 types of elementary row operations.

1. Exchanging 2 rows, Ri ↔ Rj ,

2. Adding a multiple of a row to another, Ri + cRj , c ∈ R,

3. Multiplying a row by a nonzero constant, aRj , a ̸= 0.

Remark
Performing elementary row operations to the augmented matrix of a linear system preserves the solutions.
Examples

1. Exchange row 2 and row 4    

1 1 2 5 1 1 2 5
 0 0 2 4  R2 ↔R4  0 1 4 3 
 −−−−→  
 0 1 1 2   0 1 1 2 
0 1 4 3 0 0 2 4
2. Add 2 times row 3 to row 2    
1 1 2 5 1 1 2 5
 0 0 2 4  R2 +2R3  0 2 4 8 
 − −−−→  
 0 1 1 2   0 1 1 2 
0 1 4 3 0 1 4 3
3. Multiply row 2 by 1/2    
1 1 2 5 1 1 2 5
 0 1
 0 2 4 
 −2−R→
2 
 0 0 1 2 

 0 1 1 2   0 1 1 2 
0 1 4 3 0 1 4 3
Example

Consider the following linear system

  
x + y = 2 1 1 2
, Augmented Matrix: .
x − y = 0 1 −1 0

x = 1, y = 1 is the only solution. It should be clear that if we swap the rows, the solution would remain unchanged.

Now if we take a multiple of row 2, say takes 3 times of row 2, we have

   
1 1 2 3R2 1 1 2
−−→ .
1 −1 0 3 −3 0

The solution is still x = y = 1. Finally, if we take a multiple of row 1 and add it to row 2, say -1 times of row 1 to
row 2, we have    
1 1 2 R2 −R1 1 1 2
−−−−→ .
1 −1 0 0 −2 −2
The solution is still x = y = 1.
Row Equivalent Matrices

Definition
Two (augmented) matrices are row equivalent if one can be obtained from the other by performing a set of
elementary row operations.

Theorem
Two linear systems have the same solutions if their augmented matrices are row equivalent.

The proof of the theorem can be found in the appendix of chapter 2.

Remarks

Recall that when multiplying a row by a constant, aRi , we require the multiple to be nonzero, a ̸= 0.

Consider the following linear system.

x + y = 2
x − y = 0
It has a unique solution x = y = 1. Now suppose we multiple the row two of the augmented matrix by 0,
   
1 1 2 0R2 1 1 2
−−→
1 −1 0 0 0 0

we obtain a system with infinitely many solution; the general solution is x = 2 − s, y = s, s ∈ R.

This demonstrates that if we multiply a row in the augmented matrix by 0, we may change the solution of the linear
system.
Remarks

The order by which we perform the row operations matters (that is row operations do not commute).

Consider the following augmented matrix

 
1 0 0
0 1 0
By taking 2 times of row two, then exchanging the row, we obtain
   
1 0 0 2R1 R2 ↔R1 0 1 0
−−→−−−−→ .
0 1 0 2 0 0
However, if we first exchange the rows then take 2 times of row two, we have
   
1 0 0 R2 ↔R1 2R1 0 2 0
−− −−→−−→ .
0 1 0 1 0 0
Remarks

Hence, when performing a few row operations on an augmented matrix, we have to write the row operations from
left to right.

However, if the row operations commute, that is, in some cases where it does not matter which row operations we
perform first, we may stack them together as such
   
1 0 0 2R2 2 0 0
−−→
0 1 0 2R1 0 2 0
Remarks

The notation Ri + cRj acts on Ri , but keeps Rj unchanged.

For example    
1 0 0 R1 +R2 1 1 0
−−−−→ ,
0 1 0 0 1 0
whereas    
1 0 0 R +R 1 0 0
−−2−−→
1
.
0 1 0 1 1 0
Remarks

Here is another example. R1 + 2R2 is an elementary row operation,

   
1 0 0 R1 +2R2 1 2 0
−−−−→ ,
0 1 0 0 1 0

which is different from the row operation 2R2 + R1

   
1 0 0 2R2 +R1 1 0 0
−−− −→ .
0 1 0 1 2 0

In fact, 2R2 + R1 is not an elementary row operation, but a combination of 2 elementary row operations, first
perform 2R2 , then perform R2 + R1 ;
   
1 0 0 2R2 R2 +R1 1 0 0
−−→−−−−→ .
0 1 0 1 2 0
Discussion

Consider the following elementary row operations.

     
1 0 0 1 1 0 0 1 1 0 0 1
R2 +3R1 ??
 0 1 0 2 − −−−→  3 1 0 5  −→  0 1 0 2 
0 0 1 3 0 0 1 3 0 0 1 3

What is the (unique) elementary row operation that when performed, would change the middle augmented matrix
back to its original one?

Since the second row in the augmented matrix in the middle had an extra 3 times of row 1, to undo it, we need to
remove the extra 3 times of row 1. That is, if we perform R2 − 3R1 , we will get the original augmented matrix. This
shows that the elementary row operation Ri + cRj can be undone, or “reversed” if we perform Ri − cRj . This is true
for all elementary row operations.
Reverse of Elementary Row Operations

Every elementary row operation has a reverse elementary row operation. The reverse of the row operations are given
as such.
1. The reverse of exchanging 2 rows, Ri ↔ Rj , is itself.

2. The reverse of adding a multiple of a row to another, Ri + cRj is subtracting the multiple of that row, Ri − cRj .

3. The reverse of multiplying a row by a nonzero constant, aRj is the multiplication of the reciprocal of the
constant, 1a Rj .

Readers should convinced themselves that the reverse of the elementary row operations are indeed the ones given
above. In fact, this is the definitive property of an elementary row operation; all (linear) operations that can be
reversed are composed (made up of a series) of elementary row operations.
Examples
1. The reverse of R2 ↔ R4 is itself,
     
1 0 0 0 1 1 0 0 0 1 1 0 0 0 1
 0 1 0 0 2  R2 ↔R4  0 0 0 1 4  R2 ↔R4  0 1 0 0 2 
 −−−−→  −−−−→  .
 0 0 1 0 3   0 0 1 0 3   0 0 1 0 3 
0 0 0 1 4 0 1 0 0 2 0 0 0 1 4

2. The reverse of R2 + 3R3 is R2 − 3R3

     
1 0 0 0 1 1 0 0 0 1 1 0 0 0 1
 0 1 0 0 2  R2 +3R3  0 1 3 0 11  R2 −3R3  0 1 0 0 2 
 0 0 1 0 3  −−−−→ 
   −−−−−→  .
0 0 1 0 3   0 0 1 0 3 
0 0 0 1 4 0 0 0 1 4 0 0 0 1 4
1
3. The reverse of 2R3 is 2 R3
     
1 0 0 0 1 1 0 0 0 1 1 0 0 0 1
 0 1
1 0 0 2 
−2R3  0 1 0 0 2 
 −2−R→
3  0 1 0 0 2 

 0 −→   .
0 1 0 3   0 0 2 0 6   0 0 1 0 3 
0 0 0 1 4 0 0 0 1 4 0 0 0 1 4
Question

What is the reverse of the elementary row operation R2 − 12 R1 ?

Remark

▶ Since performing elementary row operations do not change the solution set of a linear system, and

▶ the solutions of a linear system can be obtained easily from the augmented matrix in row-echelon form,

▶ our aim is therefore to use elementary row operations to reduce the augmented matrix of a linear system until it
is in row-echelon or even better, in reduced row-echelon form.

▶ This will be the discussion of the next section.

1.4 Row Reduction, Gaussian and Gauss-Jordan Elimination
Introduction

▶ Aim: to reduce an augmented matrix to a row-echelon, or reduced row-echelon form using elementary row
operations.

▶ Gaussian or Gauss-Jordan elimination are algorithms that do so.

▶ These algorithm guarantees to reduce the augmented matrix, but might not be the most efficient.

▶ May have to modify the algorithms when the augmented matrix contains unknown coefficients.
Example
Solve the following linear system 
x + y + 2z = 4
−x + 2y − z = 1
2x + 3z = −2

 
1 1 2 4
The augmented matrix of the linear system is  −1 2 −1 1 .
2 0 3 −2

The aim is to reduce it to either in row-echelon, or reduced row-echelon form. Recall that in the row-echelon form,
the leading entries are further to the right as we move down the rows. This would mean that in the first column, the
entries in the second and third row must be 0. So, we use the first entry in the first row to eliminate the first entry in
the second and third row.
   
1 1 2 4 1 1 2 4
R2 +R1
 −1 2 −1 1  −−−−−→  0 3 1 5 .
R3 −2R1
2 0 3 −2 0 −2 −1 −10

Note that since these 2 operations commute, we may stack them.

Example

Next, we need the leading entry in the third row to be in the third column.
   
1 1 2 4 2
R3 + R2
1 1 2 4
 0 3 1 5  −−−−3−→  0 3 1 5 .
0 −2 −1 −10 0 0 − 13 − 20
3

The augmented matrix is now in row-echelon form, and one can perform back substitution to obtain the solution.
5 − 20
z = 20, y= = −5, x = 4 − (−5) − 2(20) = −31.
3
Alternatively, we may continue to reduce the augmented matrix to its reduced row-echelon form. To proceed, we
need the leading entry of the third row to be 1.
   
1 1 2 4 1 1 2 4
−3R3
 0 3 1 5  −−−→  0 3 1 5 .
0 0 − 13 − 203
0 0 1 20
Example

Next, we need the entries above the leading entry in the third row to be 0. And finally, we need the leading entry in
the second row to be 1, and the entry above it to be 0.
       
1 1 2 4 1 1 2 4 1
R
1 1 2 4 1 0 0 −31
R2 −R3 2 R1 −R2
 0 3 1 5  −−−−→  0 3 0 −15  −3−→  0 1 0 −5  −−−−−→  0 1 0 −5  .
R1 −2R3
0 0 1 20 0 0 1 20 0 0 1 20 0 0 1 20

The augmented matrix is now in reduced row-echelon form and we arrive at the conclusion that the unique solution
to the linear system is
x = −31, y = −5, z = 20,
which is consistent to the result obtained via back-substitution.
Example

In summary, these are the steps we performed to reduce the augmented matrix to its RREF.
     
1 1 2 4 1 1 2 4 R + 2
R
1 1 2 4
R2 +R1 3 2 −3R3
 −1 2 −1 1  −− −−−→  0 3 1 5  −−−−3−→  0 3 1 5  −−−→
R3 −2R1
2 0 3 −2 0 −2 −1 −10 0 0 − 13 −320

       
1 1 2 4 1 1 2 4 1
R
1 1 2 4 1 0 0 −31
R2 −R3 2 R1 −R2
 0 3 1 5 − −−−→  0 3 0 −15  −3−→  0 1 0 −5  −−− −−→  0 1 0 −5 
R1 −2R3
0 0 1 20 0 0 1 20 0 0 1 20 0 0 1 20
Gaussian Elimination

Step 1: Locate the leftmost column that does not consist entirely of zeros.
Step 2: Interchange the top row with another row, if necessary, to bring a nonzero entry to the top of the column found
in Step 1.

Example
   
0 3 9 ··· ∗ 1 2 −3 · · · ∗
R1 ↔R2
(i)  1 2 −3 ··· ∗  −− −−→  0 3 9 · · · ∗ 
4 1 0 ··· ∗ 4 3 0 ··· ∗
   
0 0 5 ··· ∗ 0 1 2 ··· ∗
R1 ↔R2
(ii)  0 1 2 ··· ∗  −− −−→  0 0 5 · · · ∗ 
0 −3 2 ··· ∗ 0 −3 2 · · · ∗
 
1 0 2 ··· ∗
(iii)  0 5 −1 ··· ∗  no interchanging of rows needed.
0 4 1 ··· ∗
Gaussian Elimination

Step 3: For each row below the top row, add a suitable multiple of the top row to it so that the entry below the leading
entry of the top row becomes zero.
   
1 2 −3 · · · ∗ 1 2 −3 · · · ∗
R −4R1
(i)  0 3 9 · · · ∗  −−3−−−→  0 3 9 ··· ∗ 
4 1 0 ··· ∗ 0 −7 12 · · · ∗
   
0 1 2 ··· ∗ 0 1 2 ··· ∗
R3 +3R1
(ii)  0 0 5 · · · ∗  −− −−→  0 0 5 · · · ∗ 
0 −3 2 · · · ∗ 0 0 8 ··· ∗
 
1 0 2 ··· ∗
(iii)  0 5 −1 · · · ∗  no change required.
0 4 1 ··· ∗
Gaussian Elimination

Step 4: Now cover the top row in the augmented matrix and begin again with Step 1 applied to the submatrix that
remains. Continue this way until the entire matrix is in row-echelon form.
   
1 2 −3 · · · ∗ R3 + 37 R2
1 2 −3 · · · ∗
(i)  0 3 9 · · · ∗  −−−−−→  0 3 9 · · · ∗ 
0 −7 12 · · · ∗ 0 0 33 · · · ∗
   
0 1 2 ··· ∗ R3 − 85 R2
0 1 2 ··· ∗
(ii)  0 0 5 · · · ∗  −−−−−→  0 0 5 · · · ∗ 
0 0 8 ··· ∗ 0 0 0 ··· ∗
   
1 0 2 ··· ∗ R3 − 45 R2
1 0 2 ··· ∗
(iii)  0 5 −1 · · · ∗  −−−−−→  0 5 −1 · · · ∗ 
0 4 1 ··· ∗ 0 0 9/5 · · · ∗

The result of step 1 to 4 reduces to (augmented) matrix to a row-echelon form. The process up to step 4 is called
Gaussian Elimination.
Gauss-Jordan Elimination

Step 5: Multiply a suitable constant to each row so that all the leading entries become 1.
Step 6: Beginning with the last nonzero row and working upward, add suitable multiples of each row to the rows above
to introduce zeros above the leading entries.
 
1 ∗ 0 ∗ 0 ∗
 0 ··· 0 1 ∗ 0 ∗ 
 
 0 ··· 0 0 ··· 0 1 ∗ 
.
 
 0 0 0 0 0 
 
 .. .. .. .. .. 
 . . . . . 
0 ··· 0 ··· 0 0 0

If we continue from step 1 to 6, then the entire process is known as Gauss-Jordan elimination.
Question

Can we still perform Gaussian and/or Gauss-Jordan elimination if the coefficients contains unknown? For example,

 ax + y − z = 1
x + y + 2z = 3
x + (a − 1)y − z = 0


for some constant a ∈ R?

Challenge

1. Is it possible to reduce an augmented matrix to 2 different row-echelon forms?

2. Is it possible to reduce an augmented matrix to 2 different reduced row-echelon form?

1.5 More on Linear Systems
Summary

1. Write the linear system in its standard form.

2. Form the augmented matrix of the linear system.

3. Reduce the augmented matrix to either a row-echelon form or reduced row echelon form. May use
Gaussian/Gauss-Jordan elimination.

4. Decide if the system is consistent

▶ If the last column is a pivot column, the system is inconsistent.
▶ Otherwise, the system is consistent, assign the variables corresponding to the nonpivot columns to be
parameters, s, t, s, t ∈ R, etc.

5. If the system is in reduced row-echelon form, read off the solutions directly.

6. If the system is in row-echelon form only, do back substitution, starting from the lowest nonzero row.

7. Write down the (general) solution to the system.

More Examples
Solve the following linear system

x1 + 3x2 − 2x3 + + 2x5 = 0

2x1 + 6x2 − 5x3 − 2x4 + 4x5 − 3x6 = −1
5x3 + 10x4 + 15x6 = 5
2x1 + 6x2 + 8x4 + 4x5 + 18x6 = 6
   
1 3 −2 0 2 0 0 1 3 −2 0 2 0 0
 2 6
 −5 −2 4 −3 −1  R 2 −2R 1 R4
 −−−−−→−−−−−→ −2R 1
 0 0 −1 −2 0 −3 −1 

 0 0 5 10 0 15 5   0 0 5 10 0 15 5 
2 6 0 8 4 18 6 0 0 4 8 0 18 6
   
1 3 −2 0 2 0 0 1 3 −2 0 2 0 0
R3 +5R2 R4 +4R2  0 0 −1 −2 0 −3 −1  R3 ↔R4  0 0 −1 −2 0 −3 −1 
−−−−→−−−−→   0 0
− −−−→  
0 0 0 0 0   0 0 0 0 0 6 2 
0 0 0 0 0 6 2 0 0 0 0 0 0 0
The augmented matrix is now in row-echelon form. One may choose to use back-substitution to obtain the answer.
Which variables should we assign as parameters?
More Examples

Let us continue to reduce the system to its reduced row-echelon form.

   
1 3 −2 0 2 0 0 1 3 −2 0 2 0 0
−R2 (1/6)R3  0 0 1 2 0 3 1  R2 −3R3 0 0 1 2 0 0 0 
−−→−−−−→   0 0 0 0 0 1
− −−−−→  
1/3  0 0 0 0 0 1 1/3 
0 0 0 0 0 0 0 0 0 0 0 0 0 0
 
1 3 0 4 2 0 0
R1 +2R2  0 0 1 2 0 0 0 
−−−−→   0

0 0 0 0 1 1/3 
0 0 0 0 0 0 0

Let x2 = r , x4 = s, x5 = t. Then the general solution is

x1 = −3r − 4s − 2t, x2 = r , x3 = −2s, x4 = s, x5 = t, x6 = 1/3, r , s, t ∈ R.

Linear System with Unknowns in the Coefficient and Constants

Consider the following linear system

x1 + ax2 + 2x3 = 0
x1 + x3 = 1
x1 + ax3 = 2
for some fixed real number a ∈ R. Here we want to determine the value of a such that the system is
(a) inconsistent,
(b) consistent with a unique solution and find the unique solution, or
(c) consistent with infinitely many solutions and write down the general solution.
     
1 a 2 0 1 a 2 0 1 a 2 0
R2 −R1 R3 −R1 R3 −R2
 1 0 1 1  −−−−→−−−−→  0 −a −1 1  −−−−→  0 −a −1 1 .
1 0 a 2 0 −a a−2 2 0 0 a−1 1
Observe that if we want to continue the Gauss-Jordan elimination, we would need to multiple row 2 by − 1a and row
1
3 by a−1 . But this is not well defined if a = 0 and a = 1, respectively. Hence, we need to consider if a = 0, 1.
Linear System with Unknowns in the Coefficient and Constants

If a = 1, the augmented matrix becomes  

1 1 2 0
 0 −1 −1 1 .
0 0 0 1
The system is inconsistent.

If a = 0, the augmented matrix becomes

   
1 0 2 0 1 0 0 2
R −R2 R1 +2R2
 0 0 −1 1  −−3−−→ −−−−→  0 0 −1 1 .
0 0 −1 1 0 0 0 0
The system is consistent with infinitely many solution. The general solution is

x1 = 2, x2 = s, x3 = −1, s ∈ R.
Linear System with Unknowns in the Coefficient and Constants

Finally, suppose a ̸= 0, 1. By back substitution or reducing the augmented matrix to its reduced row-echelon form,
we obtain the unique solution
a−2 1 1
x1 = , x2 = , x3 = .
a−1 1−a a−1
The details are left as an exercise. Note that there are infinitely many values of a such that the system has a unique
solution, and the solution depends on these values of a. Do not pick a particular value of a for the solution.

(a) The system has no solution if a = 1.

(b) The system has a unique solution when a ̸= 0, 1. The unique solution is

a−2 1 1
x1 = , x2 = , x3 = .
a−1 1−a a−1
(c) The system has infinitely many solutions when a = 0. The general solution is

x1 = 2, x2 = s, x3 = −1, s ∈ R.
Another Example

Consider the following linear system


 ax + 2ay − z = 2a + 2
x + y = 1
x + y + (a − 1)z = 3


for some fixed real number a ∈ R.

 
a 2a −1 2a + 2
The augmented matrix of the system is 1 1 0 1 . Here, if we follow Gaussian elimination strictly,
1 1 a−1 3
one might consider the operation R2 − 1a R1 . However, this is not well-defined if a = 0. Hence, instead of considering
cases, we might consider doing a row swap.
     
a 2a −1 2a + 2 1 1 0 1 1 1 0 1
R2 ↔R1 R2 −aR1
 1 1 0 1 − −−−→  a 2a −1 2a + 2  −− −−→  0 a −1 a+2 
R3 −R1
1 1 a−1 3 1 1 a−1 3 0 0 a−1 2
Another Example

We should consider whether a = 0, 1 or not.

(a) The system is inconsistent if a = 1.

(b) The system has a unique solution if a ̸= 0, 1. The unique solution is

2 a+1 2
x= , y= , z= .
1−a a−1 a−1

(c) The system has infinitely many solutions with 1 parameter if a = 0. The general solution is

x = 1 − s, y = s, z = −2, s ∈ R.

The details are left as an exercise.

Hard Example

Consider the following linear system



 x1 + 3x3 + x4 = 2
3x1 + ax2 + 9x3 = 6


 2x1 + (a + 6)x3 + ax4 = b+2
2x1 + 6x3 + bx4 = b+2


where a and b are some constants.

(a) Find the conditions on a and b such that the system has no solution.
(b) Find the conditions on a and b such that the system has a unique solution, and write down the unique solution.
(c) Find the conditions on a and b such that the system has infinitely many solutions with 1 parameter, and write
down a general solution.
(d) Find the conditions on a and b such that the system has infinitely many solutions with 2 parameter, and write
down a general solution.
Hard Example
   
1 0 3 1 2 1 0 3 1 2
 3 a 9 0 6  −3R1
 −−R−2−
 0 a 0 −3 0 
 −−−→  .
 2 0 a+6 a b + 2  R3 − 2R1  0 0 a a−2 b−2 
2 0 6 b b+2 R4 − 2R1 0 0 0 b−2 b−2
The cases to consider are a = 0 or not and b = 2 or not; a total of 4 cases.

Suppose b = 2. The augmented matrix becomes

 
1 0 3 1 2
 0
 a 0 −3 0 

 0 0 a a−2 0 
0 0 0 0 0
If a = 0 then columns 2 and 3 are non-pivots. Hence, the system has infinitely many solutions with 2 parameters.The
general solution is
x1 = 2 − 3t, x2 = s, x3 = t, x4 = 0, s, t, ∈ R.
If a ̸= 0, then column 4 is non-pivot, and hence the system has infinitely many solutions with 1 parameters. The
general solution is
2a − 6 3 (2 − a)
x1 = 2 + s, x2 = s, x3 = s, x4 = s, s ∈ R.
a a a
Hard Example

Suppose b ̸= 2. We may proceed to reduce the augmented matrix.

     
1 0 3 1 2 1 0 3 1 2 1 0 3 1 2
1
 0 a 0 −3 0  b−2 R4
 −−
 0 a 0 −3 0  R3 −(a−2)R4  0 a 0 0 3 
 − −→  − −−−−−−→  
 0 0 a a−2 b−2   0 0 a a−2 b − 2  R2 +3R4  0 0 a 0 b−a 
0 0 0 b−2 b−2 0 0 0 1 1 0 0 0 1 1

From row 2, we can see that the system is inconsistent if a = 0.

Otherwise, if a ̸= 0, all 4 columns on the LHS will be pivot columns, and hence, the system has a unique solution
4a − 3b 3 b−a
x1 = , x2 = , x3 = , x4 = 1.
a a a
Hard Example

(a) The system has no solution if a = 0 and b ̸= 2.

(b) The system has a unique solution if a ̸= 0 and b ̸= 2. The unique solution is

4a − 3b 3 b−a
x1 = , x2 = , x3 = , x4 = 1.
a a a
(c) The system has infinitely many solutions with 1 parameter if a ̸= 0 and b = 2. The general solution is

2a − 6 3 (2 − a)
x1 = 2 + s, x2 = s, x3 = s, x4 = s, s ∈ R.
a a a
(d) The system has infinitely many solutions with 2 parameters if a = 0 and b = 2. The general solution is

x1 = 2 − 3t, x2 = s, x3 = t, x4 = 0, s, t, ∈ R.
Constructing a Linear System from the General Solution
Find a linear system with 2 equations such that

x1 = 1 − 2s + t, x2 = s, x3 = t, x4 = 0, s, t ∈ R

is the general solution.

We will substitute x2 = s and x3 = t into the first equation and get

x1 = 1 − 2x2 + x3 .

Include the equation x4 = 0, and changing it to a standard form, we obtain the linear system

x1 + 2x2 − x3 = 1
.
x4 = 0

The augmented matrix of the system is  

1 2 −1 0 1
0 0 0 1 0
which is in reduced row-echelon form.
Another Example
Construct an augmented matrix with 3 variables and 4 equations such that it has the following general solution

x = 3 − 5s, y = 2 + 2s, z = s, s ∈ R.

Substituting z = s, into the first 2 equations, we get


x + 5z = 3
x = 3 − 5z and y = 2 + 2z ⇒
y − 2z = 2

However, we need 4 equations. Since we have used all the given information, the other 2 more equations should not
contribute anymore information, that is, they should be derived from the first 2. We may consider taking multiples or
adding the first 2 equations to obtain more equations. For example.
  

 x + 5z = 3 1 0 5 3
y − 2z = 2  0 1 −2 2 

which corresponds to the augmented matrix  

 x + y + 3z = 5  1 1 3 5 
2y − 4z = 4 0 2 −4 4

Question

Construct an augmented matrix with 3 variables and 3 equations such that it has the following solution

x = 3, y = 2, z = 1.