ES 221e – Numerical Solution to CE Problem 1

3 Solving System of Linear Equations

OVERVIEW
This module will discuss methods in solving system of linear equations. The direct methods
such as Gaussian elimination method and LU decomposition and iterative methods such as Jacobi
and Gauss-Seidel Method will be the major contents in these module. Sample problems and
exercises will be cited for clear understanding of this topic.

LEARNING OBJECTIVES
At the end of this module, you are expected to:
1. Know how to use the different methods in solving more complex linear equations.
2. Use the knowledge in matrix analysis to perform the different methods in solving linear
equations.
3. Solve linear equations using direct methods and iterative methods.
4. Write the augmented matrix of a system of equations.
5. Perform row operations on a matrix.
6. Solve a system of linear equations using matrices.

KEY TERMS

Augmented Matrix
Direct Methods
Gaussian Elimination
Gauss-Seidel Method
Iterative Methods
Jacobi Method
Linear Equations
LU Decomposition
Non homogenous system
Row Echelon Form

3.1. Unique and Multiple Solutions, Trivial and Non-trivial Solutions, No Solution
For square systems of equations (i.e. those with an equal number of equations and
unknowns), the most powerful tool for determining the number of solutions the system has is
the determinant. Suppose we have two equations and two unknowns: ax+by=c and dx+ey=f with
b and e non-zero (i.e. the system is nonhomogeneous). These are two lines with slope -a/b and -
d/e, respectively. Let's define the determinant of a 2x2 system of linear equations to be the
determinant of the matrix of coefficients A of the system. For this system

Suppose this determinant is zero. Then, this last equation implies a/b=d/e; in other words,
the slopes are equal. From our discussion above, this means the lines are either identical (there is
an infinite number of solutions) or parallel (there are no solutions). If the determinant is non-zero,

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 2

then the slopes must be different and the lines must intersect in exactly one point. This leads us to
the following result:
A nxn nonhomogeneous system of linear equations has a unique non-trivial solution if and
only if its determinant is non-zero. If this determinant is zero, then the system has either no
nontrivial solutions or an infinite number of solutions.

Example
The nonhomogeneous system of equations 2x+3y=-8 and -x+5y=1 has determinant which
implies the system has a unique solution (the point (-43/13,-28/75)). This corresponds to our
intuition: the lines have different slopes and therefore must intersect in exactly one point.
Example
The nonhomogeneous system of equations x-4y+6z=3, -2x+8y-12z=-6, and 2x-y+3z=1
does not have a unique solution since its determinant is

In fact, this system has an infinite number of solutions. To see this, think geometrically. The
three equations represent 3 planes. Two of the equations (the first and the second) represent the
same plane (why?); so, there are only two distinct non-parallel planes. Since two non-parallel
planes intersect in a line, there are an infinite number of points which lie on all three of these planes
(i.e. the system has an infinite number of solutions). To actually find the solutions, we must use a
method such as Gaussian elimination to solve the system of equations.

Determining the Number of Solutions of a Homogeneous System of Equations

For a homogeneous system of equations ax+by=0 and cx+dy=0, the situation is slightly
different. These lines pass through the origin. Thus, there is always at least one solution, the point
(0,0). If the slopes -a/b and -c/d are equal then there are an infinite number of solutions since the
lines are identical. But as we have seen, the slopes of these lines are equal when the determinant
of the coefficient matrix is zero. Thus, for homogeneous systems we have the following result:
A nxn homogeneous system of linear equations has a unique solution (the trivial solution)
if and only if its determinant is non-zero. If this determinant is zero, then the system has an infinite
number of solutions.
Example
The system of equations 2x+y=0 and x-y=0 has exactly one solution since the slopes of
the lines are different, i.e. the determinant

is non-zero. The solution is of course (0,0).

Numerical methods for solving linear equations can generally be divided into two classes (direct
and iterative method).

3.2. Direct Methods

Direct methods compute the solution to a problem in a finite number of steps. The
basic direct method for solving linear systems of equations is Gaussian elimination. The bulk of

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 3

the algorithm involves only the matrix A and amounts to its decomposition into a product of two
matrices that have a simpler form. This is called an LU decomposition.

3.2.1. Gauss Elimination Method

Writing the Augmented Matrix of a System of Equations
A matrix can serve as a device for representing and solving a system of equations. To
express a system in matrix form, we extract the coefficients of the variables and the constants, and
these become the entries of the matrix. We use a vertical line to separate the coefficient entries
from the constants, essentially replacing the equal signs. When a system is written in this form, we
call it an augmented matrix.
For example, consider the following 2×22×2 system of equations.

We can write this system as an augmented matrix:

We can also write a matrix containing just the coefficients. This is called the coefficient matrix.

A three-by-three system of equations such as

has a coefficient matrix

and is represented by the augmented matrix

Notice that the matrix is written so that the variables line up in their own columns: x-terms
go in the first column, y-terms in the second column, and z-terms in the third column. It is very
important that each equation is written in standard form ax+by+cz=dax+by+cz=d so that the
variables line up. When there is a missing variable term in an equation, the coefficient is 0.

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 4

EXAMPLE 1: Write the augmented matrix for the given system of equations.

Solution:
The augmented matrix displays the coefficients of the variables, and an additional
column for the constants.

Performing Row Operations on a Matrix

Now that we can write systems of equations in augmented matrix form, we will examine the
various row operations that can be performed on a matrix, such as addition, multiplication by a
constant, and interchanging rows.
Performing row operations on a matrix is the method we use for solving a system of
equations. In order to solve the system of equations, we want to convert the matrix to row-echelon
form, in which there are ones down the main diagonal from the upper left corner to the lower right
corner, and zeros in every position below the main diagonal as shown.

We use row operations corresponding to equation operations to obtain a new matrix that is
row-equivalent in a simpler form. Here are the guidelines to obtaining row-echelon form.
(1) In any nonzero row, the first nonzero number is a 1. It is called a leading 1.
(2) Any all-zero rows are placed at the bottom on the matrix.
(3) Any leading 1 is below and to the right of a previous leading 1.
(4) Any column containing a leading 1 has zeros in all other positions in the column.

To solve a system of equations we can perform the following row operations to convert the
coefficient matrix to row-echelon form and do back-substitution to find the solution.
(1) Interchange rows. (Notation: Ri ↔ Rj )
(2) Multiply a row by a constant. (Notation: cRi )
(3) Add the product of a row multiplied by a constant to another row. (Notation: Ri + cRj )

Each of the row operations corresponds to the operations we have already learned to solve
systems of equations in three variables. With these operations, there are some key moves that will
quickly achieve the goal of writing a matrix in row-echelon form. To obtain a matrix in row-echelon
form for finding solutions, we use Gaussian elimination, a method that uses row operations to
obtain a 1 as the first entry so that row 1 can be used to convert the remaining rows.

The Gaussian elimination method refers to a strategy used to obtain the row-echelon form
of a matrix. The goal is to write matrix A with the number 1 as the entry down the main diagonal
and have all zeros below.

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 5

The first step of the Gaussian strategy includes obtaining a 1 as the first entry, so that row
1 may be used to alter the rows below.

How to perform row operations to achieve row-echelon form with the given augmented matrix?
(1) The first equation should have a leading coefficient of 1 . Interchange rows or multiply by a
constant, if necessary.
(2) Use row operations to obtain zeros down the first column below the first entry of 1 .
(3) Use row operations to obtain a 1 in row 2, column 2.
(4) Use row operations to obtain zeros down column 2, below the entry of 1.
(5) Use row operations to obtain a 1 in row 3, column 3.
(6) Continue this process for all rows until there is a \(1 in every entry down the main diagonal and
there are only zeros below.
(7) If any rows contain all zeros, place them at the bottom.

EXAMPLE 2: Solve the given system by Gaussian elimination.

Solution:
First, we write this as an augmented matrix.

We want a 1 in row 1, column 1. This can be accomplished by interchanging row 1 and row 2.

We now have a 1 as the first entry in row 1, column 1. Now let’s obtain a 0 in row 2, column 1.
This can be accomplished by multiplying row 1 by −2 , and then adding the result to row 2.

We only have one more step, to multiply row 2 by 1/5 .

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 6

Use back-substitution. The second row of the matrix represents y=1 . Back-substitute y=1 into
the first equation.

The solution is the point (3/2,1) .

EXAMPLE 3: Use Gaussian elimination to solve the given 2×2 system of equations.

Solution
Write the system as an augmented matrix.

Obtain a 1 in row 1, column 1. This can be accomplished by multiplying the first row by 1/2 .

Next, we want a 0 in row 2, column 1. Multiply row 1 by −4 and add row 1 to row 2.

The second row represents the equation 0=4 . Therefore, the system is inconsistent and has no
solution.

EXAMPLE 4: Solve the system of equations.

Solution
Perform row operations on the augmented matrix to try and achieve row-echelon form.

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 7

The matrix ends up with all zeros in the last row: 0y=0 . Thus, there are an infinite number of
solutions and the system is classified as dependent. To find the generic solution, return to one of
the original equations and solve for y .

So the solution to this system is (x, 3 − (3/4)x) .

EXAMPLE 5:
Perform row operations on the given matrix to obtain row-echelon form.

Solution
The first row already has a 1 in row 1, column 1. The next step is to multiply row 1 by −2 and
add it to row 2. Then replace row 2 with the result.

Next, obtain a zero in row 3, column 1.

Next, obtain a zero in row 3, column 2.

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 8

The last step is to obtain a 1 in row 3, column 3.

Solving a System of Linear Equations Using Matrices

We have seen how to write a system of equations with an augmented matrix, and then how
to use row operations and back-substitution to obtain row-echelon form. Now, we will take row-
echelon form a step farther to solve a 3 by 3 system of linear equations. The general idea is to
eliminate all but one variable using row operations and then back-substitute to solve for the other
variables.

EXAMPLE 6: Solve the system of linear equations using matrices.

Solution
First, we write the augmented matrix.

Next, we perform row operations to obtain row-echelon form.

The easiest way to obtain a 1 in row 2 of column 1 is to interchange R2 and R3 .

Then

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 9

The last matrix represents the equivalent system.

Using back-substitution, we obtain the solution as (4, −3, 1).

EXAMPLE 7:
Solve the following system of linear equations using matrices.

Solution
Write the augmented matrix.

First, multiply row 1 by −1 to get a 1 in row 1, column 1. Then, perform row operations to obtain
row-echelon form.

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 10

The last matrix represents the following system.

We see by the identity 0=0 that this is a dependent system with an infinite number of solutions.
We then find the generic solution. By solving the second equation for y and substituting it into the
first equation we can solve for z in terms of x .

Now we substitute the expression for z into the second equation to solve for y in terms of x .

The generic solution is

CHECK YOUR UNDERSTANDING EXERCISE 1

INSTRUCTIONS: Evaluate the following problem. Write your answer in your notebook.
1) Write the system of equations in row-echelon form.
a) x – 2y + 3z = 9
-x + 3y = - 4
2x – 5y + 5z = 17

b) 2x + 5y + 2z = −38
3x – 2y + 4z = 17
−6x + y – 7z = −12

2) Solve the system using Gaussian elimination

a) x + 4y – z = 4
2x + 5y + 8z = 15

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 11

5x + y - 3z = 3

b) 3x – 9z = 33
7x – 4y – z = −15
4x + 6y + 5z = −6

c) 2x1 – 3x2 – 7x3 + 10x4 = 15

x2 + x3 – 2x4 = 13
x1 – 8x2 + x4 = -11
3x1 + x2 – 5x3 + x4 = -6

3.3. Iterative Method

The word Iterative or Iteration refers to the technique that solve any linear system
problems with successive approximation at each step. In computational mathematics, an iterative
method is a mathematical procedure that uses an initial value to generate a sequence of improving
approximate solutions for a class of problems, in which the n-th approximation is derived from the
previous ones.

3.3.1. Jacobi Method

Two assumptions made on Jacobi Method:
1. The system given by

has a unique solution.

2. The coefficient matrix has no zeros on its main diagonal, namely a11, a12 , and ann are nonzeros.

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 12

EXAMPLE 8:
Apply the Jacobi method to solve
5x1 – 2x2 + 3x3 = -1
-3x1 + 9x2 + x3 = 2
2x1 – x2 – 7x3 = 3
Continue iterations until two successive approximations are identical when rounded to three
significant digits.

Solution:
There is no need to rearrange the system because it was already diagonally dominant
(coefficients 5, 9, 7)
To begin, rewrite the system

Choose the initial guess x1 = 0, x2 = 0, x3 = 0. The first approximation is

Second Approximation
−1 2 3
x1(2) = + (0.222) – (-0.429) = 0.146
5 5 5
2 3 1
x2(2) = + (-0.200) – (-0.429) = 0.203
9 9 9
−3 2 1
x3(2) = + (-0.200) – (0.222) = -0.517
7 7 7

Third Approximation
−1 2 3
x1(3) = + (0.203) – (-0.517) = 0.191
5 5 5
2 3 1
x2(3) = + (0.146) – (-0.517) = 0.328
9 9 9
−3 2 1
x3(3) = + (0.146) – (0.203) = -0.416
7 7 7

Fourth Approximation
−1
2 3
x1(4) = + (0.328) – (-0.416) = 0.181
5 5 5
2 3 1
x2(4) = + (0.191) – (-0.416) = 0.332
9 9 9

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 13

−32 1
x3(4) = + (0.191) – (0.328) = -0.421
7 7 7
Fifth Approximation
−12 3
x1(5) = + (0.332) – (-0.421) = 0.185
5 5 5
2 3 1
x2(5) = + (0.181) – (-0.421) = 0.329
9 9 9
−3 2 1
x3(5) = + (0.181) – (0.332) = -0.424
7 7 7

Sixth Approximation
−12 3
x1(6) = + (0.329) – (-0.424) = 0.186
5 5 5
2 3 1
x2(6) = + (0.185) – (-0.424) = 0.331
9 9 9
−3 2 1
x3(6) = + (0.185) – (0.329) = -0.422
7 7 7
Since 5th and 6th approximations are identical when rounded to three significant digits, the
solution is approximately equal to (0.186, 0.331, -0.422)

3.3.2. Gauss-Seidel Method

In numerical linear algebra, the Gauss–Seidel method, also known as the Liebmann method or
the method of successive displacement, is an iterative method used to solve a system of linear
equations. It is named after the German mathematicians Carl Friedrich Gauss and Philipp Ludwig von
Seidel, and is similar to the Jacobi method. Though it can be applied to any matrix with non-zero
elements on the diagonals, convergence is only guaranteed if the matrix is either strictly diagonally
dominant, or symmetric and positive definite.
A square matrix is said to be diagonally dominant if, for every row of the matrix, the magnitude
of the diagonal entry in a row is larger than or equal to the sum of the magnitudes of all the other (non-
diagonal) entries in that row.

EXAMPLE 9: Let us solve the previous example using Gauss-Seidel method. Round off
upto 3 decimal places
5x1 – 2x2 + 3x3 = -1
-3x1 + 9x2 + x3 = 2
2x1 – x2 – 7x3 = 3

Solution:
The system is diagonally dominant.
To begin, rewrite the system

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 14

Choose the initial guess x1 = 0, x2 = 0, x3 = 0. The first approximation is

−1 2 3
x1(2) = + (0) – (0) = -0.200
5 5 5
2 3 1
x2(2) = + (-0.200) – (0) = 0.156
9 9 9
−3 2 1
x3(2) = + (-0.200) – (0.156) = -0.508
7 7 7

Second Approximation
−1 2 3
x1(2) = + (0.156) – (-0.508) = 0.167
5 5 5
2 3 1
x2(2) = + (0.167) – (-0.508) = 0.334
9 9 9
−3 2 1
x3(2) = + (0.167) – (0.334) = -0.428
7 7 7

Third Approximation
−1 2 3
x1(3) = + (0.334) – (-0.428) = 0.190
5 5 5
2 3 1
x2(3) = + (0.190) – (-0.428) = 0.333
9 9 9
−3 2 1
x3(3) = + (0.190) – (0.333) = -0.422
7 7 7

Fourth Approximation
−1 2 3
x1(4) = + (0.333) – (-0.422) = 0.186
5 5 5
2 3 1
x2(4) = + (0.186) – (-0.422) = 0.331
9 9 9
−3 2 1
x3(4) = + (0.186) – (0.331) = -0.423
7 7 7

Since 5th and 6th approximations are identical when rounded to three significant digits, the
solution is approximately equal to (0.186, 0.331, -0.423).
Notice that the solution is shorter than using Jacobi Method.

CHECK YOUR UNDERSTANDING EXERCISE 2

INSTRUCTIONS: Evaluate the following linear equations. Write your answer in your notebook.
1) Use Jacobi method in solving the following:
a) x – 3y + 8z = 11

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 15

-x + 5y – z = 7
7x – y + 5z = 15

b) 5x + y + 2z = 8
3x – 2y + 7z = 17
−2x + 6y – z = −12

2) Solve by Gauss-Seidel method

9x1 – 5x2 + x3 + 2x4 = -3
3x1 – 2x2 + x3 – 7x4 = 8
x1 - 11x2 + 3x3 – x4 = 15
-4x1 – 6x2 – 8x3 + 3x4 = -9

3) Solve the given system of equation below using Gaussian Elimination and Gauss-Seidel
Method.
−3x + 7y + 2z = 1
−2x + 5y – 9z = −30
8x − 2y - 3z = 21

SUMMARY

Here are the important topics which can help you in reviewing this module:
➢ An augmented matrix is one that contains the coefficients and constants of a system of
equations.
➢ A matrix augmented with the constant column can be represented as the original system of
equations.
➢ Row operations include multiplying a row by a constant, adding one row to another row, and
interchanging rows.
➢ We can use Gaussian elimination to solve a system of equations.
➢ Row operations are performed on matrices to obtain row-echelon form.
➢ To solve a system of equations, write it in augmented matrix form. Perform row operations to
obtain row-echelon form. Back-substitute to find the solutions.

SUGGESTED READINGS AND MATERIALS

Gaussian Elimination. Retrieved from

https://math.libretexts.org/Bookshelves/Precalculus/Book%3A_Precalculus_(OpenStax)/09%3A_
Systems_of_Equations_and_Inequalities/9.06%3A_Solving_Systems_with_Gaussian_Eliminatio
n

Gauss-Seidel Method. Retrieved from https://www.youtube.com/watch?v=ajJD0Df5CsY

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio
 ES 221e – Numerical Solution to CE Problem 16

REFERENCES

Student Solutions Manual and Study Guide for Numerical Analysis, 9th Edition
Richard L. Burden, Cengage Learning, 2011

Numerical Mathematics and Computing, 7th Edition

Ward Cheney, Cengage Learning, 2013

Numerical Solution of Partial Differential Equations, 3rd Edition

Granville Sewell, Wspc, 2014

Differential Equations, 4th Edition

Paul Blanchard, Robert L. Devaney, Glen R. Hall, Cengage Learning, 2011

Engr. Michael John T. Guillermo, RME

Subject Teacher/ Special Lecturer This Review Guide is a product of
research made by Engr. Ariel E. Novio