34

This chapter covers the numerical solution of linear and non-linear systems of equations.
Systems of linear equations that have to be solved simultaneously arise in problems that include
several (possibly many) variables that are dependent to one another. Such problems occur not only
in engineering and science, which is the focus of this module, but also in any discipline
(economics, statistics, business, etc.). A system of two (or three) equations with two (or three)
unknowns can be solved manually by substitution or other mathematical methods (e.g. Cramer’s
rule). Solving a system in this way is practically impossible as the number of unknowns (or
equations) increases beyond three.

At the end of this module, it is expected that the students will be able to:
1. Know the fundamental differences between Gauss elimination and the Gauss-Jordan
elimination.
2. Apply Gauss elimination and Gauss-Jordan elimination method in solving systems of
linear equations.
3. Apply Gauss-Seidel and Jacobi iteration in solving systems of linear equations.
4. Know how to assess diagonal dominance of a system of equations and how it related
whether the system can be solved with the Gauss-Seidel and Jacobi iteration.

MODULE 3 ENGG 414 NUMERICAL METHODS AND ANALYSIS

 35

SYSTEMS OF LINEAR EQUATIONS

A linear system of n algebraic equations in n unknowns 𝑥1 , 𝑥2 , … . , 𝑥𝑛 is in the form

𝑎11 𝑥1 + 𝑎12 𝑥2 + ⋯ . +𝑎1𝑛 𝑥𝑛 = 𝑏1

𝑎 𝑥 + 𝑎22 𝑥2 + ⋯ . +𝑎2𝑛 𝑥𝑛 = 𝑏2
{ 21 1 eq. 3.1
…..
𝑎31 𝑥1 + 𝑎31 𝑥2 + ⋯ . +𝑎3𝑛 𝑥𝑛 = 𝑏𝑛

Where 𝑎𝑖𝑗 (𝑖, 𝑗 = 1, 2, … . , 𝑛) and 𝑏𝑘 (𝑘 = 1, 2, … . , 𝑛) are known constants, and 𝑎𝑖𝑗 are the
coefficients. If every 𝑏𝑘 is zero, the system is homogeneous, otherwise it is nonhomogeneous.
Expressing equation 3.1 into its equivalent matrix form, we have:

𝑨𝒙 = 𝒃 eq. 3.2

With:

𝑎11 𝑎22 … 𝑎1𝑛 𝑥1 𝑏1

𝑎21 𝑎22 … 𝑎2𝑛 𝑥2 𝑏
𝑨=[… … … …] , 𝒙 = { … } , 𝒃 = { 2}
…
𝑎31 𝑎32 … 𝑎3𝑛 𝑛𝑥𝑛 𝑥𝑛 𝑛𝑥1 𝑏𝑛 𝑛𝑥1

Where A is the coefficient of the matrix. A set of values for 𝑥1 , 𝑥2 , … . , 𝑥𝑛 satisfying equation 3.2
forms a solution of the system. The vector x with components 𝑥1 , 𝑥2 , … . , 𝑥𝑛 is the solution vector
for equation 3.2. If 𝑥1 = 0 = 𝑥2 = 𝑥𝑛 , the solution 𝒙 = 0𝑛𝑥1 is called the trivial solution. The
augmented matric for the equation 3.2 is defined as:

𝑎11 𝑎22 … 𝑎1𝑛 𝑏1

𝑎21 𝑎22 … 𝑎2𝑛 𝑏2
[𝑨|𝒃] = [ … … … … | …] eq. 3.3
𝑎31 𝑎32 … 𝑎3𝑛 𝑏𝑛 𝑛𝑥(𝑛+1)

NUMERICAL SOLUTION OF LINEAR SYSTEMS

Numerical methods for solving linear systems of equations are divided into two categories
(Figure 3.1): direct methods and indirect methods.
A direct method computes the solution of equation 3.2 by performing a predetermined
number of operations. These method transforms the original system into an equivalent system in
which the coefficient matrix is upper-triangular, lower-triangular, or diagonal, making the system
much easier to solve.
On the other hand, indirect methods are those that uses iterations to find the approximate
solution. This is done with an initial vector and then successive approximations are generated that

MODULE 3 ENGG 414 NUMERICAL METHODS AND ANALYSIS

 36

eventually converges to the actual solution. Unlike direct methods, the number of operations
required by iterative methods is not known in advance.

NUMERICAL SOLUTION
OF LINEAR SYSTEMS
𝑨𝒙 = 𝒃

Indirect or
Direct Methods Iterative
Methods

Gauss
Jacobi Iteration
Elimination

Gauss-Jordan Gauss-Seidel
Elimination Method

Figure 3.1 Classification of Methods to solve Systems of Linear Equations

➢ GAUSS ELIMINATION METHOD

The Gauss elimination method is the most familiar method for solving a system of linear
equations. This method consists of two parts: the elimination phase and the solution phase. In this
method, a system of equations that is given in a general form is manipulated to be in the upper
triangular form, which is then be solved by using the back substitution.
Gaussian elimination, also known as row reduction, is an algorithm in linear algebra for
solving a system of linear equations. It is usually understood as a sequence of operations performed
on the corresponding matrix of coefficients.

To perform row reduction on a matrix, one uses a sequence of elementary row operations to
modify the matrix until the lower left-hand corner of the matrix is filled with zeros, as much as
possible. There are three types of elementary row operations:

• Swapping two rows,

• Multiplying a row by a nonzero number,
• Adding a multiple of one row to another row.

MODULE 3 ENGG 414 NUMERICAL METHODS AND ANALYSIS

 37

EXAMPLE NO. 3.1

Solve for the values of 𝑥1 , 𝑥2 and 𝑥3 using Gauss elimination.

5𝑥1 − 2𝑥2 + 3𝑥3 = −1

−3𝑥1 + 9𝑥2 + 𝑥3 = 2
2𝑥1 − 𝑥2 − 7𝑥3 = 3

MODULE 3 ENGG 414 NUMERICAL METHODS AND ANALYSIS

 38

➢ GAUSS-JORDAN METHOD

The Gauss-Jordan method is the refinement of the Gauss elimination method. Gauss–
Jordan elimination is a procedure for converting a matrix to reduced row echelon form using
elementary row operations. It is a refinement of Gaussian elimination. A matrix is in reduced-
row echelon form is also known as row canonical form. The reduced row echelon form of a matrix
is unique, but the steps of the procedure are not.

STEPS FOR GAUSS-JORDAN ELIMINATION

To perform Gauss-Jordan Elimination:

1. Swap the rows so that all rows with all zero entries are on the bottom
2. Swap the rows so that the row with the largest, leftmost nonzero entry is on top.
3. Multiply the top row by a scalar so that top row's leading entry becomes 1.
4. Add/subtract multiples of the top row to the other rows so that all other entries in the
column containing the top row's leading entry are all zero.
5. Repeat steps 2-4 for the next leftmost nonzero entry until all the leading entries are 1.
6. Swap the rows so that the leading entry of each nonzero row is to the right of the leading
entry of the row above it.

EXAMPLE NO. 3.2

Solve for the problem in example no. 3.1 using the Gauss-Jordan Method

MODULE 3 ENGG 414 NUMERICAL METHODS AND ANALYSIS

 39

MODULE 3 ENGG 414 NUMERICAL METHODS AND ANALYSIS