College of Engineering and Computer Science

Department of Computer Engineering/ IT

Lebanese French University

Numerical Analysis
Lesson #: 4
Title: Iterative numerical method

Fall Semester 2023 – 2024

Instructor: Dr. Mohammed Fadhil Hama
Direct vs Iterative numerical method
• Direct: for example, solving for the unknows in a set of
linear equations by using Gauss elimination method & LU
decomposition. This is solved by a finite number of steps.
• Iterative: the steps can continue until a sufficient
amount of error is achieved. For example, by using
Bisection method, Newton’s method and Jacobi’s
method. For all of them, an initial guess is needed to
begin solving.
Direct method needs more computation though very
good for small matrices. Gives exact results.
Iterative method we can tell when to stop computing
(accuracy vs computation time). Gives approximate results.
Diagonally dominant matrix
• To be sure our square matrix will converge when using
iterative method.

[ ][ ] [ ]
Condition check:
𝑎 𝑏 𝑐 𝑥 𝐵1
ALL
𝑑 𝑒 𝑓 𝑦 = 𝐵2
𝑔 h 𝑖 𝑧 𝐵3
And for at least 1 row:
At least 1 of
them
Example 1 – check if it diagonally
dominant

[ ][ ] [ ]
9 1 2 𝑥 𝐵1
1 7 4 𝑦 = 𝐵2
2 3 7 𝑧 𝐵3
• OK This means that all row diagonals are
• OK dominant > this matrix is diagonally
• OK dominant.
When solving using iterative
method, convergence is
guaranteed.
Example 2 – check if it diagonally
dominant
Example 2 - solution

• OK
• FAIL
• Therefore, this matrix is not diagonally dominant, use
direct method.
 [ ][ ] [ ]
𝑎 𝑏 𝑐 𝑥 𝐵1
Jacobi iteration 𝑑 𝑒 𝑓 𝑦 = 𝐵2
Process of Jacobi iteration: 𝑔 h 𝑖 𝑧 𝐵3
1) Guess a value for x, y, z
• Normally start at x=0, y=0, z=0
2) Rewrite our system of equation so what we find for x, y, z.




3) Make table (next slide)

 Jacobi iteration
3) Make table
Put the guessed values, x=0, y=0, z=0 in the equations, the results are
put in iteration 1. Then those results will be put into the equations again,
and the results are iteration 2 and so on.
Iterati Iterati
Variabl Initial Iterati Iterati
on …. on
e guess on 1 on 2
3 n

X 0

Y 0

z 0

4) Repeat this until |(right side of equation) – (left side of equation)| ≤

acceptable error
Example Jacobi iteration
• Solve for x, y, z by using Jacobi iteration method

[ ][ ] [ ]
9 2 3 𝑥 7
1 12 9 𝑦 = 2
4 6 14 𝑧 1
 Example Jacobi iteration - solution
Check if it is diagonally dominant

[ ][ ] [ ]
• 9 > 2 + 3 OK 9 2 3 𝑥 7
• 12 > 1 + 9 OK 1 12 9 𝑦 = 2
• 14 > 4 + 6 OK 4 6 14 𝑧 1
Make separated equations for x, y, and z

97  Guess an initial value

of:
2 X=0
1 Y=0
Z=0
 Example Jacobi iteration - solution
Guess an initial value
of:
X=0 Solution for
Y=0 Iteration 1
Z=0

Iterati Iterati Iterati

Variabl Initial Iterati Iterati
on on on
e guess on 1 on 2
3 4 5

X 0 0.778

Y 0 0.167

z 0 0.071
 Example Jacobi iteration - solution
From iteration
1:
X = 0.778
Solution for
Y = 0.167
Z = 0.071 Iteration 2

Iterati Iterati Iterati

Variab Initial Iterati Iterati
on on on
le guess on 1 on 2
3 4 5
X 0 0.778 0.717

Y 0 0.167 0.049

z 0 0.071 -0.222
 Example Jacobi iteration - solution
From iteration
2:
X = 0.717
Solution for
Y = 0.049
Z = -0.222 Iteration 3

Iterati Iterati Iterati

Variab Initial Iterati Iterati
on on on
le guess on 1 on 2
3 4 5
X 0 0.778 0.717 0.841

Y 0 0.167 0.049 0.273

z 0 0.071 -0.222 -0.154

 Example Jacobi iteration - solution
From iteration
3:
X = 0.841
Solution for
Y = 0.273
Z = -0.154 Iteration 4

Iterati Iterati Iterati

Variab Initial Iterati Iterati
on on on
le guess on 1 on 2
3 4 5
X 0 0.778 0.717 0.841 0.768

Y 0 0.167 0.049 0.273 0.212

z 0 0.071 -0.222 -0.154 -0.286

 Example Jacobi iteration - solution
From iteration
4:
X = 0.768
Solution for
Y = 0.212
Z = -0.286 Iteration 5

Iterati Iterati Iterati

Variab Initial Iterati Iterati
on on on
le guess on 1 on 2
3 4 5
X 0 0.778 0.717 0.841 0.768 0.826

Y 0 0.167 0.049 0.273 0.212 0.317

z 0 0.071 -0.222 -0.154 -0.286 0.239

Example
Jacobi Variabl
e
Initial
guess
Iterati
on 1
Iterati
on 2
Iterati
on
3
Iterati
on
4
Iterati
on
5

iteration - X

Y
0

0
0.778

0.167
0.717

0.049
0.841

0.273
0.768

0.212
0.826

0.317

solution z 0 0.071 -0.222 -0.154 -0.286 -0.239

Validate Iteration
5 in original
equations
Iteration Error in equation 1 Since error
1 9(0.778)+2(0.137)+3(0.071)-
is
9 7=0.489
2 1.117 decreasing
3 -0.655 , it is
4 0.519 converging
5 0.352 . It should
6 0.265
eventually
Gauss-seidel method
• It is very similar to Jacobi method
• However, use the most updated values immediately.
• In the first step, guess that y=0 and z=0. whatever
answer you get for x, use that in the second equation,
then whatever answer you get for y, use that in the
third equation and so on.
Example – use Gauss seidel method
• Solve for x, y, and z

[ ][ ] [ ]
9 2 3 𝑥 7
1 12 9 𝑦 = 2
4 6 14 𝑧 1
 Example gauss seidel- solution
Check if it is diagonally dominant

[ ][ ] [ ]
• 9 > 2 + 3 OK 9 2 3 𝑥 7
• 12 > 1 + 9 OK 1 12 9 𝑦 = 2
• 14 > 4 + 6 OK 4 6 14 𝑧 1
Make separated equations for x, y, and z

97  Guess an initial value

of:
2 X=0
1 Y=0
Z=0
Example gauss seidel- solution

Variabl Initial Iterati Iterati

Iterati Iterati Iterati Solution
on on on
e guess on 1 on 2
3 4 5 for
Iteration
X 0 0.778
1
Y 0 0.102

z 0 -0.195
Example gauss seidel- solution

Variabl Initial Iterati Iterati

Iterati Iterati Iterati Solution
on on on
e guess on 1 on 2
3 4 5 for
Iteration
X 0 0.778 0.820
2
Y 0 0.102 0.244

z 0 -0.195 -0.267
Example gauss seidel- solution

Variabl Initial Iterati Iterati

Iterati Iterati Iterati Solution
on on on
e guess on 1 on 2
3 4 5 for
Iteration
X 0 0.778 0.820 0.813
3
Y 0 0.102 0.244 0.299

z 0 -0.195 -0.267 -0.289

Example gauss seidel- solution

Variabl Initial Iterati Iterati

Iterati Iterati Iterati Solution
on on on
e guess on 1 on 2
3 4 5 for
Iteration
X 0 0.778 0.820 0.813 0.808
4
Y 0 0.102 0.244 0.299 0.316

z 0 -0.195 -0.267 -0.289 -0.295

Example gauss seidel- solution

Variabl Initial Iterati Iterati

Iterati Iterati Iterati Solution
on on on
e guess on 1 on 2
3 4 5 for
Iteration
X 0 0.778 0.820 0.813 0.808 0.805
5
Y 0 0.102 0.244 0.299 0.316 0.322

z 0 -0.195 -0.267 -0.289 -0.295 -0.297

Example Variabl Initial Iterati Iterati
Iterati Iterati Iterati

gauss seidel-
on on on
e guess on 1 on 2
3 4 5
X 0 0.778 0.820 0.813 0.808 0.805

solution Y

z
0

0
0.102
-0.195
0.244
-0.267
0.299
-0.289
0.316
-0.295
0.322
-0.297

Validate Iteration
5 in original
equations
Iteration Error in equation 1
Since error
1 9(0.778)+2(0.102)+3(-0.195)- is
9 7=-0.38
decreasing,
2 0.07 it is
3 0.05 converging.
4 0.02 It should
5 0.002
eventually
Note:
reach zero.
this method reached near zero quicker than
the Jacobi
Classwork: gauss seidel method
• Find x. y and z, up to 3 iterations

[ ][ ] [ ]
3 1 2 𝑥 1
3 6 2 𝑦 = 2
2 4 8 𝑧 3
 End of Class

Thank you, Any questions?