Part 3

Chapter 12

Iterative Methods

Prof. Hae-
Hae-Jin Choi

[email protected]

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 1
 Chapter Objectives
l Understanding the difference between the Gauss-Seidel and
Jacobi methods.

l Knowing how to assess diagonal dominance

and knowing what it means.

l Recognizing how relaxation can be used

to improve convergence of iterative methods.

l Understanding how to solve systems of nonlinear equations

with successive substitution and Newton-Raphson.

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 2
 Gauss--Seidel Method
Gauss
l The Gauss-Seidel method is the most commonly used
iterative method for solving linear algebraic equations [A]{x}={b}.
l For a 3x3 system with nonzero elements along the diagonal,
for example, the jth iteration values are found from the j-1th
iteration using:

é a11 a12 a13 ù ì x1 ü ì b1 ü b - a x j-1

- a x j-1
êa a22
ï ï ï ï
a23 úú í x2 ý = íb2 ý x1j = 1 12 2 13 3

ê 21 a11
êë a31 a23 a33 úû ïî x3 ïþ ïîb3 ïþ j j-1
b - a x - a x
x2j = 2 21 1 23 3

a11 x1 = b1 - a12 x2 - a13 x3 a22

j j
a22 x1 = b2 - a21 x1 - a23 x3 b - a x - a x
x3j = 3 31 1 32 2

a33 x1 = b3 - a31 x1 - a32 x2 a33

School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 3
 Jacobi Iteration
l The Jacobi iteration is similar to the Gauss-Seidel method,
except the j-1th information is used to update all variables in the jth
iteration:
a) Gauss-Seidel
b) Jacobi
j -1 j -1
b - a x - a x
x1j = 1 12 2 13 3
a11
j -1 j -1
b - a x - a x
x2j = 2 21 1 23 3
a22
j -1 j -1
b - a x - a x
x3j = 3 31 1 32 2
a33
School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 4
 Convergence
l The convergence of an iterative method can be calculated by
determining the relative percent change of each element in {x}.
For example, for the ith element in the jth iteration,

xij - xij-1
e a,i = j
´100%
xi

l The method is ended when all elements have converged to a set

tolerance. �

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 5
 Example 12.1
Q. Use the Gauss-Seidel Method to solve this set of
equations.
3 x1 - 0.1x2 - 0.2 x3 = 7.85
0.1x1 + 7 x2 - 0.3 x3 = - 19.3
0.3 x1 - 0.2 x2 + 10 x3 = 71.4

Note : True solution is {x}T = [3 -2.5 7 ]

Assume that x2 and x3 are zero in the first computation.

7.85 + 0.1x2 + 0.2 x3 -19.3 - 0.1x1 + 0.3x3 71.4 - 0.3x1 + 0.2 x2

x1 = x2 = x3 =
3 7 10

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 6
 j 7.85 + 0.1x2 j -1 + 0.2 x3 j -1 j -19.3 - 0.1x1 j + 0.3 x3 j -1
x1 = x2 =
3 7

j 71.4 - 0.3 x1 j + 0.2 x2 j

x3 =
10

1st iteration

7.85 + 0.1(0) + 0.2(0)

x1 = = 2.616667
3
-19.3 - 0.1(2.616667) + 0.3(0)
x2 = = -2.794524
7
71.4 - 0.3(2.616667) + 0.2(-2.794524)
x3 = = 7.005610
10
School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 7
 2nd iteration

7.85 + 0.1(-2.794524) + 0.2(7.005610)

x1 = = 2.990557
3
-19.3 - 0.1(2.990557) + 0.3(7.005610)
x2 = = -2.499625
7
71.4 - 0.3(2.990557) + 0.2(-2.499625)
x3 = = 7.000291
10

2.990557 - 2.616667
e a ,1 = ´ 100% = 12.5%
2.990557
ea,2 = 11.8%; ea,3 = 0.076%;

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 8
 Diagonal Dominance
l The Gauss-Seidel method may diverge,
but if the system is diagonally dominant,
it will definitely converge.
l Diagonal dominance means:
n
aii > å aij
j=1
j¹i

l Many engineering problems satisfy this requirement

�

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 9
 MATLAB Program
MATLAB M-file: GaussSeidel

b1 a12 old a13 old

x1new = - x2 - x3
a11 a11 a11
b2 a21 new a23 old
x2new = - x1 - x3
a22 a22 a22
b3 a31 new a32 new
x3new = - x1 - x2
a33 a33 a33

{x} = {d } - [C ]{x}

ì b1 / a11 ü é 0 a12 / a11 a13 / a11 ù

ï
{d } = íb2 / a22 ý
ï [C ] = êêa21 / a22 0 a23 / a22 úú
ïb / a ï êë a31 / a33 a32 / a33 0 úû
î 3 33 þ

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 10
 School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 11
 Relaxation
l To enhance convergence, an iterative program can
introduce relaxation where the value at a particular iteration
is made up of a combination of the old value and the newly
calculated value (update for the new one):
xinew = lxinew + (1- l )xiold
where λ is a weighting factor that is assigned a value
between 0 and 2.
�
l 0< λ <1: underrelaxation
l λ =1: no relaxation
l 1< λ ≤2: overrelaxation
School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 12
 Nonlinear Systems
l Nonlinear systems can also be solved using
the same strategy as the Gauss-Seidel
method - solve each system for one of the
unknowns and update each unknown using
information from the previous iteration.

l This is called successive substitution.

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 13
 Example 12.2
Q. Use successive substitution to determine the roots of
the following equation. A correct pair of roots is
x1 = 2 and x2 = 3.
Use the initial guesses of x1 = 1.5 and x2 = 3.5.

x12 + x1 x2 = 10 10 - x12
x1 = x2 = 57 - 3x1 x22
x2 + 3 x1 x22 = 57 x2
First iteration
10 - (1.5) 2 x2 = 57 - 3(2.21429)(3.5) 2 = -24.37516
x1 = = 2.21429
3.5
Second iteration
10 - (2.21429) 2
x1 = = -0.20910 x2 = 57 - 3(-0.20910)(-24.37516) 2 = 429.709
- 24.37516
Seems to be diverging
School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 14
 Use the same equation but with a different format
57 - x2
x1 = 10 - x1 x2 x2 =
3 x1
First iteration
57 - 3.5
x1 = 10 - 1.5(3.5) = 2.17945 x2 = = 2.86051
3(2.17945)
Second iteration

x1 = 10 - 2.17945(2.86051) = 1.94053 57 - 2.86051

x2 = = 3.04955
3(1.94053)

The approach is converging on the true values.

à The most serious shortcoming of substitution, which depends on the

manner in which the equations are formulated.

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 15
 Newton--Raphson
Newton
l Nonlinear systems may also be solved using the Newton-
Raphson method for multiple variables.
l For a one-variable system, the Taylor series approximation
and resulting Newton-Raphson equations are:
f ( xi )
f ( xi +1 ) = f ( xi ) + ( xi +1 - xi ) f ¢( xi ) xi +1 = xi -
f ¢( xi )
l For a two-variable system,
¶f2,i ¶f
f1,i - f2,i 1,i
¶f1,i ¶f ¶x2 ¶x2
f1,i+1 = f1,i + (x1,i+1 - x1,i ) + (x2,i+1 - x2,i ) 1,i x1,i+1 = x1,i -
¶x1 ¶x2 ¶f1,i ¶f2,i ¶f1,i ¶f2,i
-
¶x1 ¶x2 ¶x2 ¶x1
¶f1,i ¶f2,i
f2,i -f
¶f2,i ¶f ¶x1 1,i ¶x1
f2,i+1 = f2,i + (x1,i+1 - x1,i ) + (x2,i+1 - x2,i ) 2,i x2,i+1 = x2,i -
¶x1 ¶x2 ¶f1,i ¶f2,i ¶f1,i ¶f2,i
-
¶x1 ¶x2 ¶x2 ¶x1
School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2
16
�
 ¶ f1,i ¶ f1,i
f1,i +1 = f1,i + ( x1,i +1 - x1,i ) + ( x2,i +1 - x2,i )
¶ x1 ¶ x2
¶ f 2,i ¶ f 2,i
f 2,i +1 = f 2,i + ( x1,i +1 - x1,i ) + ( x2,i +1 - x2,i )
¶ x1 ¶ x2
[ Z ]{xi +1} = -{ f } + [ Z ]{xi }, where [ Z ] = Jacobian matrix
[ Z ]{xi +1 - xi } = -{ f }
é ¶f1,i ¶f1,i ¶f1,i ù
ê L ú
ê ¶x1 ¶x2 ¶xn ú
{ x }T
= éë x1,i x2,i L xn ,i ùû
ê ¶f 2,i ¶f 2,i ¶f 2,i ú i
L
[ Z ] = ê ¶x1 ¶x2 ¶xn ú
ë 1,i +1 x2,i +1 L xn ,i +1 ùû
ê ú {x }T = é x
ê M M M ú i +1

ê ¶f n ,i ¶f n ,i ¶f n ,i ú
L
¶xn úû { f } = éë f1,i f 2,i L f n ,i ùû
ê ¶x T
ë 1 ¶x2

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 17
 School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 18
 Example 12.3 (1/2)
Q. Use the Newton-Raphson method to determine the
roots of the equations.
Use the initial guesses of x1 =1.5 and x2 = 3.5.

x12 + x1 x2 = 10
x2 + 3 x1 x22 = 57

¶f1, 0 ¶f1, 0
= 2 x1 + x2 = 2(1.5) + 3.5 = 6.5 = x1 = 1.5
¶x1 ¶x2
¶f 2, 0 2 2 ¶f 2, 0
= 3x = 3(3.5) = 36.75
2 = 1 + 6 x1 x2 = 1 + 6(1.5)(3.5) = 32.5
¶x1 ¶x2
Jacobian = 6.5(32.5) - 1.5(36.75) = 156.125
School of Mechanical Engineering
Chung-Ang University
Numerical Methods 2010-2 19
 Example 12.3 (2/2)
The values of the functions can be evaluated at the initial
guesses as
f1, 0 = (1.5) 2 + 1.5(3.5) - 10 = -2.5

f 2, 0 = 3.5 + 3(1.5)(3.5) 2 - 57 = 1.625

These values can be substituted to give

-2.5(32.5) - 1.625(1.5)
x1 = 1.5 - = 2.03603
156.125
1.625(6.5) - (-2.5)(36.75)
x2 = 3.5 - = 2.84388
156.125

The computation can be repeated until an acceptable accuracy

is obtained.

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 20
 MATLAB Program

School of Mechanical Engineering

Chung-Ang University
Numerical Methods 2010-2 21