0% found this document useful (0 votes)
22 views

Non Linear Iterative Methods

1) The document discusses methods for solving nonlinear systems of equations, including fixed point iteration, Gauss-Seidel, and Newton's methods. 2) It provides an example of using fixed point iteration to solve a system of 3 nonlinear equations by rewriting the system as an equivalent fixed point problem. 3) The example transforms the equations to a form where each variable is defined in terms of the others, and iterates this process until convergence within a set tolerance is achieved.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
22 views

Non Linear Iterative Methods

1) The document discusses methods for solving nonlinear systems of equations, including fixed point iteration, Gauss-Seidel, and Newton's methods. 2) It provides an example of using fixed point iteration to solve a system of 3 nonlinear equations by rewriting the system as an equivalent fixed point problem. 3) The example transforms the equations to a form where each variable is defined in terms of the others, and iterates this process until convergence within a set tolerance is achieved.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

SUPPLEMENTARY LECTURE NOTES

System of non-linear equations

Objectives
At the end of this section you will learn how to solve non-linear systems using
1. functional fixed-point iteration,
2. the Gauss-Seidel(Extra Reading) and
3. Newton’s methods.

Introduction
Solutions x  x0 to equations of the form f ( x)  0 are often required where it is
impossible or infeasible to find an analytical expression for the vector x. If the scalar
function f depends on n independent variables x1 , x2 , , xn , then the solution x0 will
describe a surface in n – 1 dimensional space. Alternatively we may consider the vector
function f(x) = 0, the solutions of which typically collapse to particular values of x. For
this course we restrict our attention to n independent variables x1 , x2 , , xn and seek
solutions to F (x)  0 where F is vector valued.

Fixed Point Method for Functions of Several Variables


The general of a system of nonlinear equations is
f1 ( x1 , x2 , , xn )  0
f 2 ( x1 , x2 , , xn )  0

f n ( x1 , x2 , , xn )  0
where each function f i maps n-dimensional space, R n , into the real line R. The above
system can be defined alternatively by defining the function F (x)  0 , where
F : Rn  Rn , x  ( x1 , x2 , , xn ) and
F ( x1 , x2 , , xn )   f1 ( x1 , x2 , , xn ), f 2 ( x1, x2 , , xn ), , f n ( x1, x2 , , xn )  .

1.1 Functional or Fixed Point Iteration


Suppose a nonlinear system of the form F (x)  0 has been transformed into an
equivalent fixed point problem G(x)  x . The functional or fixed point iteration process
applied to G is as follows:
1. Select x (0)   x1(0) , x2(0) , , xn(0)  .
2. Generate the sequence of vectors x ( k )   x1( k ) , x2( k ) , 
, xn( k ) by
 
x ( k )  G x ( k 1) for each i  1, 2,3, or, component-wise,

1
Numerical Analysis Non-Linear Iterative Methods


x1( k )  g1 x1( k 1) , x2( k 1) , , xn( k 1) 
x2( k )  g x 2
( k 1)
1 , x2( k 1) , , xn( k 1) 


xn( k )  g n x1( k 1) , x2( k 1) , , xn( k 1) 
The following theorem provides conditions for the iterative process to converge.

Theorem
Let D  ( x1 , x2 , , xn ) : ai  xi  bi , for each i  1, 2, , n , for some collection of
constants (a1 , a2 , , an ) and (b1 , b2 , , bn ) . Suppose G is a continuous function with
continuous first partial derivatives from D  Rn into R n with the property that
G(x)  D whenever x  D . If a constant K  1 exists with
gi (x) K
 whenever x  D
x j n

 

for each j  1, 2, , n and each component function g i , then the sequence x ( k )
k 0

defined by x (k )
G x  ( k 1)
 for each i  1, 2,3, converges to the unique fixed point
Kj
p  D , for any x (0) in D , and x ( j )  p  x (1)  x (0) .
 1 K 

Example 1
3x1  cos( x2 x3 )  12  0
x12  81( x2  0.1) 2  sin x3  1.06  0
e x1x2  20 x3  1033  0

f1 ( x1 , x2 , , xn )  3x1  cos( x2 x3 )  12
f 2 ( x1 , x2 , , xn )  x12  81( x2  0.1) 2  sin x3  1.06
f3 ( x1 , x2 , , xn )  e  x1x2  20 x3  1033

F ( x1 , x2 , , xn )   f1 ( x1 , x2 , , xn ), f 2 ( x1 , x2 , , xn ), , f n ( x1 , x2 , , xn ) 

 3x1  cos( x2 x3 )  12 , x12  81( x2  0.1) 2  sin x3  1.06, e x1x2  20 x3  1033 
Example 2
3x1  cos( x2 x3 )  12  0
x12  81( x2  0.1) 2  sin x3  1.06  0
e x1x2  20 x3  1033  0

2
Numerical Analysis MATH351/352

If the ith equation is solved for xi , the system can be changed into the fixed point
problem
x1  13 cos( x2 x3 )  16 ,
x2  19 x12  sin x3  1.06  0.1,
x3   201 e x1x2  1060
 3
.

Let G : R3  R3 be defined by G(x)   g1 (x), g2 (x), g3 (x)  where


g1 ( x1 , x2 , x3 )  13 cos( x2 x3 )  16 ,
g 2 ( x1 , x2 , x3 )  19 x12  sin x3  1.06  0.1,
g3 ( x1 , x2 , x3 )   201 e  x1x2  1060
 3
.

g1 ( x1 , x2 , x3 )  13 cos( x2 x3 )  16  12 ,

g 2 ( x1 , x2 , x3 )  1
9 x12  sin x3  1.06  0.1

 19 1  sin1  1.06  0.1  0.90,


g3 ( x1 , x2 , x3 )  1
20 e  x1x2  1060
 3

 1
20 e  1060
 3
 0.61
.
so 1  gi ( x1 , x2 , x3 )  1, for each i  1, 2,3. Thus, G(x)  D whenever x  D .
Finding bounds on the partial derivatives on D gives the following:

g1 g1 1 g1 1


 0,  x3 sin( x2 x3 )  13 sin1  0.281,  x2 sin( x2 x3 )  13 sin1  0.281,
x1 x2 3 x3 3

g 2 x1 1 g 2
   0.238,  0,
x1 9 x 2  sin x  1.06 9 0.218 x2
1 3

g 2 cos x3 1
   0.119,
x3 18 x 2  sin x  1.06 18 0.218
1 3

g3 x g3 x
 2 e x1x2  201 e  0.14,  1 e x1x2
x1 20 x2 20

3
Numerical Analysis Non-Linear Iterative Methods

Since the partial derivatives are bounded on D, the above Theorem implies that these
functions are continuous on D. Consequently, G is continuous on D. Moreover, for every
xD
gi (x)
 0.281 for each i  1, 2,3 and j  1, 2,3, and the condition in the second part of
x j
Theorem 9.7 holds for K  0.843 . It can be shown that
gi (x) x j for each i  1, 2,3 and j  1, 2,3 is continuous on D. Consequently, G has a
unique fixed point on D and the nonlinear system has a solution in D.

Example 3
3x1  cos( x2 x3 )  12  0
x  81( x2  0.1)  sin x3  1.06  0
2
1
2

e x1x2  20 x3  1033  0

Solution
If the ith equation is solved for xi , the system can be changed into the fixed point
problem as
x1  13 cos( x2 x3 )  16 ,
x2  19 x12  sin x3  1.06  0.1,
x3   201 e x1x2  1060  3
.
and write the iterative process as
x1( k )  13 cos  x2( k 1) x3( k 1)   16 ,

x 
2
( k 1)
x2( k )  19 1  sin x3( k 1)  1.06  0.1,
( k 1) ( k 1)
x3( k )   201 e x1 x2
 1060
 3
.

x ( k )  x ( k 1)  105

k x1( k ) x2( k ) x3( k ) x ( k )  x ( k 1)


0 0.10000000 0.10000000 0.10000000 --


1 0.49998333 0.00944115 0.52310127 0.423
2 0.49999593 0.00002557 0.52336331 9.4 103
3 0.50000000 0.00001234 0.52359814 2.3 104
4 0.50000000 0.00000003 0.52359847 1.2 105
5 0.50000000 0.00000002 0.52359877 3.1107

4
Numerical Analysis MATH351/352

(0.843)5
x (5)  p  0.423  1.15
 1  0.843


x (3)  0.50000000,1.234 105 , 0.52359814 
(0.843)5
x (5)  p  (1.20 105 )  5.5 105
 1  0.843

p   0.5,0,  6    0.5,0, 0.5235987757 

x (5)  p  2 108


x1( k )  13 cos x2( k 1) x3( k 1)  16 , 
x 
2
x2( k )  19 (k )
1  sin x3( k 1)  1.06  0.1,

x3( k )   201 e  x1
(k ) (k )
x2
 1060
 3
.

k x1( k ) x2( k ) x3( k ) x ( k )  x ( k 1)


0 0.10000000 0.10000000 0.10000000 --


1 0.49998333 0.02222979 0.52304613 0.423
2 0.49997747 0.00002815 0.52359807 2.2 102
3 0.50000000 0.00000004 0.52359877 2.8 105
4 0.50000000 0.00000000 0.52359877 3.8 108

Newton’s Method
In order to construct an algorithm that led to an appropriate fixed-point method
 a11 (x) a12 (x) a1n (x) 
 a21 (x) a22 (x) a2 n (x) 
A(x)   
 
 an1 (x) an 2 (x) ann (x) 
where each of the entries aij (x) is a function from Rn  R . The procedure requires
that A(x) be found so that G(x)  x  A(x)1 F (x) gives quadratic convergence to the
solution F (x)  0 , provided that A(x) is nonsingular at the fixed point.

5
Numerical Analysis Non-Linear Iterative Methods

Theorem
Suppose p is a solution of G(x)  x for some function G  ( g1 , g2 , , gn ) , mapping R n
into R n . If a number   0 exists with the property that
i) gi x j is continuous on N  x : x  p    for each
i  1, 2, , n and j  1, 2, ,n,
ii)  2 gi (x) x j xk is continuous, and  2 gi (x) x j xk  M for some constant M
whenever x  N for each i  1, 2, , n, j  1, 2, , n and k  1, 2, ,n,
iii) gi (p) x j  0 for each i  1, 2, , n and j  1, 2, , n ,
then the sequence generated by x ( k )  G  x ( k 1)  converges quadratically to p
n2 M ( k 1) 2
for any choice of x (0)  N and x ( k )  p  x p for each k  1 .
 2 

n
Since G(x)  x  A(x)1 F (x) , gi (x)  xi   bij (x) f j (x) ;
j 1

 n
 f j bij 
1    bij (x) (x)  (x) f j (x)  , if i  k ,
g (x)  j 1  xk xk 
so i  n
xk  f 
   b (x) j (x)  ij (x) f (x)  , if i  k.
b
   ij
xk xk
j 
 j 1  

n f j
0  1   bij (p) (p)
j 1 xk

n f j
so  b (p) x
j 1
ij (p)  1
k

n f j
0   bij (p) (p)
j 1 xk

n f j
So  b (p) x
j 1
ij (p)  0
k

6
Numerical Analysis MATH351/352

 f1 (x) f1 (x) f1 (x) 


 x x2 xn 
 1 
 f 2 (x) f 2 (x) f 2 (x) 
J (x)   x1 x2 xn 
 
 
 f n (x) f n (x) f n (x) 
 x x2 xn 
 1

A(p)1 J (p)  I

so J (p)  A(p)
G(x)  x  J (x)1 F (x)
x ( k )  G  x ( k -1)   x (k -1)  J  x (k -1)  F  x (k -1) 
1

Example 1
Solve the nonlinear system
3x1  cos( x2 x3 )  12  0
x  81( x2  0.1)  sin x3  1.06  0
2
1
2

e x1x2  20 x3  1033  0
Solution
The Jacobian matrix for the system is given by
 3 x3 sin x2 x3 x2 sin x2 x3 
J ( x1 , x2 , x3 )   2 x1 162( x2  0.1) cos x3 
  x1 x2  x1 x2

  x2e  x1e 20 
and
 x1( k )   x1( k 1)   h1( k 1) 
 x ( k )    x ( k 1)    h( k 1) 
 2( k )   2( k 1)   2( k 1) 
 x3   x3   h3 
     
where
 h1( k 1) 
 h( k 1)     J x ( k 1) , x ( k 1) , x ( k 1)  1 F x( k 1) , x( k 1) , x( k 1) .
 2( k 1)    1 2 3   1 2 3 
 h3 
 

Thus, at the kth step, the linear system

7
Numerical Analysis Non-Linear Iterative Methods

3 x3( k 1) sin x2( k 1) x3( k 1) x2 sin x2( k 1) x3( k 1)   h1( k 1) 
 ( k 1)   ( k 1) 
 2 x1 162( x2( k 1)  0.1) cos x3( k 1)   h2( k 1) 
 x e 1 2
( k 1)  x ( k 1) ( k 1)
x
 x1 e 1 2
( k 1)  x ( k 1) ( k 1)
x   h3 
 2 20  
 3x1( k 1)  cos( x2( k 1) x3( k 1) )  12 
 ( k 1) 2 
   x1   81( x2( k 1)  0.1) 2  sin x3( k 1)  1.06 
  x1( k 1) x2( k 1) 
e  20 x3( k 1)  1033 
must be solved. The results obtained using the above iterative procedure is as shown
below

k x1( k ) x2( k ) x3( k ) x ( k )  x ( k 1)


0 0.10000000 0.10000000 0.10000000 --


1 0.50003702 0.01946686 0.52152047 0.422
2 0.50004593 0.00158859 0.52355711 1.79 102
3 0.50000034 0.00001244 0.52359845 1.58 103
4 0.50000000 0.00000000 0.52359877 1.24 105
5 0.50000000 0.00000000 0.52359877 0

p   0.5,0,  6    0.5,0, 0.5235987757 

Examples
1. The nonlinear system
 x 2  10 x  y 2  8
F( x, y )   2 0
 xy  x  10 y  8
can be transformed into the fixed point problem
 x   10 
x 2  y 2 8
G( x, y)      xy2  x 8 
 y   10 

(a) Starting with the initial estimates x0  y0  0 , apply functional iteration to


G to approximate the solution to an accuracy of 105 .
(b) Does Gauss-Seidel Method accelerate convergence?

8
Numerical Analysis MATH351/352

Solution
(a)
i x y g1(x,y) g2(x,y) Tol
0 0 0 0.8 0.8
1 0.8 0.8 0.928 0.9312 0.8
2 0.928 0.9312 0.972832 0.97327 0.1312
3 0.972832 0.97327 0.989366 0.989435 0.044832
4 0.989366 0.989435 0.995783 0.995794 0.016534
5 0.995783 0.995794 0.998319 0.998321 0.006417
6 0.998319 0.998321 0.999328 0.999329 0.002536
7 0.999328 0.999329 0.999732 0.999732 0.00101
8 0.999732 0.999732 0.999893 0.999893 0.000403
9 0.999893 0.999893 0.999957 0.999957 0.000161
10 0.999957 0.999957 0.999983 0.999983 6.44E-05
11 0.999983 0.999983 0.999993 0.999993 2.58E-05
12 0.999993 0.999993 0.999997 0.999997 1.03E-05
13 0.999997 0.999997 0.999999 0.999999 4.12E-06
14 0.999999 0.999999 1 1 1.65E-06

(b)
i x y g1(x,y) g2(x,y) Tol
0 0 0 0.8 0.88
1 0.8 0.88 0.94144 0.967049 0.88
2 0.94144 0.967049 0.982149 0.990064 0.14144
3 0.982149 0.990064 0.994484 0.99693 0.040709
4 0.994484 0.99693 0.998287 0.999045 0.012335
5 0.998287 0.999045 0.999467 0.999703 0.003803
6 0.999467 0.999703 0.999834 0.999907 0.00118
7 0.999834 0.999907 0.999948 0.999971 0.000367
8 0.999948 0.999971 0.999984 0.999991 0.000114
9 0.999984 0.999991 0.999995 0.999997 3.56E-05
10 0.999995 0.999997 0.999998 0.999999 1.11E-05
11 0.999998 0.999999 1 1 3.46E-06

(c)
From (a) and (b) it is seen that Gauss-Seidel Method accelerate convergence.
2. Convert the nonlinear system
3x  cos( yz )  12  0
x 2  81 y  101   sin z  1.06  0
2

e xy  20 z  1033  0
to a fixed point problem and use both functional iteration and the Gauss-Seidel
variant of functional iteration to approximate the root to within 105 in the l
norm, starting the initial estimate x0  y0  0.1 and z0  0.1 . [Note the exact
 12 , 0,  6 
T
root is ]

9
Numerical Analysis Non-Linear Iterative Methods

Solution
cos( yz )  12 x 2  sin z  1.06 1 e xy  1033
x , y  , z
3 81 10 20
For k  1, 2, , the functional iteration and the Gauss-Seidel variant of functional
iteration are given as
cos( y ( k 1) z ( k 1) )  12
x(k ) 
3
( x )  sin z ( k 1)  1.06 1
( k 1) 2
y(k )   and
81 10
( k 1)
e  x y ( k 1)  103 3
z (k )

20
( k 1) ( k 1)
cos( y z )  12
x(k ) 
3
( x )  sin z ( k 1)  1.06 1
(k ) 2
y(k )   respectively for the above fixed point
81 10
e x y  103 3
(k ) (k )

z (k )

20
problem.
Using the functional iteration we have the following table:

i x y z g1(x,y,z) g2(x,y,z) g3(x,y,z) Tol


0 0.1 0.1 -0.1 0.499983 0.009441 -0.5231013
1 0.499983 0.009441 -0.523101 0.499996 2.56E-05 -0.5233633 0.399983
2 0.499996 2.56E-05 -0.523363 0.5 1.23E-05 -0.5235981 0.009154
3 0.5 1.23E-05 -0.523598 0.5 3.42E-08 -0.5235985 4.07E-06
4 0.5 3.42E-08 -0.523598 0.5 1.65E-08 -0.5235988 1.2E-05
5 0.5 1.65E-08 -0.523599 0.5 4.57E-11 -0.5235988 6.95E-12
6 0.5 4.57E-11 -0.523599 0.5 2.2E-11 -0.5235988 1.6E-08
7 0.5 2.2E-11 -0.523599 0.5 6.1E-14 -0.5235988 0
8 0.5 6.1E-14 -0.523599 0.5 2.94E-14 -0.5235988 2.14E-11
9 0.5 2.94E-14 -0.523599 0.5 0 -0.5235988 0
10 0.5 0 -0.523599 0.5 0 -0.5235988 2.87E-14

Using the Gauss-Seidel variant of functional iteration we have the following


table:

i x y z g1(x,y,z) g2(x,y,z) g3(x,y,z) Tol


0 0.1 0.1 -0.1 0.499983 0.02223 -0.5230461
1 0.499983 0.02223 -0.523046 0.499977 2.82E-05 -0.5235981 0.399983
2 0.499977 2.82E-05 -0.523598 0.5 3.76E-08 -0.5235988 0.02165
3 0.5 3.76E-08 -0.523599 0.5 5.03E-11 -0.5235988 2.74E-05
4 0.5 5.03E-11 -0.523599 0.5 6.72E-14 -0.5235988 3.66E-08
5 0.5 6.72E-14 -0.523599 0.5 0 -0.5235988 4.9E-11
6 0.5 0 -0.523599 0.5 0 -0.5235988 6.55E-14

10
Numerical Analysis MATH351/352

3. Starting with the initial guess x0  y0  1.0 , use fixed point (functional iteration
to approximate the solution to the system
2 x2  y 2  4.32, x2  y 2  0
by performing 5 iterations.
4. Consider the nonlinear system
2 x  xy  1  0
2 y  xy  1  0
which has a unique root x  (1, 1)T . Starting with the initial estimate x0  y0  0
, compare the methods of functional iteration, Gauss-Seidel Newton when
approximating the root of this system (perform 5 iterations in each case).
5. Use Newton’s Method to approximate the solution of the nonlinear system
x 2  2 x  y  12  0
x2  4 y 2  4  0
starting with the initial estimate  x0 , y0    2, 14  and computing 3 iterations.
Solution
Using the Newton’s iterative method
x( k )  x( k 1)  J 1 ( x( k 1) ) F ( x( k 1) ), for k  1, with x(0)   x0 , y0    2, 14  , we
have
1
 2   2 1  0.25 
x (1)  x (0)  J 1 ( x (0) ) F ( x(0) )      
 0.25   4 2   0.25 
 2  1  2 1  0.25   2  1  0.75  1.90625 
        
 0.25  8  4 2  0.25   0.25  8  0.50   0.3125 
1
1 1.90625   1.8125 1   0.00879 
x  x  J ( x )F (x )  
(2) (1) (1) (1)
   
 0.3125   3.8125 2.5   1.70312 
1.90625  1  2.5 1  0.00879   2.10773 
    3.8125 1.8125  1.70312    0.68232 
 0.3125  8.34375     
6. Use Newton’s Method to approximate the two solutions of the nonlinear system
ye x  2, x2  y 2  4
by computing 2 iterations for each of the given initial estimates
(a)  x0 , y0    0.6, 3.7 
(b)  x0 , y0    1.9, 0.4
7. Use Newton’s Method to approximate the solution of the nonlinear system
x 2  y 2  0.6 y  0.16  0
x 2  y 2  x  1.6 y  0
by computing 3 iterations with the initial estimate of  x0 , y0    0.6,0.25 .
Using the more accurate initial estimate of  x0 , y0    0.3,0.1 , repeat the
process using the modified Newton’s method whereby the Jacobian is evaluated
and held constant for subsequent iterations. Compare the two results.

11

You might also like