Example

a  6, b  7,   0.0005
How many iterations are needed such that : x - r   ?

log(b  a)  log( ) log(1)  log(0.0005)

n   10.9658
log(2) log(2)

 n  11

41
Example
 Use Bisection method to find a root of the
equation x = cos (x) with absolute error <0.02
(assume the initial interval [0.5, 0.9])

Question 1: What is f (x) ?

Question 2: Are the assumptions satisfied ?
Question 3: How many iterations are needed ?
Question 4: How to compute the new estimate ?

42
CISE301_Topic2 43
Bisection Method
Initial Interval

f(a)=-0.3776 f(b) =0.2784

Error < 0.2
a =0.5 c= 0.7 b= 0.9

44
Bisection Method

-0.3776 -0.0648 0.2784

Error < 0.1
0.5 0.7 0.9

-0.0648 0.1033 0.2784

Error < 0.05
0.7 0.8 0.9

45
Bisection Method

-0.0648 0.0183 0.1033

Error < 0.025
0.7 0.75 0.8

-0.0648 -0.0235 0.0183

Error < .0125
0.70 0.725 0.75

46
Summary
 Initial interval containing the root:
[0.5,0.9]

 After 5 iterations:
 Interval containing the root: [0.725, 0.75]
 Best estimate of the root is 0.7375
 | Error | < 0.0125

47
A Matlab Program of Bisection Method
a=.5; b=.9; c=
u=a-cos(a); 0.7000
fc =
v=b-cos(b);
-0.0648
for i=1:5
c=
c=(a+b)/2 0.8000
fc=c-cos(c) fc =
if u*fc<0 0.1033
b=c ; v=fc; c=
else 0.7500
a=c; u=fc; fc =
end 0.0183
end c=
0.7250
fc =
-0.0235
48
Example
Find the root of:

f ( x)  x 3  3 x  1 in the interval : [0,1]

* f(x) is continuous
* f( 0 )  1, f (1)  1  f (a ) f (b)  0
 Bisection method can be used to find the root

49
Example
c= (a+b) (b-a)
Iteration a b f(c)
2 2

1 0 1 0.5 -0.375 0.5

2 0 0.5 0.25 0.266 0.25

3 0.25 0.5 .375 -7.23E-2 0.125

4 0.25 0.375 0.3125 9.30E-2 0.0625

5 0.3125 0.375 0.34375 9.37E-3 0.03125

50
Bisection Method
Advantages
 Simple and easy to implement
 One function evaluation per iteration
 The size of the interval containing the zero is reduced
by 50% after each iteration
 The number of iterations can be determined a priori
 No knowledge of the derivative is needed
 The function does not have to be differentiable

Disadvantage
 Slow to converge
 Good intermediate approximations may be discarded

51
 Lecture 3
Newton-Raphson Method
 Assumptions
 Interpretation
 Examples
 Convergence Analysis

52
Newton-Raphson Method
(Also known as Newton’s Method)
Given an initial guess of the root x0,
Newton-Raphson method uses information
about the function and its derivative at
that point to find a better guess of the
root.

Assumptions:
 f(x) is continuous and the first derivative is
known
 An initial guess x0 such that f’(x0)≠0 is given

53
Newton Raphson Method
- Graphical Depiction -
 If the initial guess at
the root is xi, then a
tangent to the
function of xi that is
f’(xi) is extrapolated
down to the x-axis
to provide an
estimate of the root
at xi+1.

54
Derivation of Newton’s Method
Given : xi an initial guess of the root of f ( x )  0
Question : How do we obtain a better estimate xi 1?
__________ __________ __________ ______
Taylor Therorem : f ( x  h)  f ( x )  f ' ( x ) h
Find h such that f ( x  h)  0.
f ( x)
h Newton  Raphson Formula
f ' ( x)
f ( xi )
A new guess of the root : xi 1  xi 
f ' ( xi )
55
Newton’s Method
Given f ( x ), f ' ( x ), x0 C FORTRAN PROGRAM
Assumputio n f ' ( x0 )  0 F ( X )  X * *3  3 * X * *2  1
FP ( X )  3 * X * *2  6 * X
__________ __________ __
X 4
for i  0: n DO 10 I  1, 5
f ( xi ) X  X  F ( X ) / FP ( X )
xi 1  xi 
f ' ( xi ) PRINT *, X
end 10 CONTINUE
STOP
END

56
Newton’s Method
Given f ( x ), f ' ( x ), x0 F.m
function [ F ]  F ( X )
F  X ^3  3 * X ^ 2  1
Assumputio n f ' ( x0 )  0
__________ __________ __ function [ FP]  FP ( X )
FP.m
for i  0 : n FP  3 * X ^ 2  6 * X
f ( xi ) % MATLAB PROGRAM
xi 1  xi 
f ' ( xi ) X 4
end for i  1 : 5
X  X  F ( X ) / FP ( X )
end

57
Example
Find a zero of the function f(x)  x 3  2 x 2  x  3 , x0  4
f ' (x)  3 x 2  4 x  1
f ( x0 ) 33
Iteration 1 : x1  x0   4 3
f ' ( x0 ) 33
f ( x1 ) 9
Iteration 2 : x2  x1   3   2.4375
f ' ( x1 ) 16
f ( x2 ) 2.0369
Iteration 3 : x3  x2   2.4375   2.2130
f ' ( x2 ) 9.0742

58
Example
k (Iteration) xk f(xk) f’(xk) xk+1 |xk+1 –xk|

0 4 33 33 3 1

1 3 9 16 2.4375 0.5625

2 2.4375 2.0369 9.0742 2.2130 0.2245

3 2.2130 0.2564 6.8404 2.1756 0.0384

4 2.1756 0.0065 6.4969 2.1746 0.0010

59
Convergence Analysis
Theorem :
Let f(x), f ' (x) and f ' ' (x) be continuous at x  r
where f(r)  0. If f ' (r)  0 then there exists   0
xk 1-r
such that x0 -r    2
C
xk -r
max f ' ' ( x)
1 x0 -r 
C
2 min f ' ( x)
x0 -r 

60
Convergence Analysis
Remarks

When the guess is close enough to a simple

root of the function then Newton’s method is
guaranteed to converge quadratically.

Quadratic convergence means that the number

of correct digits is nearly doubled at each
iteration.

61
Problems with Newton’s Method

• If the initial guess of the root is far from

the root the method may not converge.
• Newton’s method converges linearly near
multiple zeros { f(r) = f’(r) =0 }. In such a
case, modified algorithms can be used to
regain the quadratic convergence.

62
Multiple Roots
f ( x)  x 3
2
f ( x)   x  1

f(x) has three f(x) has two

zeros at x  0 zeros at x  -1
63
Problems with Newton’s Method
- Runaway -

x0 x1

The estimates of the root is going away from the root.

64
Problems with Newton’s Method
- Flat Spot -

x0

The value of f’(x) is zero, the algorithm fails.

If f ’(x) is very small then x1 will be very far from x0.
65
Problems with Newton’s Method
- Cycle -

x1=x3=x5

x0=x2=x4

The algorithm cycles between two values x0 and x1

66
Newton’s Method for Systems of
Non Linear Equations

Given : X 0 an initial guess of the root of F ( x)  0

Newton ' s Iteration
X k 1  X k  F ' ( X k ) F ( X k )
1

 f1 f1 
 x 
 f1 ( x1 , x2 ,...)  x2
 1 
f f 2
F ( X )   f 2 ( x1 , x2 ,...) , F'(X )   2 
 x1 x2 
     
 
 
67
Example
 Solve the following system of equations:

y  x 2  0.5  x  0
x 2  5 xy  y  0
Initial guess x  1, y  0

 y  x 2  0.5  x   2x 1 1  1
F  2 , F '    , X0   
 x  5 xy  y  2 x  5 y  5 x  1 0 

68
Solution Using Newton’s Method
Iteration 1 :
 y  x 2  0.5  x   0.5  2x 1 1  1 1 
F  2 
   , F '     
 x  5 xy  y   1   2 x  5 y  5 x  1  2  6 
1
1 1 1   0.5 1.25 
X1        1   0.25
0   2  6     
Iteration 2 :
0.0625  1.5 1 
F    , F '  1.25  7.25
 - 0.25   
1
1.25   1.5 1  0.0625 1.2332 
X2       - 0.25   0.2126
 0 .25   1.25  7. 25    

69
Example
Try this

 Solve the following system of equations:

y  x2 1 x  0
x2  2 y 2  y  0
Initial guess x  0, y  0

 y  x 2  1  x 2 x  1 1  0 
F  2 2 , F '    , X0   
 x  2y  y   2x  4 y  1 0 

70
Example
Solution

Iteration 0 1 2 3 4 5
_____________________________________________________________
0   1  0.6  0.5287   0.5257   0.5257 
Xk 0  0  0.2   0.1969   0.1980   0.1980 
           

71
Newton’s Method (Review)
Assumptions : f ( x), f ' ( x), x0 are available,
f ' ( x0 )  0
Newton' s Method new estimate :
f ( xi )
xi 1  xi 
f ' ( xi )
Problem :
f ' ( xi ) is not available,
or difficult to obtain analytically.
72
Secant Method
f ( x  h)  f ( x )
f ' ( x) 
h
if xi and xi 1 are two initial points :
f ( xi )  f ( xi 1 )
f ' ( xi ) 
( xi  xi 1 )

f ( xi ) ( xi  xi 1 )
xi 1  xi   xi  f ( xi )
f ( xi )  f ( xi 1 ) f ( xi )  f ( xi 1 )
( xi  xi 1 )
73
Secant Method
Assumptions :
Two initial points xi and xi 1
such that f ( xi )  f ( xi 1 )
New estimate (Secant Method) :
( xi  xi 1 )
xi 1  xi  f ( xi )
f ( xi )  f ( xi 1 )
74
Secant Method
2
f ( x)  x  2 x  0.5
x0  0
x1  1
( xi  xi 1 )
xi 1  xi  f ( xi )
f ( xi )  f ( xi 1 )

75
Secant Method - Flowchart
x0 , x1 , i  1

( xi  xi 1 )
xi 1  xi  f ( xi ) ;
f ( xi )  f ( xi 1 )
i  i 1

NO Yes
xi 1  xi   Stop

76
 Modified Secant Method
In this modified Secant method, only one initial guess is needed :
f ( xi   xi )  f ( xi )
f ' ( xi ) 
 xi

f ( xi )  x i f ( xi )
xi 1  xi   xi 
f ( xi   xi )  f ( x i ) f ( xi   xi )  f ( x i )
 xi
Problem : How to select  ?
If not selected properly, the method may diverge .

77
Example
50

40

Find the roots of : 30

f ( x)  x 5  x 3  3 20

Initial points 10

x0  1 and x1  1.1 0

-10

-20
with error  0.001
-30

-40
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

78
Example
x(i) f(x(i)) x(i+1) |x(i+1)-x(i)|

-1.0000 1.0000 -1.1000 0.1000

-1.1000 0.0585 -1.1062 0. 0062

-1.1062 0.0102 -1.1052 0.0009

-1.1052 0.0001 -1.1052 0.0000

79
Convergence Analysis
 The rate of convergence of the Secant method
is super linear:
xi 1  r

 C,   1.62
xi  r
r : root xi : estimate of the root at the i th iteration.

 It is better than Bisection method but not as

good as Newton’s method.

80
Summary
Method Pros Cons
Bisection - Easy, Reliable, Convergent - Slow
- One function evaluation per - Needs an interval [a,b]
iteration containing the root, i.e.,
- No knowledge of derivative is f(a)f(b)<0
needed
Newton - Fast (if near the root) - May diverge
- Two function evaluations per - Needs derivative and an
iteration initial guess x0 such that
f’(x0) is nonzero

Secant - Fast (slower than Newton) - May diverge

- One function evaluation per - Needs two initial points
iteration guess x0, x1 such that
- No knowledge of derivative is f(x0)- f(x1) is nonzero
needed

81
Example

Use Secant method to find the root of :

f ( x)  x 6  x  1
Two initial points x0  1 and x1  1.5

( xi  xi 1 )
xi 1  xi  f ( xi )
f ( xi )  f ( xi 1 )

82
Solution
_______________________________
k xk f(xk)
_______________________________
0 1.0000 -1.0000
1 1.5000 8.8906
2 1.0506 -0.7062
3 1.0836 -0.4645
4 1.1472 0.1321
5 1.1331 -0.0165
6 1.1347 -0.0005

83
Example

Use Newton' s Method to find a root of :

f ( x)  x 3  x  1
Use the initial point : x0  1.
Stop after three iterations, or
if xk 1  xk  0.001, or
if f ( xk )  0.0001.

84
Five Iterations of the Solution
 k xk f(xk) f’(xk) ERROR
 ______________________________________
 0 1.0000 -1.0000 2.0000
 1 1.5000 0.8750 5.7500 0.1522
 2 1.3478 0.1007 4.4499 0.0226
 3 1.3252 0.0021 4.2685 0.0005
 4 1.3247 0.0000 4.2646 0.0000
 5 1.3247 0.0000 4.2646 0.0000

85
Example

Use Newton' s Method to find a root of :

f ( x)  e  x  x
Use the initial point : x0  1.
Stop after three iterations, or
if xk 1  xk  0.001, or
if f ( xk )  0.0001.

86
Example
Use Newton' s Method to find a root of :
f ( x )  e  x  x, f ' ( x )  e  x  1

f ( xk )
xk f ( xk ) f ' ( xk )
f ' ( xk )
1.0000 - 0.6321 - 1.3679 0.4621
0.5379 0.0461 - 1.5840 - 0.0291
0.5670 0.0002 - 1.5672 - 0.0002
0.5671 0.0000 - 1.5671 - 0.0000

87
Example
Estimates of the root of: x-cos(x)=0.

0.60000000000000 Initial guess

0.74401731944598 1 correct digit
0.73909047688624 4 correct digits
0.73908513322147 10 correct digits
0.73908513321516 14 correct digits

88
Example
In estimating the root of: x-cos(x)=0, to
get more than 13 correct digits:

 4 iterations of Newton (x0=0.8)

 43 iterations of Bisection method (initial
interval [0.6, 0.8])
 5 iterations of Secant method
( x0=0.6, x1=0.8)

89