Rootfinding

Indian Institute of Information Technology,

Design & Manufacturing, Kancheepuram

Rootfinding
Fixed Point Iteration Method

Theorem
Let
1 g be a continuous function on [a, b]
2 g (α) , α > 1 continuous derivatives on (a, b)
Further, let p ∈ (a, b) be a fixed point of g , i.e., g (p) = p . If

g ′ (p) = g ′′ (p) = · · · = g α−1 = 0, but g (α) ̸= 0,

then there exists a δ > 0 such that for any x0 ∈ [p − δ, p + δ], the sequence xn+1 = g (xn )
converges to the fixed point p of order α with the asymptotic error constant

|en+1 | |g (α) (p)|

lim α
= .
n→∞ |en | α!

Rootfinding
Note:
Using Mean Value Theorem,

|xn − p| = |g (xn−1 ) − g (p)| = |g ′ (ξ)(xn−1 − p)|

= |g ′ (ξ)||xn−1 − p|
en = |g ′ (ξ)|en−1 = |g ′ (ξ)|2 en−2 = · · · = |g ′ (ξ)|n e0

Clearly, if |g ′ (ξ)| < 1, |g ′ (ξ)|n → 0 as n → ∞. So, xn → p as n → ∞

Rootfinding
Comparison of Fixed Point Method
f (x) = x 3 + x 2 − 3x − 3 = 0
Transform: x = gi (x), where

x3 + x2 − 3 3x + 3
g1 (x) = , g2 (x) = −1 +
3 r x2
p3 3 + 3x − x 2
g3 (x) = 3 + 3x − x 2 , g4 (x) =
x
x 3 + x 2 − 3x − 3
g5 (x) = x −
3x 2 + 2x − 3

g1′ (p) = 4.15

g2′ (p) = −3.15
g3′ (p) = −0.052
g4′ (p) = −0.577
g5′ (p) = 0, |g5′′ (p)| = 1.3
Rootfinding
Newton’s Method
Start with a initial value x0
1st Iteration:
y
Draw a tangent at (x0 , f (x0 ))
Hence, equation of the tangent:

y − f (x0 ) = f ′ (x0 )(x − x0 )

Say, 1st iteration is obtained by the

x-intercept x = x1 of the tangent f(x)
x*
Substitute x = x1 , y = 0 x2 x1 x0 x
−f (x0 ) = f ′ (x0 )(x1 − x0 )

which gives

f (x0 )
x1 = x0 −
f ′ (x0 )
Rootfinding
2nd Iteration:
Draw a tangent at (x1 , f (x1 ))
Hence, equation of the tangent: y − f (x1 ) = f ′ (x1 )(x − x1 )
Say, 2nd iteration is obtained by the x-intercept x = x2 of the tangent
Substitute x = x2 , y = 0

−f (x1 ) = f ′ (x1 )(x2 − x1 )

which gives

f (x1 )
x2 = x1 −
f ′ (x1 )

(n+1)-th Iteration:

f (xn )
xn+1 = xn − , n = 0, 1, 2, 3, . . .
f ′ (xn )

Rootfinding
Example

Find the root of the function f (x) = x 3 + 2x 2 − 3x − 1 on the interval (1, 2) by applying
Newton’s method.
Solution: Compute the derivative of f : f ′ (x) = 3x 2 + 4x − 3. By Newton’s method,

f (xn )
xn+1 = xn − , n = 0, 1, 2, 3, . . .
f ′ (xn )

Initialization: Say, x0 = 1.
1st Iteration: Compute f (x0 ) = f (1) = −1, f ′ (1) = 4. Hence

f (x0 ) −1
x1 = x0 − ′
=1− = 1.25
f (x0 ) 4

Rootfinding
2nd Iteration: Compute f (x1 4) = f (1.25) = 0.321, f ′ (1) = 6.6875. Hence

f (x1 ) 0.321
x2 = x1 − ′
=1− = 1.2009345794
f (x1 ) 6.6875

In a similar way,

f (x2 )
x3 = x2 − = 1.1986958411
f ′ (x2 )
f (x3 )
x4 = x3 − ′ = 1.1986912435
f (x3 )

Rootfinding
Influence of initial guess, x0

Rootfinding
Order of convergence
If p is simple the roots of f , f (p) = 0 and f ′ (p) ̸= 0
f (x)
In the view of Newton’s method, we have g (x) = x −
f ′ (x)
Clearly, g (p) = p
Derivatives:
f (x)f ′′ (x)
g ′ (x) =
[f ′ (x)]2
′′
f (x) f ′ (x)f ′′′ (x) − 2[f ′ (x)]2
g ′′ (x) = ′ + f (x)
f (x) [f ′ (x)]3
f ′′ (p)
At x = p, g ′ (p) = 0 and g ′′ (p) = ̸= 0
f ′ (p)
By the Theorem, Newton’s method has order of convergence 2 with
|en+1 | |g ′′ (p)| 1 f ′′ (p)
lim = = .
n→∞ |en |2 2! 2 f ′ (p)

Rootfinding
Newton’s method is one of the most powerful tools for finding roots, but it has some
disadvantages too:
Requires calculating derivatives of f . In many cases, it may not be easy for some complex
form function.

Newton’s method will fail in the cases where the derivative f ′ is zero.

Rootfinding
Newton’s method does not always converge

Consider the function:

 √
− r − x, x <r
f (x) =
 √
x − r, x ≥r

No amount of iteration brings us closer to

the root than our first guess.

Rootfinding
Newton’s method converges to a root, it may not be the root you have in
your mind

If you start too far away, Newton’s method may miss the root you want.

Rootfinding
Newton’s Method with roots of multiplicity > 1
Issue: f ′ (p) = 0,
Suppose x = p is a root that appears m-times (m ≥ 2), then f can be written as

f (x) = (x − p)m q(x), with q(p) ̸= 0.

Making Newton’s method as a fixed point iteration function: xn+1 = g (xn ), where

f (x)
g (x) = x −
f ′ (x)
(x − p)q(x)
=x−
(x − p)q ′ (x) + mq(x)

Its derivative
m(m − 1)q(x) + 2m(x − p)q ′ (x) + (x − p)2 q ′′ (x) q(x)
 
′
g (x) =  2
(x − p)q ′ (x) + mq(x)

Rootfinding
 1 m−1
It is clear that g (p) = p and g ′ (p) = 1 − = ̸= 0
m m
Since rate of convergence of fixed point iteration method is linear, Newton’s method
provides only linear convergence for roots of multiplicity > 1.
Can we make it better ? rate of convergence is 2!!!
1
Yes, we can do that by modifying g (x) = [f (x)] m
Applying Newton’s method on g
1
[f (xn )] m f (xn )
xn+1 = xn − n = xn − m ′
d 1
o
f (xn )
[f (x)] m
dx x=xn

How to find m ?
f (xn )
If we know m, suppose m = 2, we will use xn+1 = xn − 2
f ′ (xn )

Rootfinding
 f (xn )
If m is unknown, the how use xn+1 = xn − m ?
f ′ (xn )
Assign a new function u(x) = f (x)/f ′ (x), so

(x − p)q(x) f ′ (x)f ′ (x) − f (x)f ′′ (x)

u(x) = , u ′ (x) =
mq(x) + (x − p)q ′ (x) [f ′ (x)]2

If f (x) = 0 is having x = p as a root, then x = p is also root of u(x)

Applying Newton’s method on u(x):

u(xn ) f (xn )f ′ (xn )

xn+1 = xn − = x n −
u ′ (xn ) f ′ (xn )f ′ (xn ) − f (xn )f ′′ (xn )

Note that value of m is not used in the above formula (modified)

Rootfinding
Example: Newton method vs Modified Newton Method
Evaluate the multiple roots of f (x) = x 3 − 5x 2 + 7x − 3 with an initial guess x0 = 0.
(roots:1,1,3)

Newton method Modified Newton method

f (xn ) f (xn )f ′ (xn )

xn+1 = xn − xn+1 = xn −
f ′ (xn ) f ′ (x
n )f
′ (x ′′
n ) − f (xn )f (xn )

Rootfinding
Secant Method

Newton’s method is very powerful tool, but it has a major weakness:

need to know f ′ at each iteration
f (xn−1 ) − f (xn )
To address this, we substitute f ′ (xn ) by
xn−1 − xn

f (xn ) − f (x) xn−1 is near to xn f (xn−1 ) − f (xn )

f ′ (xn ) = lim −→ f ′ (xn ) ≈
x→xn xn − x xn−1 − xn

Hence, Secant method is given by

xn − xn−1
xn+1 = xn − f (xn )
f (xn ) − f (xn−1 )

Note: Secant method needs two initial guess x0 and x1

Only one function evaluation is required, where Newton’s method needs two functions
evaluation

Rootfinding
Successive Secants

Rootfinding
Example

Find the root of the function f (x) = x 3 + 2x 2 − 3x − 1 on the interval (1, 2) by applying
Secant method.
Solution: Taking initial guesses x0 = 2 and x1 = 1
1st Iteration:
x1 − x0 1−2
x2 = x1 − f (x1 ) = 1 − (−1) = 1.1
f (x1 ) − f (x0 ) −1 − 9
2nd Iteration:
x2 − x1 1.1 − 1
x3 = x2 − f (x2 ) = 1.1 − (−0.549) = 1.2217294900
f (x2 ) − f (x1 ) −0.549 − (−1)
Next 4 Iterations:
x4 = 1.1964853266
x5 = 1.198453684
x6 = 1.1986913364
x7 = 1.1986912435
Rootfinding
Newton’s method vs Secant method

On the same accuracy, Secant method achieved in 6 iterations, but Newton’s method
achieved in only 4 iterations
However, Secant method has six function evaluations, but Newton method has 8 function
evaluations
Thus, Secant method took more iterations than Newton’s method to achieve a given level
of accuracy, the Secant method used fewer function evaluations.

For another example, f (x) = tan(πx) − x − 6 with level of accuracy of 9th decimal places

Method Initial guess Total Iterations Total function evaluations

Newton’s Method x0 = 0.48 5 10
Secant method x0 = 4, x1 = 0.48 8 8

Rootfinding
Secant method converges at faster than a linear rate. (superlinear)
Do not combine the secant formula and write it in the form
f (xn )xn−1 − f (xn−1 )xn
xn+1 = ,
f (xn ) − f (xn−1 )

This has enormous loss of significance errors as compared with the earlier formulation.

Rootfinding
Newton’s method in the multivariable case
n equations and n unknowns:

f1 (x1 , x2 , . . . , xn ) = 0,
    
 f1 (x) x1
f2 (x1 , x2 , . . . , xn ) = 0,

 f2 (x)  x2 
⇒ f(x) = 0, where f =  .  , x =  . 
   
.. .  .. 
. 


 . 
fn (x) xn


fn (x1 , x2 , . . . , xn ) = 0.
 −1
Newton’s method: x(k+1) = x(k) − J(x(k) ) f(x(k) ), k = 0, 1, . . . , with initial guess
x(0) ∈ Rn
where J(x(k) ) is the Jacobian matrix at x = x(k) of size n × n such that
 ∂fi (k) 
J(x(k) ) = (x ) 1≤i,j≤n
∂xj
h i−1
J(x(k) ) is the inverse of the Jacobian matrix
Rootfinding
Example in 2D

Problem: Find a intersection points of the

circle x 2 + y 2 = 4 and the hyperbola
4x 2 − y 2 = 4.

Solution. Equivalently, solutions to the system

f1 (x, y ) = x 2 + y 2 − 4 = 0
f2 (x, y ) = 4x 2 − y 2 − 4 = 0

So,
  2
x + y2 − 4
   
f1 (x) x
f(x) = = ,x=
f2 (x) 4x 2 − y 2 − 4 y

Rootfinding
Example contd1.
Find the jacobian matrix
" #
∂f1 ∂f1  
∂x ∂y 2x 2y
J(x) = ∂f2 ∂f2 =
∂x ∂y
8x −2y

Find the inverse of the Jacobian matrix

   1 1 
 −1 1 −2y −2y
J(x) = = 10x
2
10x
1
−20xy −8x 2x 5y − 10y
 −1
Compute xk+1 = xk − J(xk ) f(xk )
 
5xk2 − 8
 xk − 10x 
    1 1
xk2 + yk2 − 4
  
xk+1 x
= k − 10xk 10xk
= k
2 1
5y 2 − 12 
2 2

yk+1 yk − 10y 4xk − yk − 4 
5yk k yk − k
10yk

Rootfinding
Example contd2.
   
x0 1
Consider the initial guess x0 = =
y0 1
1st Iteration: when k = 0
  
5x02 − 8 5 ∗ 12 − 8

 
x1  0x −  1−
 
1.3
= 10x 0 = 10 ∗ 1 
=
5y02 − 12  
 
y1 5 ∗ 12 − 12  1.7
y0 − 1−
10y0 10 ∗ 1

2nd Iteration: when k = 1

5 ∗ (1.3)2 − 8
   
5x12 − 8
 
 1x − 1.3 −  
x2 10x1   10 ∗ (1.3)  1.265385
= = =
5y 2 − 12  
  
y2 5 ∗ (1.7)2 − 12  1.555882
y1 − 1

1.7 −
10y1 10 ∗ (1.7)

Rootfinding
Example contd3.

3rd, 4th and 5th . . . iterations:

   
x3 1.26491115
= ,
y3 1.54920772
   
x4 1.26491106
= ,
y4 1.54919334
   
x5 1.26491106
= ,······
y5 1.54919334

Rootfinding
Problem Set
−5
1 Use Newton’s method to find solutions accurate within √ 10 on [−2, √−1]: (a)
2 −x −2x
f (x) = x − 2xe + e = 0 on [0, 1], (b) cos(x + 2) + x(x/2 + 2) = 0.
2 Use Secant method (x0 = 0.5, x1 = π/4) to find the solution cos x = x, and compare the
approximations with those given by Newton’s method with x0 = π/4.
3 Discuss the influence of x0 and x1 in Secant method.
4 Establish a comparison between Newton’s method and the Secant method by considering
the following function:

f (x) = x 3 + 2x 2 − 3x − 1, on (−3, −2).

5 Let f (x) = e x − x − 1. Show that f has a zero of multiplicity 2 at x = 0. Show that
Newton’s method with x0 = 1 converges to this zero but not quadratically. Then show
that modification of Newton’s method improves the rate of convergence.
6 In example of Newton’s 2D, take initial guesses (1, −1), (−1, −1), (−1, 1) and (0, 0), then
discuss the convergence of roots.
7 Use Newton’s method to find roots:
x 3 − 5x 2 + 2x − y + 13 = 0, x 3 + x 2 − 14x − y − 19 = 0.
Rootfinding