2: Root Finding

The problem is to solve f (x) = 0 for x when an explicit analytical solution is impossible. All methods compute a sequence
x , x , x , … that converges to the root x = r satisfying f (r) = 0 .
0 1 2

Bisection Method
The bisection method is conceptually the simplest method and almost always works. However, it is also the slowest method, and
most of the time should be avoided.
We first choose x and x such that x 0 1 0 < r < x1 . We say that x and x bracket the root. With f (r) = 0 , we want f (x
0 1 0) and
f (x ) to be of opposite sign, so that
1 f (x0 ) f (x1 ) < 0 . We then assign x to be the midpoint of x and x , that is
2 0 1 x2 =

(x + x ) /2 , or
1 0

x1 − x0
x2 = x1 − .
2

The sign of f (x ) is then determined, and the value of x is chosen as either the midpoint of x and x or as the midpoint of x
2 3 2 0 2

and x , depending on whether x and x bracket the root, or x and x bracket the root. The root, therefore, stays bracketed at all
1 2 0 2 1

times. The algorithm proceeds in this fashion and is typically stopped when the absolute value of the increment to the last best
approximation to the root (above, given by |x − x | /2) is smaller than some required precision.
1 0

−
Estimate √2 = 1.41421 … using x 0 = 1 and x 1 = 2

–
Now √2 is the zero of the function f (x) = x 2
−2 . We iterate with x 0 =1 and x 1 =2 . We have
2 −1 3
x2 = 2 − = = 1.5.
2 2

Now, f (x 2) = 9/4 − 2 = 1/4 > 0 so that x and x bracket the root. Therefore,
2 0

3
3 −1 5
2
x3 = − = = 1.25.
2 2 4

Now, f (x 3) = 25/16 − 2 = −7/16 < 0 so that x and x bracket the root. Therefore,
3 2

5 3
5 − 11
4 2
x4 = − = = 1.375,
4 2 8

and so on.

Newton’s Method
This is the fastest method, but requires analytical computation of the derivative of f (x) . If the derivative is known, then this
method should be used, although convergence to the desired root is not guaranteed.
We can derive Newton’s Method from a Taylor series expansion. We again want to construct a sequence x , x , x , … that 0 1 2

converges to the root x = r . Consider the x member of this sequence, and Taylor series expand f (x ) about the point x .
n+1 n+1 n

We have
′
f (xn+1 ) = f (xn ) + (xn+1 − xn ) f (xn ) + … .

To determine x n+1 , we drop the higher-order terms in the Taylor series, and assume f (x n+1 ) =0 . Solving for x n+1 , we have
f (xn )
xn+1 = xn − .
f ′ (xn )

Starting Newton’s Method requires a guess for x , to be chosen as close as possible to the root x = r .
0

−
Estimate √2 using x 0 = 1

2.1 https://math.libretexts.org/@go/page/93741
Again, we solve f (x) = 0 , where f (x) = x
2
−2 . To implement Newton’s Method, we use ′
f (x) = 2x . Therefore, Newton’s
Method is the iteration
2
xn − 2
xn+1 = xn − .
2xn

With our initial guess x 0 =1 , we have

−1 3
x1 = 1 − = = 1.5,
2 2
9
3 −2 17
4
x2 = − = = 1.416667,
2 3 12
2
17
−2
17 12
2 577
x3 = − = = 1.41426.
17
12 408
6

Secant Method
The Secant Method is second fastest to Newton’s Method, and is most often used when it is not possible to take an analytical
derivative of the function f (x). We write in place of f (x ), ′
n

f (xn ) − f (xn−1 )
′
f (xn ) ≈ .
xn − xn−1

Starting the Secant Method requires a guess for both x0 and x1 . These values need not bracket the root, but convergence is not
guaranteed.

−
Estimate √2 using x 0 = 1 and x 1 = 2

Again, we solve f (x) = 0 , where f (x) = x 2

−2 . The Secant Method iterates
2
(xn − 2) (xn − xn−1 )
xn+1 = xn −
2 2
xn − x
n−1

2
xn − 2
= xn − .
xn + xn−1

With x 0 =1 and x1 =2 , we have

4 −2 4
x2 = 2 − = = 1.33333
3 3
16
4 −2 21
9
x3 = − = = 1.4
4
3 +2 15
3

2
21
21 ( ) −2 174
15
x4 = − = = 1.41463.
21 4
15 + 123
15 3

A fractal from Newton’s Method

Consider the complex roots of the equation f (z) = 0 , where
3
f (z) = z − 1.

These roots are the three cubic roots of unity. With

i2πn
e = 1, n = 0, 1, 2, …

the three unique cubic roots of unity are given by

i2π/3 i4π/3
1, e ,e .

2.2 https://math.libretexts.org/@go/page/93741
With
iθ
e = cos θ + i sin θ,

–
and cos(2π/3) = −1/2, sin(2π/3) = √3/2, the three cubic roots of unity are
– –
1 √3 1 √3
r1 = 1, r2 = − + i, r3 = − − i.
2 2 2 2

The interesting idea here is to determine in the complex plane which initial values of z converge using Newton’s method to which
0

of the three cube roots of unity.

Newton’s method generalized to the complex plane is
3
zn − 1
zn+1 = zn − .
2
3zn

If the iteration converges to r , we color z red; r , blue; r , green. The result will be shown in lecture.
1 0 2 3

Order of convergence
Let r be the root and x be the n th approximation to the root. Define the error as
n

ϵn = r − xn .

If for large n we have the approximate relationship

p
| ϵn+1 | = k| ϵn | ,

with k a positive constant and p ≥ 1 , then we say the root-finding numerical method is of order p. Larger values of p correspond to
faster convergence to the root. The order of convergence of bisection is one: the error is reduced by approximately a factor of 2
with each iteration so that
1
| ϵn+1 | = | ϵn | .
2

We now find the order of convergence for Newton’s Method and for the Secant Method.

Newton’s Method
We start with Newton’s Method
f (xn )
xn+1 = xn − ,
f ′ (xn )

where the root r satisfies f (r) = 0 . Subtracting both sides from r, we have
f (xn )
r − xn+1 = r − xn + ,
f ′ (xn )

or
f (xn )
ϵn+1 = ϵn + .
f ′ (xn )

We use Taylor series to expand the functions f (x n) and f ′

(xn ) about the root r, using f (r) = 0 . We have

2.3 https://math.libretexts.org/@go/page/93741
 ′
1 2 ′′
f (xn ) = f (r) + (xn − r) f (r) + (xn − r) f (r) + … ,
2

′
1 ′′
2
= −ϵn f (r) + ϵn f (r) + … ; (2.1)
2

′ ′ ′′
1 2 ′′′
f (xn ) = f (r) + (xn − r) f (r) + (xn − r) f (r) + … ,
2

′ ′′
1 ′′′
2
= f (r) − ϵn f (r) + ϵn f (r) + … .
2

To make further progress, we will make use of the following standard Taylor series:
1 2
= 1 +ϵ+ϵ +… ,
1 −ϵ

which converges for |ϵ| < 1 . Substituting (2.21) into (2.20), and using (2.22) yields
f (xn )
ϵn+1 = ϵn +
′
f (xn )

′ 1 2 ′′
−ϵn f (r) + ϵn f (r) + …
2
= ϵn +
′ ′′ 1 2 ′′′
f (r) − ϵn f (r) + ϵn f (r) +…
2

1 2 ′′
−ϵn + ϵn f (r)
2
= ϵn + ′′
f (r)
1 − ϵn ′
+…
f (r)

′′ ′′
1 f (r) f (r)
2
= ϵn + (−ϵn + ϵn + …) (1 + ϵn + …)
′ ′
2 f (r) f (r)
′′ ′′
1 f (r) f (r)
2
= ϵn + (−ϵn + ϵn ( − ) + …)
′ ′
2 f (r) f (r)
′′
1 f (r)
2
=− ϵn + …
2 f ′ (r)

Therefore, we have shown that

2
| ϵn+1 | = k| ϵn |

as n → ∞ , with
′′
1 ∣ f (r) ∣
k = ∣ ∣,
2 ∣ f ′ (r) ∣

provided f ′
(r) ≠ 0 . Newton’s method is thus of order 2 at simple roots.

Secant Method
Determining the order of the Secant Method proceeds in a similar fashion. We start with
(xn − xn−1 ) f (xn )
xn+1 = xn − .
f (xn ) − f (xn−1 )

We subtract both sides from r and make use of

xn − xn−1 = (r − xn−1 ) − (r − xn )

= ϵn−1 − ϵn

and the Taylor series

1
′ 2 ′′
f (xn ) = −ϵn f (r) + ϵn f (r) + … ,
2
1
′ 2 ′′
f (xn−1 ) = −ϵn−1 f (r) + ϵ f (r) + … ,
n−1
2

2.4 https://math.libretexts.org/@go/page/93741
so that

′
1 ′′
2 2
f (xn ) − f (xn−1 ) = (ϵn−1 − ϵn ) f (r) + (ϵn − ϵ )f (r) + …
n−1
2

′
1 ′′
= (ϵn−1 − ϵn ) ( f (r) − (ϵn−1 + ϵn ) f (r) + …)
2

We therefore have
′ 1 2 ′′
−ϵn f (r) + ϵn f (r) + …
2
ϵn+1 = ϵn +
′ 1 ′′
f (r) − (ϵn−1 + ϵn ) f (r) + …
2
′′

1 f (r)
1− ϵn +…
2 1
1− ( ϵn−1 +ϵn )
2

= ϵn − ϵn +…
f ′′ (r)
′′ ′′
1 f (r) 1 f (r)
= ϵn − ϵn (1 − ϵn + …) (1 + (ϵn−1 + ϵn ) + …)
′ ′
2 f (r) 2 f (r)
′′
1 f (r)
=− ϵn−1 ϵn + …
′
2 f (r)

or to leading order
′′
1 ∣ f (r) ∣
| ϵn+1 | = ∣ ∣ | ϵn−1 | | ϵn | .
2 ∣ f ′ (r) ∣

The order of convergence is not yet obvious from this equation, but should be less than quadratic because | ϵn−1 | > | ϵn | . To
determine the scaling law we look for a solution of the form
p
| ϵn+1 | = k| ϵn | .

From this ansatz, we also have

p
| ϵn | = k| ϵn−1 | ,

and therefore
2
p+1 p
| ϵn+1 | = k | ϵn−1 | .

Substitution into (2.26) results in

′′
p
2 k ∣ f (r) ∣ p+1
p+1
k | ϵn−1 | = ∣ ∣ | ϵn−1 | .
2 ∣ f ′ (r) ∣

Equating the coefficient and the power of ϵ n−1 results in

′′
1∣f (r) ∣
p
k = ∣ ∣,
′
2 ∣ f (r) ∣

and
2
p = p + 1.

The order of convergence of the Secant Method, given by p , is therefore the positive root of the quadratic equation
p − p − 1 = 0 , or
2

–
1 + √5
p = ≈ 1.618,
2

which coincidentally is a famous irrational number that is called The Golden Ratio, and goes by the symbol Φ . We see that the
Secant Method has an order of convergence lying between the Bisection Method and Newton’s Method.

2.5 https://math.libretexts.org/@go/page/93741
This page titled 2: Root Finding is shared under a CC BY 3.0 license and was authored, remixed, and/or curated by Jeffrey R. Chasnov via source
content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request.

2.6 https://math.libretexts.org/@go/page/93741