1.

5 Root-Finding without Derivatives | 61

1.5 ROOT-FINDING WITHOUT DERIVATIVES

Apart from multiple roots, Newtons Method converges at a faster rate than the bisection
and FPI methods. It achieves this faster rate because it uses more informationin partic-
ular, information about the tangent line of the function, which comes from the functions
derivative. In some circumstances, the derivative may not be available.
The Secant Method is a good substitute for Newtons Method in this case. It replaces the
tangent line with an approximation called the secant line, and converges almost as quickly.
Variants of the Secant Method replace the line with an approximating parabola, whose
axis is either vertical (Mullers Method) or horizontal (inverse quadratic interpolation). The
section ends with the description of Brents Method, a hybrid method which combines the
best features of iterative and bracketing methods.

1.5.1 Secant Method and variants

The Secant Method is similar to the Newtons Method, but replaces the derivative by a
difference quotient. Geometrically, the tangent line is replaced with a line through the two
last known guesses. The intersection point of the secant line is the new guess.
An approximation for the derivative at the current guess xi is the difference quotient

f (xi ) f (xi1 )
.
xi xi1

A straight replacement of this approximation for f (xi ) in Newtons Method yields the
Secant Method.

Secant Method

x0 , x1 = initial guesses
f (xi )(xi xi1 )
xi+1 = xi for i = 1, 2, 3, . . . .
f (xi ) f (xi1 )

Unlike Fixed-Point Iteration and Newtons Method, two starting guesses are needed to
begin the Secant Method.
It can be shown that under the assumption that the Secant Method converges to r and
f (r)  = 0, the approximate error relationship
 
 f (r) 
ei+1   ei ei1

2f (r)

holds and that this implies that

 
 f (r) 1
ei+1   ei ,
2f (r)

where = (1 + 5)/2 1.62. (See Exercise 6.) The convergence of the Secant Method
to simple roots is called superlinear, meaning that it lies between linearly and quadratically
convergent methods.
62 | CHAPTER 1 Solving Equations

x
x0 x2 1
x3 x1

Figure 1.11 Two steps of the Secant Method. Illustration of Example 1.16. Starting
with x0 = 0 and x1 = 1, the Secant Method iterates are plotted along with the secant
lines.

 EXAMPLE 1.16 Apply the Secant Method with starting guesses x0 = 0, x1 = 1 to nd the root of f (x) =
x 3 + x 1.
The formula gives
(xi3 + xi 1)(xi xi1 )
xi+1 = xi . (1.34)
xi3 + xi (xi1
3 +x
i1 )

Starting with x0 = 0 and x1 = 1, we compute

(1)(1 0) 1
x2 = 1 =
1+10 2
1 3 (1/2 1) 7
x3 = 8 3 = ,
2 8 1 11
as shown in Figure 1.11. Further iterates form the following table:

i xi
0 0.00000000000000
1 1.00000000000000
2 0.50000000000000
3 0.63636363636364
4 0.69005235602094
5 0.68202041964819
6 0.68232578140989
7 0.68232780435903
8 0.68232780382802
9 0.68232780382802 

There are three generalizations of the Secant Method that are also important. The
Method of False Position, or Regula Falsi, is similar to the Bisection Method, but where
the midpoint is replaced by a Secant Methodlike approximation. Given an interval [a, b]
that brackets a root (assume that f (a)f (b) < 0), dene the next point
f (a)(a b) bf (a) af (b)
c=a =
f (a) f (b) f (a) f (b)
as in the Secant Method, but unlike the Secant Method, the new point is guaranteed to
lie in [a, b], since the points (a, f (a)) and (b, f (b)) lie on separate sides of the x-axis.
 1.5 Root-Finding without Derivatives | 63

The new interval, either [a, c] or [c, b], is chosen according to whether f (a)f (c) < 0 or
f (c)f (b) < 0, respectively, and still brackets a root.

Method of False Position

Given interval [a, b] such that f (a)f (b) < 0
for i = 1, 2, 3, . . .
bf (a) af (b)
c=
f (a) f (b)
if f (c) = 0, stop, end
if f (a)f (c) < 0
b=c
else
a=c
end
end

The Method of False Position at rst appears to be an improvement on both the Bisection
Method and the Secant Method, taking the best properties of each. However, while the
Bisection Method guarantees cutting the uncertainty by 1/2 on each step, False Position
makes no such promise, and for some examples can converge very slowly.

 EXAMPLE 1.17 Apply the Method of False Position on initial interval [1, 1] to nd the root r = 0 of
f (x) = x 3 2x 2 + 32 x.
Given x0 = 1, x1 = 1 as the initial bracketing interval, we compute the new point

x1 f (x0 ) x0 f (x1 ) 1(9/2) (1)1/2 4

x2 = = = .
f (x0 ) f (x1 ) 9/2 1/2 5

Since f (1)f (4/5) < 0, the new bracketing interval is [x0 , x2 ] = [1, 0.8]. This completes
the rst step. Note that the uncertainty in the solution has decreased by far less than a factor
of 1/2. As Figure 1.12(b) shows, further steps continue to make slow progress toward the
root at x = 0.

y y
1 x4 1 x4
x x
1 x3 x2 1 1 x2 1
x0 x1 x0 x3 x1
1 1
2 2
3 3
4 4
5 5
(a) (b)

Figure 1.12 Slow convergence in Example 1.17. Both the (a) Secant Method and (b)
Method of False Position converge slowly to the root r = 0. 

Mullers Method is a generalization of the Secant Method in a different direction.

Instead of intersecting the line through two previous points with the x-axis, we use three pre-
vious points x0 , x1 , x2 , draw the parabola y = p(x) through them, and intersect the parabola
64 | CHAPTER 1 Solving Equations

with the x-axis. The parabola will generally intersect in 0 or 2 points. If there are two inter-
section points, the one nearest to the last point x2 is chosen to be x3 . It is a simple matter of
the quadratic formula to determine the two possibilities. If the parabola misses the x-axis,
there are complex number solutions. This enables software that can handle complex arith-
metic to locate complex roots. We will not pursue this idea further, although there are several
sources in the literature that follow this direction.
Inverse Quadratic Interpolation (IQI) is a similar generalization of the Secant
Method to parabolas. However, the parabola is of form x = p(y) instead of y = p(x),
as in Mullers Method. One problem is solved immediately: This parabola will intersect the
x-axis in a single point, so there is no ambiguity in nding xi+3 from the three previous
guesses, xi , xi+1 , and xi+2 .
The second-degree polynomial x = P (y) that passes through the three points (a, A),
(b, B), (c, C) is

(y B)(y C) (y A)(y C) (y A)(y B)

P (y) = a +b +c . (1.35)
(A B)(A C) (B A)(B C) (C A)(C B)

This is an example of Lagrange interpolation, one of the topics of Chapter 3. For now, it
is enough to notice that P (A) = a, P (B) = b, and P (C) = c. Substituting y = 0 gives a
formula for the intersection point of the parabola with the x-axis. After some rearrangement
and substitution, we have

r(r q)(c b) + (1 r)s(c a)

P (0) = c , (1.36)
(q 1)(r 1)(s 1)

where q = f (a)/f (b), r = f (c)/f (b), and s = f (c)/f (a).

For IQI, after setting a = xi , b = xi+1 , c = xi+2 , and A = f (xi ), B = f (xi+1 ),
C = f (xi+2 ), the next guess xi+3 = P (0) is

r(r q)(xi+2 xi+1 ) + (1 r)s(xi+2 xi )

xi+3 = xi+2 , (1.37)
(q 1)(r 1)(s 1)

where q = f (xi )/f (xi+1 ), r = f (xi+2 )/f (xi+1 ), and s = f (xi+2 )/f (xi ). Given three ini-
tial guesses, the IQI method proceeds by iterating (1.37), using the new guess xi+3 to replace
the oldest guess xi . An alternative implementation of IQI uses the new guess to replace one
of the previous three guesses with largest backward error.
Figure 1.13 compares the geometry of Mullers Method with Inverse Quadratic Inter-
polation. Both methods converge faster than the Secant Method due to the higher-order
interpolation. We will study interpolation in more detail in Chapter 3. The concepts of the
Secant Method and its generalizations, along with the Bisection Method, are key ingredients
of Brents Method, the subject of the next section.

1.5.2 Brents Method

Brents Method [Brent, 1973] is a hybrid methodit uses parts of solving techniques
introduced earlier to develop a new approach that retains the most useful properties of each.
It is most desirable to combine the property of guaranteed convergence, from the Bisection
Method, with the property of fast convergence from the more sophisticated methods. It was
originally proposed by Dekker and Van Wijngaarden in the 1960s.
The method is applied to a continuous function f and an interval bounded by a and b,
where f (a)f (b) < 0. Brents Method keeps track of a current point xi that is best in the
sense of backward error, and a bracket [ai , bi ] of the root. Roughly speaking, the Inverse
 1.5 Root-Finding without Derivatives | 65

Quadratic Interpolation method is attempted, and the result is used to replace one of xi , ai , bi
if (1) the backward error improves and (2) the bracketing interval is cut at least in half. If not,
the Secant Method is attempted with the same goal. If it fails as well, a Bisection Method
step is taken, guaranteeing that the uncertainty is cut at least in half.

xIQI
x0 x2 x x1 x
M

Figure 1.13 Comparison of Mullers Method step with Inverse Quadratic Iteration
step. The former is determined by an interpolating parabola y = p(x); the latter, by an
interpolating parabola x = p(y).

Matlabs command fzero implements a version of Brents Method, along with a

preprocessing step, to discover a good initial bracketing interval if one is not provided by
the user. The stopping criterion is of a mixed forward/backward error type. The algorithm
terminates when the change from xi to the new point xi+1 is less than 2mach max(1, xi ), or
when the backward error |f (xi )| achieves machine zero.
The preprocessing step is not triggered if the user provides an initial bracketing
interval. The following use of the command enters the function f (x) = x 3 + x 1 and
the initial bracketing interval [0, 1] and asks Matlab to display partial results on each
iteration:

>> f=@(x) x3+x-1;

>> fzero(f,[0 1],optimset(Display,iter))

Func-count x f(x) Procedure

1 0 -1 initial
2 1 1 initial
3 0.5 -0.375 bisection
4 0.636364 -0.105935 interpolation
5 0.684910 0.00620153 interpolation
6 0.682225 -0.000246683 interpolation
7 0.682328 -5.43508e-007 interpolation
8 0.682328 1.50102e-013 interpolation
9 0.682328 0 interpolation
Zero found in the interval: [0, 1].

ans=

0.68232780382802

Alternatively, the command

>> fzero(f,1)

looks for a root of f (x) near x = 1 by rst locating a bracketing interval and then applying
Brents Method.
66 | CHAPTER 1 Solving Equations

1.5 Exercises
1. Apply two steps of the Secant Method to the following equations with initial guesses x0 = 1
and x1 = 2. (a) x 3 = 2x + 2 (b) ex + x = 7 (c) ex + sin x = 4
2. Apply two steps of the Method of False Position with initial bracket [1, 2] to the equations of
Exercise 1.

3. Apply two steps of Inverse Quadratic Interpolation to the equations of Exercise 1. Use initial
guesses x0 = 1, x1 = 2, and x2 = 0, and update by retaining the three most recent
iterates.
4. A commercial sher wants to set the net at a water depth where the temperature is 10 degrees
C. By dropping a line with a thermometer attached, she nds that the temperature is 8 degrees
at a depth of 9 meters, and 15 degrees at a depth of 5 meters. Use the Secant Method to
determine a best estimate for the depth at which the temperature is 10.
5. Derive equation (1.36) by substituting y = 0 into (1.35).
6. If the Secant Method converges to r, f (r) = 0, and f (r) = 0, then the approximate error
relationship ei+1 |f (r)/(2f (r))|ei ei1 can be shown to hold. Prove that if in addition

limi ei+1 /ei exists and is nonzero for some > 0, then = (1 + 5)/2 and
ei+1 |(f (r)/2f (r))|1 ei .
7. Consider the following four methods for calculating 21/4 , the fourth root of 2. (a) Rank
them for speed of convergence, from fastest to slowest. Be sure to give reasons for your
ranking.
(A) Bisection Method applied to f (x) = x 4 2
(B) Secant Method applied to f (x) = x 4 2
x 1
(C) Fixed Point Iteration applied to g(x) = + 3
2 x
x 1
(D) Fixed Point Iteration applied to g(x) = + 3
3 3x
(b) Are there any methods that will converge faster than all above suggestions?

1.5 Computer Problems

1. Use the Secant Method to nd the (single) solution of each equation in Exercise 1.
2. Use the Method of False Position to nd the solution of each equation in Exercise 1.
3. Use Inverse Quadratic Interpolation to nd the solution of each equation in Exercise 1.

4. Set f (x) = 54x 6 + 45x 5 102x 4 69x 3 + 35x 2 + 16x 4. Plot the function on the
interval [2, 2], and use the Secant Method to nd all ve roots in the interval. To which of the
roots is the convergence linear, and to which is it superlinear?
5. In Exercise 1.1.6, you were asked what the outcome of the Bisection Method would be for
f (x) = 1/x on the interval [2, 1]. Now compare that result with applying fzero to the
problem.
6. What happens if fzero is asked to nd the root of f (x) = x 2 near 1 (do not use a
bracketing interval)? Explain the result. (b) Apply the same question to f (x) = 1 + cos x
near 1.