CS 323 — Numerical analysis and computing, Spring 2025

Root finding: Newton’s method

Peng Zhang
2025-02-04
 Today’s plan
• Root-finding: Newton’s method

• Reference: [Atkinson-Han] Chapter 3.2

• Our first homework assignment:

• Deadline: Thursday (Feb. 13, 11:59 pm)

• The deadline is hard. No points after deadline.

 Root finding
• Given a function f : ℝ → ℝ, we want to find x* ∈ ℝ such that f(x*) = 0.

• x* is called a root or zero of f.

• Easy to solve analytically:

• f(x) = 3x - 2, f(x) = x2 + 2x - 3

• Hard to solve analytically:

• f(x) = x2 + ecos(x) - 3

• Last lecture: the bisection method

 Idea of Newton’s method
• Consider f : ℝ → ℝ with a root x*.
• We have an estimate of x*, denoted by x0.

• To improve on x0, consider the straight line

that is tangent to the graph of f at the point
(x0, f(x0)). If x0 is near x*, this line should
be nearly coincident with the graph of f
around x*. The root of the tangent line
(denoted by x1) should be near x*. Root of f Our estimate of x*

x*
Root of the
tangent line
 Formula for x1
f :
derivative of f

• The tangent line: y = f′(x0)(x − x0) + f(x0)

-

• Find x1: !

• Solve f′(x0)(x1 − x0) + f(x0) = 0.

f(x0) ↓
V
x
• 1 = x0 − f(x)
f′(x0)
(x .
-

Xo) = =

f(Xo)
=> X -Xo
, = =)

f(Xo)
Root of f Our estimate of x*
Cassume f(x) +
0)
x*
Root of the
tangent line
 Formula for x1
• The tangent line: y = f′(x0)(x − x0) + f(x0)
• Find x1:

• Solve f′(x0)(x1 − x0) + f(x0) = 0.

f(x0)
x
• 1 = x0 −
f′(x0)
• Since x1 is expected to be an improvement
over x0 as an estimate of x*, this process Root of f Our estimate of x*

can be repeated with x1 as the initial guess. x*

Root of the
tangent line
 General iteration formula
f(x0) f(x1)
x
• 1 = x0 − , x2 = x1 −
f′(x0) f′(x1)
• Repeating this process, we obtain a
sequence of numbers x1, x2, x3, x4, … that
we hope will approach the root x*.

• In general, these numbers are defined by

f(xn)
xn+1 = xn − , n = 0,1,2,…
f′(xn) Root of f Our estimate of x*

x*
• This is called Newton’s method. Root of the
tangent line
Example 3.2.1 in [Atkinson-Han]
6
• Find a root of f(x) = x − x − 1 using Newton’s method.
Example 3.2.1 in [Atkinson-Han]
6
• Find a root of f(x) = x − x − 1 using Newton’s method.

• What do we need?

• An initial guess x0 for the root

5
• Derivative: f′(x) = 6x − 1
Example 3.2.1 in [Atkinson-Han]
6
• Find a root of f(x) = x − x − 1 using Newton’s method.

• What do we need?

• An initial guess x0 for the root

5
• Derivative: f′(x) = 6x − 1
6
f(xn) xn − xn − 1
Iteration formula: xn+1 = xn − = x − , n = 0,1,2,…
• f′(xn)
n
6xn − 1
5

• Matlab
True root: 1.134724138…
Newton’s method converges faster
Example 3.2.2 in [Atkinson-Han]
• Consider a procedure that was used to carry out divisions on some early
computers. These computers had hardware for addition, subtraction, and
multiplication, but division had to be implemented by software. This
procedure is also used on some present-day supercomputers.

• Suppose we want to compute a/b.

• (Approximately) compute 1/b using Newton’s method.

• Multiply a and 1/b.

Example 3.2.2 in [Atkinson-Han]
• (Approximately) compute 1/b using Newton’s method.

• Convert the problem to a root-finding problem:

1
• Let f(x) = b − (assume b > 0). Then, 1/b is a root of f(x) .
x
le+ b -

y 0 = X 5
= =
Example 3.2.2 in [Atkinson-Han]
1
• Find the root of f(x) = b − (assume b > 0 ) using Newton’s method.
x
• What do we need?
Xn += Xn-(bxi-Xn)
• An initial guess x0 for the root
1 2xn-bX
=

• Derivative: f′(x) =
x 2 xxn
=
xn(2 -

bxn)
-
•
-
Iteration formula:

O
f(xn) b − 1/xn
xn+1 = xn − = xn − = xn (2 − bxn ), n = 0,1,2,…
f′(xn) 1/xn2
Example 3.2.2 in [Atkinson-Han]
1
• Find the root of f(x) = b − (assume b > 0 ) using Newton’s method.
x
• What do we need?
• An initial guess x0 for the root
1
• Derivative: f′(x) =
x 2
Only has multiplication and subtraction,
• Iteration formula: no division
f(xn) b − 1/xn
xn+1 = xn − = xn − = xn (2 − bxn ), n = 0,1,2,…
f′(xn) 1/xn2
Example 3.2.2 in [Atkinson-Han]
1
• Initial guess x0 for the root of f(x) = b − (assume b > 0 ). Of course, we
x
should have x0 > 0.
relative error
↑ 1/b − x
2 n
• Claim. Rel(xn+1 ) = Rel(xn ) , where rel(xn ) = is the relative
1/b
error when considering xn as an approximation to the root 1/b.

If.
=
Rel(Xuy) X

-ex by
+
=
Example 3.2.2 in [Atkinson-Han]
1
• Initial guess x0 for the root of f(x) = b − (assume b > 0 ). Of course, we
x
should have x0 > 0.
Rel (Xn 14
(Rellxn-1) (2 1/b − x
= -

2 n
• Claim. Rel(xn+1) = Rel(xn) , where rel(xn) = is the relative
-
1/b
error when considering xn as an approximation to the root 1/b.
n
2
• So, Irel(xn) 111
= rel(x0) for n = 1,2,…

• If we choose x0 such that | Rel(x0) | < 1, then Rel(xn) → 0 as n → ∞

-

=>
Example 3.2.2 in [Atkinson-Han]
1
• Initial guess x0 for the root of f(x) = b − (assume b > 0 ). Of course, we
x
should have x0 > 0.

• If we choose x0 such that | Rel(x0) | < 1, then Rel(xn) → 0 as n → ∞

• The condition | Rel(x0) | < 1 means and reduces to

the equivalent condition
E bX
- y < l -

E) -
2 -

bx o

- 0x7
Example 3.2.2 in [Atkinson-Han]
• Convergence condition:

=
y
Example 3.2.2 in [Atkinson-Han]
2
• The formula Rel(xn+1) = Rel(xn) shows the convergence is very rapid,
once we have a somewhat accurate initial guess. For example,
| Rel(x0) | = 0.1 (10% error in x0), then
 Convergence
• Assumption on f:

• Has two continuous derivatives for all x in some interval around x* (root)

• f′(x*) ≠ 0 (aka, the graph of y = f(x) is not tangent to the x-axis when the
graph intersects it at x = x*; it implies that x* is a simple root)

• Since f′ is continuous, f′(x) ≠ 0 for all x near x*

• The case f′(x*) = 0 is treated in [AH] Chapter 3.5 (won’t cover in class)
 Convergence
straight line :
Y = fixn)(X- Xn) + f(xn)

/
(x*) (x* (x*
-
O

x* (x*)

I x* (x*

Recall Newton's iteration : Xu =

.
Xn-
Xn)2
*
o
=> = X -

Xn + 1 + (x
*
-
 2f(Xn)

Convergence
f(xn)

-
Recall that xn+1 = xn − . We have
f′(xn)
x* x*

x*

x* x*
-
 Convergence
f(xn)
Recall that xn+1 = xn − . We have
f′(xn)
x* x*

x*

x* x*

The error in xn+1 is nearly proportional to the square of the error in xn.
When the error in x0 is sufficiently small, the error in the succeeding iterations
will decrease very rapidly.
 Example 3.2.1 in [Atkinson-Han]
6
• Find a root of f(x) = x − x − 1 using Newton’s method.

• Newton’s method converges very rapidly.

5
• f′(x) = 6x − 1 X
*

t
4
• f′′(x) = 30x

Xn)( =
* *
x -

xn+ 1
=
(X -

-
Example 3.2.3 in [Atkinson-Han]
*

(x*) (x*) 4 x = 1 ,
13472438
..

(x*) 6(x*)5

x* x*

x*

x*

x*
 Convergence
• Recall x* x*

• If we assume xn is near the root,

(x*)
(x*)
-

x* x*

x* x*
 Convergence
x*

x* x*

x*

x*
-

(x*)
x*
(x*)
 Find x0
• If | M | is large, then x0 will have to be chosen very close to the root to
obtain convergence.

• The choice of x0 is very important for Newton’s method. Unfortunately,

there is no single strategy that is always effective in choosing x0.

• In most instances, a choice of x0 arises from the physical situation that

led to the root-finding problem.

• In other instances, graphing y = f(x) will probably be needed, possibly

combined with the bisection method for a few iterations.
Linear vs quadratic convergence
• Let En := | x* − xn |
2
• Newton’s method: En+1 ≈M⋅ En x* x*

• Quadratic convergence (En+1 is quadratic in En)

1
• Bisection method: En+1 ≤ ⋅ E n
2
• Linear convergence (En+1 is linear in En)

• If we have the same En ≪ 1 for Newton and bisection methods, the error
of Newton in iteration n+1 is much smaller than that of bisection.
True root: 1.134724138…
Newton’s method converges faster
 Comparison
• Newton’s method • Bisection method

• Need f to be both continuous • Need f to be continuous

and differentiable
• Linear convergence
• Quadratic convergence
• Don’t need f’ at each iteration
• Need f’ at each iteration
Stopping criteria for Newton’s method
• Our goal is to find xn such that | x* − xn | < tol (pre-specified tolerance). Issue:
don’t know x*

• By the mean-value theorem:

• f(xn) − f(x*) = f′(c)(xn − x*) for some c between x* and xn

-

B f(xn) f(xn)
Since f(x*) = 0, we have x* − xn = − ≈ − = x − x
• f′(c) f′(xn)
n+1 n

p
Newton's iteration
Stopping criteria for Newton’s method
• Our goal is to find xn such that | x* − xn | < tol (pre-specified tolerance). Issue:
don’t know x*

• By the mean-value theorem:

• f(xn) − f(x*) = f′(c)(xn − x*) for some c between x* and xn

f(xn) f(xn)
Since f(x*) = 0, we have x* − xn = − ≈ − = x − x
• f′(c) f′(xn)
n+1 n

• Stop the iterations when | xn+1 − xn | < tol

-

• Usually fairly accurate, not good when f′(x*) = 0

Stopping criteria for Newton’s method

*
X
X