CS 323 — Numerical analysis and computing, Spring 2025

Root-finding: Fixed-point
iteration
Peng Zhang
2025-02-14
 Today’s plan
• Root-finding: fixed point iteration

• Reference: [Atkinson-Han] Chapter 3.4;

• Book: Numerical methods: design, analysis, and computer

implementation of algorithms, by Greenbaum and Chartier, Chapter 4.5
Thi 45 2
.
.

for the 2nd sufficient

condition for convergence
 Root finding
• Given a function f : ℝ → ℝ, we want to find x* ∈ ℝ such that f(x*) = 0.

• x* is called a root or zero of f.

• Easy to solve analytically:

• f(x) = 3x - 2, f(x) = x2 + 2x - 3

• Hard to solve analytically:

• f(x) = x2 + ecos(x) - 3

• We have covered: bisection method, Newton’s method, secant method

 Root finding
• Recall Newton’s method:

• start with an initial guess x0

f(xn)
iteratively improve our guess: xn+1 = xn − , for n ≥ 0
• f′(xn)
~ def
=
g(xN)
 Root finding
• Recall Newton’s method:

• start with an initial guess x0

f(xn)
iteratively improve our guess: xn+1 = xn − , for n ≥ 0
• f′(xn)

• In general, we can define any function g : ℝ → ℝ, and find

xn+1 = g(xn), for n ≥ 0
 Motivational example
2
• Consider solving the equation x − 5 = 0 for the root
x* = 5 = 2.2361...

• Four iteration methods:

 Motivational example
2
• Consider solving the equation x − 5 = 0 for the root
x* = 5 = 2.2361...

• Four iteration methods: Yo .

X.
,
Xz , . .

x = 5+ x x
All four iterations have the property:
-

If the sequence x0, x1, x2, … has a

limit α, then α is one of the roots.
C = 1+ x
ja
Replace xn, xn+1 with α, solving the
-

equations, we get α =± 5.
d= E(x 5)()x E
+
=
 Motivational example
2
• Consider solving the equation x − 5 = 0 for the root
x* = 5 = 2.2361...

• Four iteration methods:

 Motivational example
2
• Consider solving the equation x − 5 = 0 for the root
x* = 5 = 2.2361...

• Four iteration methods:

Question: Under what conditions does

the sequence converge to a limit?
 Motivational example
ag(x) =
5 + X -

x2

• All the four iterations have the form xn+1 = g(xn)

for a continuous function g.
 Motivational example
• All the four iterations have the form xn+1 = g(xn)
for a continuous function g.
-

• If xn converges to α, then
lim Xn + lim
1 =
g(Xn)
n n-
>
-

9([MXn) (1 :
g is continuous)
=> d =
g(x)
 Motivational example
• All the four iterations have the form xn+1 = g(xn)
for a continuous function g.

• If xn converges to α, then

lim xn+1 = lim g(xn)

n→∞ n→∞

α = g(α)

• α is a solution of x = g(x), and α is called a fixed

point of g.
 More examples
a fixed point X X
g(x)
Iteration: xn+1 = g(xn), n ≥ 0
: =

• Geometrically, fixed points lie at the intersection of the line y = x and the
graph of y = g(x).

= X -

M
(Xo ,
g(Xo))
Y g(X)
= ·
(Xi , XI)

X5 X4
 Sufficient Condition for Convergence
iteration : X =
g(Xn) , #2 o

• Theorem. Assume that g(x), g′(x) are continuous and | g′(x) | < 1 in
some interval [x* − δ, x* + δ] centered about a fixed point x* of g. If x0
is in this interval, then the fixed point iteration xn+1 = g(xn), n ≥ 0
converges to x*.

[ 2
x* *S
X
Sufficient Condition for Convergence
ll -

2x(x = 0xX)

• Theorem. Assume that g(x), g′(x) are continuous and | g′(x) | < 1 in
some interval [x* − δ, x* + δ] centered about a fixed point x* of g. If x0
[5-8 55 S
is in this interval, then the fixed point iteration xn+1 = g(xn), n ≥ 0
.

converges to x*. [1 :
g(x) =
5+ X -
x2

g((x) =
1 -

2x
*
X =
5 = 2 .
2361 ...

19'125) 1 =
11 5 -4)
-
=
> 1
Xo
[2 :
g(x) = g'(x =
-

*
(g(x))) =>
(14) = X +
(5 3)
,
+

13 :
g(x) = 1 + x =

jxz UK & .
-

55)
 g'(x) = 1 -

EX .
19ix , / <
1, = 11 -

Ex)4
Sufficient Condition for Convergence
=> -

11 -

EX() = 0 < X 5
,
[E -
1 , F +]
Xo 2 5
Theorem. Assume that g(x), g′(x) are continuous and | g′(x) | < 1 in
=

•
.

some interval [x* − δ, x* + δ] centered about a fixed point x* of g. If x0

is in this interval, then the fixed point iteration xn+1 = g(xn), n ≥ 0
converges to x*.
&

If Xn+ 1 =
9(Xn) = g(X ) *
+ g'(n)(Xn -

X
*
) where In is
,

1 :
*

X is a fixed *
between Xn ,
X
*

= x +
g'(n)(Xn *
X )
point)
-

=> Xn + X
g((n)(Xn *
-

~ -
-
X
)
def en + def
= en
Sufficient Condition for Convergence
-

= en =
9 (n) en

consider
lem 1
+
=
19 n))-len

Xo[x* 8
#
n = 0
.
,
x*+ 5)

(e
=>
. ) max Igia)) ·
1eol IX.-x 1- (X
*
.
-

**
ce[X* 8 .
*
x+ 8]
-

-1
*
=> X , e[X -

8 x
*
+ S]
,
In . if Xne[X
*

general -
8 .
*** 5)

=>
/en + / = max 1g)1 ·

Jen
ce[X * 6 .
x +
*
6]
-

def M M <
/

=> Lentil a
M **, e.
~
-
> constant
=> **
As n >
-

a
,
M 12 . >
-
0
.

=>
len1 0
limit, al
In the lim
n> -
- (en)
=
9'(x *
)
Another Sufficient Condition for Convergence

• In fact, g′(x) doesn’t need to exist in order to have convergence.

• The fixed-point iteration will converge if g is a contraction; that is, if there

-

exists a constant L < 1 such that

X X
| g(x) − g(y) | ≤ L | x − y | , ∀x, 3y ∈ ℝ
*
X

XI contraction

• Theorem. If g is a contraction, then it has a unique fixed point x* and

=
the iteration xn+1 = g(xn) converges to x* from any x0.

g,x
X

not contraction
Rough idea :

(ax =

Xn) = 19(Xn) -

g(Xn 1)) -
*
G(Xn -

Xa ) ,

n21

"
=> (Xn +
-

Xn) = ( (X ,
-

Xo)

See the next page Theorem 4.5.2 for a rigorous proof for this theorm.
 (b) Next consider . In this case, |φ′2(x)| = /(x + 6)3/2,
which is less than 1 for x > 21/3 − 6 ≈ −4.74. Since we know that there
is a fixed point x* somewhere in the interval [1, 2], the distance from x0
= 1.5 to this fixed point is at most 0.5; hence the interval [x* − 0.5, x* +
0.5] ⊂ [0.5, 2.5] contains x0 and is an interval on which |φ′2(x)| < 1.
Therefore convergence is guaranteed. One also could establish
convergence by noting that φ2 maps the interval [1, 2] that contains a
fixed point to the interval [1, ] ⊂ [1, 2]. Therefore, by the
discussion after theorem 4.5.1, the iteration must converge.
(c) Finally consider . In this case |φ′3(x)| = (x2/4) ×
. It can be checked that this is less than 1 for x less than
about 1.6, but if we know only that there is a fixed point in the interval
[1, 2], we cannot conclude that fixed point iteration applied to φ3 will
converge. Since we know from running fixed point iteration with φ2,
however, that the fixed point x* is at about 1.06, we can again conclude
that in the interval [x* − 0.5, x* + 0.5], which contains the initial guess
x0 = 1.5, we have |φ′3(x)| < 1; hence the fixed point iteration will
converge to x*.

Actually, φ′(x*) need not exist in order to have convergence. The iteration
(4.21) will converge to a fixed point x* if φ is a contraction; that is, if there
exists a constant L < 1 such that for all x and y

Theorem 4.5.2. If φ is a contraction (on all of R), then it has a unique fixed
point x* and the iteration xk+1 = φ(xk) converges to x* from any x0.

Proof. We will show that the sequence is a Cauchy sequence and

hence converges. To see this, let j and k be positive integers with k > j and
use the triangle inequality to write

Each difference |xm − xm−1| can be written as |φ(xm−1) − φ(xm−2)|, which is

bounded by L|xm−1 − xm−2|, where L < 1 is the constant in (4.22). Repeating
this argument for |xm−1 − xm−2|, we find that |xm − xm−1| ≤ L2|xm−2 − xm−3|, and
continuing in this way gives |xm − xm−1| ≤ Lm−1|x1 − x0|. Making these
substitutions in (4.23) we find

If k and j are both greater than or equal to some positive integer N, we will
have

and this quantity goes to 0 as N → ∞. This shows that the sequence xk, k = 0,
1, . . . is a Cauchy sequence and hence converges.
To show that xk converges to a fixed point of φ, we first note that the fact
that φ is a contraction implies that it is continuous. Hence φ(limk→∞ xk) =
limk→∞ φ(xk). Letting x* denote limk→∞ xk, we have

so that the limit of the sequence xk is indeed a fixed point. Finally, if y* is also
a fixed point, then since φ is a contraction, we must have
should be <=
|x* − y*| = |φ(x*) − φ(y*)| − L|x* − y*|,

where L < 1. This can hold only if y* = x*, so the fixed point is unique.
Theorem 4.5.2 could be modified to assume only that φ is a contraction on
some interval [a, b], where φ maps [a, b] into itself. The conclusion would
then be that there is a unique fixed point x* in [a, b] and that if fixed point
iteration is started with an initial guess x0 ∈ [a, b], then it converges to x*.

4.6 FRACTALS, JULIA SETS, AND MANDELBROT