1 Error Analysis for Iterative Methods

We will investigate the order of convergence of functional iteration schemes.

1.1 Order of Convergence

∞
Definition: Suppose {pn }n=0 is a sequence that converges to p, with pn 6= p
for all n. If positive constants λ and α exist with

|pn+1 − p|
lim α =λ
n→∞ |pn − p|
∞
then {pn }n=0 converges to p of order α, with asymptotic error constant λ.

In general, a sequence with a high order of convergence converges more

rapidly than a sequence with a lower order.

• If α = 1, the sequence is linearly convergent

• If α = 2, the sequence is quadratically convergent

1
Example: Show that the sequence pn = n2 , n ≥ 1 converges linearly to
p = 0.

Solution:
1
|pn+1 − 0| (n+1)2
lim = lim 1 =1
n→∞ |pn − 0| n→∞
n2
So pn → 0 linearly.

NOTE: Quadratically convergent sequences are expected to converge much

quicker than those that converge only linearly.
∞
Illustration: Suppose that {pn }n=0 is linearly convergent to 0 with

|pn+1 |
lim = 0.5
n→∞ |pn |

∞
and that {epn }n=0 is quadratically convergent to 0 with the same asymptotic
error constant,
|e
pn+1 |
lim 2 = 0.5
n→∞ |e pn |
For simplicity we assume that for each n we have

|pn+1 | |e
pn+1 |
≈ 0.5 and 2 ≈ 0.5
|pn | |e
pn |

1
For the linearly convergent scheme, this means that
2 n
|pn − 0| = |pn | ≈ 0.5 |pn−1 | ≈ (0.5) |pn−2 | ≈ ... ≈ (0.5) |p0 |

whereas the quadratically convergent procedure has

h i2
2 2 3 4
|e
pn − 0| = |e
pn | ≈ 0.5 |e
pn−1 | ≈ (0.5) 0.5 |e pn−2 | = (0.5) |e
pn−2 |
h i4
3 2 7 8
≈ (0.5) 0.5 |e pn−3 | = (0.5) |e
pn−3 |
2n −1 2n
≈ ... ≈ (0.5) |e
p0 |

If |p0 | = |e
p0 | = 1

n 2n −1
n Lineer Convergence: (0.5) → 0 Quadratic Convergence: (0.5) →0
1 0.5 0.5
2 0.25 0.125
3 0.125 0.0078125
4 0.0625 0.000030518
5 0.03125 4.6566x10−10
6 0.015625 1.0842x10−19
7 0.0078125 5.8775x10−39
−38
The quadratically convergent sequence is within 10 of 0 by the seventh
term. At least 126 terms are needed to ensure this accuracy for the linearly
convergent sequence.

Theorem: Let g ∈ C[a, b] be such that g(x) ∈ [a, b], for all x ∈ [a, b].
Suppose, in addition, that g 0 is continuous on (a, b) and a positive constant
k < 1 exists with
|g 0 (x)| ≤ k forall x ∈ (a, b)
If g 0 (p) 6= 0, then for any number p0 6= p in [a, b], the sequence

pn = g(pn−1 ) for n ≥ 1

converges only linearly to the unique fixed point p in [a, b]

Proof : We know from the Fixed-Point Theorem that the sequence con-
verges to p. Since g 0 exists on (a, b), we can apply the Mean Value Theorem to
g to show that for any n,

pn+1 − p = g(pn ) − g(p) = g 0 (ξn ) (pn − p)

where ξn is between pn and p. Since {pn }∞ n=0 converges to p, we also have

{ξn }∞
n=0 converging to p. Since g 0
is continuous on (a, b), we have

lim g 0 (ξn ) = g 0 (p)

n→∞

2
Thus
pn+1 − p |pn+1 − p|
lim = lim g 0 (ξn ) = g 0 (p) and lim = |g 0 (p)|
n→∞ pn − p n→∞ n→∞ |pn − p|

Hence, if g 0 (p) 6= 0, fixed-point iteration exhibits linear convergence with asymp-

totic error constant |g 0 (p)|.

NOTE: Theorem implies that higher-order convergence for fixed-point

methods of the form g(p) = p can occur only when g 0 (p) = 0

Theorem: Let p be a solution of the equation x = g(x). Suppose that

g 0 (p) = 0 and g 00 is continuous with |g 00 (x)| < M on an open interval I contain-
ing p. Then there exists a δ > 0 such that, for p0 ∈ [p − δ, p + δ], the sequence
defined by pn = g(pn−1 ), when n ≥ 1, converges at least quadratically to p.
Moreover, for sufficiently large values of n,
M 2
|pn+1 − p| < |pn − p|
2

NOTE: This theorem describes additional conditions that ensure the

quadratic convergence.

SUMMARY:
(i) For a fixed point method to converge quadratically we need to have both
g(p) = p and g 0 (p) = 0

(ii) If f (p) = 0 and f 0 (p) 6= 0, then for starting values sufficiently close to p,
Newton’s method will converge at least quadratically.

Proof : Consider the sequence

pn = g(pn−1 ) for n ≥ 1

for g in the form

g(x) = x − φ(x)f (x)
where φ is a differentiable function that will be chosen later. It is easy to see
that g(p) = p when f (p) = 0. Because

g 0 (x) = 1 − φ0 (x)f (x) − φ(x)f 0 (x)

and f (p) = 0, we have

g 0 (p) = 1 − φ0 (p)f (p) − φ(p)f 0 (p) = 1 − φ(p)f 0 (p)

Therefore, g 0 (p) = 0 if and only if φ(p) = 1/f 0 (p).

3
 If we let φ(x) = 1/f 0 (x), then we will ensure that φ(p) = 1/f 0 (p) and g is
quadratically convergent. Hence
f (pn−1 )
pn = g(pn−1 ) =pn−1 −
f 0 (pn−1 )
This, of course, is simply Newton’s method.

Example: Let f (x) = ex − x − 1. Show that Newton’s method with p0 = 1

converges to this zero but not quadratically.

Solution: We have

f (x) = ex − x − 1 and f 0 (x) = ex − 1

The first two terms generated by Newton’s method applied to f with p0 = 1 are
f (p0 ) e−2
p1 = p0 − =1− ≈ 0.58198
f 0 (p0 ) e−1
f (p1 ) 0.20760
p2 = p1 − 0
= 0.58198 − ≈ 0.31906
f (p1 ) 0.78957

n pn
0 1
1 0.58198
2 0.31906
3 0.16800
4 0.08635
5 0.04380
6 0.02206
7 0.01107
8 0.005545
9 2.7750x10−3
10 1.3881x10−3
11 6.9411x10−4
12 3.4703x10−4
13 1.7416x10−4
14 8.8041x10−5
15 4.2610x10−5
The sequence is clearly converging to 0, but not quadratically since f 0 (0) = 0.

Definition: The function f ∈ C m [a, b] has a zero of multiplicity m at p

in (a, b) if

f (p) = f 0 (p) = f 00 (p) = ... = f (m−1) (p) = 0, but f (m) (p) 6= 0

4
Example: f (x) = x3 − x4 has a zero of multiplicity 3 at x = 0. Because

f (0) = 0,
f 0 (x) = 3x2 − 4x3 ⇒ f 0 (0) = 0,
f 00 (x) = 6x − 12x2 ⇒ f 00 (0) = 0,
f 000 (x) = 6 − 24x ⇒ f 000 (0) = 6 6= 0.