Outline

Math 305 (Numerical Analysis) Classification of Fixed Points (Continued)

Week 2 - Lecture 3

Solution of a Nonlinear Equation (Part 3) Rate of Convergence

Classification of Fixed Points

x∗ is a fixed-point of g satisfying g(x∗ ) = x∗ .

Simple Iteration: (xk ), xk +1 = g(xk ) for a given x0

x∗ is a Stable Fixed Point

Classification of Fixed Points (Continued) (xk ) converges to x∗ (provided x0 is close to x∗ ).

x∗ is an Unstable Fixed Point

(xk ) cannot converge to x∗ (unless xk = x∗ ∃k ).

Assume g 0 (x) is continuous in an open interval containing x∗ .

I If |g 0 (x∗ )| < 1, x∗ is stable.
I If |g 0 (x∗ )| > 1, x∗ is unstable.

Classification of Fixed Points Stable Fixed Points

15
Theorem
Let g : R → R be continuous on [a, b], and x∗ ∈ [a, b] be a
fixed-point of g such that
10 1. g 0 (x) is continuous in an open neighborhood of x∗ ,
2. |g 0 (x∗ )| < 1.

There exists an open neighborhood I of x∗ such that

lim xk = x∗ ∀x0 ∈ I,
5 k →∞
so x∗ is a stable fixed point of g.

Ex: g(x) = −(x 2 − 2)/4 + x

0 √ √
0 2 4 6 8 10 I g 0 ( 2) = 1 − 1/ 2
√
g(x) = 2cosh(x/4), where cosh(x) := (ex + e−x )/2 I 2 is stable.
Unstable Fixed Points Unstable Fixed Points
Proof. Let L be such that |g 0 (x∗ )| > L > 1, and
I = (x∗ − δ, x∗ + δ) s.t. |g 0 (x)| ≥ L ∀x ∈ I.
Theorem
Let g : R → R and x∗ be as in the previous theorem with the
Suppose xk ∈ I, xk 6= x∗ . MVT implies the existence of an
only exception |g 0 (x∗ )| > 1.
εk ∈ I such that
For all x0 , x0 6= x∗ , we have (xk ) does not converge to x∗ , unless
xk = x∗ for some k, so x∗ is an unstable fixed point of g. |xk +1 − x∗ | = |g(xk ) − g(x∗ )| = |g 0 (εk )||xk − x∗ | ≥ L|xk − x∗ |

/ I ∃m ∈ Z+ .
implying xk +m ∈

Ex: g(x) = x 2 − 2 + x
√ √ This shows that no matter how large k is
I g 0 ( 2) = 2 2 + 1
√ |xκ − x∗ | ≥ δ ∃κ > k ,
I 2 is unstable.
so (xk ) does not converge to x∗ .

Rate of Convergence
To quantity how quick convergence is, would like to relate
|xk − x∗ | and |xk +1 − x∗ |.

Assumptions

Rate of Convergence 1. g 0 (x) is continuous in an open neighborhood of x∗ .

2. limk →∞ xk = x∗ .

By MVT
|xk +1 − x∗ | = |g(xk ) − g(x∗ )| = |g 0 (εk )||xk − x∗ |
for some εk in open interval with end-points xk , x∗ . That is
|xk +1 − x∗ |
lim = lim |g 0 (εk )| = |g 0 ( lim εk )| = |g 0 (x∗ )|.
k →∞ |xk − x∗ | k →∞ k →∞

Examples Rate of Convergence

√ Definition
2−xk2
   
2−x 2
1. g(x) = 4 + x, xk +1 = 4 + xk , g 0 ( 2) = 1 − √1 . Suppose limk →∞ xk = x∗ , and there exists εk such that
2
|xx − x∗ | ≤ εk , as well as
√ |xk +1 − x∗ | 1
lim xk = 2 =⇒ lim = 1− √ |εk +1 |
k →∞ k →∞ |xk − x∗ | 2 lim = µ.
k →∞ |εk |
Linear Convergence
1. If µ ∈ (0, 1), then (xk ) converges to x∗ at least linearly.
√ 2. If εk = |xk − x∗ | ∀k and µ ∈ (0, 1), then (xk ) converges to
2−xk2
   
2−x 2
2. g(x) = 2x + x, xk +1 = − 2xk + xk , g 0 ( 2) = 0. x∗ linearly.
3. If µ = 0, then (xk ) converges to x∗ superlinearly.
√ |xk +1 − x∗ |
lim xk = 2, =⇒ lim = 0 4. If εk = |xk − x∗ | ∀k and µ = 1, then (xk ) converges to x∗
k →∞ k →∞ |xk − x∗ |
sublinearly.
Superlinear Convergence      
1 1 1
Linear: , Superlinear: , Sublinear:
2k k! k
Rate of Convergence Rate of Convergence
Suppose x∗ = limk →∞ xk , and
xk +p − x∗ ≈ µp |xk − x∗ |
|xk +1 − x∗ |
lim =µ
k →∞ |xk − x∗ | Find p such that |xk +p − x∗ | ≈ 0.1|xk − x∗ |
log10 0.1 −1
for some µ ∈ [0, 1]. (Linear, superlinear, sublinear convergence) µp = 0.1 =⇒ p= =
log10 µ log10 µ

Then |xk +1 − x∗ | ≈ µ|xk − x∗ | for large k .

A SYMPTOTIC R ATE OF C ONVERGENCE
Applying this repeatedly, Suppose x∗ = limk →∞ xk , and
|xk +1 − x∗ |
|xk +2 − x∗ | ≈ µ|xk +1 − x∗ | ≈ µ2 |xk − x∗ | lim =µ
k →∞ |xk − x∗ |
for some µ ∈ [0, 1].
..
. I ρ = −log10 µ is asymptotic rate of convergence.
I About 1/ρ iterations to attain one decimal digit accuracy
|xk +p − x∗ | ≈ µp |xk − x∗ |

Examples Rate of Convergence

xk +1 = −(xk2
− 2)/4 + xk
√
suppose lim xk = 2
k →∞
I µ = 0, rate = − log10 µ = ∞
g(x) = −(x 2
− 2)/4 + x
√ √
g (x) = 1 − x/2, g 0 ( 2) = 1 − 2/2 = µ
0
Very quick convergence
√
asymptotic rate of convergence = −log10 (1 − 2/2) ≈ 0.293

xk +1 = −(xk2 − 2)/(2xk ) + xk I µ = 1, rate = − log10 µ = 0

√
suppose lim xk = 2
k →∞ Very slow converge
g(x) = −(x 2 − 2)/(2x) + x
√
g 0 (x) = (x 2 − 2)/(2x 2 ), g 0 ( 2) = 0 = µ
rate of convergence = −log10 0 ≈ ∞