MATH3230B Numerical Analysis

Tutorial 3

1 Recall:
1. Quasi-Newton method:
Recall the Newton’s method
f (xk )
xk+1 = xk − , k = 0, 1, 2, . . . (1)
f ′ (xk )
we see that at each iteration, we need to compute the derivative f ′ (xk ). This might be a big trouble for some
cases. For instance
(a) The expression f (x) is unknown;
(b) The derivative f ′ (x) is very expensive to compute;
(c) The value of function f may be the result of a long numerical calculation, so the derivative has no formula
available.
2. Constant slope method: By approximating f ′ (xk ) by a constant gk = g, the Newton’s method becomes

f (xk )
xk+1 = xk − , k = 0, 1, 2, . . . (2)
g
This is called the constant slope method. In particular, we might take g = f ′ (x0 ).
3. Fixed-point iterative methods: Both Newton’s method and the Quasi-Newton’s method can be seen as
some special case of fixed-point iterative methods. For a given function φ(x), x∗ is called its fixed point if x∗
satisfies
φ(x∗ ) = x∗ .
The iterative method
xk+1 = φ(xk ), k = 0, 1, 2, . . .
is called a fixed-point iteration associated with the function φ(x), and φ(x) is called the iterative function.
Theorem 1 (Convergence of fixed-point iterative method). If the iterative function φ(x) satisfies the condition

|φ′ (x∗ )| < 1,

then there exists a δ > 0 such that for any x0 ∈ [x∗ − δ, x∗ + δ], the fixed-point iteration converges. If
φ′ (x∗ ) ̸= 0, the convergence is linear with convergence rate ρ = |φ′ (x∗ )|. If

φ′ (x∗ ) = φ′′ (x∗ ) = · · · = φ(p−1) (x∗ ) = 0, but φ(p) ̸= 0,

then the fixed-point interation converges with order p.

4. Cases with multiple zeros: A point x∗ is called a zero of the function f (x) with multiplicity m ≥ 1 if it
holds
f (x∗ ) = f ′ (x∗ ) = · · · = f (m−1) (x∗ ) = 0,
but f (m) (x∗ ) ̸= 0.
5. Convergence of Newton’s method in the case of multiplicity: We have the following local convergence
of the Newton’s method when it is applied for solving a nonlinear equation with a zero of multiplicity m:
(a) It converges quadratically for m = 1, namely when x∗ is a single zero;

1
 (b) It converges only linearly to the multiple zero x∗ with rate ρ = 1− m
1
for m > 1. If m = 2, the convergence
rate is 1/2, the same as the convergence rate of the bisection algorithm. The Newton’s method converges
more slowly when m is larger.
6. Secant method: By approximating f ′ (xk ) by

f (xk ) − f (xk−1 )
gk = , (3)
xk − xk−1

the Newton’s method becomes

f (xk )
xk+1 = xk − , k = 0, 1, 2, . . . (4)
gk
This is called the secant method. Two initial values x0 and x1 are needed to start.

2
2 Exercises:
Please do the star problem (*) in tutorial class and finish the rest after class.

1. Consider the following nonlinear equation

f (x) := x3 − 2x2 + x = 0,

and the following iterative sequence {

xn+1 = ϕ(xn ),
x0 is given,
where ϕ : R → R is an iteration function.
(a) Consider the iterative function
f (x)
ϕ(x) := x − 2 .
f ′ (x)
Determine if the fixed-point iteration has local convergence near x = 1. If so, find the order of conver-
gence.
(b) Consider the iterative function
f (x)
ϕ(x) := x + 10 .
f ′ (x)
Determine if the fixed-point iteration has local convergence near x = 1. If so, find the order of conver-
gence.

Solution. Note that

f = x(x − 1)2 ,
f ′ = (x − 1)(3x − 1),
f ′′ = 6x − 4,
f ′′′ = 6.
(a) Consider

[f ′ ]2 − f ′′ f 2f ′′ f
ϕ′ (x) = 1 − 2 = −1 +
[f ′ ]2 (f ′ )2
2x(x − 1) (6x − 4)
2
= −1 +
(x − 1)2 (3x − 1)2
4x(3x − 2)
= −1 + .
(3x − 1)2
Hence, we have ϕ′ (1) = 0.
Also consider that
4 ( )
ϕ′′ (x) = 2(3x − 1)3 − 6(3x − 1)(3x − 2)x (5)
(3x − 1) 4

8
= . (6)
(3x − 1)3
Hence, ϕ′′ (1) ̸= 0. We conclude that the fixed-point iteration has convergence of order 2.
(b) Note that

[f ′ ]2 − f ′′ f f ′′ f
ϕ′ (x) = 1 + 10 = 11 − 10 (7)
[f ′ ]2 (f ′ )2
2x(3x − 2)
= 11 − 10 . (8)
(3x − 1)2
Because ϕ′ (1) = 11 − 5 = 6 > 1. this iteration can not ensure local convergence near x = 1.

3
2. Prove the local convergence of the fixed-point iterative method. (Theorem 1 first part)

Solution. Assume that

|φ′ (x∗ )| < 1.
Then
xk+1 − x∗ = φ(xk ) − φ(x∗ ) = φ′ (ξk )(xk − x∗ ),
where ξk lies between xk and x∗ . As |φ′ (x∗ )| < 1., there exists an δ > 0 such that

|φ′ (x)| ≤ α < 1 ∀x ∈ [x∗ − δ, x∗ + δ].

Then it is easy to see that {xk } will lie inside the interval [x∗ − δ, x∗ + δ] as long as the initial guess x0 ∈
[x∗ − δ, x∗ + δ]. this implies

|xk+1 − x∗ | = |φ′ (ξk )||(xk − x∗ )| ≤ α|xk − x∗ |,

or
|xk+1 − x∗ | ≤ αk+1 |x0 − x∗ |.
Thus we know xk → x∗ as k → ∞, or the fixed-point iterative method converge locally.

3. Show that the constant slope method converges locally and show that it converges linearly.

Solution. Please refer to the lecture notes.

4. Prove that the function f (x) has a zero of multiplicity m at x∗ if and only if f (x) = (x − x∗ )m g(x), for some
g(x) ∈ C m (R) with g(x∗ ) ̸= 0.

Solution. Assume that the function f (x) ha a zero of multiplicity m at x∗ .

By Tayor expansion,

f ′ (x∗ ) f (m−1) (x∗ ) f (m) (ξ)

f (x) = f (x∗ ) + (x − x∗ ) + · · · + (x − x∗ )m−1 + (x − x∗ )m ,
1! (m − 1)! m!

where ξ lies between x and x∗ . So if x∗ is a multiple zero of order m, then

f (m) (ξ)
f (x) = (x − x∗ )m = (x − x∗ )m g(x),
m!
with g(x∗ ) ̸= 0.
Assume that f (x) = (x − x∗ )m g(x), with g(x∗ ) ̸= 0.
∑k i ∗ m
Then we have f (k) (x∗ ) = i=0 d (x−x
dxi
)
|x=x∗ g (k−i) (x∗ ) if k is non-negative integer.
i ∗ m
Since d (x−x
dxi
)
|x=x∗ = 0 if i < m, then we have f (k) (x∗ ) = 0 if k < m.
Also, f (x ) = m!g(x∗ ) ̸= 0.
(m) ∗

Hence, f (x) ha a zero of multiplicity m at x∗

5. Let x∗ be a double root of f (x) (i.e. f (x) can be written as f (x) = (x − x∗ )2 g(x)), where f is continuously
differentiable up to second order.

(a) Let
f (x)
φ(x) = x − a , 0 < a ≤ 2.
f ′ (x)
Show that
d a
lim (φ(x)) = 1 − .
x→x∗ dx 2

4
(b) Explain why φ converges in a neighbourhood of x∗ .
(c) From (b) we know that the iteration

f (xk )
xk+1 = xk − a , 0 < a ≤ 2,
f ′ (xk )
converges to x∗ for all initial value x0 near to x∗ . Let ϵk = xk − x∗ . Find the Taylor series of

φ(x∗ + ϵk )

and prove that

|ϵk+1 | < Cϵ2k
if a = 2.
(d) Apply the above iteration to find the root of

f (x) = x4 − 2x3 − 20x2 + 66x − 45

with initial value x0 =2.5. (Find the approximate solution at the 4th iteration)

Solution. (a)
d f (x)f ′′ (x)
lim∗ (φ(x)) = 1 − a + a lim∗
x→x dx x→x f ′ (x)2
Since x∗ is a double root of f (x), f (x) can be written as f (x) = (x − x∗ )2 g(x), where g(x) is two times
continuously differentiable and g(x∗ ) ̸= 0.
Then

f ′ (x) = 2(x − x∗ )g(x) + (x − x∗ )2 g ′ (x)

f ′′ (x) = 2g(x) + 4(x − x∗ )g ′ (x) + (x − x∗ )2 g ′′ (x)

f (x)f ′′ (x) (x − x∗ )2 g(x)(2g(x) + 4(x − x∗ )g ′ (x) + (x − x∗ )2 g ′′ (x))

lim∗ = lim∗
x→x f ′ (x)2 x→x (2(x − x∗ )g(x) + (x − x∗ )2 g ′ (x))2
2g (x) + 4(x − x∗ )g(x)g ′ (x) + (x − x∗ )2 g(x)g ′′ (x)
2
= lim∗
x→x 4g 2 (x) + 4(x − x∗ )g(x)g ′ (x) + (x − x∗ )2 g ′ (x)2
1
=
2
(b) The iteration
f (xk )
xk+1 = φ(xk ) = xk − a
f ′ (xk )
Since 0 6 φ′ (x∗ ) = 1 − a2 < 1, there exists a (closed) neighbourhood of x∗ ,say B(x∗ , δ) such that
0 6 φ′ (x) 6 c < 1, for ∀x ∈ B(x∗ , δ). Note that φ(x∗ ) = x∗ as f (x∗ ) = 0. Define ek+1 = xk+1 − x∗ =
φ(xk ) − φ(x∗ ). By mean value theorem, we derive the error equation for ek :

ek+1 = φ′ (ζk )(xk − x∗ ) = φ′ (ζk )ek

where ζk lies between xk and x∗ . Therefore we have ek → 0 as c < 1.

(c) By Taylor expansion, there is some ξ such that

φ′′ (ξ)
xk+1 = φ(xk ) = φ(x∗ ) + φ′ (x∗ )(xk − x∗ ) + (xk − x∗ )2
2
if a = 2, φ′ (x∗ ) = 1 − a
2 = 0 , since φ(x∗ ) = x∗ , we have

|φ′′ (ξ)| 2
|ϵk+1 | = |xk+1 − x∗ | = ϵk < Cϵ2k
2
φ′′ (ξ)
Here we assume 2 is bounded in the neighbourhood of x∗ .

5
(d) f ′ (x) = 4x3 − 6x2 − 40x + 66, so the iteration is

x4k − 2x3k − 20x2k + 66xk − 45

xk+1 = xk − 2
4x3k − 6x2k − 40xk + 66

The first four iterations are as follows:

x0 = 2.5
x1 = 3.125
x2 = 3.004472077
x3 = 3.000006234