1

Numerical Analysis
Lecture-3

Dr. Ilyas Fakhir

CS Dept. GCU, Lahore

February 3, 2020

Dr. Ilyas Fakhir Numerical Analysis

 2

Basic Idea Behind Fixed-Point Iteration

What is a fixed-point?
α is a fixed-point of g(x) provided g (α) = α
Here, α is being ”fixed” by g(x) since it maps to itself.
The root finding problem → fixed-point finding problem.

f (x) = 0 → f (x) + x = x ⇒ g (x) = x

Dr. Ilyas Fakhir Numerical Analysis

 3

Towards the Design of Fixed-Point Iteration

Dr. Ilyas Fakhir Numerical Analysis

 4

Towards the Design of Fixed-Point Iteration

Dr. Ilyas Fakhir Numerical Analysis

 5

Towards the Design of Fixed-Point Iteration

Dr. Ilyas Fakhir Numerical Analysis

 6

Towards the Design of Fixed-Point Iteration

Dr. Ilyas Fakhir Numerical Analysis

 7

Performance of the 4 Methods

Dr. Ilyas Fakhir Numerical Analysis

 8

Functional (Fixed-Point) Iteration

Prime Objective
In what follows, it is important not to lose sight of our prime
objective:
Given a function f (x) where a ≤ x ≤ b, find values p such
that
f (p) = 0
Given such a function, f (x), we now construct an auxiliary
function g(x) such that

p = g (p)

whenever f (p) = 0(this construction is not unique).

The problem of finding p such that p = g(p) is known as the
fixed point problem.

Dr. Ilyas Fakhir Numerical Analysis

 9

Functional (Fixed-Point) Iteration

A Fixed Point
If g is defined on [a, b] and g(p) = p for some p ∈ [a, b], then the
function g is said to have the fixed point p in [a, b].

Note
The fixed-point problem turns out to be quite simple both
theoretically and geometrically.
The function g(x) will have a fixed point in the interval [a, b]
whenever the graph of g(x) intersects the line y = x.

Dr. Ilyas Fakhir Numerical Analysis

 10

Functional (Fixed-Point) Iteration

The Equation f (x) = x − cos(x) = 0

If we write this equation in the form:

x = cos(x)

then g (x) = cos(x).

Dr. Ilyas Fakhir Numerical Analysis

 11

Single Nonlinear Equation f (x) = x − cos(x) = 0

Dr. Ilyas Fakhir Numerical Analysis

 12

Functional (Fixed-Point) Iteration

x = cos(x)

Dr. Ilyas Fakhir Numerical Analysis

 13

Functional (Fixed-Point) Iteration

p = cos(p) p ≈ 0.739

Dr. Ilyas Fakhir Numerical Analysis

 14

Functional (Fixed-Point) Iteration

Existence of Fixed Point

If g ∈ C [a, b] and g (x) ∈ [a, b] for all x ∈ [a, b] then the function
g has a fixed point in [a, b].

Proof
If g (a) = a or g (b) = b the existence of a fixed point is obvious.
Suppose not; then it must be true that g (a) > a and g (b) < b.
Define h(x) = g (x) − x; h is continuous on [a, b] and, moreover,

h(a) = g (a) − a > 0, h(b) = g (b) − b < 0.

The Intermediate Value Theorem implies that there exists p ∈ (a, b)

for which h(p) = 0.
Thus g (p) − p = 0 and p is a fixed point of g.

Dr. Ilyas Fakhir Numerical Analysis

 15

g (x) is Defined on [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 16

g (x) ∈ [a, b] for all x ∈ [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 17

g (x) has Fixed Point in [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 18

g (x) has a Fixed Point in [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 19

Illustration
Consider the function g (x) = 3−x on 0 ≤ x ≤ 1. g(x) is
continuous and since

g 0 (x) = −3−x log 3 < 0 on[0, 1]

g(x) is decreasing on [0, 1].

Hence
1
g (1) = ≤ g (x) ≤ 1 = g (0)
3
i.e. g (x) ∈ [0, 1] for all x ∈ [0, 1] and therefore, by the
preceding result, g(x) must have a fixed point in [0, 1].

Dr. Ilyas Fakhir Numerical Analysis

 20

Functional (Fixed-Point) Iteration

g (x) = 3−x

Dr. Ilyas Fakhir Numerical Analysis

 21

Functional (Fixed-Point) Iteration

An Important Observation
It is fairly obvious that, on any given interval I = [a, b], g(x)
may have many fixed points (or none at all).
In order to ensure that g(x) has a unique fixed point in I, we
must make an additional assumption that g(x) does not vary
too rapidly
Thus we have to establish a uniqueness result.

Dr. Ilyas Fakhir Numerical Analysis

 22

Functional (Fixed-Point) Iteration

Uniqueness Result
Let g ∈ C [a, b] and g (x) ∈ [a, b] for all x ∈ [a, b]. Further if g 0 (x)
exists on (a, b) and

|g 0 (x)| ≤ k < 1, ∀x ∈ [a, b],

then the function g has a unique fixed point p in [a, b].

Dr. Ilyas Fakhir Numerical Analysis

 23
0
g (x) is Defined on [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 24
0
−1 ≤ g (x) ≤ 1 for all x ∈ [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 25
0
Unique Fixed Point: |g (x)| ≤ 1 for all x ∈ [a, b]

Dr. Ilyas Fakhir Numerical Analysis

 26

Functional (Fixed-Point) Iteration

Now that we have established a condition for which g(x) has a

unique fixed point in I, there remains the problem of how to find it.
The technique employed is known as fixed-point iteration.

Basic Approach
To approximate the fixed point of a function g, we choose an
initial approximation p0 and generate the sequence {pn }∞
n=0
by letting pn = g (pn−1 ), for each n ≥ 1.
If the sequence converges to p and g is continuous, then

p = lim pn = lim g (pn−1 ) = g ( lim pn−1 ) = g (p),

n→∞ n→∞ n→∞

and a solution to x = g(x) is obtained.

This technique is called fixed-point, or functional iteration.

Dr. Ilyas Fakhir Numerical Analysis

 27

Functional (Fixed-Point) Iteration

Dr. Ilyas Fakhir Numerical Analysis

 28

Functional (Fixed-Point) Iteration

Fixed-Point Algorithm
To find the fixed point of g in an interval [a, b], given the equation
x = g(x) with an initial guess p0 ∈ [a, b]:
1 n = 1;
2 pn = g (pn−1 );
3 If |pn − pn−1 | <  then 5;
4 n → n + 1; go to 2.
5 End of Procedure.

Dr. Ilyas Fakhir Numerical Analysis

 29

A Single Nonlinear Equation

Example
The equation x 3 + 4x 2 − 10 = 0 has a unique root in [1,2]. Its value is
approximately 1.365230013.

Dr. Ilyas Fakhir Numerical Analysis

 30
3 2
f (x) = x + 4x − 10 = 0 on [1,2]

Possible Choices for g(x)

There are many ways to change the equation to the
fixed-point form x = g(x) using simple algebraic manipulation.
For example, to obtain the function g described in part (c),
we can manipulate the equation x 3 + 4x 2 − 10 = 0 as follows:
1 1
4x 2 = 10 − x 3 , so x 2 = (10 − x 3 ), and x = ± (10 − x 3 )1/2 .
4 2
We will consider 5 such rearrangements and, later in this
section, provide a brief analysis as to why some do and some
not converge to p = 1.365230013.

Dr. Ilyas Fakhir Numerical Analysis

 31
3 2
Solving f (x) = x + 4x − 10 = 0

5 Possible Transpositions to x = g(x)

x = g1 (x) = x − x 3 − 4x 2 + 10
q
x = g2 (x) = 10 x − 4x
√
x = g3 (x) = 12 10 − x 3
q
10
x = g4 (x) = 4+x
x 3 +4x 2 −10
x = g5 (x) = x − 3x 2 +8x

Dr. Ilyas Fakhir Numerical Analysis

 32
3 2
Numerical Results for f (x) = x + 4x − 10 = 0

Dr. Ilyas Fakhir Numerical Analysis