Chapter 2.

SOLUTIONS OF EQUATIONS

IN ONE VARIABLE
Bisection method (I)

f(b)

a b

f(a)

Calculus: Intermediate Value Theorem Let f (x) be continuous

(f (x) ∈ C ([a, b])). If f (a)f (b) < 0, then there exists one zero in
the interval (a, b).
Bisection method (II)

Suppose that on (a, b) we only have one root for f (x) = 0.

Algorithm to find a root:

• Take the mid-point p1 = (a + b)/2

• If f (p1 ) = 0, then p1 is the root.
• If not then the root is either on (a, p1 ) or (p1 , b).
If f (a)f (p1 ) < 0, then the root is in (a, p1 ), we repeat the
process on (a, p1 )
Otherwise, the root is in (p1 , b), we repeat the process on (p1 , b).
Bisection method (III)

f(b)

p3
a p2 p1 b

f(a)

p
Bisection method (IV)

Algorithm
1. We let a1 = a and b1 = b; then compute the value of f at
p1 = (a1 + b1 )/2.

2. Then we find the half that contains the root.

• If f (p1 ) = 0 we have the root

• If f (a1 )f (p1 ) < 0 then the root is in (a1 , p1 ) so we set a2 = a1
and b2 = p1 .
• Otherwise the root is in (p1 , b1 ) so we set a2 = p1 and b2 = b1 .

3. We go back to step 2, and take p2 = (a2 + b2 )/2.

4. We repeat until we reach the stopping condition |an − bn |/2 < ε

for a chosen ε; the approximated root is p = (an + bn )/2.
Bisection method (V)

f(b)
a4 b4
a3 b3
a2 b2

a1 p3 b1
a p2 p1 b

f(a)

p
Bisection method (VI)

Pseudo code: We carry the algorithm out until |an − bn |/2 < ε

INPUT endpoints a, b and the tolerance EPSILON

Step 1: Set FA = f (a)
Step 2: Set p = (a + b)/2; FP = f (p).
Step 3: If FP = 0 or (b − a)/2 < EPSILON then OUTPUT (p)
and STOP.
Step 6: If FA.FP < 0 then b = p else set a = p.
Step 7: Go back to Step 1.
STOP.
Bisection method (VII)

A Matlab code (a slightly different code will be given in the labs):

function p=Bisection(a,b,EPSILON)
while 1
FA=f(a);
p=(a+b)/2;
FP=f(p);
if (FP==0)|(b-a)/2<EPSILON
break;
else
if FA*FP<0
b=p;
else
a=p;
end;
end;
end;
Bisection method (VIII)

Example:
Find the root of f (x) = x 2 − 2 in (0, 2).

We use the bisection method:

1. Let p1 = 1. Since f (0) = −2 and f (1) = −1 so p is in (1, 2).
2. Let p2 = 1.5. Since f (1.5) = 0.25 so p is in (1, 1.5).
3. Let p3 = 1.25. Since f (1.25) = −0.4375 so p is in (1.25, 1.5).
√
Now p4 = 1.375. The exact root is 2 = 1.414.

If we carry on further, we get better approximations.

Bisection method (IX)
Theorem: Let f (x) ∈ C ([a, b]) with f (a)f (b) < 0. Suppose that
p is the only root of f in [a, b]. Let {pn } be the sequence
generated by the bisection method. Then
b−a
|pn − p| ≤ , n = 1, 2, . . . .
2n
Proof

|a1 − b1 | = |a − b|
|a1 − b1 | |a − b|
|a2 − b2 | = =
2 2
|a2 − b2 | |a − b|
|a3 − b3 | = =
2 4
...
|an−1 − bn−1 | |a − b|
|an − bn | = = n−1
2 2
Note that the root p is always in [an , bn ], so with pn = 12 (an + bn ):
|pn − p| ≤ 12 (bn − an ) = 21n (b − a).
Bisection method (X)

Given the tolerance ε, we stop when

b−a |an − bn |
= < ε,
2n 2
so
|pn − p| < ε.
To achieve this, we need ln(b − a) − n ln 2 < ln ε, i.e.

ln(b − a) − ln ε
n> .
ln 2
Bisection method (XI)

Example: Solve x 2 − 2 = 0 on (0, 2) with accuracy 10−3 .

We should have

|pn − p| ≤ 2−n (b − a) = 2−n+1 < 10−3 .

Thus 2n > 2000. Thus n > 10.97.

We need 11 iterations.
Fixed-point iteration (I)

Definition A number p is a fixed point of a function g if

g (p) = p, i.e. p is the root of the equation g (x) − x = 0.

Examples: If g (x) = x, then all the points are fixed point.

If g (x) = x 2 − 2, then a fixed point p satisfies p 2 − 2 = p, so

p = −1 and p = 2.

The roots of f (x) = 0 are fixed points of g (x) = x − f (x).

Fixed-point iteration (II)

When does a function possess a fixed point?

Theorem If a continuous function g satisfies a ≤ g (x) ≤ b for all

x in [a, b], then g has a fixed point on [a, b].

y=x
y
b Proof Consider h(x) = g (x) − x.
g(b) Then h(a) = g (a) − a ≥ 0 and
g(p) h(b) = g (b) − b ≤ 0
g(a) so from the intermediate value theo-
rem, h has a zero p in [a, b], at which
a
g (p) = p.
a p b x
Fixed-point theorem (III)

When does a function has a unique fixed point?

Theorem If there is a constant k < 1 such that |g ′ (x)| ≤ k for all

x ∈ [a, b], then the fixed point is unique.

Proof Assume that g has two fixed points p and q. From the
Mean value theorem (see calculus), we can find a value ξ such that

g (p) − g (q)
= g ′ (ξ).
p−q
Thus

|p − q| = |g (p) − g (q)| = |g ′ (ξ)||p − q| ≤ k|p − q| < |p − q|,

which is a contradiction. Thus g has a unique fixed point.

Fixed-point iteration (IV)

Sometimes g still has a unique fixed point in [a, b] when these

conditions do not hold, (the conditions are not necessary)
√
Example: Let g (x) = 2x − 1 on [1/2, 2].
g (1/2) = 0 ∈
/ [1/2, 2].
The derivative of g is
1
g ′ (x) = √ ,
2x − 1

which converges to ∞ when x → 1/2.

But g has a unique fixed point p = 1.

Fixed-point iteration (V)

Fix an initial point p0 .

Then we build a sequence {pn } by the iteration scheme

p1 = g (p0 ),
p2 = g (p1 ),
...,
pn = g (pn−1 ),
...

If {pn } converges to p, then g (pn ) → g (p). However,

g (pn ) = pn+1 → p so g (p) = p, i.e p is a fixed point of g .
Fixed-point iteration (VI)

y y=x

p1=g(p0)

p2=g(p1)
p3=g(p2)

p p3 p2 p1 p0 x
Fixed-point iteration (VII)

Some first values p0 do not give us a fixed point.

y=g(x)

y y=x

p3=g(p2)

p2=g(p1)

p1=g(p0)

p=g(p)

p p0 p1 p2 p3 x
Fixed-point iteration (VIII)

When does the fixed point iteration converge?

Fixed point theorem

Theorem Let a continuous function g satisfy a ≤ g (x) ≤ b for all
a ≤ x ≤ b. Suppose that there is a constant 0 < k < 1 such that
|g ′ (x)| ≤ k for all x ∈ (a, b). Then the fixed point iteration
converges to the unique fixed point p in [a, b] for all p0 ∈ [a, b].
Fixed-point iteration (IX)

Proof of the fixed point theorem

From the Mean Value Theorem, there is a ξ ∈ (a, b) such that

|pn − p| = |g (pn−1 ) − g (p)| = |g ′ (ξ)||pn−1 − p| ≤ k|pn−1 − p|.

Repeating this for |pn−1 − p|, . . . , we get

|pn − p| ≤ k|pn−1 − p| ≤ k 2 |pn−2 − p| ≤ . . . ≤ k n |p0 − p|.

Since 0 < k < 1, limn→∞ k n = 0, so

lim |pn − p| = 0.
n→∞
Fixed-point iteration (X)

We now look at the rate of convergence.

We have seen: |pn − p| ≤ k n |p0 − p|.

Obviously |p0 − p| ≤ b − a, so |pn − p| ≤ k n (b − a).

The rate of convergence is determined by k n .

The fixed point iteration converges faster when k is small.

We can also show that

kn
|pn − p| ≤ |p1 − p0 |.
1−k
Fixed-point iteration (XI)

Example:
The equation f (x) = x 3 + 4x 2 − 10 = 0 has a unique root
p = 1.365230013 in [1, 2].

The equation is equivalent to:

1
x 2 = (10 − x 3 ),
4
i.e.
1
x = (10 − x 3 )1/2 = g1 (x). (1)
2
The root of the equation is a fixed point of g1 (x). We find a fixed
point of g1 (x) by taking p0 = 1.5.
From table 2.2, page 58 in the text book, we see that it converges
after 30 iterations.
Fixed-point iteration (XII)

We can also write x 3 + 4x 2 − 10 = 0 as

x 2 (x + 4) = 10,

so  10 1/2
x= = g2 (x). (2)
4+x
We find a fixed point of g2 . Take p0 = 1.5.
The scheme converges after 15 iterations. Why is it faster?
Fixed-point iteration (XIII)

Look at g1 (x) in (1):

3
g1′ (x) = − x 2 (10 − x 3 )−1/2 ,
4
which is negative on [1, 2] so g1 (x) is decreasing on [1, 2].

When 1 ≤ x ≤ 1.5, g1 (x) ≥ g1 (1.5) = 1.287 > 1 and

g1 (x) ≤ g1 (1) = 1.5.
As p0 = 1.5, p1 = g (p0 ) ∈ [1, 1.5], p2 = g (p1 ) ∈ [1, 1.5], ... so we
only need to consider the interval [1, 1.5].

On [1, 1.5] |g1′ (x)| ≤ |g1′ (1.5)| ≈ 0.66, here k = 0.66.

Fixed-point iteration (XIV)

For g2 (x) in (2):

−5
g2′ (x) = √ ,
10(4 + x)3/2
so
|g2′ (x)| ≤ |g2′ (1)| ≤ 0.15.
Here k = 0.15; 0.15n << 0.66n . The scheme (2) converges faster.
Fixed-point iteration (XV)

Another equivalent form of f (x) = x 3 + 4x 2 − 10 = 0 is

x 3 + 4x 2 − 10
x =x− = g (x).
3x 2 + 8x
We find the fixed point of g (x). Take p0 = 1.5. The scheme
converges after only 4 iterations.

How do we know this choice of g (x)? It is the Newton iteration.