Chapter 1

Numerical methods for solving algebraic equations

MATH2052

Chengjie Cai
School of Mathematical Sciences
2022-23
 Lecture 2  An introduction to numerical methods

Last time, we:

revised what DEs are and how to describe them;

revised solving rst-order ODEs via separation of variables and the
integrating factor method.

Today, we will:

appreciate why and how numerical methods work for solving both
algebraic and dierential equations;
learn about the bisection method and the Newton-Raphson method
for solving algebraic equations; how they work, and pros and cons.

Chengjie Cai (UoN) MATH2052 2022-23 2 / 19

Numerical methods  motivation
Many interesting equations are either very dicult or impossible to
solve analytically.
However, we can approximate the solution using numerical methods.
Many numerical methods are iterative; i.e. a repeated rule/formula.

We will explore several common numerical methods for solving DEs on

this course. But let's start simple rst; let's look at numerical methods
for solving algebraic equations.
Two such simple methods are the bisection method and the
Newton-Raphson method.

F Suppose we wish to solve x = cos x , for x ∈ [0; 1] . . . we can't solve

this exactly, but we can approximate the solution.

Chengjie Cai (UoN) MATH2052 2022-23 3 / 19

The bisection method
General idea: for a function of the form f (x) = 0:
1 Choose an interval [a; b] which we know contains the solution
(therefore f (a) and f (b) have ).
2 Calculate f (x) at the mid-point of the interval, m.
3 Choose our new interval to be the half that contains the root. We can
work this out by considering sign changes:
If f (a) and f (m) have opposite signs, choose interval [a; m].
If f (m) and f (b) have opposite signs, choose interval [m; b].
4 Repeat until a desired accuracy has been achieved (e.g. approximation
correct to 4 d.p.). We consider m as the approximation for the root at
each iteration.

To keep track of where we are, it helps to tabulate our ndings.

Note: We may need to rst rearrange our equation into the form f (x) = 0.
Chengjie Cai (UoN) MATH2052 2022-23 4 / 19
The bisection method  example
Approximate the root x ∈ [0; 1] for the equation x = cos x to 2 d.p..

First, rearrange to get RHS = 0:

Our rst interval is
Calculate f (m)
Choose whichever of [0; 0:5]
or [0:5; 1] contains the root.
This is , as
and have opposite
signs (see graph).
Repeat until we have m correct
to 2 d.p.!
Let's tabulate our data.
Chengjie Cai (UoN) MATH2052 2022-23 5 / 19
The bisection method  example

Approximate the root x ∈ [0; 1] for the equation x = cos x to 2 d.p..

Iter a m b f (a) f (m) f (b) New interval

1 0 0.5 1 -1 -0.38 0.46 [0:5; 1]
2
3
4 0.625 0.6875 0.75 -0.19 -0.09 0.02 [0:6875; 0:75]

See the method graphically with this GeoGebra le.

Chengjie Cai (UoN) MATH2052 2022-23 6 / 19
The bisection method  example

Approximate the root x ∈ [0; 1] for the equation x = cos x to 2 d.p..

Iter a m b f (a) f (m) f (b) New interval

1 0 0.5 1 -1 -0.38 0.46 [0:5; 1]

2 0.5 0.75 1 -0.38 0.02 0.46 [0:5; 0:75]
3 0.5 0.625 0.75 -0.38 -0.19 0.02 [0:625; 0:75]
4 0.625 0.6875 0.75 -0.19 -0.09 0.02 [0:6875; 0:75]
5 0.6875 0.718... 0.75 -0.09 -0.03 0.02 [0:718:::; 0:75]
6 0.718... 0.734... 0.75 -0.03 −0:008 0.02 [0:734:::; 0:75]
7 0.734... 0.742... 0.75 -0.008 0:005 0.02 [0:734:::; 0:742:::]
8 0.734... 0.738... 0.742... -0.008 −0:001 0.005 [0:738:::; 0:742:::]

We have found that x ∈ [0:738:::; 0:742:::]; therefore,

. . . and it took 8 iterations to nd this.

Chengjie Cai (UoN) MATH2052 2022-23 7 / 19
The bisection method  review

Provided f (x) is continuous over our initial interval [a; b], the method
will always work (a desirable feature that not all numerical methods
have).
However, as we saw, the method is very tedious to do by hand, even
with a calculator. This is because the method was hard to automate
with our calculator.
Furthermore, the method converges very slowly; it took 8 iterations
just to get 2 decimal places of accuracy!
Due to these disadvantages, the bisection method is not typically used.

Another method which we can automate with our calculator, and

(usually) converges faster, is the Newton-Raphson method.

Chengjie Cai (UoN) MATH2052 2022-23 8 / 19

The Newton-Raphson method
General idea: for a function of the form f (x) = 0:
1 Choose a starting x value point near the root, say x0 .
2 Draw a tangent to f (x) at x0 . Label where this tangent cuts the
x -axis as x1 .
3 x1 is (typically) closer to the root than x0 (see gure).
4 Repeat the process starting from x = x1 . Label the new point x2 .
5 Repeat until the desired accuracy is reached.

But how do we
automate this process?
We can write down an
iterative formula for
this process.

Chengjie Cai (UoN) MATH2052 2022-23 9 / 19

The Newton-Raphson method
The tangent to f (x) at x0 has equation
The tangent passes through . Substitute this point in:

Therefore, the tangent has equation

The tangent also passes through . Substitute this point in and
rearrange for x1 :

F Iterating this formula:

Chengjie Cai (UoN) MATH2052 2022-23 10 / 19

The Newton-Raphson method  example
Approximate the root x ∈ [0; 1] for the equation x = cos x to 4 d.p..
Rearrange equation:
f 0 (x) =
Choose x0 = 1.
So our iterative formula is
Pro-gamer move: We can abuse the `Ans' button on our calculator
to vastly speed this process up:
Type 1 and press equals (this logs x0 = 1 on the calculator).

Type the RHS of the above formula, but replace xn with `Ans'.

Then just keep pressing `=' (writing down our answers as we go).

Repeat until the desired accuracy is obtained.

x0 x1 x2 x3 x4
1
Chengjie Cai (UoN) MATH2052 2022-23 11 / 19
The Newton-Raphson method  review

To see the Newton-Raphson method working graphically, check out

this GeoGebra le.
The Newton-Raphson method took 4 iterations to get 4 d.p.
. . . compare this with the bisection method, which took 8 iterations to
get just 2 d.p.!
Also, the method was easy to automate on our calculator!

Now you try!

(a) Show that f (x) = x 5 − 4x 2 + 2 has a root in-between 1 and 2.

(b) Use the Newton-Raphson method with x0 = 2 to nd this root,
correct to 4 dp.

Chengjie Cai (UoN) MATH2052 2022-23 12 / 19

Solution:

Therefore:

Choose x0 = 2.
x0 x1 x2 x3 x4 x5 x6
2

So after 6 iterations, we found that the root is , correct to 4 d.p..

Chengjie Cai (UoN) MATH2052 2022-23 13 / 19

The Newton-Raphson method  shortcomings

The N-R method typically converges a lot faster than the bisection
method.
However, unlike the bisection method, the Newton-Raphson method
does not always converge correctly.
For example, consider the above example, but starting with x0 = 1:
x0 x1 x2 x3 x4
1 0.66667... 0.74811... 0.74713... 0.74713...

We've converged to a dierent root!

See this happen graphically with this GeoGebra le.
Knowledge about how many solutions there are, and roughly where they
are, will help us choose x0 so that we converge to the correct root.
Chengjie Cai (UoN) MATH2052 2022-23 14 / 19
The Newton-Raphson method  shortcomings
The N-R method immediately fails if at any of our xn .
(why?)
It also often fails if at any of our xn . (why?)

Example:
1
With x0 = 5, approximate the solution to f (x) = − 2 = 0.
x −3
Note: the actual solution nearby is x = 3:5.

f 0 (x) =

x0 x1 x2 x3
5
Uh oh! Let's see what went wrong.
Chengjie Cai (UoN) MATH2052 2022-23 15 / 19
The Newton-Raphson method  shortcomings

f 0 (x0 ) ≈ 0; i.e. our rst tangent had a very shallow gradient.

This took us quite far away from the root.

The next tangent was even shallower . . . and took us in the wrong
direction!
This process then repeated, sending us o to −∞... not where we
want to be!

F The N-R method tends to (understandably) struggle for functions with

sudden changes in gradient.

Chengjie Cai (UoN) MATH2052 2022-23 16 / 19

N-R method  miraculous recoveries
Having said that, sometimes the N-R method can recover from a
terrible start to converge to the correct root.
Consider x0 = −0:6 for the function e5x − 2 = 0 on GeoGebra.

The only solution to this equation is x =

The rst tangent is very shallow, sending us far away from the root...
...but the rest of the tangents are suciently steep, and we slowly but
surely converge to the root.

F When using the N-R method, we should choose x0 to be a point that:

is fairly near the root,

does not make a shallow tangent to the curve.

We will learn how to write programs in Python for both the bisection
method and the N-R method in a future workshop.
Chengjie Cai (UoN) MATH2052 2022-23 17 / 19
Lecture 2  summary
Numerical methods allow us to approximate the solutions to equations
that are too dicult/impossible to solve exactly.
In the bisection method, we bisect the interval which contains the
solution, and choose the half that still contains the solution.
The Newton-Raphson method works by considering intersections
between tangents to the function and the x -axis. The formula is
f (xn )
xn+1 = xn − :
f 0 (xn )
Both of these methods are iterative; they provide a rule that we
repeat, with our approximation (hopefully) becoming more accurate
with each repeat.
The bisection method works every time (if the function is continuous
over the original interval), but is slow.
The Newton-Raphson method converges much faster, but is prone to
failure at points with shallow gradients.
Chengjie Cai (UoN) MATH2052 2022-23 18 / 19
Lecture 2  optional feedback question

(a) Show that the equation 3 sin x = x has a root x ∈ [2; 3] (in radians).
(b) Using the Newton-Raphson method, approximate this root to 4 d.p..
Note how many iterations it takes.
(c) Using this GeoGebra le (or otherwise), approximate the same root to
4 d.p. using the bisection method, starting with the interval [2; 3].
Note how many iterations it takes.
(d) Comment on the success of the two methods.

Chengjie Cai (UoN) MATH2052 2022-23 19 / 19