APM2613/TUT1/0/2023

Tutorial Letter TUT1/0/2023

Numerical Analysis I
APM2613

Year Module

Mathematical Sciences Department

This tutorial letter contains Tutorial 1 to compliment Assignment 01. Please read it along
Lesson 2 and Chapter 2 of the textbook.

BARCODE

Open Rubric
1 Tutorial 1: Solutions of Non-linear Equations
Reading: Chapter 2 and Chapter 10.2 of textbook; Lesson 2
Other resources:

https://sites.google.com/site/numericalanalysis1burden/

1.1 Objective
To go over (an) example(s) and give exercises to practise on the use of various methods to
solve a non-linear equation.

1.1.1 The methods

1. Bisection method;

2. Fixed point method;

3. Newton’s method and its extensions: Secant method; Regula falsi method;

4. Horner’s method and Muller’s method

5. Newton’s method for systems of two equations in two unknowns.

As these methods are applied, the following will be highlighted where appropriate:

ˆ Key aspects of investigating convergence and error of these methods;

ˆ Key points of other methods used for specific purposes in the study of solutions of
nonlinear equations;

Nature of the problem The methods are used to solve a nonlinear equation in two
forms, namely,

f (x) = 0 referred to as the root (or root-finding) problem (1)

and the
x = g(x) referred to as thefixed − pointproblem (2)
The root problem involves rearranging the given equation so that one side of the equation
is 0. Whereas a solution of this problem is the usual x-intercept of the graph of f (x), the
solution of the fixed-point problem is x-value at the intersection of y = x and y = g(x).
It is also worth noting that the use of the terms root or, zero and fixed-point in the
context of these two equations.

1
Example 1
Consider the problem of finding the zeros of the function f (x) = x3 − 9x2 + 12 using the
various methods.

Before we solve this problem, let us analyse the function itself.

First we note that the function is smooth; i.e. it is defined everywhere, continuous and
differentiable. This is typical of polynomial functions.
The function has three roots: one in the interval (−2, 0) another one in (0, 2) and the other
in (4, 10).
The function has two extrema (critical points/turning points): one maximum and one min-
imum, from which it follows that there is a point of inflection. (It is left as an exercise to
verify these facts)

f ( x) = x 3- 9x 2+ 12
150

100

50
f ( x)
0 x= - 1. 0905
x= 1. 2438 x= 8. 8467

-50

-100

-150

-200
-4 -2 0 2 x 4 6 8 10

Figure 1: The function f (x) of the root equation Nflat

A quick Octave check for the roots using the code

f=inline(’x.^3-9*x.^2+12’);
c=[1 -9 0 12]; % Coefficients of the polynomial
r=roots(c)
gives the roots as x = −1.0905; 1.2438; 8.8467. Let us then go through the various methods
to find the root in 4 < x < 10.

1.1.2 Bisection Method

Note that the iterative formula for the bisection method for a root in [a, b] is
xk − xk−1
xk+1 =
2
Let us, for simplicity, use endpoints a and b and new iterate c. so we start with x0 = a = 4
and x1 = b = 10.
A quick check using the Intermediate Value Theorem with f (x) = x3 − 9x2 + 12 yields
f (a) = −68; f (b) = 112, from which we see that f (a) · f (b) < 0

2
which confirms that there is a root between 4 and 10; i.e. f (4) and f (10) have opposite
signs).

First iteration: The new iterate is c = (a + b)/2 = (4 + 10)/2 = 7. So x2 = c = 7

For this, f (7) = −86 < 0, so f (c) · f (b) < 0 and hence we discard x0 = a = 4 and replace
it with c = 7. The new interval that contains the root is thus 7 < x < 10, and now
a = 7, b = 10.
Second iteration:
The new iterate is c = (7 + 10)/2 = 8.5, and
f (a) · f (c) = (−86)(−24.125) > 0 and f (b) · f (c) = (112)(−24.125) < 0.
We observe that, still the function has the same sign at the left endpoint, a as at c. So we
replace x2 = 7 with c = x3 = 8.5, and repeat the bisection.
If we do this bisection and the intermediate value test we get
i a b c f (a) f (b) f (c)
0 4.0000 10.0000 7.0000 −68.0000 112.0000 −86.0000
1 7.0000 10.0000 8.5000 −86.0000 112.0000 −24.1250
2 8.5000 10.0000 9.2500 −24.1250 112.0000 33.3906
3 8.5000 9.2500 8.8750 −24.1250 33.3906 2.1543
4 8.5000 8.8750 8.6875 −24.1250 2.1543 −11.5852
5 8.6875 8.8750 8.7812 −11.5852 2.1543 −4.8679
6 8.7812 8.8750 8.8281 −4.8679 2.1543 −1.3952
7 8.8281 8.8750 8.8516 −1.3952 2.1543 0.3699
8 8.8281 8.8516 8.8398 −1.3952 0.3699 −0.5151
.. .. .. .. .. ..
. . . . . . ···
We can observe that the method converges to a value between 8 and 9, the last bracket above
is [8.8281, 8.8398]tobeprecise, at which point the value of f is f (c) = f (8.8398) = 0.5151
in absolute value.
Convergence The nice thing about the Bisection Method is that the repeated applica-
tion of the intermediate point test after each calculation of the midpoint ensures that the
iterations converge to the root because each time it is bracketed by the endpoints.

Furthermore, the convergence rate of the method is simple and given in Theorem 2.1.

Termination criterion
Since the next iterate is obtained simply as the midpoint of the current interval, it is
possible to determine the number of iterations n required to attain a specified tolerance ϵ
by solving the inequality
b−a
< ϵ, b > a (3)
2n
for n.

Example 2 on p 52 illustrates this calculation.

Note that this calculation does not involve the function f (x).

1.1.3 Fixed-point Iteration

Fixed-point iteration is the core of the discussion of the methods presented in Chapter
2. In simple terms, a fixed-point iteration formula involves rearranging the root equation

3
f (x) = 0 in various ways to obtain the fixed-point equation x = g(x) associated with an
iteration formula xk+1 = g(xk ). The different ways of rearranging f (x) = 0 lead to different
expressions of g(x) in x = g(x).

Example
For our example, the root equation f (x) = x3 −9x2 +12 = 0 has solutions x = −1.0444; 1.3165; 8.7279;
i.e. can be rearranged in various expressions of x = g(x), to name one,
x3 + 12
x= ,
9x
3
where g(x) = x 9x+12
, which is a hyperbola which is asymptotic to x = 0.
Graphically, the line y = x crosses the hyperbola at the values of x corresponding to the
roots of f (x). (You can verify this for yourself.) The iterative formula is

g( x) = ( x 3+ 12) / 9x
15

10 x= 8. 8467

5 y= x
y= g( x)
y x= 1. 2438
0 x= - 1. 0905

-5

-10

-15
-2 0 2 x 4 6 8 10

Figure 2: The graph of a fixed-point equation x = g(x) fxpoint

x3 + 12
xk+1 =
9x
Various iterations using different initial values x0 lead to different results as shown in the
table:

x0 Result
-2 converges to 1.2438
2 converges to 1.2438
5 converges to 1.2438
8 converges to 1.2438
10 diverges

If you apply the Fixed-point Theorem to check the intervals from which x0 can be chosen
and obtain convergence, you would solve the inequality
2 8
|g ′ (x)| < 1 or − 1 < x + x−2 < 1
9 3

4
 x3 + 12
with g(x) = .
9x
You can think of other iteration formulae x = g(x) and try them using different initial
values.

In a sense, the rest of the methods discussed for solving nonlinear equations entails finding
different expressions of g(x).

1.1.4 Newton’s Method

The Newton’s (also known simply as Newton-Raphson’s) Method is popular for it’s fast
convergence.
It makes use of the function f (x) from the root equation f (x) = 0 and the iteration equation
f (x)
g(x) = x − ′ , leading to the iterative formula
f (x)

f (xi )
xi+1 = xi − .
f ′ (xi )

So with our example, the root equation is f (x) = x3 − 9x2 + 12 = 0, from which we get
f ′ (x) = 3x2 − 18x. The iterative formula then becomes

x3 − 9x2 + 12
xk+1 = xk −
3x2 − 18x
Having determined that there are three roots between −2 and 10, we run the algorithm for
different initial values, x0 and get the following results:

x0 = −2 x0 = 2 x0 = 7 x0 = 10
−2.0000 2.0000 7.0000 10.0000
−1.3333 1.3333 11.0952 9.0667
−1.1162 1.2460 9.5036 8.8571
−1.0909 1.2438 8.9281 8.8467
−1.0905 1.2438 8.8482 8.8467
−1.0905 . . . 8.8467 ...
We can see how fast the method converges for the various initial values. Note, in particular
that the convergence is not always monotonic; see the case for x0 = 7.

So Newton’s method is straight forward and normally much faster. We use the example
to demonstrate the variants of Newton’s method: the secant method and the regula falsi
methods. For these methods the derivative is approximated by the slope
f (xk ) − f (x + k − 1)
f ′ (xk ) ≈
(x + k − xk−1 )
to get the iterative algorithm
xk − xk−1
xk+1 = xk − f (xk ) ·
f (xk ) − f (xk−1 )
NB: These method require two initial values, x0 and x1 .

5
1.1.5 The Secant and Regula Falsi Methods
Secant Regula Falsi
4.0000 4.0000
10.0000 10.0000
6.2667 6.2667
7.9834 7.9834
10.1136 10.1136
8.6127 8.6127
8.7887 8.7887
8.8500 8.8500
8.8466 8.8466
8.8467 8.8467
8.8467 8.8467

1.2 Methods for Polynomial Functions

Horner’s method and Muller’s method have been presented as useful techniques for han-
dling polynomial functions, with specific mention of possible existence of complex roots for
Muller’s method.

1.2.1 Horner’s Method

See Example 2 on p93 for an illustration of this method.

1.2.2 Muller’s Method

An illustration for this method is given on p97.

1.3 Newton’s Method for Multi-variable Functions

Recall the similarities and differences in Newton’s formula in the two cases, using a function

F (x, y) or F (x1 , x2 ) or F (x),

where  
x1
x=
x2
While Newton’s method for f (x) = 0 uses the iteration (fixed-point) formula

x = g(x) = x − ϕ(x)f (x), with ϕ(x) = 1/f ′ (x)

Newton’s method for a function, F (x1 , x2 ) = 0 of two variables,

 
F1 (x1 , x2 )
F (x1 , x2 ) = ,
F2 (x1 , x2 )
uses  (k+1)  (k)
x1 x1
= − J −1 (x1 , x2 )F (x1 , x2 ) (4)
x2 x2

6
where J(x1 , x2 ) is now the Jacobian of F (x1 , x2 ), defined by

∂F1 (x1 , x2 ) ∂F1 (x1 , x2 )

 

JF (x1 , x2 ) =  ∂F ∂x 1 ∂x2
 
2 (x1 , x2 ) ∂F2 (x1 , x2 ) 
∂x1 ∂x2
which now replaces f ′ (x) in the case of one variable function.
Example 2
Consider the problem of approximating the solution of the nonlinear system:

5x21 = x22
0.25(sin x1 + cos x2 ) = x2

Here    
F1 (x1 , x2 ) 5x21 − x22
F (x1 , x2 ) = =
F2 (x1 , x2 ) x2 − 0.25(sin x1 + cos x2 )
The Jacobian is
 ∂F1 ∂F1   
∂x1 ∂x2 10x1 −2x2
J(x1 , x2 ) = ∂F2 ∂F2 =
∂x1 ∂x2
−0.25(cos x1 )1 + 0.25(sin x2 )

So that, using an initial guess of (x1 , x2 ) = (0, 0) we have

   
(0) (0) 0 (0) (0) 0 4
F (x1 , x2 ) = F (0, 0) = J(x1 , x2 ) =
0.2500 −4 −12

and the next iterate is

Iteration 1
" # −1 
(1)    
x1 0 0 4 0
(1) = − ∗
x2 0 −4 −12 0.2500
 
0.062500
=
0

Iteration 2
" # −1 
(2)    
x1 0.062500 −044274 4.0 0.019531
(2) = − ∗
x2 0 −3.8750 −12.0 0.234385
 
0.13560
=
−0.0040737

7
Carrying out a few more iterations yields

k x1 x2
0 0 0
1 0.0625 0
2 0.1356 −0.0041
3 0.2570 −0.0241
4 0.5430 −0.0925
5 1.6178 −0.3362
6 7.2729 −0.6543
7 9.0181 0.3750
8 9.8137 1.6723
9 10.3153 −5.2785
10 10.7114 −5.2483
11 11.0374 −5.2335
12 11.3143 −5.2308
13 11.5550 −5.2380

We can observe that there is some convergence.