ENGINEERING
MATHEMATICS III
NUMERICAL METHODS FOR SOLVING EQUATIONS
Mohsen Aghaeiboorkheili
Department of Mathematics & Computer Science
The Papua New Guinea University of Technology
February 12, 2025
Numerical Methods Introduction
Numerical methods are used to approximate solutions of equations
when exact solutions can not be determined via algebraic methods.
They construct successive approximations that converge to the exact
solution of an equation or system of equations.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 2/35
Numerical Methods Introduction
Numerical methods are used to approximate solutions of equations
when exact solutions can not be determined via algebraic methods.
They construct successive approximations that converge to the exact
solution of an equation or system of equations.
ex − sin(x) = 0.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 2/35
Numerical Methods Bisection Method
Theorem 1.1
Special case of Intermediate Value Theorem:
Let f (x) be a continuous function on the interval [a, b]. If
f (a)f (b) < 0, then there is a c ∈ [a, b] such that f (c) = 0.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 3/35
Numerical Methods Bisection Method
Consider a transcendental equation f (x) = 0 which has a zero in
the interval [a, b] and f (a) ∗ f (b) < 0. Bisection scheme computes
the zero, say c, by repeatedly halving the interval [a, b]. That is,
starting with
a+b
c= ,
2
the interval [a, b] is replaced either with [c, b] or with [a, c] depending
on the sign of f (a) ∗ f (c). This process is continued until the zero
is obtained. Since the zero is obtained numerically the value of c
may not exactly match with all the decimal places of the analytical
solution of f (x) = 0 in the interval [a, b]. Hence any one of the
following mechanisms can be used to stop the bisection iterations :
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 4/35
Numerical Methods Bisection Method
Fixing a priori the total number of bisection iterations N i.e.,
the length of the interval or the maximum error after N
iterations in this case is less than |b − a|/2N .
By testing the condition |ci − ci−1 | (where i are the iteration
number) less than some tolerance limit, say epsilon, fixed a
priori.
By testing the condition |f (ci )| less than some tolerance limit
alpha again fixed a priori.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 5/35
Numerical Methods Bisection Method Examples
Example 1
Find the root of f (x) = e−x 3.2 sin(x) − 0.5 cos(x) on the interval
[3, 4] with the help of bisection method.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 6/35
Numerical Methods Bisection Method Examples
Example 1
Find the root of f (x) = e−x 3.2 sin(x) − 0.5 cos(x) on the interval
[3, 4] with the help of bisection method.
Solution
Note that f (3) = 0.047 > 0, and f (4) = −0.038 < 0.
1st iteration:
f (3) = 0.047 3+4
⇒ f (3)f (4) < 0 ⇒ x1 = = 3.5.
f (4) = −0.038 2
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 6/35
Numerical Methods Bisection Method Examples
Solution ( Cont.)
2nd iteration:
f (3) = 0.047 3 + 3.5
f (3)f (3.5) < 0
f (3.5) = −0.019 ⇒ ⇒ x2 = = 3.25.
f (3.5)f (4) > 0 2
f (4) = −0.038
3rd iteration:
f (3) = 0.047 3.25 + 3.5
f (3)f (3.25) > 0
f (3.25) = 0.005 ⇒ ⇒ x3 = = 3.375.
f (3.25)f (3.5) < 0 2
f (3.5) = −0.019
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 7/35
Numerical Methods Bisection Method Examples
Example 2
√
Compute numerical approximations for the 2 with the help of the
bisection method.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 8/35
Numerical Methods Bisection Method Examples
Example 2
√
Compute numerical approximations for the 2 with the help of the
bisection method.
Solution
We set f (x) = x2 − 2. Let us start with an interval of length
one: a = 1 and b = 2. Note that f (a) = f (1) = −1 < 0, and
f (b) = f (2) = 2 > 0.
1st iteration:
f (1) = −1 1+2
⇒ f (1)f (2) < 0 ⇒ x1 = = 1.5.
f (2) = 2 2
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 8/35
Numerical Methods Bisection Method Examples
Solution ( Cont.)
2nd iteration:
f (1) = −1 1 + 1.5
f (1)f (1.5) < 0
f (1.5) = 0.25 ⇒ ⇒ x2 = = 1.25.
f (1.5)f (2) > 0 2
f (2) = 2
3rd iteration:
f (1) = −1 1.25 + 1.5
f (1)f (1.25) > 0
f (1.25) = −0.43 ⇒ ⇒ x3 = = 1.375.
f (1.25)f (1.5) < 0 2
f (1.5) = 0.25
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 9/35
Numerical Methods Bisection Method Examples
Example 3
Find a root of f (x) = 3x + sin(x) − exp(x) = 0.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 10/35
Numerical Methods Bisection Method Examples
Example 3
Find a root of f (x) = 3x + sin(x) − exp(x) = 0.
Solution
The graph of this equation is given in the figure.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 10/35
Numerical Methods Bisection Method Examples
Solution ( Cont.)
Its clear from the graph that there are two roots, one lies between 0 and 0.5 and the other lies between 1.5 and
2.0. Consider the function f (x) in the interval [0, 0.5].
1st iteration:
f (0) = −1 0 + 0.5
⇒ f (0)f (0.5) < 0 ⇒ x1 = = 0.25.
f (0.5) = 0.33 2
2nd iteration:
f (0) = −1 0.25 + 0.5
f (0)f (0.25) > 0
f (0.25) = −0.28 ⇒ ⇒ x2 = = 0.375.
f (0.25)f (0.5) < 0 2
f (0.5) = 0.33
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 11/35
Numerical Methods Bisection Method Examples
Solution ( Cont.)
3rd iteration:
f (0.25) = −0.28
f (0.25)f (0.375) < 0
f (0.375) = 0.03 ⇒ ⇒ x3 = 0.25+0.375 = 0.3125
f (0.375)f (0.5) > 0 2
f (0.5) = 0.33
4th iteration:
f (0.25) = −0.28
f (0.25)f (0.3125) > 0
f (0.3125) = −0.12 ⇒ ⇒ x4 = 0.3125+0.375 = 0.34375
f (0.3125)f (0.375) < 0 2
f (0.375) = 0.03
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 12/35
Numerical Methods Bisection Method Examples
Definition 1
In every iteration of bisection(or any other methods) the value of
|f (xi )| is called ”absolute error of i−th iteration. It will be shown
by abs . Moreover the width of i−th interval is called step of that
iteration.
Absolute error of bisection method for the example 1 in the 3−rd
iteration is:
abs = |f (x3 )| = |f (3.375)| = | − 0.00868108| = 0.00868108,
step = 3.5 − 3.25 = 0.25
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 13/35
Numerical Methods Bisection Method Examples
Remark 1
Using the Bisection Method, in order to calculate an approximate
root of a function within tolerance , the number n of iterations we
need to perform is:
b − a
log
n≥
log(2)
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 14/35
Numerical Methods Bisection Method Examples
Remark 1
Using the Bisection Method, in order to calculate an approximate
root of a function within tolerance , the number n of iterations we
need to perform is:
b − a
log
n≥
log(2)
Example 4
What is the minimum calculated number of iterations using bisection
method for the following function on [−2, 5] to converge the solution
within the 0.01 tolerance?
f (x) = 10 − x2 .
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 14/35
Numerical Methods Bisection Method Examples
Solution
5 − (−2)
a = −2 log
b=5 ⇒n≥ 0.01 ⇒ n ≥ 9.45
log(2)
= 0.01
The result shown that we need at least 10 iterations to converge the
solution within predefined tolerance.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 15/35
Numerical Methods Bisection Method Examples
Remark 2
The Bisection Method is a simple root finding method, easy to
implement and very robust. The disadvantages of this method is
that its relatively slow. Because of this, most of the time, the
bisection method is used as a starting point to obtain a rough value
of the solution which is used later as a starting point for more rapidly
converging methods.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 16/35
Numerical Methods Newton-Raphson Method Examples
Let x0 be a good estimation for the root of f (x) = 0. The
next estimate xn+1 using Newton-Raphson Method(or Newton’s
method) is given by
f (xi )
xi+1 = xi − i = 0, 1, 2, 3, · · ·
f 0 (xi )
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 17/35
Numerical Methods Newton-Raphson Method Examples
Let x0 be a good estimation for the root of f (x) = 0. The
next estimate xn+1 using Newton-Raphson Method(or Newton’s
method) is given by
f (xi )
xi+1 = xi − i = 0, 1, 2, 3, · · ·
f 0 (xi )
Example 5
√
Compute numerical approximations for the 2 with help of
Newton-Raphson method.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 17/35
Numerical Methods Newton-Raphson Method Examples
Let x0 be a good estimation for the root of f (x) = 0. The
next estimate xn+1 using Newton-Raphson Method(or Newton’s
method) is given by
f (xi )
xi+1 = xi − i = 0, 1, 2, 3, · · ·
f 0 (xi )
Example 5
√
Compute numerical approximations for the 2 with help of
Newton-Raphson method.
Solution
Let’s start with a close approximation, say x0 = 1.5. In this case,
we have f (x) = x2 − 2 and f 0 (x) = 2x. So
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 17/35
Numerical Methods Newton-Raphson Method Examples
Solution ( Cont.)
1st iteration:
x0 = 1.5 f (x0 ) 0.25
f (x0 ) = 0.25 ⇒ x1 = x0 − = 1.5 − = 1.41667,
f 0 (x0 ) = 3 f 0 (x0 ) 3
2nd iteration:
x1 = 1.41667 f (x1 ) 0.00695389
f (x1 ) = 0.00695389 ⇒ x2 = x1 − = 1.41667 − = 1.41422
f 0 (x1 ) = 2.83334 f 0 (x1 ) 2.83334
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 18/35
Numerical Methods Newton-Raphson Method Examples
Solution ( Cont.)
3rd iteration:
x2 = 1.41422 f (x2 ) 0.0000182084
f (x2 ) = 0.0000182084 ⇒ x3 = x2 − = 1.41422 − = 1.41421
f 0 (x2 ) = 2.82844 f 0 (x2 ) 2.82844
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 19/35
Numerical Methods Secant Method
The Secant Method is the most popular of the many variants of
the Newton Method. We start with two estimates of the root, x0
and x1 . The iterative formula, for i ≥ 1 is
f (xi )
xi+1 = xi − i = 1, 2, 3, · · ·
Q(xi−1 , xi )
where
f (xi−1 ) − f (xi )
Q(xi−1 , xi ) = i = 1, 2, 3, · · ·
xi−1 − xi
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 20/35
Numerical Methods Secant Method Examples
Example 6
√
Compute numerical approximations for the 2 with the help of the Secant method.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 21/35
Numerical Methods Secant Method Examples
Example 6
√
Compute numerical approximations for the 2 with the help of the Secant method.
Solution
Let’s start with close approximations, say x0 = 1 and x1 = 1.5. In this case, we have f (x) = x2 − 2. So
1st iteration:
x0 = 1
x1 = 1.5
f (x0 ) = −1 f (x1 )
f (x1 ) = 0.25 ⇒ x2 = x1 − = 1.4
Q(x0 , x1 )
f (x0 ) − f (x1 )
Q(x0 , x1 ) = = 2.5
x0 − x1
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 21/35
Numerical Methods Secant Method Examples
Solution ( Cont.)
2nd iteration:
x1 = 1.5
x2 = 1.4
f (x1 ) = 0.25 f (x2 )
f (x2 ) = −0.04 ⇒ x3 = x2 − = 1.41379
Q(x1 , x2 )
f (x1 ) − f (x2 )
Q(x1 , x2 ) = = 2.9
x1 − x2
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 22/35
Numerical Methods Secant Method Examples
The Secant Method has some advantages over the Newton Method.
It is more stable, less subject to the wild gyrations that can afflict the
Newton Method(The differences are not great, since the geometry
is nearly the same). To use the Secant Method, we do not need the
derivative, which can be expensive to calculate. The Secant Method,
when it is working well, which is most of the time, is fast. Usually
we need about 45 percent more iterations than with the Newton
Method to get the same accuracy, but each iteration is cheaper.
Your mileage may vary.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 23/35
Numerical Methods Secant Method Examples
Consider a transcendental equation f (x) = 0 which has a zero in
the interval [a, b] and f (a) ∗ f (b) < 0. False Position Technique
computes the zero, say c, by repeatedly halving the interval [a, b].
That is, starting with
af (b) − bf (a)
c= ,
f (b) − f (a)
the interval [a, b] is replaced either with [c, b] or with [a, c] depending
on the sign of f (a) ∗ f (c). This process is continued until the zero
is obtained. Since the zero is obtained numerically the value of c
may not exactly match with all the decimal places of the analytical
solution of f (x) = 0 in the interval [a, b].
Hence any one of the following mechanisms can be used to stop the
false position iterations :
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 24/35
Numerical Methods Secant Method Examples
Fixing a prior the total number of false position iterations N
i.e., the length of the interval or the maximum error after N
iterations in this case is less than |b − a|/2N .
By testing the condition |ci − ci−1 | (where i are the iteration
number) less than some tolerance limit, say epsilon, fixed a
prior.
By testing the condition |f (ci )| less than some tolerance limit
alpha again fixed a prior.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 25/35
Numerical Methods Secant Method Examples
Example 7
Find the root of f (x) = e−x 3.2 sin(x) − 0.5 cos(x) on the interval [3, 4] with the help
of false position method.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 26/35
Numerical Methods Secant Method Examples
Example 7
Find the root of f (x) = e−x 3.2 sin(x) − 0.5 cos(x) on the interval [3, 4] with the help
of false position method.
Solution
Note that f (3) = 0.047 > 0, and f (4) = −0.038 < 0.
1st iteration:
f (3) = 0.047 3f (4) − 4f (3)
⇒ f (3)f (4) < 0 ⇒ x1 = = 3.55121.
f (4) = −0.038 f (4) − f (3)
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 26/35
Numerical Methods Secant Method Examples
Solution ( Cont.)
2nd iteration:
f (3) = +0.047 3f (3.55121) − 3.55121f (3)
f (3)f (3.55121) < 0
f (3.55121) = −0.0234049 ⇒ ⇒ x2 =
f (3.55121)f (4) > 0 f (3.55121) − f (3)
f (4) = −0.038
x2 = 3.3683
3rd iteration:
f (3) = +0.047 3f (3.3683) − 3.3683f (3)
f (3)f (3.3683) < 0
f (3.3683) = −0.00799401 ⇒ ⇒ x3 =
f (3.3683)f (3.55121) > 0 f (3.3683) − f (3)
f (3.55121) = −0.0234049
x3 = 3.31489
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 27/35
Numerical Methods Secant Method Examples
The final method we will use in this section to determine the roots
of the function is the Fixed-point method. In this approach, the
equation is rearranged into the form x = g(x). After that, starting
from an initial value x0 , the iteration formula for finding subsequent
values x1 , x2 , x3 and so on, is given by:
xn+1 = g(xn )
When selecting g(x), we must be very careful and choose it ap-
propriately. There are two important conditions to consider when
determining a suitable g(x) :
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 28/35
Numerical Methods Secant Method Examples
1 If x lies between a and b, then g(x) must also be within the
interval [a, b].
a < x < b =⇒ a < g(x) < b
2 The absolute value of the derivative |g 0 (x)| should be less
than 1 for the iteration process to converge to the root.
a < x < b =⇒ |g 0 (x)| < 1
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 29/35
Numerical Methods Secant Method Examples
Example 8
Apply the fixed-point method to determine the roots of the function
f (x) = x3 − x − 1, on the interval 1 < x < 2 using an initial value
of x0 = 1.5.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 30/35
Numerical Methods Secant Method Examples
Example 8
Apply the fixed-point method to determine the roots of the function
f (x) = x3 − x − 1, on the interval 1 < x < 2 using an initial value
of x0 = 1.5.
Solution
Given the function f (x) = x3 − x − 1, we can rearrange it as:
√
x= 3x+1
This implies that the iteration function is:
√
g(x) = 3 x + 1
Before proceeding, let’s check whether this function g(x) converges.
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 30/35
Numerical Methods Secant Method Examples
1 < g(x) < 2
1 < x < 2 =⇒ 1
|g 0 (x)| = p <1
3 (x + 1)2
3
1st iteration:
√
3
x1 = g(x0 ) = g(1.5) = 1.5 + 1 = 1.35721
2nd iteration:
√
3
x2 = g(x1 ) = g(1.35721) = 1.35721 + 1 = 1.33086
3rd iteration:
√
3
x3 = g(x2 ) = g(1.33086) = 1.33086 + 1 = 1.32588
Mohsen Aghaei PNG Unitech ENG. MATHEMATICS III February 12, 2025 31/35
Thanks for your attention...
