Chapter 3

Numerical Solutions of ODE

School of Mathematical Sciences, Sunway University 1 MAT2034 Mathematical Methods

 Introduction

• A first order differential equation (DE) where

𝑑𝑦
= 𝑓 𝑥, 𝑦
𝑑𝑥
is a source of information.
• Previously, we only examined them analytically and solved it
analytically.
• DE can possess a solution, may not be possible to obtain it
analytically for certain equations.
• But, we can “solve” the differential equation numerically.
• A method that utilizes the idea that a tangent line can be used.
• We approximate the values of a function in a small neighborhood
of the point of tangency.

School of Mathematical Sciences, Sunway University 2 MAT2034 Mathematical Methods

 Introduction
• Let assume that the first-order initial-value problem

𝑦 ′ = 𝑓 𝑥, 𝑦 , 𝑦 0 = 𝑦0
possess a solution.
• One way of approximating this solution is to use tangent lines.
• For example, let 𝑦(𝑥) denote the unknown solution of the first-order
initial value problem

𝑦 ′ = 0.1 𝑦 + 0.4𝑥 2 , 𝑦 2 = 4

• The nonlinear differential equation in this IVP cannot be solved directly

by any of the methods considered in previous chapters.
• We can find approximate numerical values of the unknown 𝑦(𝑥).

School of Mathematical Sciences, Sunway University 3 MAT2034 Mathematical Methods

 Introduction

Figure 3.1 Magnification of a neighborhood about the point (2,4).

• Figure 3.1(a) - generated with lineal elements passing through

points in a grid with integer coordinates.
• As the solution curve passes through the initial point (2, 4), the
lineal element at this point is a tangent line with slope given by
𝑓 2,4 = 0.1 4 + 0.4 0.2 2 = 1.8

School of Mathematical Sciences, Sunway University 4 MAT2034 Mathematical Methods

 Introduction

Figure 3.1 Magnification of a neighborhood about the point (2,4).

• When 𝑥 is close to 2 (zoom from Figure 3.1 (a)), the points on the
solution curve are close to the points on the tangent line (the lineal
element).
• Using the point (2, 4), the slope 𝑓 2,4 = 1.8, and the point-slope
form of a line, we find that an equation of the tangent line is
𝑦 = 𝐿(𝑥), where 𝑳 𝒙 = 𝟏. 𝟖𝒙 + 𝟎. 𝟒.

School of Mathematical Sciences, Sunway University 5 MAT2034 Mathematical Methods

 Introduction

Figure 3.1 Magnification of a neighborhood about the point (2,4).

• The equation is called a linearization of 𝑦(𝑥) at 𝑥 = 2, can be used

to approximate values of 𝑦(𝑥) within a small neighborhood of 𝑥 =
2.
• If 𝑦1 = 𝐿(𝑥1 ) denotes the y-coordinate on the tangent line and
𝑦 𝑥1 is the 𝒚-coordinate on the solution curve corresponding to
an 𝑥-coordinate 𝑥1 that is close to 𝑥 = 2, then 𝒚(𝒙𝟏 ) ≈ 𝒚𝟏 .

School of Mathematical Sciences, Sunway University 6 MAT2034 Mathematical Methods

 Euler’s Method for 1st Order ODEs

Definition 3.1.1

A first order differential equation with an initial value can be stated as

follows:
𝑑𝑦
= 𝑓 𝑥, 𝑦 , 𝑦 = 𝑦0 at 𝑥 = 𝑥0. (1)
𝑑𝑥

The description in equation (1) consists of the differential equation

itself and a given solution 𝑦0 at initial location 𝑥0 .

• We then obtain the solution 𝒚 as 𝑥 ranges from its initial value to

some other value.
• Euler’s method (forward Euler method): single step, explicit method
to solve first-order ODE.

School of Mathematical Sciences, Sunway University 7 MAT2034 Mathematical Methods

 Euler’s Method for 1st Order ODEs

• The method uses the following equations to obtain approximate

solutions at 𝑥𝑛 , 𝑦𝑛 ,

𝑥𝑛+1 = 𝑥𝑛 + ℎ (2)
𝑦𝑛+1 = 𝑦𝑛 + Slope ℎ (3)

where the value of the (𝑆𝑙𝑜𝑝𝑒) in Eq. (3) is the slope of 𝑦(𝑥) at point
𝑥𝑛 , 𝑦𝑛 .

• Let’s consider a slope computed from the

differential equation:
𝑑𝑦
𝑆𝑙𝑜𝑝𝑒 = ቤ = 𝑓(𝑥0 , 𝑦0 )
𝑑𝑥 𝑥=𝑥
0

School of Mathematical Sciences, Sunway University 8 MAT2034 Mathematical Methods

 Euler’s Method for 1st Order ODEs

𝑑𝑦 𝑓(𝑥𝑛 , 𝑦𝑛 )
𝑆𝑙𝑜𝑝𝑒 = ቤ = 𝑓(𝑥0 , 𝑦0 )
𝑑𝑥 𝑥=𝑥 𝑦𝑛+1 − 𝑦𝑛
0
ℎ 𝒚𝒏+𝟏 − 𝒚𝒏
𝑦 𝒇 𝒙𝒏 , 𝒚𝒏 =
𝒉
Exact Solution
𝑦(𝑥)
Numerical Solution
𝑓(𝑥𝑛+1 )

𝑦𝑛+1 𝑆𝑙𝑜𝑝𝑒: 𝑓(𝑥𝑛 , 𝑦𝑛 )

𝑦𝑛

𝑥
𝑥𝑛 ℎ 𝑥𝑛+1 𝒙𝒏+𝟏 = 𝒙𝒏 + 𝒉
School of Mathematical Sciences, Sunway University 9 MAT2034 Mathematical Methods
 Euler’s Method for 1st Order ODEs
𝑦
𝑦(𝑥)
A smaller ℎ 𝒚𝒏+𝟏 − 𝒚𝒏
𝑓(𝑥𝑛+1 ) 𝒇 𝒙𝒏 , 𝒚𝒏 =
𝒉
error
𝑦𝑛+1
𝑆𝑙𝑜𝑝𝑒: 𝑓(𝑥𝑛 , 𝑦𝑛 )
𝑦𝑛 𝒚𝒏+𝟏 = 𝒚𝒏 + 𝒉𝒇 𝒙𝒏 , 𝒚𝒏
𝑥 𝒙𝒏+𝟏 = 𝒙𝒏 + 𝒉
𝑥𝑛 ℎ 𝑥𝑛+1

Euler’s method assumes for a short distance ℎ near (𝑥𝑛 , 𝑦𝑛 ), the

function 𝑦(𝑥) has a constant slope equal to the slope at 𝑥𝑛 , 𝑦𝑛 .

The error in this method depends on the value of ℎ.

The error is small when the value for 𝒉 is small.

School of Mathematical Sciences, Sunway University 10 MAT2034 Mathematical Methods

 Euler’s Method for 1st Order ODEs
𝑥𝑛+1 = 𝑥𝑛 + ℎ (4)
𝑦𝑛+1 = 𝑦𝑛 + ℎ𝑓 𝑥𝑛 , 𝑦𝑛 (5)

Equation (5) can be derived as follows:

Consider the DE
𝑑𝑦
= 𝑓 𝑥, 𝑦 (6)
𝑑𝑥

with initial condition 𝑦 𝑥0 = 𝑦0 .

Integrating equation (6), we obtain

𝑥
𝑦 = 𝑦0 + න 𝑓 𝑥, 𝑦 𝑑𝑥 (7)
𝑥0

School of Mathematical Sciences, Sunway University 11 MAT2034 Mathematical Methods

 Euler’s Method for 1st Order ODEs
Approximation of value 𝑦, let’s 𝑦
say 𝑦(𝑥)
𝑦𝑛 when 𝑥 = 𝑥𝑛 Exact Solution
Numerical Solution
We divide the interval [𝑥0 , 𝑥𝑛 ] 𝑦𝑛
𝑦3
into 𝑛 subintervals of equal 𝑦2
𝑦1
length, say 𝒉 with the division 𝑦0
point 𝒙𝟎 , 𝒙𝟏 , 𝒙𝟐 , … , 𝒙𝒏 where 𝑥
𝑥0 𝑥1 𝑥2 𝑥3 𝑥𝑛
𝑥 = 𝑥𝑟 = 𝑥0 + 𝑟ℎ,
𝑟 = 1,2,3, … Assume that 𝑓 𝑥, 𝑦 = 𝑓 𝑥0 , 𝑦0
in 𝑥0 ≤ 𝑥 ≤ 𝑥1 , the equation (8)
Then, from equation (7), we have leads to
𝑥1
𝑦1 = 𝑦0 + න 𝑓 𝑥, 𝑦 𝑑𝑥 (8) 𝑦1 = 𝑦0 + ℎ𝑓 𝑥0 , 𝑦0 (9)
𝑥0 Equation (9) is called the Euler’s
formula.
School of Mathematical Sciences, Sunway University 12 MAT2034 Mathematical Methods
 Euler’s Method for 1st Order ODEs
Similarly, for the range of 𝑥1 ≤ 𝑥 ≤ 𝑥2 , we have
𝑥2
𝑦2 = 𝑦1 + න 𝑓 𝑥, 𝑦 𝑑𝑥 = 𝑦1 + ℎ𝑓(𝑥1 , 𝑦1 )
𝑥1
and for the range 𝑥2 ≤ 𝑥 ≤ 𝑥3 , we get
𝑦3 = 𝑦2 + ℎ𝑓 𝑥2 , 𝑦2
and so on.
Finally we obtain

𝑦𝑛+1 = 𝑦𝑛 + ℎ𝑓 𝑥𝑛 , 𝑦𝑛 , 𝑛 = 0,1,2,3, … . . (10)

Note:
Euler’s method accumulates a large error as the process proceeds. The
process is known to be very slow and in order to obtain reasonable
accuracy, the value of 𝒉 needs to be smaller.
School of Mathematical Sciences, Sunway University 13 MAT2034 Mathematical Methods
 Euler’s Method for 1st Order ODEs
Example 3.1.1

Consider the initial-value problem

𝑦 ′ = 2𝑥𝑦, 𝑦 1 =1
(a) Find an explicit solution to the IVP.
(b) Use Euler’s method to obtain an approximation of 𝑦(1.5) using first
ℎ = 0.1 and then ℎ = 0.05.

Solution:
(a) Separate variables and integrating, we have
𝑑𝑦
= 2𝑥 and ln y = x 2 + C
𝑦
Using 𝑦 1 = 1, 𝐶 = −1
𝟐
ln 𝑦 = 𝑥 2 − 1 ⟹ 𝒚 = 𝒆𝒙 −𝟏
is the explicit solution of the IVP.
School of Mathematical Sciences, Sunway University 14 MAT2034 Mathematical Methods
 Euler’s Method for 1st Order ODEs
Example 3.1.1

Consider the initial-value problem

𝑦 ′ = 2𝑥𝑦, 𝑦 1 =1
(a) Find an explicit solution to the IVP.
(b) Use Euler’s method to obtain an approximation of 𝑦(1.5) using first
ℎ = 0.1 and then ℎ = 0.05.

Solution:
(b) Using Euler’s formula
𝑦𝑛+1 = 𝑦𝑛 + ℎ𝑓 𝑥𝑛 , 𝑦𝑛 , 𝑛 = 0,1,2,3, …
or 𝑦𝑛+1 = 𝑦𝑛 + ℎ 2𝑥𝑛 , 𝑦𝑛
Then, ℎ = 0.1, 𝑥0 = 1, 𝑦0 = 1, 𝑛 = 0
𝑦1 = 𝑦0 + ℎ 2𝑥0 , 𝑦0 = 1 + 0.1 2 1 1 = 𝟏. 𝟐𝟎𝟎𝟎
which estimate 𝒚 𝟏. 𝟏 .
School of Mathematical Sciences, Sunway University 15 MAT2034 Mathematical Methods
 Euler’s Method for 1st Order ODEs
Example 3.1.1

Consider the initial-value problem

𝑦 ′ = 2𝑥𝑦, 𝑦 1 =1
(a) Find an explicit solution to the IVP.
(b) Use Euler’s method to obtain an approximation of 𝑦(1.5) using first
ℎ = 0.1 and then ℎ = 0.05.

Solution:
(b) If we use ℎ = 0.05, we will take two steps to reach 𝑦(1.1) such as

𝑦1 = 1 + 0.05 2 1 1 = 𝟏. 𝟏𝟎𝟎𝟎
𝑦2 = 1.1000 + 0.05 2 1.05 1.10000 = 𝟏. 𝟐𝟏𝟓𝟓

Here, 𝒚𝟏 ≈ 𝒚 𝟏. 𝟎𝟓 and 𝒚𝟐 ≈ 𝒚 𝟏. 𝟏 .
School of Mathematical Sciences, Sunway University 16 MAT2034 Mathematical Methods
 Euler’s Method for 1st Order ODEs
Solution: |𝑎𝑐𝑡𝑢𝑎𝑙 − 𝑦𝑛 | 𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝑒𝑟𝑟𝑜𝑟
× 100
𝒚𝒏+𝟏 = 𝒚𝒏 + 𝒉𝒇 𝒙𝒏 , 𝒚𝒏 , 𝒏 = 𝟎, 𝟏, 𝟐, 𝟑, … . . 𝑎𝑐𝑡𝑢𝑎𝑙 𝑣𝑎𝑙𝑢𝑒

𝑓 𝑥, 𝑦 = 2𝑥𝑦, ℎ = 0.1 MS Excel

𝒏 𝒙𝒏 𝒚𝒏 Actual Absolute % Relative
value error error
0 1.00 1.0000 1.0000 0.0000 0.00
1 1.10 1.2000 1.2337 0.0337 2.73
2 1.20 1.4640 1.5527 0.0887 5.71
3 1.30 1.8154 1.9937 0.1784 8.95
4 1.40 2.2874 2.6117 0.3243 12.42
5 1.50 2.9278 3.4903 0.5625 16.12

School of Mathematical Sciences, Sunway University 17 MAT2034 Mathematical Methods

 Euler’s Method for 1st Order ODEs
Solution: 𝒚𝒏+𝟏 = 𝒚𝒏 + 𝒉𝒇 𝒙𝒏 , 𝒚𝒏 , 𝒏 = 𝟎, 𝟏, 𝟐, 𝟑, … . . ℎ = 0.05

𝒏 𝒙𝒏 𝒚𝒏 Actual value Error % Error

0 1.00 1.0000 1.0000 0.0000 0.00
1 1.05 1.1000 1.1079 0.0079 0.72
2 1.10 1.2155 1.2337 0.0182 1.47
3 1.15 1.3492 1.3806 0.0314 2.27
4 1.20 1.5044 1.5527 0.0483 3.11
5 1.25 1.6849 1.7551 0.0702 4.00
6 1.30 1.8955 1.9937 0.0982 4.93
7 1.35 2.1419 2.2762 0.1343 5.90
8 1.40 2.4311 2.6117 0.1806 6.92
9 1.45 2.7714 3.0117 0.2403 7.98
10 1.50 3.1733 3.4903 0.3171 9.08
School of Mathematical Sciences, Sunway University 18 MAT2034 Mathematical Methods
 Error in Numerical Methods

• In choosing numerical method for an IVP, we must be aware various

sources of errors.
• Accumulation of errors may reduce the accuracy of an approximation
to the point, the computation will be useless.
• All depends on which numerical solution is applied to, extreme
accuracy may not be needed due to its expenses and complication.

Definition 3.2.1 Round-off Error

• Round-off error is always present in calculations (approximation

representation of numbers).
• Calculator or computer may cause the error results because only
finite number or digits are used.
1
• E.g. a calculator that uses base 10 arithmetic, carries 4 digits, so 3 is
1
represented in the calculator as 0.3333 and 9 = 0.1111.
School of Mathematical Sciences, Sunway University 19 MAT2034 Mathematical Methods
 Error in Numerical Methods
Example on round-off errors
For this equation
1 So, we will look into
𝑥2 − 9
𝑦= errors introduced by
1 formula or algorithm
𝑥−
3 to approximate the
solution for errors.
For 𝑥 = 0.3334, y =? With calculator, we have
(0.3334)2 −0.1111 0.1112 − 0.1111
𝑦= = =𝟏
0.3334 − 0.3333 0.3334 − 0.3333
But,
1 1 1
𝑥 2 − 9 (𝑥 − 3)(𝑥 + 3) 1
𝑦= = =𝑥+
1 1 3
𝑥−3 𝑥−3
𝑦 ≈ 0.3334 + 0.3333 = 𝟎. 𝟔𝟔𝟔𝟕

School of Mathematical Sciences, Sunway University 20 MAT2034 Mathematical Methods

 Error in Numerical Methods
Definition 3.2.2 Truncation Error
A truncation error occurs when approximation is involved in a
mathematical procedure.

E.g., Taylor series expansion for 𝑒 𝑥 at 𝑎 = 0 is

𝑥
𝑥2 𝑥3
𝑒 = 1+𝑥+ + +⋯
2! 3!
Then,
0.5
0.52 0.53
𝑒 = 1 + 0.5 + + +⋯
2! 3!

truncation error

School of Mathematical Sciences, Sunway University 21 MAT2034 Mathematical Methods

 Truncation Errors in Euler’s Method

• In the sequence of values 𝑦1 , 𝑦2 , 𝑦3 , … generated from

𝒚𝒏+𝟏 = 𝒚𝒏 + 𝒉𝒇 𝒙𝒏 , 𝒚𝒏 , 𝒏 = 𝟎, 𝟏, 𝟐, 𝟑, …
• Usually the value of 𝑦1 will not agree with the actual solution at 𝑥1 -
namely , 𝑦 𝑥1 because the algorithm gives only a straight – line
approximation to the solution.

• The error is called the local truncation error, formula error or

discretization error.

• It occurs at each step; that is, if we assume that 𝑦𝑛 will contain

local truncation error.

• To derive a formula for the local truncation error for Euler’s

method, we use Taylor’s formula with remainder.

School of Mathematical Sciences, Sunway University 22 MAT2034 Mathematical Methods

 Truncation Errors in Euler’s Method

If a function 𝑦(𝑥) possess 𝑘 + 1 derivatives that

Taylor’s
are continuous on an open interval containing 𝑎
formula and 𝑥, then

𝑥−𝑎 𝑥−𝑎 2 𝑥−𝑎 𝑘 𝑥 − 𝑎 𝑘+1

𝑦 𝑥 =𝑦 𝑎 + 𝑦′ a + 𝑦 ′′ 𝑎 + ⋯+ 𝑦𝑘 𝑎 + 𝑦 𝑘+1 𝒄
1! 2! 𝑘! 𝑘+1 !
where 𝒄 is some point between 𝑎 and 𝑥.

When we set 𝒌 = 𝟏, 𝒂 = 𝒙𝒏 and 𝒙 = 𝒙𝒏+𝟏 = 𝒙𝒏 + 𝒉, we get

′
ℎ ′′
ℎ2
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 + 𝑦 𝑥𝑛 +𝑦 𝑐
1! 2!
or
′′
ℎ2
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 + ℎ𝑓 𝑥𝑛 , 𝑦𝑛 + 𝑦 𝑐
2!
𝒚𝒏+𝟏

School of Mathematical Sciences, Sunway University 23 MAT2034 Mathematical Methods

 Truncation Errors in Euler’s Method
𝒉𝟐
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 + ℎ𝑓 𝑥𝑛 , 𝑦𝑛 + 𝒚′′ 𝒄
𝟐!
• If the last term is removed, it is an Euler’s formula.
• Hence, the local truncation error in 𝑦𝑛+1 is

ℎ2
𝑦′′(𝑐) , where 𝑥𝑛 < 𝑐 < 𝑥𝑛+1
2!

• The value of 𝑐 is usually unknown (it exists theoretically), so the

exact error cannot be calculated, but an upper bound on the
absolute value of the error is
𝑴𝒉𝟐
𝟐!

where 𝑴 = 𝒎𝒂𝒙𝒙𝒏 <𝒙<𝒙𝒏+𝟏 𝒚′′(𝒄)

School of Mathematical Sciences, Sunway University 24 MAT2034 Mathematical Methods

 Truncation Errors in Euler’s Method
• Errors from numerical method, we use notation 𝑶 𝒉𝒏 .
• Let 𝑒(ℎ) denote the error in a numerical method calculation depending
on ℎ.
• The, 𝑒(ℎ) is the order of ℎ𝑛 denoted by 𝑂(ℎ𝑛 ), if there is a constant 𝐶
and a positive integer 𝑛 such that
𝑒 ℎ ≤ 𝐶ℎn
for ℎ sufficiently small.
• Thus, the local truncation error for Euler’s method is 𝑶 𝒉𝟐 .
• In general, if 𝑒(ℎ) in a numerical method is of order ℎ𝑛 and ℎ is halved,
the new error is approximately
𝑛
ℎ 𝐶ℎ𝑛
𝐶 = 𝑛
2 2
• The error is reduced by a factor of
𝟏
𝟐𝒏

School of Mathematical Sciences, Sunway University 25 MAT2034 Mathematical Methods

 Truncation Errors in Euler’s Method
Example 3.3.1 Bound for Local Truncation Errors
Find a bound for the local truncation errors for Euler’s method applied to
𝑦 ′ = 2𝑥𝑦, 𝑦 1 = 1.
Solution:
2
From Example 3.1.1, the solution to the IVP is 𝑦 = 𝑒 𝑥 −1. Hence,
2
𝒚′ = 𝟐𝒙𝑒 𝑥 −1
2 2 𝟐
𝒚′′ = 2𝑥 2𝑥𝑒 𝑥 −1 + 2𝑒 𝑥 −1 = 𝒆𝒙 −𝟏 (𝟒𝒙𝟐 + 𝟐)
So, the local truncation error is
′′
ℎ2 𝑐 2 −1 2
ℎ2
𝑦 𝑐 = 𝑒 4𝑐 + 2
2 2
where 𝑐 is between 𝑥𝑛 and 𝑥𝑛 + ℎ.
For ℎ = 0.1, we can get an upper bound on the local truncation error
for 𝑦1 by taking 𝑐 = 1.1.

School of Mathematical Sciences, Sunway University 26 MAT2034 Mathematical Methods

 Truncation Errors in Euler’s Method
Example 3.3.1 Bound for Local Truncation Errors
Solution:
For ℎ = 0.1, we can get an upper bound on the local truncation
error for 𝑦1 by taking 𝑐 = 𝟏. 𝟏.
1.1 2 −1 2
0.1 2
𝑒 4 1.1 + 2 = 𝟎. 𝟎𝟒𝟐𝟐
2
From the Table 3.1.1, we see that the error after the first step is
0.0337, less than the value given by the bound.

Similarly, we can get any bounds for the local truncation error for
any 5 steps given in Table 3.1.1. Let say 𝑐 = 𝟏. 𝟓. then we have
1.5 2 −1 2
0.1 2
𝑒 4 1.5 + 2 = 𝟎. 𝟏𝟗𝟐𝟎
2
If ℎ is halved to 0.05 in this example, then the error bound is
0.0480, its one-fourth as much as shown 0.1920.

School of Mathematical Sciences, Sunway University 27 MAT2034 Mathematical Methods

 Improved Euler’s Method
The numerical method defined by the formula
∗
𝑓 𝑥𝑛 , 𝑦𝑛 + 𝑓(𝑥𝑛+1 , 𝑦𝑛+1 )
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 + ℎ (1)
2
where
∗
𝑦𝑛+1 = 𝑦𝑛 + ℎ𝑓(𝑥𝑛 , 𝑦𝑛 ) (2)

is commonly known as the improved Euler’s method.

To compute 𝑦𝑛+1 for 𝑛 = 0,1,2, … from (1), we must at each step, first
∗
use Euler’s formula (2) to obtain an initial estimate 𝒚𝒏+𝟏 .

For example, with 𝑛 = 0, equation (2) gives

𝑦1∗ = 𝑦0 + ℎ𝑓 𝑥0 , 𝑦0
and then, knowing this value, we use (1) to get,
𝑓 𝑥0 , 𝑦0 + 𝑓(𝑥1 , 𝑦1∗ )
𝑦1 = 𝑦0 + ℎ
2
where 𝑥1 = 𝑥0 + ℎ.
School of Mathematical Sciences, Sunway University 28 MAT2034 Mathematical Methods
 Improved Euler’s Method
𝑦
Solution curve

𝑚𝑎𝑣𝑒
𝒙𝟏 , 𝒚 𝒙𝟏 𝑚1 = 𝑓(𝑥1 , 𝑦1∗ )
Exact
(𝑥1 , 𝑦1 ) 𝑚0 = 𝑓(𝑥0 , 𝑦0 )
(𝑥1 , 𝑦1∗ )
(𝑥0 , 𝑦0 ) 𝑓 𝑥0 , 𝑦0 + 𝑓(𝑥1 , 𝑦1∗ )
𝑚𝑎𝑣𝑒 =
2
𝑥
𝑥0 𝑥1

Figure 3.4.1 Slope of red dashed line

is the average of 𝑚0 and 𝑚1
School of Mathematical Sciences, Sunway University 29 MAT2034 Mathematical Methods
 Improved Euler’s Method

These equation can be readily

visualized. In Figure 3.4.1, we
observe that
𝑚0 = 𝑓 𝑥0 , 𝑦0 , 𝑚1 = 𝑓(𝑥1 , 𝑦1∗ )
are slopes of the solid straight lines
shown passing through points
(𝑥0 , 𝑦0 ) and (𝑥1 , 𝑦1∗ ) respectively.

So, the average of these slopes,

that is,
𝑓 𝑥0 , 𝑦0 + 𝑓(𝑥1 , 𝑦1∗ )
𝑚𝑎𝑣𝑒 =
2
we obtain the slope of the parallel
Figure 3.4.1 Slope of red dashed line dashed skew lines.
is the average of 𝑚0 and 𝑚1

School of Mathematical Sciences, Sunway University 30 MAT2034 Mathematical Methods

 Improved Euler’s Method

With the first step, rather

advancing along the line through
(𝑥0 , 𝑦0 ) with slope 𝑓(𝑥0 , 𝑦0 ) to the
point with y-coordinate 𝑦1∗ obtained
by Euler’s method,

we advance instead along the

pink dashed line through (𝑥1 , 𝑦1∗ )
with slope 𝑚𝑎𝑣𝑒 until we reach 𝑥1 .

It seems plausible from inspection

of the figure that 𝒚𝟏 is an
∗
Figure 3.4.1 Slope of red dashed line improvement over 𝒚 𝟏.
is the average of 𝑚0 and 𝑚1

School of Mathematical Sciences, Sunway University 31 MAT2034 Mathematical Methods

 Improved Euler’s Method

In general, the improved Euler’s method is an example of a

predictor-corrector method.
∗
𝑓 𝑥𝑛 , 𝑦𝑛 + 𝑓(𝑥𝑛+1 , 𝑦𝑛+1 )
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 +ℎ
2
where
∗
𝑦𝑛+1 = 𝑦𝑛 + ℎ 𝑓(𝑥𝑛 , 𝑦𝑛 )

∗
The value of 𝑦𝑛+1 given by (2) predicts a value of 𝑦(𝑥𝑛 ) whereas
the value of 𝑦𝑛+1 defined by formula (1) corrects this estimate.

School of Mathematical Sciences, Sunway University 32 MAT2034 Mathematical Methods

 Improved Euler’s Method
Example 3.4.1 Improved Euler’s Method
Use the improved Euler’s method to obtain the approximate value
of 𝑦(1.5) for the solution of the initial-value problem
𝑦 ′ = 2𝑥𝑦, 𝑦 1 = 1.
Compare the results for ℎ = 0.1 and ℎ = 0.05.
Solution:
With 𝑥0 = 1, 𝑦0 = 1, 𝑓 𝑥𝑛 , 𝑦𝑛 = 2𝑥𝑛 𝑦𝑛 , 𝑛 = 1 and ℎ = 0.1, we first
compute (2),
𝒚∗𝟏 = 𝒚𝟎 + 𝒉𝒇 𝒙𝟎 , 𝒚𝟎 = 1 + 0.1 2 1 1 = 𝟏. 𝟐
We use this last value in (1) along with 𝑥1 = 1 + ℎ = 1 + 0.1 = 𝟏. 𝟏,
𝑓 𝑥0 , 𝑦0 + 𝑓(𝑥1 , 𝑦1∗ )
𝑦1 = 𝑦0 + ℎ
2
2 1 1 + 2(𝟏. 𝟏)(𝟏. 𝟐)
𝑦1 = 1 + 0.1 = 𝟏. 𝟐𝟑𝟐
2

School of Mathematical Sciences, Sunway University 33 MAT2034 Mathematical Methods

 Improved Euler’s Method
Example 3.4.1 Improved Euler’s Method
Solution: Table 3.4.1 Improved Euler’s Method with ℎ = 0.1
MS Excel
n xn y*(n) y(n) actual error % relative error

0 1.00 1.0000 1.0000 0.0000 0.00

1 1.10 1.2000 1.2320 1.2337 0.0017 0.14

2 1.20 1.5030 1.5479 1.5527 0.0048 0.31

3 1.30 1.9194 1.9832 1.9937 0.0106 0.53

4 1.40 2.4988 2.5908 2.6117 0.0209 0.80

5 1.50 3.3162 3.4509 3.4903 0.0394 1.13

School of Mathematical Sciences, Sunway University 34 MAT2034 Mathematical Methods

 Improved Euler’s Method
Example 3.4.1 Improved Euler’s Method MS Excel

Solution: Table 3.4.2 Improved Euler’s Method with ℎ = 0.05

n xn y*(n) y(n) actual error % relative error

0 1.00 1.0000 1.0000 0.0000 0.00
1 1.05 1.1000 1.1078 1.1079 0.0002 0.02
2 1.10 1.2241 1.2332 1.2337 0.0004 0.04
3 1.15 1.3689 1.3798 1.3806 0.0008 0.06
4 1.20 1.5384 1.5514 1.5527 0.0013 0.08
5 1.25 1.7376 1.7531 1.7551 0.0020 0.11
6 1.30 1.9722 1.9909 1.9937 0.0029 0.14
7 1.35 2.2497 2.2721 2.2762 0.0041 0.18
8 1.40 2.5789 2.6060 2.6117 0.0057 0.22
9 1.45 2.9708 3.0038 3.0117 0.0079 0.26
10 1.50 3.4394 3.4795 3.4903 0.0108 0.31

School of Mathematical Sciences, Sunway University 35 MAT2034 Mathematical Methods

 Global Truncation Error

Definition 3.4.1 Global Truncation Error

From the analysis, we assume the value of 𝒚𝒏 was exact in the calculation
of 𝑦𝑛+1 but it is not because it contains local truncation errors from
previous steps.
The total error in 𝑦𝑛+1 is an accumulation of the errors in each of the
previous steps. This total error is called the global truncation error.

It can be shown that the global truncation error for Euler’s method is 𝑂 ℎ .

Expectation; In Euler’s method, the step size is halved,

Global the error will be approximately halved as well. From
Table 3.1.1 and Table 3.1.2, when 𝑥 = 1.50, the
truncation absolute error when
error ℎ = 0.1 is 𝟎. 𝟓𝟔𝟐𝟓 and ℎ = 0.05 is 0.3171
* approximately half as large.

School of Mathematical Sciences, Sunway University 36 MAT2034 Mathematical Methods

 Global Truncation Error

In general, it can be shown that of a method for the numerical

solution of a differential equation has local truncation error 𝑶(𝒉𝒏 ),
then the global truncation error is 𝑶 𝒉𝒏−𝟏 .

Truncation errors for the improved Euler’s method

• Local truncation error for the improved Euler’s method is 𝑶(𝒉𝟑 ).
• The global truncation error is 𝑶 𝒉𝟐 .
• In Example 3.4.1, when step size is halved from
𝒉 = 𝟎. 𝟏 ⇒ 𝒉 = 𝟎. 𝟎𝟓

the absolute error at 𝑥 = 1.50 is reduced from

0.0394 to 0.0108

𝟏 𝟐 1
• A reduction of approximately 𝟐
=4

School of Mathematical Sciences, Sunway University 37 MAT2034 Mathematical Methods

 Practice

Use the improved Euler’s method to obtain a four decimal

approximation of the indicated value. Use ℎ = 0.1
𝑦
𝑦 ′ = 𝑥𝑦 2 − , 𝑦 1 = 1; 𝑦(1.5)
𝑥
𝑦
With 𝑥0 = 1, 𝑦0 = 1 ; 𝑓 𝑥𝑛 , 𝑦𝑛 = 𝑥𝑛 𝑦𝑛2 − 𝑥𝑛
𝑛
So, 𝑛 = 0, ℎ = 0.1, we have
1
𝑦1∗ = 1 + 0.1 1 1 2
−=1
1
And 𝑥1 = 1 + ℎ = 1 + 0.1 = 1.1
Then,
𝑦𝑛 ∗ 2 𝑦1∗
𝑥𝑛 𝑦𝑛2
− 𝑥 + 𝑥1 𝑦1 − 𝑥
𝑛 1
𝑦1 = 𝑦0 + 0.1
2
1
1 − 1 + ( 1.1 1 2 − 1.1)
𝑦1 = 1 + 0.1 = 𝟏. 𝟎𝟎𝟗𝟓
2
School of Mathematical Sciences, Sunway University 38 MAT2034 Mathematical Methods
 Practice

Use the improved Euler’s method to obtain a four decimal

approximation of the indicated value. Use ℎ = 0.1
𝑦
𝑦 ′ = 𝑥𝑦 2 − , 𝑦 1 = 1; 𝑦(1.5)
𝑥

n xn y*(n) y(n)
0 1 1.0000
1 1.1 1.0000 1.0095
2 1.2 1.0299 1.0404
3 1.3 1.0836 1.0967
4 1.4 1.1687 1.1866
5 1.5 1.2989 1.3260

School of Mathematical Sciences, Sunway University 39 MAT2034 Mathematical Methods

 Runge-Kutta Method

Carl Runge Martin Kutta

(1856-1927) (1867- 1944)
School of Mathematical Sciences, Sunway University 40 MAT2034 Mathematical Methods
 Runge-Kutta Method

• One of the most popular and accurate numerical procedures used

• To approximate solutions to a first-order initial-value problem
𝑦 ′ = 𝑓 𝑥, 𝑦 , 𝑦 𝑥0 = 𝑦0
is the fourth-order Runge-Kutta method (RK4).

• As the name suggests, there are Runge-Kutta methods of different

orders.
• All Runge-Kutta methods have been generalized from basic Euler
formula
• The slope function 𝑓 is replaced by a weighted average slopes over
the interval 𝑥𝑛 ≤ 𝑥 ≤ 𝑥𝑛+1 , that is

𝑦𝑛+1 = 𝑦𝑛 + ℎ (𝑤1 𝑘1 + 𝑤2 𝑘2 + ⋯ + 𝑤𝑚 𝑘𝑚 ) (1)

weighted average

School of Mathematical Sciences, Sunway University 41 MAT2034 Mathematical Methods

 Runge-Kutta Method

𝑦𝑛+1 = 𝑦𝑛 + ℎ (𝑤1 𝑘1 + 𝑤2 𝑘2 + ⋯ + 𝑤𝑚 𝑘𝑚 ) (1)

weighted average
Here, the weights 𝑤𝑖 , 𝑖 = 1,2, … , 𝑚 are constants that generally satisfy
𝑤1 + 𝑤2 + 𝑤3 + ⋯ + 𝑤𝑚 = 1
for each 𝑘𝑖 , 𝑖 = 1,2, … , 𝑚 is the function 𝑓 evaluated at a selected point
(𝒙, 𝒚) for which 𝑥𝑛 ≤ 𝑥 ≤ 𝑥𝑛+1 .

We shall see that the 𝒌𝒊 are defined recursively. The number 𝒎 is called
the order of the method.

Observe that by taking 𝑚 = 1, 𝑤1 = 1 and 𝑘1 = 𝑓 𝑥𝑛 , 𝑦𝑛 , we get the

familiar Euler formula.

Hence, Euler’s methods is said to be a first-order Runge-Kutta method.

School of Mathematical Sciences, Sunway University 42 MAT2034 Mathematical Methods

 Runge-Kutta Method

• The parameters are chosen so that (1) agrees with a Taylor polynomial
of degree 𝑚.
• If a function 𝑦(𝑥) possess 𝑘 + 1 derivatives that are continuous on an
open interval containing 𝑎 and 𝑥, then we can write

𝑥−𝑎 𝑥−𝑎 2 𝑥−𝑎 𝑘 𝑥−𝑎 𝑘+1

𝑦 𝑥 = 𝑦 𝑎 + 𝑦′ 𝑎 + 𝑦 ′′ 𝑎 + ⋯+ 𝑦 𝑘 𝑎 +𝑦 𝑘+1 𝑐
1! 2! 𝑘! 𝑘+1 !

where 𝑐 is some number between 𝑎 and 𝑥.

• If we replace 𝑎 by 𝑥𝑛 and 𝑥 by 𝑥𝑛+1 = 𝑥𝑛 + ℎ, then the foregoing

formula becomes
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 + ℎ
ℎ2 ′′ ℎ𝑘 ℎ𝑘+1
= 𝑦 𝑥𝑛 + ℎ𝑦 ′ 𝑥𝑛 + 𝑦 𝑥𝑛 + ⋯ + 𝑦 𝑘 𝑥𝑛 + 𝑦 𝑘+1 (𝑐)
2 𝑘! 𝑘+1 !
where 𝑐 is now some number between 𝑥𝑛 and 𝑥𝑛+1 .

School of Mathematical Sciences, Sunway University 43 MAT2034 Mathematical Methods

 Runge-Kutta Method
When 𝑦(𝑥) is a solution of 𝑦 ′ = 𝑓(𝑥, 𝑦) in the case 𝑘 = 1 and the
remainder is small, we see that a Taylor polynomial
𝑦 𝑥𝑛+1 = 𝑦 𝑥𝑛 + ℎ = 𝑦 𝑥𝑛 + ℎ𝑦′(𝑥𝑛 )
of degree one agrees with the approximation formula of Euler’s method
𝑦𝑛+1 = 𝑦𝑛 + ℎ𝑦𝑛′ = 𝑦𝑛 + ℎ𝑓 𝑥𝑛 , 𝑦𝑛 .
A Fourth-Order Runge-Kutta Method
The procedure consists of finding parameters so that the formula

𝑦𝑛+1 = 𝑦𝑛 + ℎ 𝑤1 𝑘1 + 𝑤2 𝑘2 + 𝑤3 𝑘3 + 𝑤4 𝑘4
𝑘1 = 𝑓 𝑥𝑛 , 𝑦𝑛
𝑘2 = 𝑓 𝑥𝑛 + 𝛼1 ℎ, 𝑦𝑛 + 𝛽1 ℎ𝑘1
𝑘3 = 𝑓 𝑥𝑛 + 𝛼2 ℎ, 𝑦𝑛 + 𝛽2 ℎ𝑘1 + 𝛽3 ℎ𝑘2 (2)
𝑘4 = 𝑓(𝑥𝑛 + 𝛼3 ℎ, 𝑦𝑛 + 𝛽4 ℎ𝑘1 + 𝛽5 ℎ𝑘2 + 𝛽6 ℎ𝑘3 )

agrees with a Taylor polynomial of degree four. This results in a system of

11 equations in 13 unknowns.
School of Mathematical Sciences, Sunway University 44 MAT2034 Mathematical Methods
 Runge-Kutta Method

A Fourth-Order Runge-Kutta Method

The most commonly used set of values for the parameters yields the
following result;
ℎ
𝑦𝑛+1 = 𝑦𝑛 + 𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4
6
𝑘1 = 𝑓 𝑥𝑛 , 𝑦𝑛
1 1
𝑘2 = 𝑓 𝑥𝑛 + ℎ, 𝑦𝑛 + ℎ𝑘1 (3)
2 2
1 1
𝑘3 = 𝑓 𝑥𝑛 + ℎ, 𝑦𝑛 + ℎ𝑘2
2 2
𝑘4 = 𝑓 𝑥𝑛 + ℎ, 𝑦𝑛 + ℎ𝑘3

School of Mathematical Sciences, Sunway University 45 MAT2034 Mathematical Methods

 Runge-Kutta Method

Example 3.5.1
Use the RK4 to obtain the approximate value of 𝑦(1.5) for the
solution of the initial-value problem with ℎ = 0.1
𝑦 ′ = 2𝑥𝑦, 𝑦 1 = 1.
Solution:
For 𝑛 = 0, we find 𝑘1 = 𝑓 𝑥0 , 𝑦0 = 2𝑥0 𝑦0 = 𝟐
1 1 1 1
𝑘2 = 𝑓 𝑥0 + ℎ, 𝑦0 + ℎ𝑘1 = 2 1 + 0.1 1 + 0.1 𝟐 = 2.31
2 2 2 2
1 1 1 1
𝑘3 = 𝑓 𝑥0 + ℎ, 𝑦0 + ℎ𝑘2 = 2 1 + 0.1 1 + 0.1 2.31
2 2 2 2
= 2.34255
𝑘4 = 𝑓 𝑥0 + ℎ, 𝑦0 + ℎ𝑘3 = 2 1 + 0.1 1 + 0.1 2.34255 = 2.715361
0.1
Thus, 𝑦1 = 1 + 6 𝟐 + 2 2.31 + 2 2.34255 + 2.715361 = 1.23367435

School of Mathematical Sciences, Sunway University 46 MAT2034 Mathematical Methods

 Runge-Kutta Method

Example 3.5.1
Solution:
Table 3.5.1 RK4 Method with ℎ = 0.1
Actual
n x(n) k1 k2 k3 k4 y(n) value
0 1.00 2.0000 2.3100 2.3426 2.7154 1.0000 1.0000
1 1.10 2.7141 3.1496 3.1997 3.7287 1.2337 1.2337
2 1.20 3.7265 4.3475 4.4252 5.1876 1.5527 1.5527
3 1.30 5.1836 6.0827 6.2041 7.3195 1.9937 1.9937
4 1.40 7.3126 8.6341 8.8257 10.4826 2.6116 2.6117
5 1.50 3.4902 3.4903

School of Mathematical Sciences, Sunway University 47 MAT2034 Mathematical Methods

 Truncation Errors for RK4 Method

• Previously, we know that the global truncation errors for Euler’s

method is 𝑶(𝒉) and the improved Euler’s method is 𝑶(𝒉𝟐 )
• The equation in (3) agrees with Taylor polynomial of degree four
• So, the local truncation error for this method is
5
ℎ5
𝑦 𝑐 or 𝑂(ℎ5 )
5!
• The global truncation error is thus 𝑶(𝒉𝟒 )
• Thus, we can simplify that

Euler’s method RK first-order

Improved Euler’s method RK second-order
Equation (3) RK fourth-order

School of Mathematical Sciences, Sunway University 48 MAT2034 Mathematical Methods

 Truncation Errors for RK4 Method

Example 3.5.2
Find a bound for the local truncation errors for the RK4 method
applied to
𝑦 ′ = 2𝑥𝑦, 𝑦 1 = 1.
Solution:
2
Compute the fifth derivative of the known solution 𝑦 𝑥 = 𝑒 𝑥 −1
we get
5
ℎ5 3 5 𝑐 2 −1
ℎ5
𝑦 𝑐 = 120𝑐 + 160𝑐 + 32𝑐 𝑒 (4)
5! 5!
With 𝑐 = 1.5, (4) yields a bound of 0.00028 on the local truncation
error for each of the 5 steps when 𝒉 = 𝟎. 𝟏.

The Table 3.5.1 has shown that the error in 𝑦1 is much lesser than
this bound.

School of Mathematical Sciences, Sunway University 49 MAT2034 Mathematical Methods

 Multistep Methods
Introduction
• Euler’s method, improved Euler’s method and the Runge-Kutta
methods are example of single-step or starting methods.
• Each successive value of 𝑦𝑛+1 is computed based only on
information about the immediately preceding value 𝑦𝑛 .
• However, multistep or continuing methods use the values from
several computed steps to obtain the value of 𝑦𝑛+1 .

Adams-Bashforth-Moulton Method
• Fourth-order Adams-Bashforth-Moulton
method
• It’s also a predictor – corrector method
• One formula is used to predict a value
∗
𝑦𝑛+1 which in turn is used to obtain a
corrected value 𝑦𝑛+1 .
School of Mathematical Sciences, Sunway University 50 MAT2034 Mathematical Methods
 Multistep Methods
Adams-Bashforth-Moulton Method

The predictor in this method is the Adams Bashforth formula.

∗
ℎ ′ ′ ′
𝑦𝑛+1 = 𝑦𝑛 + 55𝑦𝑛′ − 59𝑦𝑛−1 + 37𝑦𝑛−2 − 9𝑦𝑛−3 (5)
24
𝑦𝑛′ = 𝑓 𝑥𝑛 , 𝑦𝑛
′
𝑦𝑛−1 = 𝑓 𝑥𝑛−1 , 𝑦𝑛−1
′
𝑦𝑛−2 = 𝑓 𝑥𝑛−2 , 𝑦𝑛−2
′
𝑦𝑛−3 = 𝑓 𝑥𝑛−3 , 𝑦𝑛−3 for 𝑛 ≥ 3

∗
We then substitute 𝑦𝑛+1 into Adams-Moulton corrector,
ℎ ′ ′ ′
𝑦𝑛+1 = 𝑦𝑛 + 9𝑦𝑛+1 + 19𝑦𝑛′ − 5𝑦𝑛−1 + 𝑦𝑛−2 (6)
24
′ ∗
𝑦𝑛+1 = 𝑓(𝑥𝑛+1 , 𝑦𝑛+1 )

School of Mathematical Sciences, Sunway University 51 MAT2034 Mathematical Methods

 Multistep Methods
Adams-Bashforth-Moulton Method

• Formula (5) requires that we know the values of 𝑦0 , 𝑦1 , 𝑦2 , and

𝑦3 to obtain 𝑦4 .
• The value of 𝑦0 is the given initial condition.
• The local truncation error of the Adams-Bashforth-Moulton
method is 𝑶 𝒉𝟓
• The values of 𝑦1 , 𝑦2 , and 𝑦3 are generally computed by a method
with the same error property, such as the fourth-order Runge-
Kutta method.

School of Mathematical Sciences, Sunway University 52 MAT2034 Mathematical Methods

 Multistep Methods

Example 3.6.1
Use the Adams-Bashforth-Moulton method to obtain the approximate
value of 𝑦(1.5) for the solution of the initial-value problem with ℎ = 0.1
𝑦 ′ = 2𝑥𝑦, 𝑦 1 = 1.
Solution:
For step size of ℎ = 0.1, 𝑦(1.5) will be approximated by 𝑦5 .
We use the RK4 method with 𝑥0 = 1, 𝑦0 = 1, and ℎ = 0.1 to obtain
𝑦1 = 1.23367, 𝑦2 = 1.55270, 𝑦3 = 1.99369
Now, the identifications for
𝑥0 = 1, 𝑥1 = 1.1, 𝑥2 = 1.2, 𝑥3 = 1.3, 𝑓 𝑥, 𝑦 = 2𝑥𝑦
we find
𝑦0′ = 𝑓 𝑥0 , 𝑦0 = 2 1 1 = 2
𝑦1′ = 𝑓 𝑥1 , 𝑦1 = 2 1.1 1.23367 = 2.71407
𝑦2′ = 𝑓 𝑥2 , 𝑦2 = 2 1.2 1. 55270 = 3.72648
𝑦3′ = 𝑓 𝑥3 , 𝑦3 = 2 1.3 1.99369 = 5.18359

School of Mathematical Sciences, Sunway University 53 MAT2034 Mathematical Methods

 Multistep Methods

Example 3.6.1
Solution:
Foregoing values the predictor (5) will gives
0.1
𝑦4∗ = 𝑦3 + 55𝑦3′ − 59𝑦2′ + 37𝑦1′ − 9𝑦0′ = 2.60892
24
To use the corrector (6), we first need
𝑦4′ = 𝑓 𝑥4 , 𝑦4∗ = 2 1.4 2.60892 = 7.30498
Finally, (2) yields
0.1
𝑦4 = 𝑦3 + 9𝑦4′ + 19𝑦3′ − 5𝑦2′ + 𝑦1′ = 2.61167
24
′
𝒚𝟒 = 𝒇 𝒙𝟒 , 𝒚𝟒 = 𝟐 𝟏. 𝟒 𝟐. 𝟔𝟏𝟏𝟔𝟕 = 𝟕. 𝟑𝟏𝟐𝟔𝟔
∗
0.1
𝑦5 = 𝑦4 + 55𝑦4′ − 59𝑦3′ + 37𝑦2′ − 9𝑦1′ = 3.48590
24
𝑦5 = 𝑓 𝑥5 , 𝑦5∗ = 2 1.5 3.48590 = 10.45771
′
0.1
𝑦5 = 𝑦4 + 9𝑦5′ + 19𝒚′𝟒 − 5𝑦3′ + 𝑦2′ = 3.49028
24
School of Mathematical Sciences, Sunway University 54 MAT2034 Mathematical Methods
 Multistep Methods

Example 3.6.1
0.1
Solution: 𝑦5 = 𝑦4 + 9𝑦5′ + 19𝒚′𝟒 − 5𝑦3′ + 𝑦2′ = 3.49028
24

n x(n) k1 k2 k3 k4 y(n) y*(n) y'(n) Actual value Absolute error Relative error

0 1.00 2.00000 2.3100 2.3426 2.7154 1.00000 2.00000 1.0000 0.00000 0.0000

1 1.10 2.71408 3.1496 3.1997 3.7287 1.23367 2.71408 1.2337 0.00000 0.0003

2 1.20 3.72647 4.3475 4.4252 5.1876 1.55270 3.72647 1.5527 0.00001 0.0008

3 1.30 5.18359 6.0827 6.2041 7.3195 1.99369 5.18359 1.9937 0.00003 0.0014

4 1.40 7.31266 2.61166 2.60892 7.30498 2.6117 0.00003 0.0012

5 1.50 10.47085 3.49028 3.48590 10.45771 3.4903 0.00006 0.0017

School of Mathematical Sciences, Sunway University 55 MAT2034 Mathematical Methods

 Finite Difference Method

What is finite differrence method, and why?

Many scientific and engineering problems using mathematical

modelling involve solutions of differential equations.
We have seen the solution of first order differential equation

𝑑𝑦
+ 𝑝 𝑥 𝑦 = 𝑞(𝑥)
𝑑𝑥
is given by
𝜇 𝑥 𝑦 = ∫ 𝜇 𝑥 𝑞 𝑥 𝑑𝑥 + 𝐶

where 𝜇 𝑥 = 𝑒 ∫ 𝑝 𝑥 𝑑𝑥
is known as the integrating factor.

School of Mathematical Sciences, Sunway University 56 MAT2034 Mathematical Methods

 Finite Difference Method

What is finite differrence method, and why?

• The previous ODE is linear and first order, so we can use

analytical methods to find its solution.

• Real life problems are often not linear and first order, so it will be
difficult to solve these problems analytically.

• The finite difference method is another numerical method

developed to find approximate solutions to ODE.

• In this method, the derivative in the differential equation is

approximated by a finite difference.

School of Mathematical Sciences, Sunway University 57 MAT2034 Mathematical Methods

 Finite Difference Method

Principle of Finite Difference Method

• Physically, a derivative represents the rate of change of a physical

quantity represented by a function with respect to the change of its
variable.

• So, it will approximate this rate of change by the difference of the

function value 𝑦(𝑥) corresponding to the difference in the values of the
variable 𝑥.

School of Mathematical Sciences, Sunway University 58 MAT2034 Mathematical Methods

 Finite Difference Method

Here the function 𝑦(𝑥) is graphically

𝑦(𝑥)
shown, with the function evaluated at Tangent of 𝑦(𝑥) at 𝑥 = 𝑥𝑖
3 consecutive values of 𝑥:

𝑦𝑖−1 at 𝑥 = 𝑥𝑖−1 𝑦𝑖+1 𝑦(𝑥)

𝑦𝑖 at 𝑥 = 𝑥𝑖
𝑦𝑖+1 at 𝑥 = 𝑥𝑖+1 𝑦𝑖

Since 𝑥𝑖−1 = 𝑥𝑖 − ∆𝑥 and 𝑥𝑖+1 = 𝑥𝑖 + ∆𝑥 ,

we have 𝑦𝑖−1
𝑑𝑦 ∆𝑦 ∆𝑦
= lim ≈ ∆𝑥 ∆𝑥
𝑑𝑥 ∆𝑥→0 ∆𝑥 ∆𝑥

𝑥𝑖−1 𝑥𝑖 𝑥𝑖+1 𝑥
Note:
𝑑𝑦 ∆𝑦
The differential ≠ “the difference”. (1)
𝑑𝑥 ∆𝑥

School of Mathematical Sciences, Sunway University 59 MAT2034 Mathematical Methods

 Finite Difference Method

Equation (1) depicts the 𝑦(𝑥)

principle of finite difference. Tangent of 𝑦(𝑥) at 𝑥 = 𝑥𝑖

It is important to be aware of the 𝑦𝑖+1 𝑦(𝑥)

fact that smaller the steps ∆𝐱,
the closer the values between 𝑦𝑖
the differential and difference of
the rate of change of the 𝑦𝑖−1
function
∆𝑥 ∆𝑥

𝑥𝑖−1 𝑥𝑖 𝑥𝑖+1 𝑥

School of Mathematical Sciences, Sunway University 60 MAT2034 Mathematical Methods

 Three Basic Finite Difference Scheme

𝑦(𝑥) Tangent of 𝑦(𝑥) at 𝑥 = 𝑥𝑖

The Forward Difference Scheme

The rate of the change of the function with

𝑦𝑖+1 𝑦(𝑥) respect to the variable x is accounted for
between the current value at:
𝑦𝑖
𝑥 = 𝑥𝑖
𝑦𝑖−1
and the step forward at
∆𝑥 ∆𝑥 𝑥 = 𝑥𝑖 + ∆𝑥.

𝑥𝑖−1 𝑥𝑖 𝑥𝑖+1 𝑥 Mathematical expression of the derivative of

the function 𝑦(𝑥) is

𝑑𝑦 ∆𝑦 𝑦𝑖+1 − 𝑦𝑖 𝑦𝑖+1 − 𝑦𝑖 𝑦𝑖+1 − 𝑦𝑖 (2)

ቤ ≈ ቤ = = =
𝑑𝑥 𝑥=𝑥 ∆𝑥 𝑥=𝑥 𝑥𝑖+1 − 𝑥𝑖 ∆𝑥 ℎ
𝑖 𝑖

where ℎ = ∆𝑥 is the incremental step size.

School of Mathematical Sciences, Sunway University 61 MAT2034 Mathematical Methods

 Three Basic Finite Difference Scheme

𝑦(𝑥) Tangent of 𝑦(𝑥) at 𝑥 = 𝑥𝑖

𝑑𝑦 𝑦𝑖+1 − 𝑦𝑖
ቤ ≈ (2)
𝑦𝑖+1 𝑦(𝑥) 𝑑𝑥 𝑥=𝑥 ℎ
𝑖

𝑦𝑖
The derivative of the function at other values
𝑦𝑖−1 of the variable 𝑥 in the positive direction can
be expressed following equation (2) are
∆𝑥 ∆𝑥
𝑑𝑦 𝑦𝑖+2 − 𝑦𝑖+1
𝑥𝑖−1 𝑥𝑖 𝑥𝑖+1 𝑥 ቤ ≈
𝑑𝑥 𝑥=𝑥 +1 ℎ
𝑖

𝑑𝑦 𝑦𝑖+3 − 𝑦𝑖+2
ቤ ≈
𝑑𝑥 𝑥=𝑥 +2 ℎ
𝑖

and etc.

School of Mathematical Sciences, Sunway University 62 MAT2034 Mathematical Methods

 Three Basic Finite Difference Scheme

The second order derivative of the function at 𝑥 can be derived by the following
procedure:
𝑑𝑦 𝑑𝑦
ฬ − ฬ
2
𝑑 𝑦 𝑑 𝑑𝑦 𝑑𝑥 𝑥=𝑥 +1 𝑑𝑥 𝑥=𝑥
𝑖 𝑖
2 อ = อ = lim
𝑑𝑥 𝑑𝑥 𝑑𝑥 ∆𝑥→0 ∆𝑥
𝑥=𝑥𝑖 𝑥=𝑥𝑖

∆𝑦 ∆𝑦
∆𝑥 𝑥=𝑥 +1 ∆𝑥 ฬ𝑥=𝑥
ฬ −
𝑖 𝑖
≈
∆𝑥
𝑦𝑖+2 − 𝑦𝑖+1 𝑦𝑖+1 − 𝑦𝑖
−
= ℎ ℎ
ℎ

𝑦𝑖+2 − 2𝑦𝑖+1 + 𝑦𝑖
= (3)
ℎ2

School of Mathematical Sciences, Sunway University 63 MAT2034 Mathematical Methods

 Three Basic Finite Difference Scheme

𝑦(𝑥) Tangent of 𝑦(𝑥) at 𝑥 = 𝑥𝑖

The Backward Difference Scheme

We evaluate the rate of change of the function

𝑦𝑖+1 𝑦(𝑥) values between the “current” step at xi , and
the function value at a step “back” at 𝑥𝑖−1 , i.e.
𝑦𝑖
𝑥𝑖−1 = 𝑥𝑖 − ∆𝑥
𝑦𝑖−1
Mathematically, we will have the following
∆𝑥 ∆𝑥 𝑑𝑦 ∆𝑦 𝑦𝑖 − 𝑦𝑖−1 𝑦𝑖 − 𝑦𝑖−1
ቤ ≈ ቤ = 𝑙𝑖𝑚 ≈
𝑑𝑥 𝑥=𝑥 ∆𝑥 𝑥=𝑥 ∆𝑥→0 ∆𝑥 ℎ
𝑖 𝑖
𝑥𝑖−1 𝑥𝑖 𝑥𝑖+1 𝑥
(4)
and the second order derivatives in the form:
𝑑2 𝑦 𝑦𝑖 − 2𝑦𝑖−1 + 𝑦𝑖−2
อ ≈
𝑑𝑥 2 ℎ2
𝑥=𝑥𝑖 (5)

School of Mathematical Sciences, Sunway University 64 MAT2034 Mathematical Methods

 Three Basic Finite Difference Scheme

The Central Difference Scheme

The rate of change of function 𝑓(𝑥) is

accounted for between the step at back at
𝑥 − ∆𝑥
𝑦(𝑥) Tangent of 𝑦(𝑥) at 𝑥 = 𝑥𝑖

and the step ahead of 𝑥, i.e.

𝑥 + ∆𝑥.
𝑦𝑖+1 𝑦(𝑥)

Mathematically, we will have the following: 𝑦𝑖

𝑑𝑦 𝑦𝑖+1 − 𝑦𝑖−1 𝑦𝑖+1 − 𝑦𝑖−1
ቤ ≈ = (6)
𝑑𝑥 𝑥=𝑥 𝑥𝑖+1 − 𝑥𝑖−1 2ℎ 𝑦𝑖−1
𝑖

∆𝑥 ∆𝑥
and the second order derivatives in the form:
𝑑2 𝑦 𝑦𝑖+1 − 2𝑦𝑖 + 𝑦𝑖−1 𝑥𝑖−1 𝑥𝑖 𝑥𝑖+1 𝑥
อ ≈ (7)
𝑑𝑥 2 ℎ2
𝑥=𝑥𝑖

School of Mathematical Sciences, Sunway University 65 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Consider a linear second-order boundary-value problem

𝑑2 𝑦 𝑑𝑦
+ 𝑃(𝑥) + 𝑄 𝑥 𝑦 = 𝑓 𝑥 , 𝑦 𝑎 = 𝛼, 𝑦 𝑏 = 𝛽 (8)
𝑑𝑥 2 𝑑𝑥

Suppose
𝑎 = 𝑥0 < 𝑥1 < ⋯ < 𝑥𝑛−1 < 𝑥𝑛 = 𝑏

represent a regular partition of the interval 𝑎, 𝑏 , that is,

𝑥𝑖 = 𝑎 + 𝑖ℎ, where 𝑖 = 0,1,2, … , 𝑛
and ℎ = (𝑏 − 𝑎)/𝑛.
The points
𝑥1 = 𝑎 + ℎ, 𝑥2 = 𝑎 + 2ℎ, … , 𝑥𝑛−1 = 𝑎 + 𝑛 − 1 ℎ

are called interior mesh points of the interval 𝑎, 𝑏 .

School of Mathematical Sciences, Sunway University 66 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

If we let
𝑦𝑖 = 𝑦 𝑥𝑖 , 𝑃𝑖 = 𝑃 𝑥𝑖 , 𝑄𝑖 = 𝑄 𝑥𝑖 , 𝑓𝑖 = 𝑓(𝑥𝑖 )

𝑑2 𝑦 𝑑𝑦
and if and in (8)
𝑑𝑥 2 𝑑𝑥

are replaced by the finite difference approximations (7) and (6),

respectively, we get

𝑦𝑖+1 − 2𝑦𝑖 + 𝑦𝑖−1 𝑦𝑖+1 − 𝑦𝑖−1

+ 𝑃𝑖 + 𝑄𝑖 𝑦𝑖 = 𝑓𝑖 (9)
ℎ2 2ℎ

After simplification, equation (9) can be given as follows:

ℎ ℎ
1 + 𝑃𝑖 𝑦𝑖+1 + −2 + 𝑄𝑖 ℎ2 𝑦𝑖 + 1 − 𝑃𝑖 𝑦𝑖−1 = ℎ2 𝑓𝑖 (10)
2 2

School of Mathematical Sciences, Sunway University 67 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs
ℎ ℎ
1 + 𝑃𝑖 𝑦𝑖+1 + −2 + 𝑄𝑖 ℎ2 𝑦𝑖 + 1 − 𝑃𝑖 𝑦𝑖−1 = ℎ2 𝑓𝑖 (10)
2 2

Equation (10), known as a finite difference equation, is an approximation to the

differential equation.

It enables us to approximate the solution 𝑦(𝑥) of (8) at the interior mesh points
𝑥1 , 𝑥2 , … , 𝑥𝑛−1 of the interval 𝑎, 𝑏 .

By letting i take on the values 1,2, … , 𝑛 − 1 in (10), we obtain 𝑛 − 1 equations in the

𝑛 − 1 unknowns 𝑦1 , 𝑦2 , … , 𝑦𝑛−1 .

Bear in mind that we know 𝑦0 and 𝑦𝑛 , since these are the prescribed boundary
conditions
𝑦0 = 𝑦 𝑥0 = 𝑦 𝑎 = 𝛼 and 𝑦𝑛 = 𝑦 𝑥𝑛 = 𝑦 𝑏 = 𝛽.

School of Mathematical Sciences, Sunway University 68 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Example 10.3.1
Use the difference equation (10) with 𝑛 = 4 to approximate the solution of the
boundary-value problem.
𝑦 ′′ − 4𝑦 = 0, 𝑦 0 = 0, 𝑦 1 = 5.

Solution:
𝑑2 𝑦 𝑑𝑦
+ 𝑃(𝑥) + 𝑄 𝑥 𝑦 = 𝑓 𝑥 , 𝑦 𝑎 = 𝛼, 𝑦 𝑏 = 𝛽
𝑑𝑥 2 𝑑𝑥
From (8), we have
𝑏−𝑎 1−0
𝑃 𝑥 = 0, 𝑄 𝑥 = −4, 𝑓 𝑥 = 0, ℎ = = = 0.25
𝑛 4
𝑥0 = 0, 𝑦0 = 0, 𝑥4 = 1 , 𝑦4 = 5.
ℎ ℎ
1 + 𝑃𝑖 𝑦𝑖+1 + −2 + 𝑄𝑖 ℎ2 𝑦𝑖 + 1 − 𝑃𝑖 𝑦𝑖−1 = ℎ2 𝑓𝑖
2 2

Then, we have
0.25 2
0.25
1+ (0) 𝑦𝑖+1 + −2 + −4 0.25 𝑦𝑖 + 1 − (0) 𝑦𝑖−1 = 0.25 2 (0)
2 2

School of Mathematical Sciences, Sunway University 69 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Example 10.3.1
Use the difference equation (10) with 𝑛 = 4 to approximate the solution of the
boundary-value problem.
𝑦 ′′ − 4𝑦 = 0, 𝑦 0 = 0, 𝑦 1 = 5.
Solution:

0.25 0.25
1+ (0) 𝑦𝑖+1 + −2 + −4 0.25 2 𝑦𝑖 + 1 − (0) 𝑦𝑖−1 = 0.25 2 (0)
2 2
After simplify, we have
𝑦𝑖+1 − 2.25𝑦𝑖 + 𝑦𝑖−1 = 0

Since 𝑛 = 4, the interval [0,1] has 4 partitions with 𝑥0 = 0, 𝑥4 = 1 and interior points
𝑥𝑖 = 𝑥0 + 𝑖ℎ where 𝑖 = 1,2 and 3, that is
𝑥1 = 𝑥0 + 1ℎ = 0 + 0.25 = 0.25
𝑥2 = 𝑥0 + 2ℎ = 0 + 2(0.25) = 0.50
𝑥3 = 𝑥0 + 3ℎ = 0 + 3(0.25) = 0.75

School of Mathematical Sciences, Sunway University 70 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Example 10.3.1
Use the difference equation (10) with 𝑛 = 4 to approximate the solution of the
boundary-value problem.
𝑦 ′′ − 4𝑦 = 0, 𝑦 0 = 0, 𝑦 1 = 5.

Solution:

With the boundary conditions 𝑦0 = 0 and 𝑦4 = 5, the system becomes

𝑦1+1 − 2.25𝑦1 + 𝑦1−1 = 0
𝑦2+1 − 2.25𝑦2 + 𝑦2−1 = 0
𝑦3+1 − 2.25𝑦3 + 𝑦3−1 = 0
And we obtained
−2.25𝑦1 + 𝑦2 =0
𝑦1 − 2.25𝑦2 + 𝑦3 =0
𝑦2 − 2.25𝑦3 + 5 = 0

Solving the system gives 𝑦1 = 0.7256, 𝑦2 = 1.6327, and 𝑦3 = 2.9479

School of Mathematical Sciences, Sunway University 71 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Example 10.3.2
Use the difference equation (10) with 𝑛 = 10 to approximate the solution of the
boundary-value problem.
𝑦 ′′ + 3𝑦 ′ + 2𝑦 = 4𝑥 2 , 𝑦 1 = 1, 𝑦 2 = 6.
Solution:
𝑑2 𝑦 𝑑𝑦
+ 𝑃(𝑥) + 𝑄 𝑥 𝑦 = 𝑓 𝑥 , 𝑦 𝑎 = 𝛼, 𝑦 𝑏 = 𝛽
𝑑𝑥 2 𝑑𝑥

From (8), we have

2−1
𝑃 𝑥 = 3, 𝑄 𝑥 = 2, 𝑓 𝑥 = 4𝑥 2, ℎ = = 0.1
10
and (10) becomes
0.1 0.1
1+ (3) 𝑦𝑖+1 + −2 + 2 0.1 2 𝑦𝑖 + 1 − (3) 𝑦𝑖−1 = 0.1 2 (4𝑥𝑖2)
2 2

and simplify, we have

1.15𝑦𝑖+1 − 1.98𝑦𝑖 + 0.85𝑦𝑖−1 = 0.04𝑥𝑖2

School of Mathematical Sciences, Sunway University 72 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Example 10.3.2
Use the difference equation (10) with 𝑛 = 10 to approximate the solution of the
boundary-value problem.
𝑦 ′′ + 3𝑦 ′ + 2𝑦 = 4𝑥 2 , 𝑦 1 = 1, 𝑦 2 = 6.

Solution:

1.15𝑦𝑖+1 − 1.98𝑦𝑖 + 0.85𝑦𝑖−1 = 0.04𝑥𝑖2

From 𝑛 = 10, the interior points with 𝑥𝑖 = 𝑥0 + 𝑖ℎ, 𝑖 = 1,2, … , 9 are 𝑥1 = 1.1,
𝑥2 = 1.2, 𝑥3 = 1.3, 𝑥4 = 1.4, 𝑥5 = 1.5, 𝑥6 = 1.6, 𝑥7 = 1.7, 𝑥8 = 1.8 and 𝑥9 = 1.9.

With 𝑖 = 1,2, … , 9 and 𝑦0 = 1 and 𝑦10 = 6, (10) yields a system of 9 equations and 9
unknowns:

School of Mathematical Sciences, Sunway University 73 MAT2034 Mathematical Methods

 Finite Difference Method for Higher Order ODEs

Solution:

Example 10.3.2
1.15𝑦2 − 1.98𝑦1 = −0.8016
1.15𝑦3 − 1.98𝑦2 + 0.85𝑦1 = 0.0576
1.15𝑦4 − 1.98𝑦3 + 0.85𝑦2 = 0.0676
We can solve this large system
1.15𝑦5 − 1.98𝑦4 + 0.85𝑦3 = 0.0784
using Gaussian elimination or,
1.15𝑦6 − 1.98𝑦5 + 0.85𝑦4 = 0.0900
which relative ease, by means of
1.15𝑦7 − 1.98𝑦6 + 0.85𝑦5 = 0.1024
computer algebra system. The
1.15𝑦8 − 1.98𝑦7 + 0.85𝑦6 = 0.1156
results is found to be:
1.15𝑦9 − 1.98𝑦8 + 0.85𝑦7 = 0.1296
−1.98𝑦9 + 0.85𝑦8 = −6.7556

𝒏 1 2 3 4 5 6 7 8 9
𝑥𝑖 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9
𝑦𝑖 2.4047 3.4432 4.2010 4.7469 2.1359 5.4124 5.6117 5.7620 5.8855

School of Mathematical Sciences, Sunway University 74 MAT2034 Mathematical Methods

 Discussion

• In numerical methods, we shall consider the stability of the

method.
• It is stable if small changes in the initial condition result in only
small changes in the computed solution.
• In each step after the first step of a numerical technique, we are
essentially starting over again with a new initial-value problem,
where the initial condition is the approximate solution value
computed in the preceding step.
• Because of round-off error, this value will almost certainly vary at
least slightly from the true value of the solution.
• Another common source of error occurs in the initial condition
itself; in physical applications the data are often obtained by
imprecise measurements.

School of Mathematical Sciences, Sunway University 75 MAT2034 Mathematical Methods

 Discussion

• To detect instability in the numerical solution, we can compare the

approximate solutions obtained when decreasing step sizes are
used.
• If the numerical method is unstable, the error may actually
increase with smaller step sizes.
• Another way of checking stability is to observe what happens to
solutions when the initial condition is slightly perturbed (for
example, change 𝑦(0) = 1 to 𝑦 0 = 0.999)
• In general, all of the methods that we have discussed in this
chapter have good stability characteristics.

School of Mathematical Sciences, Sunway University 76 MAT2034 Mathematical Methods

 Discussion
ADVANTAGES AND DISADVANTAGES OF MULTISTEP METHODS

• Many considerations enter into the choice of a method to solve a

differential equation numerically.
• Single-step methods, particularly the RK4 method, are often
chosen because of their accuracy and the fact that they are easy to
program.
• However, a major drawback is that the right-hand side of the
differential equation must be evaluated many times at each step.
• For instance, the RK4 method requires four function evaluations
for each step.
• On the other hand, if the function evaluations in the previous step
have been calculated and stored, a multistep method requires
only one new function evaluation for each step.
• This can lead to great savings in time and expense.

School of Mathematical Sciences, Sunway University 77 MAT2034 Mathematical Methods

 Discussion
ADVANTAGES AND DISADVANTAGES OF MULTISTEP METHODS

• Another issue that is involved with multistep methods is how

many times the Adams-Moulton corrector formula should be
repeated in each step.
• Each time the corrector is used, another function evaluation is
done, and so the accuracy is increased at the expense of losing an
advantage of the multistep method.
• In practice, the corrector is calculated once, and if the value of
𝑦𝑛+1 is changed by a large amount, the entire problem is
restarted using a smaller step size.
• This is often the basis of the variable step size methods, whose
discussion is beyond the scope of this text.

School of Mathematical Sciences, Sunway University 78 MAT2034 Mathematical Methods

 Summary of Chapter 3
Numerical Solutions of ODE

School of Mathematical Sciences, Sunway University 79 MAT2034 Mathematical Methods