Ch 6 - Numerical Methods of Solving Differential Equations

dy
What do we do when a differential equation is difficult to solve analytically? Given =f x , y  and
dx
an initial value y  x 0 = y 0 , we will use numerical methods to evaluate y  x  at values near x 0 .

6.3 Euler's Method

We know two things about y  x  :
• y  x 0 = y 0
dy
• | =f  x 0, y 0  Slope of the line tangent to y  x at the point  x 0, y 0
dx x=x 0

 We want to find y  x 1 . if x 0 and x 1 are close, the points y 1 and y  x 1 will be close
too. So, we will approximate y  x 1 by y 1 . To find y 1 , we need to find the equation of
l1 .
 Equation of l 1 : y= y 0 f  x 0, y 0  x−x 0  , so y 1 = y 0 f  x 0, y 0  x1 − x0 
 Now, how do we approximate y  x 2 ? One way is by y 21 using l 3 . But we have a
problem in doing so as we don't exactly know the slope of l 3 as we don't know y  x 1
precisely.
 But, if y 1 is a very good approximation to y  x 1 , we can use that to estimate the slope of
the tangent line at x 1 . So, practically, we will use l 2 (a line that passes through  x1, y 1 
that has a slope of f  x1, y 1  ) instead of l 3 .
 From l 2 , we will find y 2 , and use that to approximate y  x 2 .Equation of l 2 :
y= y 1f x 1, y 1 x− x1 
 y  x 2≈ y 2= y 1 f  x 1, y 1  x2 −x 1
We continue in this fashion:
y  x 3≈ y 3= y 2 f  x2, y 2  x 3− x 2
⋮
 y  x n1≈ y n1= y nf  x n , y n x n1−x n
 • Denoting f  x n , y n =f n and x n 1−x n =h , we have Euler's method as follows
y n1= y n h f n

dy
Example1. = y , y 0=1 , use Euler's method with size h=0.1 to find the approximate
dx
values of the solution at x=0.1, 0.2, 0.3, 0.4 . Also find the approximate value of y 100 .

Solution:
f  x , y = y and y 0=1
y 0.1≈ y 1= y 0f  x 0, y 0 h , but f  x 0, y 0= y 0 =1
11∗0.1=1.1

y 0.2≈ y 2 = y 1f  x1, y 1 h , but f  x 1, y 1= y1 =1.1

1.11.1∗0.1=1.21

y 0.3≈ y 3= y 2f  x 2, y 2 h , but f  x2, y 2 = y 2=1.21

1.211.21∗0.1=1.331

y 0.4 ≈ y 4 = y 3f  x 3, y 3 h , but f  x3, y 3 = y 3=1.331

1.3311.331∗0.1=1.4641

But, finding y 100 will not be an easy task if we follow the above process. We need to find all the
numbers in [ y 0.4 , y 99.9] . Since it is tiresome, we won't do that! But we can find a general
formula for out approximation, and plug in the numbers we want.

To do this, we need to solve the difference equation y n1= y n f x n , y n h . For this case,
f  x n , y n = y n , so, the difference equation becomes y n1= y n hy n=1h y n .

So, let's find the solution to the difference equation y n1−1h y n=0
This difference equation is linear, homogeneous, and with constant coefficients, which means it can be
solved using the characteristic equation method.

−h1=0
=h1

y n=c 1 n
y n=c1 h1n
To find the constant c 1 , we use the initial value
y 0=c1=1

y n=h1n

Now, that we have our general approximation formula, we can use it find the approximate value of
y nh , n=0, 1, 2,⋯ . Let's see if what we have already gotten are right.

y 0.1≈ y 1=h1=1.1
y 0.2≈ y 2 =h12=1.1 2=1.21
 3 3
y 0.3≈ y 3=h1 =1.1 =1.331
4 4
y 0.4 ≈ y 4 h1 =1.1 =1.4641

Now, what is y 100 ?

From the trend above, we can see that y n /10≈ y n . So, y 100= y 1000 /10≈ y 1000 .
1000 1000
y 1000 =1h =1.1

y a≈ y a−x y n approximates y  x 0nh .

Generally, 0 or
h

Another important question is how to find y a where a≠nh , n=0,1,2,⋯ .

One solution is to redo the whole process by changing the value of h , so that a=nh . If we do
that, we just apply our general formula, with the changed h .

What is y 0.15 ?
With h=0.1 , we don't find an integer index to approximate it. If we try, y 0.15≈ y 1.5 . But we
can change the value of h so that we get an integer index. If we make h=0.05 ,
y 0.15≈ y 0.15/ 0.05= y 3 , and y 3=1h3=1.053

What if we don't want to change the value of h ?

From the geometric approximation, we know that y  x  is approximated by one line whose equation
is y= y i f x i , y i x−x i  for x∈[x i , xi h]

0.15 is in the second interval, [0.1, 0.2]. The same line is used to do the approximation in this
interval. So, for this, x i=0.1 .To approximate y(0.15), we need to determine the equation of the line.

y= y i f x i , y i x−0.1 = y i y i  x−0.1 . what is y i ? It is the approximation of y  xi  .

y  x i = y 0.1≈ y 1=1h=1.1 . So, the equation of the line becomes y=1.11.1 x−0.1 .

So, y 0.15=1.11.10.15−0.1=1.155 .

The equation of this line can be used for any x∈[0.1,0.2] . So, we can check that
y 0.1=1.11.1 0.1−0.1=1.1 and y 0.2=1.11.1 0.2−0.1=1.21 .

dy
Example2. =x use Euler's method with size h and y  x 0 = y 0 to find a difference equation.
dx
Solve the difference equation to find a general formula for the approximation of y  x  .

Find y 1, y 2, y 3 using the general formula, and using the direct method. Also Assuming
h=0.01, x 0 =0, y 0=2 , find y 56 , y 67.015 [NB: Don't change value of h]

Solution:
y n1= y n f x n , y n h , but f  x n , y n =x n , so y n1= y n hx n . But, what is x n ? We know that
 x 1=x 0h , x 2=x 02h , x 3=x 03h ,⋯ .So, x n=x 0nh .Using this, the difference equation is:

y n1= y n h x 0nh . or y n1− y n =nh2hx 0

Solving the difference equation:
The difference equation is linear, non-homogeneous, with constant coefficients. So, we need to find the
general solution for the homogeneous part and the particular solution for the non-homogeneous.

General solution: y Gn =?
n
Homogeneous part: y n1− y n =0 . This is easy to solve. =1 So, y n=c1 1 .

Particular solution: y np=?

2 2 n
nh hx 0 can be rewritten as nh hx 0   where =1 . Since  is the solution of the
characteristic equation (multiplicity =1), our particular solution has the form y np=anb n .
Inserting this solution into the difference equation, we get:
2 2 2
a n1 b n1−an bn=nh hx 0
After a little manipulation, we get:
2
2anab=nh hx 0 .
Equating constant terms with constant terms, we get
ab=hx 0
and equating the remaining terms, we get
2a=h2 .
Solving, we get:
2 2
a=h /2 and b=hx 0−h /2 .So,

nh 2 h2 p G
y np=nanb=n  hx 0−  . Combining y n and y n , we get:
2 2

n 2 h2 nh2
y n=c1 − nhx 0
2 2
To find c 1 , use the initial value y  x 0 = y 0 .Evaluate y n at n=0 . y 0=c1
2 2 2
n h nh
Therefore, y n= y 0  − nhx 0
2 2

Direct Method:
y 1= y 0 f x 0, y 0 h= y 0hx 0
2
y 2= y 1f  x 1, y 1 h= y 0hx 0hx1 = y 0hx 0 h x 0h=h 2hx 0 y 0
y 3= y 2hf  x 2, y 2=h22hx 0 y 0hx 2=h22hx 0 y 0h x 0 2h=3h 23hx 0 y 0

Using the general formula:

2 2
y 1= y 0 h /2−h /2hx 0= y 0 hx 0
y 2= y 0 2h2−h22hx 0= y 0 h22hx 0
2 2 2
y 3= y 0 9 /2 h −3/2h 3hx0 = y 03h 3hx 0

We see that the solutions are exactly the same.

 2
0.01 2
Replacing h , x 0, y 0 by the numbers, we get: y n=2 n −n
2

0.012
y 56≈ y 56/ 0.01= y 5600=2 5600 2−5600=1569.72 .
2
x2
The exact solution y  x =2 . Using this, y 56=2562 /2=1570 . The relative error in
2
approximation is E=1570−1569.72/1570=0.0014
2
0.001
If h=0.001 , y 56≈ y 56000=2560002−56000=1569.972 . The relative error in this
2
approximation is 1570−1569.972/1570=.00001783 .

Now, what is y 67.015 ? [with h=0.01]

Since 67.015≠ x0 nh , n=0,1,2,⋯ , we need to find the equation of the line used to do the
approximation in the interval 67.015 belongs to. That interval is [67.01, 67.02] .

In a given interval [x i , x ih] , the equation of the line used for approximation is
y= y i f x i , y i x−x i  . For the current problem, y= y i xi x −xi  . For the current interval,
x i=67.01 and y i is the approximation of y 67.01≈ y 6701 =2246.835 .

The equation of the line becomes: y=2246.83567.01 x−67.01 .

y 67.015=2246.83567.0167.015−67.01=2247.17005

The exact solution is 267.0152 /2=2247.50511 . The relative error in approximation is

2247.50511−2247.17005=.33506

Example3: Use Euler's method to numerically integrate the following ODE

dy 3 2
=−2x 12x – 20x8.5 from x=0 to x=4 with a step size of h=0.5 . Initial
dx
condition at x=0 is y=1
Solution:
Given: f  x , y =−2x312x 2 – 20x8.5  f n =−2x 3n12x 2n – 20x n8.5 , x 0=0, y 0=1 and
x n=0.5n
y 0.5= y  x 1 ≈ y 1= y 0 hf 0=10.5∗8.5=5.25
y 1= y  x 2≈ y 2= y 1hf 1 =5.250.5 1.25=5.875
Continuing, we get
y 1.5≈5.125, y 2≈4.5
y 2.5≈4.75, y 3≈5.875
y 3.5≈7.125, y 4 ≈7

Taylor series expansion

 y ' ' x i  x−x i 2 y 3   xi  x−x i3
y  x = y  x i y ' x i  x−x i  ...
2! 3!
The solution y  x  , if it has continuous derivatives, can be expressed by Taylor series
expansion about a starting point  xi , y i
2  n n
y ''h y h
y i1= y i yi ' h i ⋯ i Rn
2! n!
where h=xi1−x i , and Rn = remainder term

f '  xi , y i h2 f  n−1  x i , y i  n
y i1= y if  x i , y i  h ⋯ h O hn1 
2! n!
Euler's formula : i1 i y = y f  x i , y i h
f ' x i , y i h2 f ' ' x i , yi  h3 f  n−1  x i , y i hn
Error formula: Et =  ⋯ O hn1
2! 3! n!

Example 4:
Use Et to estimate the error of the first step of the previous example
Solution:
2 3 3  4
f ' x i , y i h f ' ' x i , yi  h f  xi , y i h
For this specific example, Et =   as f  4 =0
2! 3! 4!
f  x , y =−2x312x 2−20x8.5
f '  x , y =−6x224x – 20
f ' '  x , y =−12x24
f  3  x , y =−12
f  4   x , y =0

The first step evaluates y 0.5= y  x 1 ≈ y 1= y 0 hf 0

f ' x 0, y 0 h f ' '  x 0, y 0 h3 f  3  x 0, y 0 h4
2
So, Et =   , x 0=0, y 0=1
2! 3! 4!
f '  x 0, y 0 h2 −20∗0.52
Et ,1= = =−2.5
2! 2
f ' '  x 0, y 0 h3 24∗0.53
Et ,2= = =0.5
6 6
f  3  x 0, y 0 h 4 −12∗0.54
Et ,3= = =−0.031250
4! 4!
3
Et =∑i=1 E t , i=−2.50.5 – 0.03125=−2.03125
Check that the error in evaluating y 0.5 is −2.03125 analytically.

6.4 Runge-Kutta Methods

These are methods that improve Euler's method

6.4.1. Runge-Kutta 2nd Order

• Also called the midpoint method
 • Improves Euler's method by adding a midpoint in the step, which increases accuracy.
dy
• To solve the first order ODE =f x , y  , given the initial condition y  x 0 = y 0 , and
dx
a step size h , we use the following
k 1=f  x n , y n ... as in Euler's method
h hk
k 2=f  x n , y n 1  = Slope at the midpoint of the interval using slope k 1 to
2 2
determine the value of y at the point x nh /2 using Euler's method.

y n1= y n hk 2

6.4.2. Runge-Kutta 4th Order

• Most often used ODE solving method
1
• y n1= y n  h k 12 k 22 k 3k 4 
6
where n1 is the RK4 approximation of y  x 1 and
y
k 1=f  x n , y n
h k
k 2=f  x n , y nh 1 
2 2
h k
k 3 =f x n  , y nh 2 
2 2
k 4=f  x nh , y nh k 3 

• y n1 is determined by the present value y n plus the product of the size of the
interval, h , and the estimated slope.
• The slope is the weighted average of slopes
• k 1  slope @ the beginning of the interval
• k 2  slope at the midpoint of the interval, using slope k 1 to determine the value
h
of y at the point x n using Euler's method
2
• k 3  slope at the midpoint, but now using the slope k 2 to determine the y-value
• k 4  slope at the end of the interval, with its y-value determined using k 3
• In averaging the4slopes, greater weight is given to the slopes at the midpoint
1
• Slope= k 12 k 2 2 k 3k 4 
6

dy
Example1. = y , y 0=1 , use Runge-kutta 2nd order method with size h=0.1 to find the
dx
approximate values of the solution at x=0.1 . Also find the approximate value of y 100 .
Solution:
Given: f  x n , y n = y n , x 0=0, y 0 =1
y 0.1= y  x 1 ≈ y 1= y 0hk 2
h h
k 2=f  x 0 , y 0 k 1 and k 1=f  x 0, y 0  , so
2 2
 h
k 1= y 0=1 and k 2= y 0 k 1=10.05∗1=1.05 , therefore
2
y 1=10.11.05=1.105

Now, compare the accuracy of the solution obtained using Euler's method and the one using RK2.
The ODE given in the problem is easy to solve analytically: y  x =e x y 0.1=1.10517092
| Eeuler |=1.10517092−1.1=0.00517092, | E RK2 |=1.10517092−1.105=1.7092e-04

To find y 100 , we solve the difference equation we get from RK2, which is
h2
y n1= y n hy n y n
2
2
h
y n1 – 1h  y n=0 , solving this difference equation, we get:
2
2 n
h
y n=1h 
2
0.12 1000
y 100≈ y 1000 =10.1  =1.1051000
2

Example 2:
Solve the problem using RK4:
h
y 1= y 0   k 12 k 22 k 3 k 4 
6

k 1=f  x 0, y 0 = y 0=1
h k
k 2=f  x 0 , y 0h 1 =10.05=1.05
2 2
h k 0.105
k 3 =f x 0  , y 0h 2 =1 =1.0525
2 2 2
k 4=f  x 0h , y 0h k 3 =10.10525=1.10525

0.1
y 1=1 12∗1.052∗1.05251.10525=1.105170833333333
6
E RK4=1.10517092−1.10517083=8.99999998971168e-08 , which is by far a better
approximation

To find y 100 , we need to solve the difference equation that results from RK4.
h
y n1= y n   k 12 k 22 k 3 k 4 
6
k 1=f  x n , y n= y n
h h h h h
k 2=f  x n , y n k 1 = y n k 1= y n  y n= y n 1 
2 2 2 2 2
h h h h h h2
k 3 =f x n  , y n k 2= y n  y n 1 = y n 1  
2 2 2 2 2 4
h h2 h 2 h3
k 4=f  x nh , y nhk 3= y nh y n 1  = y n 1h  
2 4 2 4
 2 h3
6 y n3 h y nh y n  y n
1 4 h h2 h3
Slope= k 12k 22k 3k 4 = = y n y n  y n y n
6 6 2 6 24
2 3 4
h h h
y n1= y n h∗slope= y n 1h   
2 6 24
h2 h3 h4
Difference equation: y n1 – y n 1h   =0
2 6 24

Solving the difference equation, we get

h2 h 3 h4 n n
y n=1h    , since h=0.1 , y n=1.10517083333333
2 6 24

y 100≈ y 1000 =1.105170833333331000=2.68791103021863e+43

Euler : y 100≈1.11000 =2.46993291800603e+41

RK2 : y 100≈1.1051000 =2.30291559690439e+43
RK4 : y 100≈1.105170833333331000 =2.68791103021863e+43
Exact : y 100=e 100 =2.68811714181612e+43

We can see that RK4 results in a better approximation than the others.