The Taylor series method

MEC3456/MAE3456
dy
The Taylor series method for solutions of ( )
= f t, y
dt

§ The numerical problem for ODE solution is: Find y(t+h) given y(t)

§ Let’s look at the Taylor series expansion of y(t+h)

dy h 2 d 2 y h 3 d 3 y
y (t + h ) = y (t ) + h + 2
+ 3
!
dt 2! dt 3! dt
h2 h3
= y (t ) + h f (t , y ) + y¢¢ + y¢¢¢!
2! 3!
§ If we IGNORE terms O(h2) and above then we have

( ) ()
y t + h ≅ y t + h f (t, y) This is Euler’s method
2
MEC3456/MAE3456
dy
The Taylor series method for solutions of ( )
= f t, y
dt

§ Let’s NOT throw away terms O(h2) and above

h2 h3
y (t + h) = y (t ) + h f (t , y ) + y¢¢ + y¢¢¢!
2! 3!

§ If we can estimate the higher order derivatives

– We can find an improved solution of the problem…

§ We have the original ODE, y! = f t, y ( )

d ∂f ∂f dy
§ Take the derivative of this to give y!! =
dt
( )
f (t, y) = +
∂t ∂y dt
∂f ∂f
y!! = + y!
∂t ∂y 3
MEC3456/MAE3456
dy
The Taylor series method for solutions of ( )
= f t, y
dt

§ How can we use this in our ODE solution? Consider a concrete example:
dy
§ Solve = −2t − y y(0) = −1 using a scheme with error O(h4)
dt 2 3
h h
()
§ For 4th order error we want to calculate y(t + h) ≅ y t + h f (t, y) +
2!
y"" + y"""
3!
§ Calculate the 2nd derivative term from the ODE
y¢ = -2t - y thus y¢¢ = -2 - y¢ or y¢¢ = -2 + 2t + y

§ Calculate the 3rd derivative term from the 2nd derivative term,

y!! = −2 + 2t + y thus y!!! = 2 + y! or y!!! = 2 − 2t − y

4
MEC3456/MAE3456
dy
The Taylor series method for solutions of ( )
= f t, y
dt

§ Substituting y! = −2t − y, y!! = −2 + 2t + y and y!!! = 2 − 2t − y

h2 h3
into y(t + h) ≅ y t + h y! + y!! + y!!! + O(h 4 )
() 2! 3!
h2 h3
§ Gives () (
y(t + h) ≅ y t + h −2t − y +) (
−2 + 2t + y + ) 2 − 2t − y + O(h 4 )
( )
2! 3!
§ Collecting terms in t and y

# h 2 h3 & " h3 % " h3 %

y(t + h) ≅ %%1− h + − (( y t + $$ −2h + h − '' t + $$ −h + '' + O(h 4 )
() 2 2

$ 2! 3! ' # 3& # 3&

5
MEC3456/MAE3456
dy
The Taylor series method for solutions of ( )
= f t, y
dt

§ While the method can sometimes be used successfully

– There are problems

§ It must be applied in a case-by-case manner

– Every problem will be different and will have a different y-update
– Mathematical analysis is required before the details are determined
– The method cannot be coded generically – a bespoke solution for each case

§ Stability is not guaranteed

– Must be assessed for each new application

6
MEC3456/MAE3456

END

7
2nd order Runge-Kutta techniques
 MEC3456/MAE3456
dy
2nd order Runge-Kutta methods for = f t, y ( )
dt

§ Runge-Kutta methods start with the same idea as the Taylor series method
h2 h3
§ Write y (ti + h) = y (ti ) + h f (ti , yi ) + y¢¢ + y¢¢¢!
2! 3!
§ Want to find a way to estimate the higher order derivative terms (y’’, y’’’, etc.)
without using the original ODE.

§ The more terms we estimate, the higher the accuracy of the method
dy! d ∂f ∂f dy ∂f ∂f
§ We know that y!! =
dt
=
dt
(f (t, y) ) = +
∂t ∂y dt
=
∂t
+f
∂y
§ OR
∂f ∂f
y!! = + f
∂t ∂y 9
 MEC3456/MAE3456
dy
2nd order Runge-Kutta methods for = f t, y ( )
dt
∂f ∂f h2 h3
§ Substitute y!! = + f into y (ti + h) = y (ti ) + h f (ti , yi ) + y¢¢ + y¢¢¢!
∂t ∂y 2! 3!
to remove the 2nd derivative term
h 2 æ ¶f (ti , yi ) ¶f (ti , yi ) ö h 3
§ Giving y (ti + h) = y (ti ) + h f (ti , yi ) + çç + f (ti , yi ) ÷÷ + y¢¢¢!
2! è ¶t ¶y ø 3!

∂f (ti , yi ) ∂f (ti , yi )
§ We know f (ti,yi). Need to find a way to estimate ,
∂t ∂y
§ We use Runge-Kutta theory to estimate them

10
 MEC3456/MAE3456

Runge-Kutta theory

§ Runge-Kutta theory makes the assumption that:

– The correct average value of the slope, f (t,y), over one step can be written as a
linear combination of values of f (t,y) at certain points (tk, yk) in the step.

§ Graphically:

f (ti,yi)

f (ti+α,yi+β)

ti ti+h ti ti+α ti+h ti ti+α ti+h 11

 MEC3456/MAE3456

Runge-Kutta theory

§ Runge-Kutta theory makes the assumption that:

– The correct average value of the slope, f (t,y), over one step can be written as a linear
combination of values of f (t,y) at certain points (tk, yk) in the step.

§ The problem is determining which points AND what linear combination of them

h 2 æ ¶f (ti , yi ) ¶f (ti , yi ) ö h 3
§ Re-order y (ti + h) = y (ti ) + h f (ti , yi ) + çç + f (ti , yi ) ÷÷ + y¢¢¢!
2! è ¶t ¶y ø 3!

ì h ¶f (ti , yi ) h ¶f (ti , yi ) ü h 3
§ To give y (ti + h) = y (ti ) + h í f (ti , yi ) + + f (ti , yi ) ý + y¢¢¢!
î 2 ¶t 2 ¶y þ 3!
12
 MEC3456/MAE3456

Runge-Kutta theory

ì h ¶f (ti , yi ) h ¶f (ti , yi ) ü h 3
§ We have y (ti + h) = y (ti ) + h í f (ti , yi ) + + f (ti , yi ) ý + y¢¢¢!
î 2 ¶t 2 ¶y þ 3!
§ The term in braces {...} is our “better estimate” of the slope over the
interval
– the red line in the figure below

ti ti+α ti+h
13
 MEC3456/MAE3456

Runge-Kutta theory

ì h ¶f (ti , yi ) h ¶f (ti , yi ) ü h 3
§ We have y (ti + h) = y (ti ) + h í f (ti , yi ) + + f (ti , yi ) ý + y¢¢¢!
î 2 ¶t 2 ¶y þ 3!
§ The term in braces {...} is our “better estimate” of the slope over the interval

§ It bears a striking resemblance to the 2-D Taylor series

∂f (t, y) ∂f (t, y)
f (t + α , y + β ) = f (t, y) + α +β + O(α 2 , β 2 ) (Eqn 1)
∂t ∂y
h h
§ Letting α = , β = f (t, y) in Eqn 1
2 2
ì æ h h ö ü
§ Gives y (t + h) = y (t ) + h í f ç t + , y + f (t , y ) ÷ý + h 3

i i i i i i y¢¢¢!
î è 2 2 øþ 3!
14
 MEC3456/MAE3456

Runge-Kutta theory

ì æ h h öü h ¢¢¢
3
§ Perhaps not clear, but y (ti + h) = y (ti ) + h í f ç ti + , yi + f (ti , yi ) ÷ý + y !
î è 2 2 øþ 3!
is EXACTLY the midpoint Method.

§ In terms of Runge-Kutta theory, the points at which we need to find the slope
f (t,y) is only the midpoint, f (ti+½, yi+½)
– Because there is only one point, there is no “linear combination”

15
 MEC3456/MAE3456

Runge-Kutta theory

§ Are there alternative (perhaps better) schemes?

ì h ¶f (ti , yi ) h ¶f (ti , yi ) ü h 3
§ Again, use y (ti + h) = y (ti ) + h í f (ti , yi ) + + f (ti , yi ) ý + y¢¢¢!
î 2 ¶t 2 ¶y þ 3!
§ Let the term in braces {...} be equal to a linear combination of
– the starting slope f (ti, yi) (which we always know) and
– the slope at some other point f (ti+α, yi+β)
" h ∂f (ti , yi ) h ∂f (ti , yi ) %
§ i.e. # f (ti , yi ) + + f (ti , yi ) & = c1 f (ti , yi ) + c2 f (ti + α , yi + β )
$ 2 ∂t 2 ∂y '
16
MEC3456/MAE3456
" h ∂f (ti , yi ) h ∂f (ti , yi ) %
Runge-Kutta theory f (t
# i i , y ) +
2 ∂t
+
2
f (ti
, yi
)
∂y
& = c1 f (ti , yi ) + c2 f (ti + α , yi + β )
$ '

§ Using the 2-D Taylor series for the RHS

" h ∂f (ti , yi ) h ∂f (ti , yi ) %
# f (ti , yi ) + + f (ti , yi ) &
$ 2 ∂t 2 ∂y '
( ∂f (ti , yi ) ∂f (ti , yi ) 2 2
+
= c1 f (ti , yi ) + c2 * f (ti , yi ) + α +β + O(α , β ) -
) ∂t ∂y ,
∂f (ti , yi ) ∂f (ti , yi )
( )
= c1 + c2 f (ti , yi ) + c2α + c2 β + O(α 2 , β 2 )
∂t ∂y

h h
§ Comparing terms gives c1 + c2 = 1, c2α = , c2 β = f (ti , yi )
2 2 17
MEC3456/MAE3456

Runge-Kutta theory

§ Once c1, c2, α and β are found, the ODE update is

()
y(ti + h) ≅ y ti + h {c1 f (ti , yi ) + c2 f (ti + α , yi + β )}

h h
c
§ Three equations, four unknowns: 1 2
+ c = 1, c2
α = , c2
β = f (ti , yi )
2 2
§ Infinite number of solutions
1
– e.g. c1 = c2 = , α = h, β = hf (ti , yi )
2
h
()
– gives y(ti + h) ≅ y ti + { f (ti , yi ) + f (ti + h, yi + h f (ti , yi ))}
2
§ This is EXACTLY Heun’s Method (you should prove this to yourself)
18
MEC3456/MAE3456

Runge-Kutta theory

h h
§ Infinite number of solutions c1 + c2 = 1, c2α = , c2 β = f (ti , yi )
2 2
h h
– e.g. c = 0, c = 1, α = , β = f (t , y )
1 2 i i
2 2
– gives the Midpoint method again (show this)

19
MEC3456/MAE3456

END

20
4th order Runge-Kutta techniques
MEC3456/MAE3456
dy
4th order Runge-Kutta methods for ( )
= f t, y
dt

§ The basic idea of Runge-Kutta theory can be extended to higher order methods
– Higher order usually means more accurate

§ The most common is the 4th order Runge-Kutta method (RK4) Error in one
h2 h3 h 4 iv h 5 v step is O(h5)
y (ti + h) = y (ti ) + h y¢ + y¢¢ + y¢¢¢ + y + y !
2! 3! 4! 5!
ì h h2 h 3 iv ü h 5 v
= y (ti ) + hí f (ti , yi ) + y¢¢ + y¢¢¢ + y ý + y !
î 2! 3! 4! þ 5!
§ We then seek to write the term in braces above as
"$ h h2 h 3 iv &$
# f (ti , yi ) + y!! + y!!! + y ' = {w1k1 + w2 k2 + w3k3 + w4 k4 }
$% 2! 3! 4! $( 22
MEC3456/MAE3456

4th order Runge-Kutta methods

§ So that the ODE update becomes y(ti + h) = y(ti ) + h w1k1 + w2 k2 + w3k3 + w4 k4

{ }
§ Here, the wi are the weights and the ki are the derivative (slope) estimates
k1 = f (ti , yi )
k2 = f (ti + a1h, yi + b1k1 )
k3 = f (ti + a2h, yi + b2 k1 + b3k2 )
k4 = f (ti + a3h, yi + b4 k1 + b5k2 + b6 k3 )

§ The mathematics for estimating wi and ki are intricate

– Expand the ki terms in Taylor series, equate like terms, etc. ...
– 13 constants (w1-4, a1-3, b1-6) and 11 equations, giving a wide choice
23
MEC3456/MAE3456

4th order Runge-Kutta methods

§ The standard RK4 coefficients give a scheme that is written

( )
k1 = f ti , yi
! h h $
k2 = f # ti + , yi + k1 &
" 2 2 % h
! h h $ 6
(
yi+1 = yi + k1 + 2k2 + 2k3 + k4 )
k3 = f # ti + , yi + k2 &
" 2 2 %
( )
k4 = f ti + h, yi + hk3
§ The coefficients are
1 1 h h
w1 = w4 = , w2 = w3 = , a1 = a2 = , a3 = h, b2 = b4 = b5 = 0, b1 = b3 = , b6 = h
6 3 2 2 24
MEC3456/MAE3456

4th order Runge-Kutta methods

§ Graphically
! h h $
k2 = f # ti + , yi + f1 &
" 2 2 %
k1
! h h $
k3 = f # ti + , yi + f 2 &
" 2 2 %
h
k2 yi+1 = yi + (k + 2k2 + 2k3 + k4
6 1
)
yi
(
k1 = f ti , yi ) k3

(
k4 = f ti + h, yi + hf 3 )

ti ti+½ ti+1
25
 MEC3456/MAE3456

Is this worth all the effort?

§ It appears so .....

26
MEC3456/MAE3456

Summary

§ Key points regarding higher order methods for ODEs:

– The Taylor series method allows higher order solutions for ODEs
§ Must be done case-by-case
§ No generic coding possible
– Runge-Kutta methods use multiple estimates of slope to improve accuracy
§ RK2 provides a “family” of methods that includes Midpoint and Heun’s method
§ You need to know how to derive RK2
§ RK4 is a good high order method that is common
§ Need to be able to code it

27
MEC3456/MAE3456

END

28
Higher order equations
MEC3456/MAE3456

Solution of higher order equations

§ All ODE solution techniques to date can only be used on 1st order ODEs

§ How often do we see 1st order ODEs in Engineering? Not often.

§ E.g. the eqns that describe a pitching and plunging flat plate are:
d 2q dq
(
A + ρB )
dt 2
+ ρ (
UD )
dt
+ (
E + ρU 2
)
F q = 0,

! h $
where q = ## && , A, B, D, E, F are matrices
" α %
§ Application in aero-elasticity: Flutter

30
MEC3456/MAE3456

Damped, non-linear oscillator

§ A damped linear spring (e.g. shock absorber) is defined by the equation


x = −Dx − Kx
– where D is the damping coefficient and K the spring constant (D, K both > 0)

§ When D, K both1constant, an analytic solution exists of the form

− Dt
x = C0 e 2
cos(ω t) ω= K
. .
§ Q1: What do we do if D=D(x,x), K=K(x,x)?

§ Q2: How do we handle second time derivative?

31
 MEC3456/MAE3456

Solving higher order ODEs

§ In the most general case, we will want to

solve an equation like y = y1,

æ ö dy dy1
dn y d n -1 y d n - 2 y = = y2 ,
n
= f ç x, y, n -1 , n - 2 , !÷÷
ç dx dx
dx è dx dx ø
d 2 y dy2
2
= = y3 ,
dx dx
d 3 y dy3
§ Basic idea is to introduce a hierarchy of 3
= = y4 ,
variables, y1, y2, y3, ... yn where we define dx dx
the yi according to !
dny dyn
= = f (x, y1," yn )
dx n
dx 32
 MEC3456/MAE3456

Solving higher order ODEs

§ In the most general case, we will want to

solve an equation like y = y1,

æ ö dy dy1
dn y d n -1 y d n - 2 y = = y2 ,
n
= f ç x, y, n -1 , n - 2 , !÷÷
ç dx dx
dx è dx dx ø
d 2 y dy2
2
= = y3 ,
dx dx
d 3 y dy3
§ Basic idea is to introduce a hierarchy of 3
= = y4 ,
variables, y1, y2, y3, ... yn where we define dx dx
the yi according to !
dny dyn
= = f (x, y1," yn )
dx n
dx 33
 MEC3456/MAE3456

Solving higher order ODEs

§ Taking the essential definitions of y1, y2, y3, ... yn from

dy1 dy2 dy3 dyn

y = y1, = y2 , = y3, = y4 ,  = f (x, y1,yn )
dx dx dx dx

dy1
= y2 § Writing the unknowns as a vector,
dx y = (y1, y2, y3, ... yn )T
§ Gives the set of dy2
= y3 § Our system can be written
1st order ODEs dx
!
dy
dyn = f (x, y)
= f (x, y1," yn ) dx
dx 34
MEC3456/MAE3456

Solving higher order ODEs

§ These can be solved, for example, using Euler’s method

y1i+1 = y1i + hy2i

y2i+1 = y2i + hy3i OR y i+1 = y i + hf (xi , y i )
!
yni+1 = yni + hf (x, y1i ," yni )

§ The vector notation will be useful when thinking about this problem in MATLAB

35
MEC3456/MAE3456

Solving higher order ODEs

d 3u d 2u! du $ 2
§ Let’s work a concrete example u 3 + 2 + sin # & + u = g(x)
dx dx " dx %
1. Let y1= u
§ Use 1-4 to re-write the original ODE
du dy1 dy
2. Then = = y2 y1 3 + y3 + sin ( y2 ) + y12 = g(x) OR
dx dx dx
d 2u dy2 dy3 g(x) y3 sin ( y2 )
3. Then 2
= = y3 = − − − y1
dx dx dx y1 y1 y1
d 3u dy3 § Finally, include eqns 2 and 3 to complete
4. And =
3
dx dx dy1 dy2
= y2 AND = y3
dx dx
36
MEC3456/MAE3456

END

37