WBMT2049-T2/WI2032TH – Numerical Analysis for ODE’s

Lecture 4

initial-value problems
single-step numerical time integration methods
stability of numerical time integration

Lecture 4 1 / 34
theory of initial-value problems

Lecture 4 2 / 34
Definition: initial-value problem
Find a function y (t) which satisfies

dy
= f (t, y ), a ≤ t ≤ b,
dt
y (a) = ya ,

where f (t, y ) is a given function and ya is a given constant – the initial

value of y (t).

The notation f (t, y ) means f (t, y (t)). For example, f (t, y ) = t 2 sin(y (t)).

Lecture 4 3 / 34
Definition: well-posed initial-value problem
The initial value problem
dy
= f (t, y ), a ≤ t ≤ b,
dt
y (a) = ya ,

is well-posed, if its solution y (t) exists, is unique and depends continuously

on the data, i.e., on a, ya and f .

Lecture 4 4 / 34
Definition: Lipschitz continuity
A function f (t, y ) is Lipschitz continuous in y on a set D ⊂ R2 if there
exists a constant L > 0 such that:

|f (t, y1 ) − f (t, y2 )| ≤ L|y1 − y2 |,

for all (t, y1 ), (t, y2 ) ∈ D.

The constant L is called the Lipschitz constant.

Here t and y in f (t, y ) are considered as two independent variables.

Lipschitz continuity is a stronger requirement than the usual continuity.

Lecture 4 5 / 34
Theorem: Differentiablity and Lipschitz continuity
If the function f (t, y ) is differentiable with respect to y and there exists
L > 0 such that
∂f (t, y )
≤ L, for all (t, y ) ∈ D,
∂y

then the function f (t, y ) is Lipschitz continuous in y .

Lecture 4 6 / 34
Theorem: conditions for well-posedness of innitial-value problem
Let D = {(t, y ) | a ≤ t ≤ b, −∞ < y < ∞} and let f (t, y ) be a
continuous function. If f (t, y ) is also Lipschitz continuous in y on D, then
the initial-value problem
dy
= f (t, y ), a ≤ t ≤ b,
dt
y (a) = ya ,

is well posed.

Lecture 4 7 / 34
idea of numerical time integration

Lecture 4 8 / 34
Example: f (t, y ) = f (t) = sin(t)

dy
= sin(t), 0 ≤ t ≤ T ,
dt
y (0) = 0.

Integrating both sides over t:

Z t Z t
dy ′
′
dt = sin(t ′ ) dt ′ ,
0 dt 0
t
y (t) − y (0) = [− cos(t ′ )]0 = − cos(t) + cos(0),
y (t) = 1 − cos(t).

Lecture 4 9 / 34
Example: f (t, y ) = −y 2 sin(t)

dy
= −y 2 sin(t), 0 ≤ t ≤ T ,
dt
y (0) = 1.

Separating the t and y vartiables, and integrating both sides:

1
dy = − sin(t) dt,
y2
Z y (t) Z t
1
2
dy = − sin(t ′ ) dt ′ ,
y (0) y 0
 y (t)
1 t
− = [cos(t ′ )]0 ,
y y (0)
1
− + 1 = cos(t) − 1
y (t)
1
y (t) =
2 − cos(t)

Lecture 4 10 / 34
When analytical methods fail?

2
f (t, y ) = e −t , integral over t is not known
f (t, y ) = t cos(y 2 ), integral over y is not known
f (t, y ) = sin(ty 2 ), non-separable equation

Lecture 4 11 / 34
Numerical time integration
Split the time interval [t0 , T ] in pieces: [t0 , t1 ], [t1 , t2 ], . . . , [tN , T ]
For each interval [tn , tn+1 ], the initial value y (tn ) is given. To find y (tn+1 )
do:
1

Z tn+1 Z tn+1
dy
dt = f (t, y ) dt
tn dt tn
Z tn+1
y (tn+1 ) − y (tn ) = f (t, y ) dt
tn

2 Approximate integral over f (t, y ) by a numerical quadrature rule.

3 Solve the resulting algebraic equation for y (tn+1 ).

Lecture 4 12 / 34
elementary single-step time integration methods

Lecture 4 13 / 34
Notation: yn = y (tn ). Uniform time stepping: tn+1 − tn = ∆t,
n = 0, . . . , N. Single-step methods involve only yn and yn+1 in each step.
Forward Euler Method
Single-point numerical approximation of the integral:
Z tn+1
yn+1 − yn = f (t, y ) dt ≈ ∆t f (tn , yn )
tn

Denoting by wn the numerical approximation of yn :

wn+1 = wn + ∆t f (tn , wn ), n = 0, 1, . . . , N

This is an explicit method, where wn+1 is directly obtained from wn without the
need to solve an additional equation.

Lecture 4 14 / 34
Backward Euler Method
Single-point numerical approximation of the integral:
Z tn+1
yn+1 − yn = f (t, y ) dt ≈ ∆t f (tn+1 , yn+1 )
tn

Denoting by wn the numerical approximation of yn :

wn+1 = wn + ∆t f (tn+1 , wn+1 ), n = 0, 1, . . . , N

This is an implicit method. The above (possibly nonlinear) equation must be

solved for wn+1 at each time step.

Lecture 4 15 / 34
Trapezoidal Method
Two-point numerical approximation of the integral:
Z tn+1
∆t ∆t
yn+1 − yn = f (t, y ) dt ≈ f (tn , yn ) + f (tn+1 , yn+1 )
tn 2 2

Denoting by wn the numerical approximation of yn :

∆t ∆t
wn+1 = wn + f (tn , wn ) + f (tn+1 , wn+1 ), n = 0, 1, . . . , N
2 2
This is an implicit method. The above (possibly nonlinear) equation must be
solved for wn+1 at each time step.

Lecture 4 16 / 34
Modified Euler Method
Two-point numerical approximation of the integral:
Z tn+1
∆t ∆t
yn+1 − yn = f (t, y ) dt ≈ f (tn , yn ) + f (tn+1 , yn+1 )
tn 2 2

Denoting by wn the numerical approximation of yn :

w n+1 = wn + ∆t f (tn , wn ), (predictor step)

∆t ∆t
wn+1 = wn + f (tn , wn ) + f (tn+1 , w n+1 ), (corrector step)
2 2
First, forward Euler method is used to predict w n+1 . Then, a corrected value
wn+1 is obtained with the Trapezoidal formula. This is an explicit method, since
f (tn+1 , w n+1 ) is known, if w n+1 is known.

Lecture 4 17 / 34
stability of numerical time integration methods

Lecture 4 18 / 34
Consider the problem
dy
= f (t, y ), t > t0 ; y (0) = y0
dt
and its perturbation:
d ỹ
= f (t, ỹ ), t > t0 ; ỹ (0) = y0 + ϵ0 ,
dt
where ϵ0 is the error in the initial data. The difference between the
solutions is defined as:

ϵ(t) = ỹ (t) − y (t)

This is the error due to the perturbation of the initial data.

Lecture 4 19 / 34
Stability of initial-value problem
The initial value problem is called stable if

|ϵ(t)| < ∞, for all t > t0 ,

and absolutely stable if

lim |ϵ(t)| = 0.
t→∞

If |ϵ(t)| is unbounded the problem is unstable.

Lecture 4 20 / 34
Stability of the test equation
Let f (t, y ) = λy (t) + g (t). Then, the so-called test equation is:

dy
= λy + g , t > t0 ; y (0) = y0 ,
dt
and its perturbation:
d ỹ
= λỹ + g , t > t0 ; ỹ (0) = y0 + ϵ0 .
dt
The difference ϵ(t) = ỹ (t) − y (t) satisfies its own initial-value problem (why?):

dϵ
= λϵ, t > t0 ; ϵ(0) = ϵ0 ,
dt
which can be solved as (how?):

ϵ(t) = ϵ0 e λ(t−t0 ) .

Lecture 4 21 / 34
Stability of the test equation: real and complex cases

Perturbation ϵ(t) of the solution of the test problem evolves as:

ϵ(t) = ϵ0 e λ(t−t0 ) .

Let λ ∈ R. Then, the test problem is stable if λ ≤ 0, absolutely stable if

λ < 0, unstable if λ > 0.
Let λ ∈ C, i.e., λ = Re(λ) + iIm(λ). Then,

|ϵ(t)| = ϵ0 e (Re(λ)+iIm(λ))(t−t0 ) = |ϵ0 | e Re(λ)(t−t0 ) e iIm(λ)(t−t0 )

= |ϵ0 |e Re(λ)(t−t0 ) ,

and the test problem is stable if Re(λ) ≤ 0, absolutely stable if Re(λ) < 0,
unstable if Re(λ) > 0.

Lecture 4 22 / 34
Numerical solution of the test equation: FE Method

Forward Euler Method for the test equation:

wn+1 = wn + ∆t (λwn + gn ) = (1 + λ∆t)wn + ∆t gn , w0 = y0 .

Forward Euler Method for perturbed test equation:

w̃n+1 = w̃n + ∆t (λw̃n + gn ) = (1 + λ∆t)w̃n + ∆t gn , w̃0 = y0 + ϵ0

Numerical approximation of ϵ(tn ), i.e., ϵn = w̃n − wn , satisfies:

ϵn+1 = (1 + λ∆t)ϵn , ϵ0 = ϵ0 .

Lecture 4 23 / 34
Numerical solution of the test equation: BE Method

Backward Euler Method for the test equation:

wn+1 = wn + ∆t (λwn+1 + gn+1 ), w0 = y0 ,

1 ∆t gn+1
wn+1 = wn + , w0 = y0 ,
1 − λ∆t 1 − λ∆t
Backward Euler Method for perturbed test equation:
1 ∆t gn+1
w̃n+1 = w̃n + , w̃0 = y0 + ϵ0 ,
1 − λ∆t 1 − λ∆t
Numerical approximation of ϵ(tn ), i.e., ϵn = w̃n − wn , satisfies:

1
ϵn+1 = ϵn , ϵ0 = ϵ0 .
1 − λ∆t

Lecture 4 24 / 34
Amplification factors of single-step methods
For all single-step methods the numerical approximation of the perturbation has
the form:

ϵn+1 = Q ϵn

with the following amplification factors:

Q = 1 + λ∆t, (Forward Euler)

1
Q= , (Backward Euler)
1 − λ∆t
1 + 12 λ∆t
Q= , (Trapezoidal)
1 − 12 λ∆t
1
Q = 1 + λ∆t + (λ∆t)2 , (Modified Euler)
2

Lecture 4 25 / 34
Stability of numerical methods for test equation
Since

ϵn+1 = Q ϵn = Q n ϵ0 ,

and

|ϵn+1 | = |Q n ϵ0 | = |Q|n |ϵ0 |,

a numerical method is stable, i.e. |ϵn | < ∞ for all n = 0, 1, . . . , if

|Q| ≤ 1,

absolutely stable, i.e. limn→∞ |ϵn | = 0, if

|Q| < 1,

and unstable, i.e. |ϵn | is unbounded, if

|Q| > 1.

Lecture 4 26 / 34
Stability condition for the Forward Euler Method

Assume that the initial-value problem is absolutely stable, i.e. λ < 0 or

λ = −|λ| and |λ| > 0. Then, the Forward Euler Method, Q = 1 + λ∆t, is
stable if:

|Q| ≤ 1,
−1 ≤Q ≤ 1,
−1 ≤1 + λ∆t ≤ 1,
−1 ≤1 − |λ|∆t ≤ 1,
−2 ≤ − |λ|∆t ≤ 0,
2
0 ≤∆t ≤ .
|λ|

Stability condition: ∆t ≤ 2/|λ|.

Forward Euler Method is only conditionally stable.

Lecture 4 27 / 34
Stability of the Backward Euler Method

The Backward Euler Method’s amplification factor is Q = 1/(1 − λ∆t).

With λ = −|λ|, |λ| > 0, we have:

|Q| ≤ 1,
−1 ≤Q ≤ 1,
1
−1 ≤ ≤ 1,
1 − λ∆t
1
−1 ≤ ≤1
1 + |λ|∆t

This holds for any ∆t > 0.

Backward Euler Method is unconditionally stable.

Lecture 4 28 / 34
Stability of general problems
For a general f (t, y )

dy
= f (t, y ), t > t0 ; y (0) = y0 ,
dt
consider the Taylor series around t̂ and ŷ :

∂f (t̂, ŷ ) ∂f (t̂, ŷ )
f (t, y ) = f (t̂, ŷ ) + (y − ŷ ) + (t − t̂) + C1 (t − t̂)2 + C2 (y − ŷ )2
∂y ∂t
= λy + g (t) + C2 (y − ŷ )2 .

Neglecting C2 (y − ŷ )2 , this can be reduced to the test equation with

f (t, y ) = λy + g (t) and

∂f
λ= (t̂, ŷ ).
∂y
∂f
Approximate (local) stability condition λ < 0, i.e., ∂y (t̂, ŷ ) < 0, depends on
(t̂, ŷ ).

Lecture 4 29 / 34
Example: analytical stability
Consider the initial-value problem
dy
= −10y 2 + 20, t > 0; y (0) = 0.
dt
This is a separable equation with the analytical solution:
√
e t20 2
− 1√
y (t) = √ 2.
e t20 2 +1
Linearization (first two terms of Taylor series) of f (t, y ) around (t̂, ŷ ):
∂
−10y 2 + 20 = −10ŷ 2 + 20 + −10y 2 + 20 (y − ŷ ) + O((y − ŷ )2 )


∂y y =ŷ

= −10ŷ 2 + 20 − 20ŷ (y − ŷ ) + O((y − ŷ )2 )

= λy + g (t) + O((y − ŷ )2 ),
where
λ = −20ŷ .

The problem is locally stable (λ < 0) for ŷ > 0.

Lecture 4 30 / 34
Example: numerical stability of the FE method

Applying the Forward Euler method with wi denoting the numerical

approximation of yi :

w0 = 0,
wi+1 = wi + ∆t (−10wi2 + 20), i = 0, 1, . . .

Stability condition on ∆t for the FE method:

2 2 2 1 1
∆t ≤ = ≤ = √ √ = √
|λ| 20|ŷ | 20 maxt |y (t)| 10 2 maxt | tanh(t 300)| 10 2

In general, y (t) is not known, and a stable ∆t is obtained experimentally

(educated trial and error). Instability is easy to spot.

Lecture 4 31 / 34
Lecture 4 32 / 34
Lecture 4 33 / 34
Questions

Derive the amplification factors for the Trapezoidal and Modified

Euler Methods applied to the test equation.
Study the stability of the Trapezoidal and Modified Euler Methods for
the test equation, either deriving the stability condition on the time
step ∆t or showing that the method is unconditionally stable.

Lecture 4 34 / 34