Stiff Systems,

Initial and Boundary Value

Problems
 STIFFNESS

 A stiff system is one involving rapidly changing components together

with slowly changing ones

 Both individual and systems of ODEs can be stiff

Example:

If y(0) = 0, the analytical solution is:

Dominated by

Dominated by
Step size required for stability

If y(0) = y0, the solution is

Explicit Euler’s method:

Condionally stable
<1

For the fast transient part of the example, the step size to maintain stability
must be < 2/1000 = 0.002
An even smaller step size would be required for an accurate solution!
Implicit Euler method:

regardless of the size of the step, |yi| → 0 as i →∞.

unconditionally stable
EXAMPLE
Use both the explicit and implicit Euler methods to solve

where y(0) = 0.

(a) Use the explicit Euler with step sizes of 0.0005 and 0.0015 to solve
for y between t = 0 and 0.006.
(b) Use the implicit Euler with a step size of 0.05 to solve for y between
0 and 0.4.
Solution.
(a) The explicit Euler’s method

If h = 0.0015, the solution

shows oscillations
(b) The implicit Euler’s method

rearrange this equation

Systems of ODEs can also be stiff

y1(0) = 52.29

y2(0) = 83.82

The exact solution:

Exponents are (-)

the large exponents respond rapidly
 heart of the system’s stiffness.
implicit Euler’s method

Solve simultaneously
 MATLAB Functions for Stiff Systems

Built-in functions for solving stiff systems of ODEs:

 ode15s: a variable-order solver based on numerical differentiation formulas. It

is a multistep solver that is used for stiff problems of low to medium accuracy.

 ode23s: is a one-step solver and may be more efficient than ode15s at crude
tolerances. It can solve some kinds of stiff problems better than ode15s.

 ode23t: is an implementation of the trapezoidal rule with a “free- interpolant”.

This is used for moderately stiff problems with low accuracy where you need a
solution without numerical damping.

 ode23tb: is an implementation of an implicit Runge-Kutta formula with a first

stage that is a trapezoidal rule and a second stage that is a backward
differentiation formula of order 2. This solver may also be more efficient than
ode15s at crude tolerances.
 MATLAB Functions for Nonstiff Systems

 ode23 function simultaneously uses second- and third-order RK

formulas to solve the ODE and make error estimates for step-size
adjustment.
1
yi 1  yi  (2k1  3k2  4k3 )h
9
k1  f ti , yi 
 1 1 
k 2  f  ti  h, yi  k1h 
 2 2 
 3 3 
k 3  f  t i  h, y i  k 2 h 
 4 4 
The error is estimated as

Ei 1 
1
(5k1  6k2  8k3  9k4 )h k4  f ti 1 , yi 1 
72
At each step, the error is compared to a tolerance:
RelTol = 10-3
E  max( RelTol  y , AbsTol )
AbsTol=10-6

If <: the value of yi+1 is accepted, and k4 becomes k1

for the next step

If >: the step is repeated with reduced step sizes until

the estimated error satisfies the tolerance.
 ode45 function simultaneously uses fourth- and fifth-order RK
formulas to solve the ODE and make error estimates for step-size
adjustment.
 MATLAB recommends that ode45 is the best function to apply as a “first try” for most
problems.

 ode113 function uses a variable-order Adams-Bashforth-Moulton

solver. It is useful for stringent error tolerances or computationally
intensive ODE functions.

Call a ode function in Matlab:

y: the solution array

t: time vector,
odefun: the name of the function of the right-hand-sides of the DEs,
tspan: the integration interval,
y0: the initial values.
 Multistep Methods

One-step methods: a single point ti used Multistep methods: previous points

to predict a value of the dependent are used as well.
variable yi+1 at a future point ti+1
 The Non-Self-Starting Heun Method

 Heun approach uses Euler’s method as a predictor

yi01  yi  f (ti , yi )h local truncation error O(h2)

 The trapezoidal rule as a corrector

f (ti , yi )  f (ti 1 , yi01 )

yi 1  yi  h local truncation error O(h3)
2

The predictor is the weak link in the method because it has the greatest
error.
Improve Heun’s method by developing a predictor with a local error of
O(h3).
yi01  yi 1  f (ti , yi )2h
Need yi-1: not a typical initial value problem!

The midpoint method is used

as a predictor.
yi01  yim1  f (ti , yim )2h

The trapezoidal rule is employed as a

corrector.
j 1
f (t , y m
)  f (t , y i 1 )
yi 1  yi 
j m i i i 1
h
2
j=1,2, ..., m
The iterations are terminated based on an estimate of the
approximate error,

yij1  yij11
a  j
100%
yi 1

When |εa| is less than an error tolerance εs, the iterations are
terminated
EXAMPLE.
Use the non-self-starting Heun method to integrate
y' = 4e0.8t − 0.5y from t = 0 to 4 with a step size of 1.

The initial condition at t = 0 is y = 2 & at t=-1 y = -0.3929953

 t=-1 0 1 2 3 4
Solution.
step 1 2 3 4
Step 1 (i=0):
The predictor to extrapolate linearly from t = -1 to 1:
yi01  yim1  f (ti , yim )2h
y10  0.3929953  [4e0.8(0)  0.5(2)]2  5.607005

The corrector to compute the value:

f (ti , yim )  f (ti 1 , yij11 ) j=1
y y 
j
i 1
m
i h
2
4e 0.8( 0)  0.5(2)  4e 0.8(1)  0.5(5.607005)
y1  2 
1
1  6.549331
2

a true percent relative error of -5.73% (true value = 6.194631).

This is smaller than the value of -8.18% found in the self-
starting Heun.
 Improve the solution: j=2
f (ti , yim )  f (ti 1 , yij11 )
y y 
j
i 1
m
i h
2
3  4e 0.8(1)  0.5(6.549331) An error of -1.92%
y1  2 
2
1  6.313749
2
An approximate error can be estimated:
yij1  yij11 6.313749  6.549331
a  j
 100%  100%  3.7%
yi 1 6.313749

iterations converge on a value of 6.36087 (εt = -2.68%)

Step 2 (i=1):
yi01  yim1  f (ti , yim )2h
The predictor
y20  2  [4e 0.8(1)  0.5(6.36087)]2  13.44346
Better than 12.0826 (εt = 18%) that
Iterations yield 15.76693 (εt = 6.8%) was computed with the original
Heun method
Non-self-starting Heun method: characteristic of most multistep methods.

- Uses an open integration formula (the midpoint method) to make an

initial estimate. This predictor step requires a previous data point.
- Then, it uses a closed integration formula (the trapezoidal
rule) that is applied iteratively to improve the solution.

How to improve the multistep methods ?

- Use higher-order integration formulas as predictors and correctors.

- For example: the higher order Newton-Cotes formulas can be used.

Let’s make a review of integration formulas.

Integration Formulas
The Newton-Cotes Formulas:
The Newton-Cotes formulas estimate the
integral over an interval spanning
several points. This integral is then used
to project from the beginning of the
interval to the end.

The Adams formulas:

The Adams formulas use a set of points
from an interval to estimate the integral
solely for the last segment in the interval.
This interval is then used to project
across this last segment.
 Newton-Cotes Formulas
Open formulas n equally spaced data points
𝑥𝑖+1
𝑦𝑖+1 = 𝑦𝑖−𝑛 + න 𝑓𝑛 𝑥 𝑑𝑥
𝑥𝑖−𝑛

For n=1,
𝑦𝑖+1 = 𝑦𝑖−1 + 2𝑓𝑖 ℎ
Equivalent to the midpoint method
For n=2,
3ℎ
𝑦𝑖+1 = 𝑦𝑖−2 + 𝑓 + 𝑓𝑖−1
2 𝑖
For n=3,
4ℎ
𝑦𝑖+1 = 𝑦𝑖−3 + 2𝑓𝑖 − 𝑓𝑖−1 + 2𝑓𝑖−2
3
Closed formulas
𝑥𝑖+1
𝑦𝑖+1 = 𝑦𝑖−𝑛+1 + න 𝑓𝑛 𝑥 𝑑𝑥
𝑥𝑖−𝑛 +1
For n=1,
ℎ
𝑦𝑖+1 = 𝑦𝑖 + 𝑓𝑖 + 𝑓𝑖+1
2
which is equivalent to trapezoidal rule.

For n=2,
ℎ
𝑦𝑖+1 = 𝑦𝑖−1 + 𝑓𝑖−1 + 4𝑓𝑖 + 𝑓𝑖+1
3

which is equivalent to Simpson’s 1/3 rule.

 Adams Formulas
Open Formulas (Adams-Bashforth):

One way to derive the Adams formulas is to write a forward Taylor series expansion
around xi:

𝑓𝑖′ 2 𝑓𝑖′′ 3
𝑦𝑖+1 = 𝑦𝑖 + 𝑓𝑖 ℎ + ℎ + ℎ +⋯
2 6

which can also be written as

ℎ ′ ℎ2 ′′
𝑦𝑖+1 = 𝑦𝑖 + ℎ 𝑓𝑖 + 𝑓𝑖 + 𝑓𝑖 + ⋯
2 3

A backward difference can be used to approximate the derivative:

𝑓𝑖 −𝑓𝑖−1 𝑓𝑖′′
𝑓𝑖′ = + ℎ +𝑂(ℎ2 )
ℎ 2

which can be substituted into above equation, or, collecting terms,

 3 1 5
𝑦𝑖+1 = 𝑦𝑖 + ℎ 𝑓𝑖 − 𝑓𝑖−1 + ℎ3 𝑓𝑖′′ + 𝑂(ℎ4 )
2 2 12

This formula is called the second order open Adams formula. Open
Adams formulas are also referred to as Adams-Bashforth formulas.

High order Adams-Bashforth formulas can be developed by substituting

higher difference approximations. The nth order open Adams formula can be
represented generally as

𝑛−1

𝑦𝑖+1 = 𝑦𝑖 + ℎ ෍ 𝛽𝑘 𝑓𝑖−𝑘 + 𝑂(ℎ𝑛+1 )

𝑘

The coefficients βk are compiled in the following Table. Notice that the first
order is Euler’s method.
 𝑛−1

𝑦𝑖+1 = 𝑦𝑖 + ℎ ෍ 𝛽𝑘 𝑓𝑖−𝑘 + 𝑂(ℎ𝑛+1 )

𝑘

Coefficients and truncation error for Adams-Bashforth

predictors
 4th order

𝑛−1

𝑦𝑖+1 = 𝑦𝑖 + ℎ ෍ 𝛽𝑘 𝑓𝑖−𝑘 + 𝑂(ℎ𝑛+1 )

𝑘
Closed Formulas (Adams-Moulton):

One way to derive the Adams formulas is to write a backward Taylor series
expansion around xi+1:
′ ′′
𝑓𝑖+1 2
𝑓𝑖+1
𝑦𝑖+1 = 𝑦𝑖+1 − 𝑓𝑖+1 ℎ + ℎ − ℎ3 + ⋯
2 3

Solving for yi+1 yields

ℎ ′ ℎ2 ′′
𝑦𝑖+1 = 𝑦𝑖 + ℎ 𝑓𝑖+1 − 𝑓𝑖+1 + 𝑓𝑖+1 + ⋯
2 6

A difference can be used to approximate the first derivative:

′′
′ 𝑓𝑖+1 − 𝑓𝑖 𝑓𝑖+1
𝑓𝑖+1 = + ℎ + 𝑂(ℎ2 )
ℎ 2

which can be substituted into above equation, and collecting terms,

 1 1 1 3 ′′
𝑦𝑖+1 = 𝑦𝑖 + ℎ 𝑓𝑖+1 + 𝑓𝑖 − ℎ 𝑓𝑖+1 − 𝑂(ℎ4 )
2 2 12

This formula is called the second order closed Adams formula.

Open Adams formulas are also referred to as Adams-Moulton formula.

Notice that it is also the Trapezoidal rule.

The nth order closed Adams formula can be represented generally as

𝑛−1

𝑦𝑖+1 = 𝑦𝑖 + ℎ ෍ 𝛽𝑘 𝑓𝑖+1−𝑘 + 𝑂(ℎ𝑛+1 )

𝑘=0
The coefficients βk are compiled in the following Table.

Coefficients and truncation error for Adams-Moulton correctors

 4th order
𝑛−1

𝑦𝑖+1 = 𝑦𝑖 + ℎ ෍ 𝛽𝑘 𝑓𝑖+1−𝑘 + 𝑂(ℎ𝑛+1 )

𝑘=0
 High Order Multistep Methods
Milne’s Method:

Milne’s method is the most common multistep method based on

Newton-Cotes integration formulas. It uses the three point Newton-Cotes
open formula as a predictor:

0 𝑚 4ℎ
𝑦𝑖+1 = 𝑦𝑖−3 + 2𝑓𝑖𝑚 − 𝑓𝑖−1
𝑚 𝑚
+ 2𝑓𝑖−2
3
and the three point Newton-Cotes formula (Simpson’s 1/3 rule) as a
corrector:
𝑗 ℎ 𝑚 𝑗−1
𝑦𝑖+1 = 𝑦𝑖−1 + 𝑓𝑖−1 + 4𝑓𝑖𝑚 + 𝑓𝑖+1
𝑚
3
where j is an index representing the number of iterations of the modifier.
Fourth-Order Adams Method:

A popular multistep method based on the Adams integration formulas

uses the fourth-order Adams-Bashforth formula as the predictor:

0 55 𝑚 59 𝑚 37 𝑚 9 𝑚
𝑦𝑖+1 = 𝑦𝑖𝑚 + ℎ 𝑓𝑖 − 𝑓𝑖−1 + 𝑓𝑖−2 − 𝑓𝑖−3
24 24 24 24

and the fourth order Adams-Moulton formula as the corrector:

𝑗 9 𝑗−1 19 𝑚 5 𝑚 1 𝑚
𝑦𝑖+1 = 𝑦𝑖𝑚 +ℎ 𝑓 + 𝑓𝑖 − 𝑓𝑖−1 − 𝑓
24 𝑖+1 24 24 24 𝑖−2
 Boundary-Value Problems
For an nth order equation, n conditions are required. If all the conditions
are specified at the same value of the independent variable, then we are
dealing with an initial value problem.

In contrast, there is another application for which the conditions are not
known at a single point, but rather, are known at different values of the
independent variable. Because these values are often specified at the
extreme points or boundaries of a system, they are customarily referred to
as boundary value problems.
The Shooting Method
The shooting method is based on converting the boundary value
problem into an equivalent initial value problem.

A trial-error approach is then implemented to solve the initial value

version.
Example.

The conservation of heat can be used to develop a heat balance for a

long, thin rod. If the rod is not insulated along its length and the system
is at a steady state, the equation that results is

Use the shooting method to solve for a 10 m rod with h = 0.01 m-2, Ta =
20 and the boundary conditions
 the second-order equation can be expressed as two first order ODEs:

We will solve these equations simultaneously. But, we require an initial

value for z. For the shooting method, we guess a value: say, z(0) = 10.
Let’s use a fourth-order RK method with a step size of 2:

(a) Obtain at the end of the interval of T(10) = 168.3797,

(approximate)
differs from the boundary condition of T(10) = 200.
(exact)

(b) we make another guess, z(0) = 20,

Calculate again: T(10) = 285.8980 is obtained.
(approximate)
Because the original ODE is linear, the values

z(0) = 10 T (10) = 168.3797

and

z(0) = 20 T (10) = 285.8980

are linearly related.

Use a linear interpolation formula for a better estimate of z(0):

Repeat the calculations again:

Note that the approximate and

the exact values match.
MATLAB examples for stiff and non-stiff systems
EXAMPLE.
The van der Pol equation is a model of an electronic circuit that arose
back in the days of vacuum tubes

The solution to this equation becomes progressively stiffer as μ gets

large. Given the initial conditions, y1(0) = dy1/dt = 1, use MATLAB to
solve the following two cases:

(a) for μ = 1, use ode45 to solve from t = 0 to 20; and

(b) for μ = 1000, use ode23s to solve from t = 0 to 6000.
Solution
(a) Convert the second-order ODE into a pair of first-order ODEs by,

2nd order 1st order

An M-file
function yp = vanderpol(t,y,mu)
yp = [y(2);mu*(1-y(1)^2)*y(2)-y(1)]; μ is passed as a parameter

>> [t,y] = ode45(@vanderpol,[0 20],[1 1],[],1);

>> plot(t,y(:,1),'-',t,y(:,2),'--')
>> legend('y1','y2'); [ ] as a place holder
 Smooth nature of the plot:
the van der Pol equation with μ = 1 is
not a stiff system
(b) If a standard solver like ode45 is used for the stiff case (μ = 1000), it
will fail. However, ode23s does an efficient job

>> [t,y] = ode23s(@vanderpol,[0 6000],[1 1],[],1000);

>> plot(t,y(:,1))

Sharper edges:
a visual manifestation of the “stiffness”
of the solution
EXAMPLE.

Solve the ODE y’ =4e0.8t − 0.5y from t = 0 to 4 with an initial condition

of y(0) = 2 using ode45 in MATLAB. The analytical solution at t = 4 is
75.33896.

>> dydt=@(t,y) 4*exp(0.8*t)-0.5*y;

>> [t,y]=ode45(dydt,[0 4],2);
>> y(length(t))
ans =
75.3390
EXAMPLE

Employ ode45 to solve the following set of nonlinear ODEs from t = 0 to

20:
dy1 dy2
 1.2 y1  0.6 y1 y2  0.8 y2  0.3 y1 y2
dt dt

where y1 = 2 and y2 = 1 at t = 0. Such equations are referred to as

predator-prey equations.
 dy1 dy2
Solution.  1.2 y1  0.6 y1 y2  0.8 y2  0.3 y1 y2
dt dt
create an M-file under the name: predprey.m
function yp = predprey(t,y)
yp = [1.2*y(1)-0.6*y(1)*y(2); -0.8*y(2)+0.3*y(1)*y(2)];

enter the following commands

>> tspan = [0 20];
>> y0 = [2, 1];
>> [t,y] = ode45(@predprey, tspan, y0);
>> plot(t,y)
>> plot(y(:,1),y(:,2)) % plot of the dependent variables
versus each other
To control the default parameters of the ode solver:

Some commonly used parameters

EXAMPLE
Use ode23 to solve the following ODE from t = 0 to 4:
dy ( t  2 ) 2 /[ 2 ( 0.075 ) 2 ]
 10e  0.6 y
dt
where y(0) = 0.5. Obtain solutions for the default (10-3) and for a more
stringent (10-4) relative error tolerance.
 dy
Solution.  10e ( t  2 ) 2 /[ 2 ( 0.075 ) 2 ]
 0.6 y
dt
create an M-file
function yp = dydt(t, y)
yp = 10*exp(-(t-2)*(t-2)/(2*.075^2))-0.6*y;

without setting the options, set the relative error

relative error tolerance = 10-3 tolerance = 10-4:
>>ode23(@dydt, [0 4], 0.5); >>options=odeset('RelTol',1e-4);
>>ode23(@dydt, [0, 4], 0.5,
options);
MATLAB Application: Bungee Jumper With
Cord

Solve 2 coupled ODEs for vertical position and velocity.

Gravitational force Spring force of the cord

Drag force Dampening force of the cord

Determine the position and velocity of a bungee jumper with

L = 30 m, g = 9.81 m/s2, m = 68.1 kg, cd = 0.25 kg/m, k = 40 N/m, and γ
= 8 N ・ s/m.
Perform the computation from t = 0 to 50 s and assume that the initial
conditions are x(0) = v(0) = 0.
M-file

function dydt = bungee(t,y,L,cd,m,k,gamma)

g = 9.81;
cord = 0;
if y(1) > L %determine if the cord exerts a force
cord = k/m*(y(1)-L)+gamma/m*y(2);
end
dydt = [y(2); g - sign(y(2))*cd/m*y(2)^2 - cord];

Equations are not stiff, we can use ode45

>> [t,y] = ode45(@bungee,[0 50],[0 0],[],30,0.25,68.1,40,8);
>> plot(t,-y(:,1),'-',t,y(:,2),':')
>> legend('x (m)','v (m/s)')
Jumper’s bouncing motion.
 Events

 Problems where we do not know the final time

Example: free-falling bungee jumper.

Suppose that the jump master neglects to attach the

cord to the jumper!

The final time  the jumper hitting ground

Determine when the jumper hits the ground.

 MATLAB’s events option provides a means to solve such problems.
 Solves differential equations until one of the dependent variables
reaches zero.

dx dv c
ODEs v  g  d vv
dt dt m
Assume that the jumper is initially located 200 m above the ground
and the initial velocity is 20 m/s in the upward direction.

That is,

x(0) = -200 and v(0) = -20.

M-file function for the odes: dx dv c
v  g  d vv
dt dt m

A function that defines the event

a script that generates the solution

the jumper hits the ground
in 9.5475 s with a velocity
of 46.2454 m/s.