Chapter 8

Dynamics and Dierential

Equations
153
154 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
Many dynamical systems are modeled by rst order systems of dierential
equations. An n component vector x(t) = (x
1
(t), . . . , x
n
(t)), models the state
of the system at time t. The dynamics are modelled by
dx
dt
= x(t) = f(x(t), t) , (8.1)
where f(x, t) is an n component function, f(x, t) = f
1
(x, t), . . . , f
n
(x, t)). The
system is autonomous if f = f(x), i.e., if f is independent of t. A trajectory
is a function, x(t), that satises (8.1). The initial value problem is to nd a
trajectory that satises the initial condition
1
x(0) = x
0
, (8.2)
with x
0
being the initial data. In practice we often want to specify initial data
at some time other than t
0
= 0. We set t
0
= 0 for convenience of discussion. If
f(x, t) is a dierentiable function of x and t, then the initial value problem has
a solution trajectory dened at least for t close enough to t
0
. The solution is
unique as long as it exists.
2
.
Some problems can be reformulated into the form (8.1), (8.2). For example,
suppose F(r) is the force on an object of mass m if the position of the object is
r R
3
. Newtons equation of motion: F = ma is
m
d
2
r
dt
2
= F(r) . (8.3)
This is a system of three second order dierential equations. The velocity at
time t is v(t) = r(t). The trajectory, r(t), is determined by the initial position,
r
0
= r(0), and the initial velocity, v
0
= v(0).
We can reformulate this as a system of six rst order dierential equations
for the position and velocity, x(t) = (r(t), v(t)). In components, this is
_
_
_
_
_
_
_
_
x
1
(t)
x
2
(t)
x
3
(t)
x
4
(t)
x
5
(t)
x
6
(t)
_
_
_
_
_
_
_
_
=
_
_
_
_
_
_
_
_
r
1
(t)
r
2
(t)
r
3
(t)
v
1
(t)
v
2
(t)
v
3
(t)
_
_
_
_
_
_
_
_
.
1
There is a conict of notation that we hope causes little confusion. Sometimes, as here,
x
k
refers to component k of the vector x. More often x
k
refers to an approximation of the
vector x at time t
k
.
2
This is the existence and uniqueness theorem for ordinary dierential equations. See any
good book on ordinary dierential equations for details.
8.1. TIME STEPPING AND THE FORWARD EULER METHOD 155
The dynamics are given by r = v and v =
1
m
F(r). This puts the equations (8.3)
into the form (8.1) where
f =
_
_
_
_
_
_
_
_
f
1
f
2
f
3
f
4
f
5
f
6
_
_
_
_
_
_
_
_
=
_
_
_
_
_
_
_
_
x
4
x
5
x
6
1
m
F
1
(x
1
, x
2
, x
3
)
1
m
F
2
(x
1
, x
2
, x
3
)
1
m
F
3
(x
1
, x
2
, x
3
)
_
_
_
_
_
_
_
_
.
There are variants of this scheme, such as taking x
1
= r
1
, x
2
= r
1
, x
3
= r
2
, etc.,
or using the momentum, p = m r rather than the velocity, v = r. The initial
values for the six compenents x
0
= x(t
0
) are given by the initial position and
velocity components.
8.1 Time stepping and the forward Euler method
For simplicity this section supposes f does not depend on t, so that (8.1) is just
x = f(x). Time stepping, or marching, means computing approximate values
of x(t) by advancing time in a large number of small steps. For example, if we
know x(t), then we can estimate x(t + t) using
x(t + t) x(t) + t x(t) = x(t) + tf(x(t)) . (8.4)
If we have an approximate value of x(t), then we can use (8.4) to get an ap-
proximate value of x(t + t).
This can be organized into a method for approximating the whole trajectory
x(t) for 0 t T. Choose a time step, t, and dene discrete times t
k
= kt
for k = 0, 1, 2, . . .. We compute a sequence of approximate values x
k
x(t
k
).
The approximation (8.4) gives
x
k+1
x(t
k+1
) = x(t
k
+ t) x(t
k
) + tf(x
k
) x
k
+ tf(x
k
) .
The forward Euler method takes this approximation as the denition of x
k+1
:
x
k+1
= x
k
+ tf(x
k
) . (8.5)
Starting with initial condition x
0
(8.5) allows us to calculate x
1
, then x
2
, and
so on as far as we like.
Solving dierential equations sometimes is called integration. This is because
of the fact that if f(x, t) is independent of x, then x(t) = f(t) and the solution
of the initial value problem (8.1) (8.2) is given by
x(t) = x(0) +
_
t
0
f(s)ds .
If we solve this using the rectangle rule with time step t, we get
x(t
k
) x
k
= x(0) + t
k1

j=0
f(t
j
) .
156 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
We see from this that x
k+1
= x
k
+tf(t
k
), which is the forward Euler method
in this case. We know that the rectangle rule for integration is rst order
accurate. This is a hint that the forward Euler method is rst order accurate
more generally.
We can estimate the accuracy of the forward Euler method using an infor-
mal error propagation equation. The error, as well as the solution, evolves (or
propagates) from one time step to the next. We write the value of the exact
solution at time t
k
as x
k
= x(t
k
). The error at time t
k
is e
k
= x
k
x
k
. The
residual is the amount by which x
k
fails to satisfy the forward Euler equations
3
(8.5):
x
k+1
= x
k
+ tf( x
k
) + tr
k
, (8.6)
which can be rewritten as
r
k
=
x(t
k
+ t) x(t
k
)
t
f(x(t
k
)) . (8.7)
In Section 3.2 we showed that
x(t
k
+ t) x(t
k
)
t
= x(t
k
) +
t
2
x(t
k
) + O(t
2
) .
Together with x = f(x), this shows that
r
k
=
t
2
x(t
k
) + O(t
2
) , (8.8)
which shows that r
k
= O(t).
The error propagation equation, (8.10) below, estimates e in terms of the
residual. We can estimate e
k
= x
k
x
k
= x
k
x(t
k
) by comparing (8.5) to
(8.6)
e
k+1
= e
k
+ t (f(x(k)) f( x
k
)) tr
k
.
This resembles the forward Euler method applied to approximating some func-
tion e(t). Being optimistic, we suppose that x
k
and x(t
k
) are close enough to
use the approximation (f

is the Jacobian matrix of f as in Section ??)

f(x
k
) = f( x
k
+ e
k
) f( x
k
) + f

( x
k
)e
k
,
and then
e
k+1
e
k
+ t (f

( x
k
)e
k
r
k
) . (8.9)
If this were an equality, it would imply that the e
k
satisfy the forward Euler
approximation to the dierential equation
e = f

(x(t)) e r(t) , (8.10)

where x(t) satises (8.1), e has initial condition e(0) = 0, and r(t) is given by
(8.8):
r(t) =
t
2
x(t) . (8.11)
3
We take out one factor of t so that the order of magnitude of r is the order of magnitude
of the error e.
8.1. TIME STEPPING AND THE FORWARD EULER METHOD 157
The error propagation equation (8.10) is linear, so e(t) should be proportional to
4
r, which is proportional to t. If the approximate e(t) satises e(t) = C(t)t,
then the exact e(t) should satisfy e(t) = O(t), which means there is a C(t)
with
e(t) C(t)t . (8.12)
This is the rst order accuracy of the forward Euler method.
It is important to note that this argument does not prove that the forward
Euler method converges to the correct answer as t 0. Instead, it assumes the
convergence and uses it to get a quantitative estimate of the error. The formal
proof of convergence may be found in any good book on numerical methods for
ODEs, such as the book by Isaiah Iserles.
If this analysis is done a little more carefully, it shows that there is an
asymptotic error expansion
x
k
x(t
k
) + tu
1
(t
k
) + t
2
u
2
(t
k
) + . (8.13)
The leading error coecient, u
1
(t), is the solution of (8.10). The higher or-
der coecients, u
2
(t), etc. are found by solving higher order error propagation
equations.
The modied equation is a dierent approach to error analysis that allows
us to determine the long time behavior of the error. The idea is to modify
the dierential equaion (8.1) so that the solution is closer to the forward Euler
sequence. We know that x
k
= x(t
k
) + O(t). We seek a dierential equation
y = g(y) so that x
k
= y(t
k
) + O(t
2
) We construct an error expansion for the
equation itself rather than the solution.
It is simpler to require y(t) so satisfy the forward Euler equation at each t,
not just the discrete times t
k
:
y(t + t) = y(t) + tf(y(t)) . (8.14)
We seek
g(y, t) = g
0
(y) + tg
1
(y) + (8.15)
so that the solution of (8.14) satises y = g(y) + O(t
2
). We combine the
expansion (8.15) with the Taylor series
y(t + t) = y(t) + t y(t) +
t
2
2
y(t) + O(t
3
) ,
to get (dividing both sides by t, O(t
3
)/t = O(t
2
).):
y(t) + t y(t) +
t
2
2
y(t) + O(t
3
) = y(t) + tf(y(t))
g
0
(y(t)) + tg
1
(y(t)) +

2
y(t) = f(y(t)) + O(t
2
)
4
The value of e(t) depends on the values of r(s) for 0 s t. We can solve u = f

(x)uw,
where w =
1
2
x, then solve (8.10) by setting e = tu. This shows that e(t) = t u(t),
which is what we want, with C(t) = u(t).
158 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
Equating the leading order terms gives the unsurprising result
g
0
(y) = f(y) ,
and leaves us with
g
1
(y(t)) +
1
2
y(t) = O(t) . (8.16)
We dierentiate y = f(y) + O(t) and use the chain rule, giving
y =
d
dt
y =
d
dt
_
f(y(t)) + O(t)
_
= f

(y) y(t) + O(t)

y = f

(y)f(y) + O(t)
Substituting this into (8.16) gives
g
1
(y) =
1
2
f

(y)f(y) .
so the modied equation, with the rst correction term, is
y = f(y)
t
2
f

(y)f(y) . (8.17)
A simple example illustrates these points. The nondimensional harmonic
oscillator equation is r = r. The solution is r(t) = a sin(t) + b cos(t), which
oscillates but does not grow or decay. We write this in rst order as x
1
= x
2
,
x
2
= x
1
, or
d
dt
_
x
1
x
2
_
=
_
x
2
x
1
_
. (8.18)
Therefore, f(x) =
_
x
2
x
1
_
, f

=
_
0 1
1 0
_
, and f

f =
_
0 1
1 0
__
x
2
x
1
_
=
_
x
1
x
2
_
, so (8.17) becomes
d
dt
_
y
1
y
2
_
=
_
y
2
y
1
_
+
t
2
_
y
1
y
2
_
=
_
t
2
t 1
1
t
2
__
y
1
y
2
_
.
We can solve this by nding eigenvalues and eigenvectors, but a simpler trick
is to use a partial integrating factor and set y(t) = e
1
2
tt
z(t), where z =
_
0 1
1 0
_
z. Since z
1
(t) = a sin(t) + b cos(t), we have our approximate nu-
merical solution y
1
(t) = e
1
2
tt
(a sin(t) + b cos(t)). Therefore
e(t)
_
e
1
2
tt
1
_
. (8.19)
This modied equation analysis conrms that forward Euler is rst order
accurate. For small t, we write e
1
2
tt
1
1
2
t t so the error is about
t t (a sin(t) + b cos(t)). Moreover, it shows that the error grows with t. For
each xed t, the error satises e(t) = O(t) but the implied constant C(t)
(in e(t) C(t)t) is a growing function of t, at least as large as C(t)
t
2
.
8.2. RUNGE KUTTA METHODS 159
8.2 Runge Kutta methods
Runge Kutta
5
methods are a general way to achieve higher order approximate
solutions of the initial value problem (8.1), (8.2). Each time step consists of
m stages, each stage involving a single evaluation of f. The relatively simple
four stage fourth order method is in wide use today. Like the forward Euler
method, but unlike multistep methods, Runge Kutta time stepping computes
x
k+1
from x
k
without using values x
j
for j < k. This simplies error estimation
and adaptive time step selection.
The simplest Runge Kutta method is forward Euler (8.5). Among the second
order methods is Heuns
6

1
= tf(x
k
, t
k
) (8.20)

2
= tf(x
k
+
1
, t
k
+ t) (8.21)
x
k+1
= x
k
+
1
2
(
1
+
2
) . (8.22)
The calculations of
1
and
2
are the two stages of Heuns method. Clearly they
depend on k, though that is left out of the notation.
To calculate x
k
from x
0
using a Runge Kutta method, we apply take k time
steps. Each time step is a transformation that may be written
x
k+1
=

S(x
k
, t
k
, t) .
As in Chapter 6, we express the general time step as
7
x

=

S(x, t, t). This

S approximates the exact solution operator, S(x, t, t). We say that x

=
S(x, t, t) if there is a trajectory satisfying the dierential equation (8.1) so that
x(t) = x and x

= x(t +t). In this notation, we would give Heuns method as

x

=

S(x, t) = x+
1
2
(
1
+
2
), where
1
= f(x, t, t), and
2
= f(x+
1
, t, t).
The best known and most used Runge Kutta method, which often is called
the Runge Kutta method, has four stages and is fourth order accurate

1
= tf(x, t) (8.23)

2
= tf(x +
1
2

1
, t +
1
2
t) (8.24)

3
= tf(x +
1
2

2
, t +
1
2
t) (8.25)

4
= tf(x +
3
, t + t) (8.26)
x

= x +
1
6
(
1
+ 2
2
+ 2
3
+
4
) . (8.27)
5
Carl Runge was Professor of applied mathematics at the turn of the 20
th
century in
G ottingen, Germany. Among his colleagues were David Hilbert (of Hilbert space) and Richard
Courant. But Courant was forced to leave Germany and came to New York to found the
Courant Institute. Kutta was a student of Runge.
6
Heun, whose name rhymes with coin, was another student of Runge.
7
The notation x

here does not mean the derivative of x with respect to t (or any other
variable) as it does in some books on dierential equations.
160 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
Understanding the accuracy of Runge Kutta methods comes down to Taylor
series. The reasoning of Section 8.1 suggests that the method has error O(t
p
)
if

S(x, t, t) = S(x, t, t) + t r , (8.28)

where r = O(t
p
). The reader should verify that this denition of the resid-
ual, r, agrees with the denition in Section 8.1. The analysis consists of ex-
panding both S(x, t, t) and

S(x, t, t) in powers of t. If the terms agree up
to order t
p
but disagree at order t
p+1
, then p is the order of accuracy of the
overall method.
We do this for Heuns method, allowing f to depend on t as well as x. The
calculations resemble the derivation of the modied equation (8.17). To make
the expansion of S, we have x(t) = x, so
x(t + t) = x + t x(t) +
t
2
2
x(t) + O(t
3
) .
Dierentiating with respect to t and using the chain rule gives:
x =
d
dt
x =
d
dt
f(x(t), t) = f

(x(t), t) x(t) +
t
f(x(t), t) ,
so
x(t) = f

(x, t)f(x, t) +
t
f(x, t) .
This gives
S(x, t, t) = x +tf(x, t) +
t
2
2
(f

(x, t)f(x, t) +
t
f(x, t)) +O(t
3
) . (8.29)
To make the expansion of

S for Heuns method, we rst have
1
= tf(x, t),
which needs no expansion. Then

2
= tf(x +
1
, t + t)
= t
_
f(x, t) + f

(x, t)
1
+
t
f(x, t)t + O(t
2
)
_
= tf(x, t) + t
2
_
f

(x, t)f(x, t) +
t
f(x, t)
_
+ O(t
3
) .
Finally, (8.22) gives
x

= x +
1
2
(
1
+
2
)
= x +
1
2
_
tf(x, t) +
_
tf(x, t) + t
2
_
f

(x, t)f(x, t) +
t
f(x, t)
_
__
+ O(t
3
)
Comparing this to (8.29) shows that

S(x, t, t) = S(x, t, t) + O(t

3
) .
which is the second order accuracy of Heuns method. The same kind of analysis
shows that the four stage Runge Kutta method is fourth order accurate, but it
would take a full time week. It was Kuttas thesis.
8.3. LINEAR SYSTEMS AND STIFF EQUATIONS 161
8.3 Linear systems and sti equations
A good way to learn about the behavior of a numerical method is to ask what
it would do on a properly chosen model problem. In particular, we can ask how
an initial value problem solver would perform on a linear system of dierential
equations
x = Ax . (8.30)
We can do this using the eigenvalues and eigenvectors of A if the eigenvectors
are not too ill conditioned. If
8
Ar

and x(t) =

n
=1
u

(t)r

, then the
components u

satisfy the scalar dierential equations

u

. (8.31)
Suppose x
k
x(t
k
) is the approximate solution at time t
k
. Write x
k
=

n
=1
u
k
r

. For a majority of methods, including Runge Kutta methods and

linear multistep methods, the u
k
(as functions of k) are what you would get
by applying the same time step approximation to the scalar equation (8.31).
The eigenvector matrix, R, (see Section ??), that diagonalizes the dierential
equation (8.30) also diagonalizes the computational method. The reader should
check that this is true of the Runge Kutta methods of Section 8.2.
One question this answers, at least for linear equations (8.30), is how small
the time step should be. From (8.31) we see that the

have units of 1/time, so

the 1/ |

| have units of time and may be called time scales. Since t has units
of time, it does not make sense to say that t is small in an absolute sense, but
only relative to other time scales in the problem. This leads to the following:
Possibility: A time stepping approximation to (8.30) will be accurate only if
max

t |

| 1 . (8.32)
Although this possibility is not true in every case, it is a dominant technical
consideration in most practical computations involving dierential equations.
The possibility suggests that the time step should be considerably smaller than
the smallest time scale in the problem, which is to say that t should resolve
even the fastest time scales in the problem.
A problem is called sti if it has two characteristics: (i) there is a wide range
of time scales, and (ii) the fastest time scale modes have almost no energy. The
second condition states that if |

| is large (relative to other eigenvalues), then

|u

| is small. Most time stepping problems for partial dierential equations are
sti in this sense. For a sti problem, we would like to take larger time steps
than (8.32):
t |

| 1
_
for all with u

signi-
cantly dierent from zero.
(8.33)
What can go wrong if we ignore (8.32) and choose a time step using (8.33)
is numerical instability. If mode u

is one of the large |

| small |u

| modes,
8
We call the eigenvalue index to avoid conict with k, which we use to denote the time
step.
162 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
it is natural to assume that the real part satises Re(

) 0. In this case
we say the mode is stable because |u

(t)| = |u

(0)| e
t
does not increase as t
increases. However, if t

is not small, it can happen that the time step ap-

proximation to (8.31) is unstable. This can cause the u
k
to grow exponentially
while the actual u

decays or does not grow. Exercise 8 illustrates this. Time

step restrictions arising from stability are called CFL conditions because the
rst systematic discussion of this possibility in the numerical solution of partial
dierential equations was given in 1929 by Richard Courant, Kurt Friedrichs,
and Hans Levy.
8.4 Adaptive methods
Adaptive means that the computational steps are not xed in advance but are
determined as the computation proceeds. Section 3.6, discussed an integration
algorithm that chooses the number of integration points adaptively to achieve
a specied overall accuracy. More sophisticated adaptive strategies choose the
distribution of points to maximize the accuracy from a given amount of work.
For example, suppose we want an

I for I =
_
2
0
f(x)dx so that

I I

< .06. It
might be that we can calculate I
1
=
_
1
0
f(x)dx to within .03 using x = .1 (10
points), but that calculating I
2
=
_
2
1
f(x)dx to within .03 takes x = .02 (50
points). It would be better to use x = .1 for I
1
and x = .02 for I
2
(60 points
total) rather than using x = .02 for all of I (100 points).
Adaptive methods can use local error estimates to concentrate computational
resources where they are most needed. If we are solving a dierential equation
to compute x(t), we can use a smaller time step in regions where x has large
accelleration. There is an active community of researchers developing systematic
ways to choose the time steps in a way that is close to optimal without having
the overhead in choosing the time step become larger than the work in solving
the problem. In many cases they conclude, and simple model problems show,
that a good strategy is to equidistribute the local truncation error. That is, to
choose time steps t
k
so that the the local truncation error
k
= t
k
r
k
is nearly
constant.
If we have a variable time step t
k
, then the times t
k+1
= t
k
+ t
k
form
an irregular adapted mesh (or adapted grid). Informally, we want to choose a
mesh that resolves the solution, x(t) being calculated. This means that knowing
the x
k
x(t
k
) allows you make an accurate reconstruction of the function x(t),
say, by interpolation. If the points t
k
are too far apart then the solution is
underresolved. If the t
k
are so close that x(t
k
) is predicted accurately by a few
neighboring values (x(t
j
) for j = k 1, k 2, etc.) then x(t) is overresolved,
we have computed it accurately but paid too high a price. An ecient adaptive
mesh avoids both underresolution and overresolution.
Figure 8.1 illustrates an adapted mesh with equidistributed interpolation
error. The top graph shows a curve we want to resolve and a set of points that
concentrates where the curvature is high. Also also shown is the piecewise linear
8.4. ADAPTIVE METHODS 163
curve that connects the interpolation points. On the graph is looks as though
the piecewise linear graph is closer to the curve near the center than in the
smoother regions at the ends, but the error graph in the lower frame shows this
is not so. The reason probably is that what is plotted in the bottom frame is
the vertical distance between the two curves, while what we see in the picture is
the two dimensional distance, which is less if the curves are steep. The bottom
frame illustrates equidistribution of errer. The interpolation error is zero at the
grid points and gets to be as large as about 6.3 10
3
in each interpolation
interval. If the points were uniformly spaced, the interpolation error would be
smaller near the ends and larger in the center. If the points were bunched even
more than they are here, the interpolation error would be smaller in the center
than near the ends. We would not expect such perfect equidistribution in real
problems, but we might have errors of the same order of magnitude everywhere.
For a Runge Kutta method, the local truncation error is (x, t, t) =

S(x, t, t)
S(x, t, t). We want a way to estimate and to choose t so that || = e, where
e is the value of the equidistributed local truncation error. We suggest a method
related to Richardson extrapolation (see Section 3.3), comparing the result of
one time step of size t to two time steps of size t/2. The best adaptive Runge
Kutta dierential equation solvers do not use this generic method, but instead
use ingenious schemes such as the Runge Kutta Fehlberg ve stage scheme that
simultaneously gives a fth order

S
5
, but also gives an estimate of the dierence
between a fourth order and a fth order method,

S
5

S
4
. The method described
here does the same thing with eight function evaluations instead of ve.
The Taylor series analysis of Runge Kutta methods indicates that (x, t, t) =
t
p+1
(x, t) + O(t
p+2
). We will treat as a constant because the all the x
and t values we use are within O(t) of each other, so variations in do not
eect the principal error term we are estimating. With one time step, we get
x

=

S(x, t, t) With two half size time steps we get rst x
1
=

S(x, t, t/2),
then x
2
=

S( x
1
, t + t/2, t/2).
We will show, using the Richardson method of Section 3.3, that
x

x
2
=
_
1 2
p
_
(x, t, t) + O(t
p+1
) . (8.34)
We need to use the semigroup property of the solution operator: If we run
the exact solution from x for time t/2, then run it from there for another time
t/2, the result is the same as running it from x for time t. Letting x be the
solution of (8.1) with x(t) = x, the formula for this is
S(x, t, t) = S(x(t + t/2), t + t/2, t/2)
= S( S(x, t, t/2) , t + t/2, t/2) .
We also need to know that S(x, t, t) = x + O(t) is reected in the Jacobian
matrix S

(the matrix of rst partials of S with respect to the x arguments with

t and t xed)
9
: S

(x, t, t) = I + O(t).
9
This fact is a consequence of the fact that S is twice dierentiable as a function of all
its arguments, which can be found in more theoretical books on dierential equations. The
Jacobian of f(x) = x is f

(x) = I.
164 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
3 2 1 0 1 2 3
0
0.5
1
1.5
x
y
A function interpolated on 16 unevenly spaced points


the function y(x)
mesh values
linear interpolant of mesh values
locations of mesh points
3 2 1 0 1 2 3
7
6
5
4
3
2
1
0
1
x 10
3
x
y
Error in interpolation from 16 nonuniformly spaced points
Figure 8.1: A nonuniform mesh for a function that needs dierent resolution
in dierent places. The top graph shows the function and the mesh used to
interpolate it. The bottom graph is the dierence between the function and the
piecewise linear approximation. Note that the interpolation error equidistributed
though the mesh is much ner near x = 0.
8.4. ADAPTIVE METHODS 165
The actual calculation starts with
x
1
=

S(x, t, t/2)
= S(x, t, t/2) + 2
(p+1)
t
(p+1)
+ O(t
(p+2)
) ,
and
x
2
=

S( x
1
, t + t, t/2)
= S( x
1
, t + t/2, t/2) + 2
(p+1)
t
(p+1)
+ O(t
(p+2)
) ,
We simplify the notation by writing x
1
= x(t+t/2)+u with u = 2
(p+1)
t
p
+
O(t
(p+2)
). Then u = O(t
(p+1)
) and also (used below) t u = O(t
(p+2)
)
and (since p 1) u
2
= O(t
(2p+2)
) = O(t
(p+2)
). Then
S( x
1
, t + t/2, t/2) = S(x(t + t/2) + u, t + t/2, t/2)
= S(x(t + t/2), t + t/2, t/2) + S

u + O
_
u
2
_
= S(x(t + t/2), t + t/2, t/2) + u + O
_
u
2
_
= S(x, t, t) + 2
(p+1)
t
p
+ uO
_
t
p+2
_
.
Altogether, since 2 2
(p+1)
= 2
p
, this gives
x
2
= S(x, t, t) + 2
p
t
p+1
+ O
_
t
p+2
_
.
Finally, a single size t time step has
x

= X(x, t, t) + t
p+1
+ O
_
t
p+2
_
.
Combining these gives (8.34). It may seem like a mess but it has a simple
underpinning. The whole step produces an error of order t
p+1
. Each half
step produces an error smaller by a factor of 2
p+1
, which is the main idea of
Richardson extrapolation. Two half steps produce almost exactly twice the
error of one half step.
There is a simple adaptive strategy based on the local truncation error es-
timate (8.34). We arrive at the start of time step k with an estimated time
step size t
k
. Using that time step, we compute x

=

S(x
k
, t
k
, t
k
) and x
2
by
taking two time steps from x
k
with t
k
/2. We then estimate
k
using (8.34):

k
=
1
1 2
p
_
x

x
2
_
. (8.35)
This suggests that if we adjust t
k
by a factor of (taking a time step of size
t
k
instead of t
k
), the error would have been
p+1

k
. If we choose to
exactly equidistribute the error (according to our estimated , we would get
e =
p+1

k
=
k
= (e/
k
)
1/(p+1)
. (8.36)
166 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
We could use this estimate to adjust t
k
and calculate again, but this may lead
to an innite loop. Instead, we use t
k+1
=
k
t
k
.
Chapter 3 already mentioned the paradox of error estimation. Once we
have a quantitative error estimate, we should use it to make the solution more
accurate. This means taking
x
k+1
=

S(x
k
, t
k
, t
k
) +
k
,
which has order of accuracy p + 1, instead of the order p time step

S. This
increases the accuracy but leaves you without an error estimate. This gives
an order p + 1 calculation with a mesh chosen to be nearly optimal for an
order p calculation. Maybe the reader can nd a way around this paradox.
Some adaptive strategies reduce the overhead of error estimation by using the
Richardson based time step adjustment, say, every fth step.
One practical problem with this strategy is that we do not know the quanti-
tative relationship between local truncation error and global error
10
. Therefore
it is hard to know what e to give to achieve a given global error. One way to es-
timate global error would be to use a given e and get some time steps t
k
, then
redo the computation with each interval [t
k
, t
k+1
] cut in half, taking exactly
twice the number of time steps. If the method has order of accuracy p, then the
global error should decrease very nearly by a factor of 2
p
, which allows us to
estimate that error. This is rarely done in practice. Another issue is that there
can be isolated zeros of the leading order truncation error. This might happen,
for example, if the local truncation error were proportional to a scalar function
x. In (8.36), this could lead to an unrealistically large time step. One might
avoid that, say, by replacing
k
with min(
k
, 2), which would allow the time
step to grow quickly, but not too much in one step. This is less of a problem
for systems of equations.
8.5 Multistep methods
Linear multistep methods are the other class of methods in wide use. Rather than
giving a general discussion, we focus on the two most popular kinds, methods
based on dierence approximations, and methods based on integrating f(x(t)),
Adams methods. Hybrids of these are possible but often are unstable. For some
reason, almost nothing is known about methods that both are multistage and
multistep.
Multistep methods are characterized by using information from previous
time steps to go from x
k
to x
k+1
. We describe them for a xed t. A simple
example would be to use the second order centered dierence approximation
x(t) (x(t + t) x(t t)) /2t to get
(x
k+1
x
k1
) /2t = f(x
k
) ,
10
Adjoint based error control methods that address this problem are still in the research
stage (2006).
8.5. MULTISTEP METHODS 167
or
x
k+1
= x
k1
+ 2tf(x
k
) . (8.37)
This is the leapfrog
11
method. We nd that
x
k+1
= x
k1
+ 2tf( x
k
) + tO(t
2
) ,
so it is second order accurate. It achieves second order accuracy with a single
evaluation of f per time step. Runge Kutta methods need at least two evalua-
tions per time step to be second order. Leapfrog uses x
k1
and x
k
to compute
x
k+1
, while Runge Kutta methods forget x
k1
when computing x
k+1
from x
k
.
The next method of this class illustrates the subtletlies of multistep methods.
It is based on the four point one sided dierence approximation
x(t) =
1
t
_
1
3
x(t + t) +
1
2
x(t) x(t t) +
1
6
x(t 2t)
_
+ O
_
t
3
_
.
This suggests the time stepping method
f(x
k
) =
1
t
_
1
3
x
k+1
+
1
2
x
k
x
k1
+
1
6
x
k2
_
, (8.38)
which leads to
x
k+1
= 3tf(x
k
)
3
2
x
k
+ 3x
k1

1
2
x
k2
. (8.39)
This method never is used in practice because it is unstable in a way that Runge
Kutta methods cannot be. If we set f 0 (to solve the model problem x = 0),
(8.38) becomes the recurrence relation
x
k+1
+
3
2
x
k
3x
k1
+
1
2
x
k2
= 0 , (8.40)
which has characteristic polynomial
12
p(z) = z
3
+
3
2
z
2
3z +
1
2
. Since one
of the roots of this polynomial has |z| > 1, general solutions of (8.40) grow
exponentially on a t time scale, which generally prevents approximate solutions
from converging as t 0. This cannot happen for Runge Kutta methods
because setting f 0 always gives x
k+1
= x
k
, which is the exact answer in this
case.
Adams methods use old values of f but not old values of x. We can integrate
(8.1) to get
x(t
k+1
) = x(t
k
) +
_
t
k+1
t
k
f(x(t))dt . (8.41)
An accurate estimate of the integral on the right leads to an accurate time step.
Adams Bashforth methods estimate the integral using polynomial extrapolation
11
Leapfrog is a game in which two or more children move forward in a line by taking turns
jumping over each other, as (8.37) jumps from x
k1
to x
k+1
using only f(x
k
).
12
If p(z) = 0 then x
k
= z
k
is a solution of (8.40).
168 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
from earlier f values. At its very simplest we could use f(x(t)) f(x(t
k
)),
which gives
_
t
k+1
t
k
f(x(t))dt (t
k+1
t
k
)f(x(t
k
)) .
Using this approximation on the right side of (8.41) gives forward Euler.
The next order comes from linear rather than constant extrapolation:
f(x(t)) f(x(t
k
)) + (t t
k
)
f(x(t
k
)) f(x(t
k1
))
t
k
t
k1
.
With this, the integral is estimated as (the generalization to non constant t is
Exercise ??):
_
t
k+1
t
k
f(x(t))dt tf(x(t
k
)) +
t
2
2
f(x(t
k
)) f(x(t
k1
))
t
= t
_
3
2
f(x(t
k
))
1
2
f(x(t
k1
))
_
.
The second order Adams Bashforth method for constant t is
x
k+1
= x
k
+ t
_
3
2
f(x
k
)
1
2
f(x
k1
)
_
. (8.42)
To program higher order Adams Bashforth methods we need to evaluate the
integral of the interpolating polynomial. The techniques of polynomial interpo-
lation from Chapter ?? make this simpler.
Adams Bashforth methods are attractive for high accuracy computations
where stiness is not an issue. They cannot be unstable in the way (8.39) is
because setting f 0 results (in (8.42), for example) in x
k+1
= x
k
, as for Runge
Kutta methods. Adams Bashforth methods of any order or accuracy require one
evaluation of f per time step, as opposed to four per time step for the fourth
order Runge Kutta method.
8.6 Implicit methods
Implicit methods use f(x
k+1
) in the formula for x
k+1
. They are used for sti
problems because they can be stable with large t (see Section 8.3) in ways
explicit methods, all the ones discussed up to now, cannot. An implicit method
must solve a system of equations to compute x
k+1
.
The simplest implicit method is backward Euler:
x
k+1
= x
k
+ tf(x
k+1
) . (8.43)
This is only rst order accurate, but it is stable for any and t if Re() 0.
This makes it suitable for solving sti problems. It is called implicit because
x
k+1
is determined implicitly by (8.43), which we rewrite as
F(x
k+1
, t) = 0 , where F(y, t) = y tf(y) x
k
, (8.44)
8.6. IMPLICIT METHODS 169
To nd x
k+1
, we have to solve this system of equations for y.
Applied to the linear scalar problem (8.31) (dropping the index), the
method (8.43) becomes u
k+1
= u
k
+ tu
k+1
, or
u
k+1
=
1
1 t
u
k
.
This shows that |u
k+1
| < |u
k
| if t > 0 and is any complex number with
Re() 0. This is in partial agreement with the qualitative behavior of the
exact solution of (8.31), u(t) = e
t
u(0). The exact solution decreases in time if
Re() < 0 but not if Re() = 0. The backward Euler approximation decreases in
time even when Re() = 0. The backward Euler method articially stabilizes a
neutrally stable system, just as the forward Euler method articially destabilizes
it (see the modied equation discussion leading to (8.19)).
Most likely the equations (8.44) would be solved using an iterative method as
discussed in Chapter 6. This leads to inner iterations, with the outer iteration
being the time step. If we use the unsafeguarded local Newton method, and let
j index the inner iteration, we get F

= I tf

and
y
j+1
= y
j

_
I tf

(y
j
)
_
1_
y
j
tf(y
j
) x
k
_
, (8.45)
hoping that y
j
x
k+1
as j . We can take initial guess y
0
= x
k
, or, even
better, an extrapolation such as y
0
= x
k
+ t(x
k
x
k1
)/t = 2x
k
x
k1
.
With a good initial guess, just one Newton iteration should give x
k+1
accurately
enough.
Can we use the approximation J I for small t? If we could, the Newton
iteration would become the simpler functional iteration (check this)
y
j+1
= x
k
+ tf(y
j
) . (8.46)
The problem with this is that it does not work precisely for the sti systems
we use implicit methods for. For example, applied to u = u, the functional
iteration diverges (|y
j
| as j ) for t < 1.
Most of the explicit methods above have implicit anologues. Among implicit
Runge Kutta methods we mention the trapezoid rule
x
k+1
x
k
t
=
1
2
_
f(x
k+1
) + f(x
k
)
_
. (8.47)
There are backward dierentiation formula, or BDF methods based on higher
order one sided approximations of x(t
k+1
). The second order BDF method uses
(??):
x(t) =
1
t
_
3
2
x(t) 2x(t t) +
1
2
x(t 2t)
_
+ O
_
t
2
_
,
to get
f(x(t
k+1
)) = x(t
k+1
) =
_
3
2
x(t
k+1
) 2x(t
k
) +
1
2
x(t
k1
)
_
+ O
_
t
2
_
,
170 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
and, neglecting the O
_
t
2
_
error term,
x
k+1

2t
3
f(x
k+1
) =
4
3
x
k

1
3
x
k1
. (8.48)
The Adams Molton methods estimate
_
t
k+1
t
k
f(x(t))dt using polynomial in-
terpolation using the values f(x
k+1
), f(x
k
), and possibly f(x
k1
), etc. The
second order Adams Molton method uses f(x
k+1
) and f(x
k
). It is the same
as the trapezoid rule (8.47). The third order Adams Molton method also uses
f(x
k1
). Both the trapezoid rule (8.47) and the second order BDF method
(8.48) both have the property of being A-stable, which means being stable for
(8.31) with any and t as long as Re() 0. The higher order implicit
methods are more stable than their explicit counterparts but are not A stable,
which is a constant frustration to people looking for high order solutions to sti
systems.
8.7 Computing chaos, can it be done?
In many applications, the solutions to the dierential equation (8.1) are chaotic.
13
The informal denition is that for large t (not so large in real applications) x(t)
is an unpredictable function of x(0). In the terminology of Section 8.5, this
means that the solution operator, S(x
0
, 0, t), is an ill conditioned function of
x
0
.
The dogma of Section ?? is that a oating point computation cannot give
an accurate approximation to S if the condition number of S is larger than
1/
mach
10
16
. But practical calculations ranging from weather forcasting to
molecular dynamics violate this rule routinely. In the computations below, the
condition number of S(x, t) increases with t and crosses 10
16
by t = 3 (see
Figure 8.3). Still, a calculation up to time t = 60 (Figure 8.4, bottom), shows
the beautiful buttery shaped Lorentz attractor, which looks just as it should.
On the other hand, in this and other computations, it truly is impossible
to calculate details correctly. This is illustrated in Figure 8.2. The top picture
plots two trajectories, one computed with t = 4 10
4
(dashed line), and the
other with the time step reduced by a factor of 2 (solid line). The dierence
between the trajectories is an estimate of the accuracy of the computations.
The computation seems somewhat accurate (curves close) up to time t 5,
at which time the dashed line goes up to x 15 and the solid line goes down
to x 15. At least one of these is completely wrong. Beyond t 5, the
two approximate solutions have similar qualitative behavior but seem to be
independent of each other. The bottom picture shows the same experiment with
t a hundred times smaller than in the top picture. With a hundred times more
accuracy, the approximate solution loses accuracy at t 10 instead of t 5. If
a factor of 100 increase in accuracy only extends the validity of the solution by
5 time units, it should be hopeless to compute the solution out to t = 60.
13
James Glick has written a nice popular book on chaos. Neil Strogatz has a more technical
introduction that does not avoid the beautiful parts.
8.7. COMPUTING CHAOS, CAN IT BE DONE? 171
The present numerical experiments are on the Lorentz equations, which are
a system of three nonlinear ordinary dierential equations
x = (y x)
y = x( z) y
z = xy z
with
14
= 10, = 28, and =
8
3
. The C/C++ program outputs (x, y, z)
for plotting every t = .02 units of time, though there many time steps in each
plotting interval. The solution rst nds its way to the buttery shaped Lorentz
attractor then stays on it, travelling around the right and left wings in a seem-
ingly random (technically, chaotic) way. The initial data x = y = z = 0 are
not close to the attractor, so we ran the dierential equations for some time
before time t = 0 in order that (x(0), y(0), z(0)) should be a typical point on
the attractor. Figure 8.2 shows the chaotic sequence of wing choice. A trip
around the left wing corresponds to a dip of x(t) down to x 15 and a trip
around the right wing corresponds to x going up to x 15.
Section ?? explains that the condition number of the problem of calculating
S(x, t) (simplifying the notation from Section ?? to leave out zeros) depends on
the Jacobian matrix A(x, t) =
x
S(x, t). This represents the sensitivity of the
solution at time t to small perturbations of the initial data. We can calculate
A(x, t) using ideas of perturbation theory similar to those we used for linear
algebra problems in Chapter 4. Since S(x, t) is the value of a solution at time
t, it satises the dierential equation
d
dt
S(x, t) = f(S(x, t)) .
We dierentiate both sides with respect to x and interchange the order of dif-
ferentiation,

x
d
dt
S(x, t) =
d
dt

x
S(x, t) =
d
dt
A(x, t) ,
to get (with the chain rule)
d
dt
A(x, t) =

x
f(S(x, t))
= f

(S(x, t))
x
S

A = f

(S(x, t))A(x, t) . (8.49)

Thus, if we have an initial value x and calculate the trajectory S(x, t), then we
can calculate the rst variation, A(x, t), by solving the linear initial value prob-
lem (8.49) with initial condition A(x, 0) = I (why?). In the present experiment,
we solved the Lorentz equations and the perturbation equation using forward
Euler with the same time step.
14
These can be found, for example, in http://wikipedia.org by searching on Lorentz at-
tractor.
172 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
In typical chaotic problems, the rst variation grows exponentially in time.
If
1
(t)
2
(t)
n
(t) are the singular values of A(x, t), then there
typically are Lyapounov exponents,
k
, so that

k
(t) e

k
t
,
More precisely,
lim
t
ln(
k
(t))
t
=
k
.
If
1
>
n
, the l
2
condition number of A grows exponentially,

l
2 (A(x, t)) =

1
(t)

n
(t)
e
(1n)t
.
Figure 8.3 gives some indication that our Lorentz system has diering Lyapoounov
exponents. The top gure shows computations of the three singular values for
A(x, t). For 0 t < 2, it seems that
3
is decaying exponentially, making a
downward sloping straight line on this log plot. When
3
gets to about 10
15
,
the decay halts. This is because it is nearly impossible for a full matrix in double
precision oating point to have a condition number larger than 1/
mach
10
16
.
When
3
hits 10
15
, we have
1
10
2
, so this limit has been reached. These
trends are clearer in the top frame of Figure 8.4, which is the same calculation
carried to a larger time. Here
1
(t) seems to be growing exponentially with a
gap between
1
and
3
of about 1/
mach
. Theory says that
2
should be close to
one, and the computations are consistent with this until the condition number
bound makes
2
1 impossible.
To summarize, some results are quantitatively right, including the buttery
shape of the attractor and the exponential growth rate of
1
(t). Some results
are qualitatively right but quantitatively wrong, including the values of x(t)
for t > 10. Convergence analyses (comparing t results to t/2 results)
distinguishes right from wrong in these cases. Other features of the computed
solution are consistent over a range of t and consistently wrong. There is no
reason to think the condition number of A(x, t) grows exponentially until t 2
then levels o at about 10
16
. Much more sophisticated computational methods
using the semigroup property show this is not so.
8.8 Software: Scientic visualization
Visualization of data is indispensable in scientic computing and computational
science. Anomolies in data that seem to jump o the page in a plot are easy
to overlook in numerical data. It can be easier to interpret data by looking at
pictures than by examining columns of numbers. For example, here are entries
500 to 535 from the time series that made the top curve in the top frame of
Figure 8.4 (multiplied by 10
5
).
0.1028 0.1020 0.1000 0.0963 0.0914 0.0864 0.0820
8.8. SOFTWARE: SCIENTIFIC VISUALIZATION 173
0 2 4 6 8 10 12 14 16 18 20
20
15
10
5
0
5
10
15
20
t
x
Lorentz, forward Euler, x component, dt = 4.00e04 and dt = 2.00e04


larger time step
smaller time step
0 2 4 6 8 10 12 14 16 18 20
20
15
10
5
0
5
10
15
20
t
x
Lorentz, forward Euler, x component, dt = 4.00e06 and dt = 2.00e06


larger time step
smaller time step
Figure 8.2: Two convergence studies for the Lorentz system. The time steps in
the bottom gure are 100 times smaller than the time steps in the to gure. The
more accurate calculation loses accuracy at t 10, as opposed to t 5 with a
larger time step. The qualitative behavior is similar in all computations.
174 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
0 2 4 6 8 10 12 14 16 18 20
10
20
10
15
10
10
10
5
10
0
10
5
10
10
t
s
i
g
m
a
Lorentz, singular values, dt = 4.00e04


sigma 1
sigma 2
sigma 3
0 2 4 6 8 10 12 14 16 18 20
10
20
10
15
10
10
10
5
10
0
10
5
10
10
t
s
i
g
m
a
Lorentz, singular values, dt = 4.00e05


sigma 1
sigma 2
sigma 3
Figure 8.3: Computed singluar values of the sensitivity matrix A(x, t) =

x
S(x, t) with large time step (top) and ten times smaller time step (bottom).
Top and bottom curves are similar qualitatively though the ne details dier.
Theory predicts that middle singular value should be not grow or decay with t.
The times from Figure 8.2 at which the numerical solution loses accuracy are
not apparent here. In higher precision arithmetic,
3
(t) would have continued
to decay exponentially. It is unlikely that computed singular values of any full
matrix would dier by less than a factor of 1/
mach
10
16
.
8.8. SOFTWARE: SCIENTIFIC VISUALIZATION 175
0 10 20 30 40 50 60
10
20
10
15
10
10
10
5
10
0
10
5
10
10
10
15
10
20
10
25
t
s
i
g
m
a
Lorentz, singular values, dt = 4.00e05


sigma 1
sigma 2
sigma 3
20
15
10
5
0
5
10
15
20
40
20
0
20
40
0
5
10
15
20
25
30
35
40
45
Lorentz attractor up to time 60
x
y
z
Figure 8.4: Top: Singular values from Figure 8.3 computed for longer time. The

1
(t) grows exponentially, making a straight line on this log plot. The computed

2
(t) starts growing with the same exponential rate as
1
when roundo takes
over. A correct computation would show
3
(t) decreasing exponentially and

2
(t) neither growing nor decaying. Bottom: A beautiful picture of the buttery
shaped Lorentz attractor. It is just a three dimensional plot of the solution curve.
176 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
0.0790 0.0775 0.0776 0.0790 0.0818 0.0857 0.0910
0.0978 0.1062 0.1165 0.1291 0.1443 0.1625 0.1844
0.2104 0.2414 0.2780 0.3213 0.3720 0.4313 0.4998
0.5778 0.6649 0.7595 0.8580 0.9542 1.0395 1.1034
Looking at the numbers, we get the overall impression that they are growing in
an irregular way. The graph shows that the numbers have simple exponential
growth with ne scale irregularities superposed. It could take hours to get that
information directly from the numbers.
It can be a challenge to make visual sense of higher dimensional data. For
example, we could make graphs of x(t) (Figure 8.2), y(t) and z(t) as functions
of t, but the single three dimensional plot in the lower frame of Figure 8.4 makes
is clearer that the solution goes sometimes around the left wing and sometimes
around the right. The three dimensional plot (plot3 in Matlab) illuminates the
structure of the solution better than three one dimensional plots.
There are several ways to visualize functions of two variables. A contour plot
draws several contour lines, or level lines, of a function u(x, y). A contour line for
level u
k
is the set of points (x, y) with u(x, y) = u
k
. It is common to take about
ten uniformly spaced values u
k
, but most contour plot programs, including the
Matlab program contour, allow the user to specify the u
k
. Most contour lines
are smooth curves or collections of curves. For example, if u(x, y) = x
2
y
2
,
the contour line u = u
k
with u
k
= 0 is a hyperbola with two components. An
exception is u
k
= 0, the contour line is an .
A grid plot represents a two dimensional rectangular array of numbers by
colors. A color map assigns a color to each numerical value, that we call c(u). In
practice, usually we specify c(u) by giving RGB values, c(u) = (r(u), g(u), b(u)),
where r, g, and b are the intensities for red, green and blue respectively. These
may be integers in a range (e.g. 0 to 255) or, as in Matlab, oating point
numbers in the range from 0 to 1. Matlab uses the commands colormap and
image to establish the color map and display the array of colors. The image is
an array of boxes. Box (i, j) is lled with the color c(u(i, j)).
Surface plots visualize two dimensional surfaces in three dimensional space.
The surface may be the graph of u(x, y). The Matlab commands surf and surfc
create surface plots of graphs. These look nice but often are harder to interpret
than contour plots or grid plots. It also is possible to plot contour surfaces of
a function of three variables. This is the set of (x, y, z) so that u(x, y, z) = u
k
.
Unfortunately, it is hard to plot more than one contour surface at a time.
Movies help visualize time dependent data. A movie is a sequence of frames,
with each frame being one of the plots above. For example, we could visualize
the Lorentz attractor with a movie that has the three dimensional buttery
together with a dot representing the position at time t.
The default in Matlab, and most other quality visualization packages, is
to render the users data as explicitly as possible. For example, the Matlab
command plot(u) will create a piecewise linear curve that simply connects
successive data points with straight lines. The plot will show the granularity of
the data as well as the values. Similarly, the grid lines will be clearly visible in a
8.9. RESOURCES AND FURTHER READING 177
color grid plot. This is good most of the time. For example, the bottom frame
of Figure 8.4 clearly shows the granularity of the data in the wing tips. Since
the curve is sampled at uniform time increments, this shows that the trajectory
is moving faster at the wing tips than near the body where the wings meet.
Some plot packages oer the user the option of smoothing the data using
spline interpolation before plotting. This might make the picture less angu-
lar, but it can obscure features in the data and introduce artifacts, such as
overshoots, not present in the actual data.
8.9 Resources and further reading
There is a beautiful discussion of computational methods for ordinary dieren-
tial equations in Numerical Methods by A]ke Bjoork and Germund Dahlquist.
It was Dahlquist who created much of our modern understanding of the sub-
ject. A more recent book is A First Course in Numerical Analysis of Dierential
Equations by Arieh Iserles. The book Numerical Solution of Ordinary Dier-
ential Equations by Lawrence Shampine has a more practical orientation.
There is good public domain software for solving ordinary dierential equa-
tions. A particularly good package is LSODE (google it).
The book by Sans-Serna explains symplectic integrators and their appli-
cation to large scale problems such as the dynamics of large scape biological
molecules. It is an active research area to understand the quantitative relation-
ship between long time simulations of such systems and the long time behavior
of the systems themselves. Andrew Stuart has written some thoughtful oapers
on the subject.
The numerical solution of partial dierential equations is a vast subject with
many deep specialized methods for dierent kinds of problems. For computing
stresses in structures, the current method of choice seems to be nite element
methods. For uid ow and wave propagation (particularly nonlinear), the ma-
jority relies on nite dierence and nite volume methods. For nite dierences,
the old book by Richtmeyer and Morton still merit though there are more up
to date books by Randy LeVeque and by Bertil Gustavson, Heinz Kreiss, and
Joe Oliger.
8.10 Exercises
1. We compute the second error correction u
2
(t) n (8.13). For simplicity,
consider only the scalar equation (n = 1). Assuming the error expansion,
we have
f(x
k
) = f( x
k
+ tu
1
(t
k
) + t
2
u
2
(t
k
) + O(t
3
))
f( x
k
) + f

( x
k
)
_
tu
1
(t
k
) + t
2
u
2
(t
k
)
_
+
1
2
f

( x
k
)t
2
u
1
(t
k
)
2
+ O
_
t
3
_
.
178 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
Also
x(t
k
+ t) x(t
k
)
t
= x(t
k
) +
t
2
x(t
k
) +
t
2
6
x
(3)
(t
k
) + O
_
t
3
_
,
and
t
u
1
(t
k
+ t) u
1
(t
k
)
t
= t u
1
(t
k
) +
t
2
2
u
1
(t
k
) + O
_
t
3
_
.
Now plug in (8.13) on both sides of (8.5) and collect terms proportional
to t
2
to get
u
2
= f

(x(t))u
2
+
1
6
x
(3)
(t) +
1
2
f

(x(t))u
1
(t)
2
+??? .
2. This exercise conrms what was hinted at in Section 8.1, that (8.19) cor-
rectly predicts error growth even for t so large that the solution has lost
all accuracy. Suppose k = R/t
2
, so that t
k
= R/t. The error equation
(8.19) predicts that the forward Euler approximation x
k
has grown by a
factor of e
R
/2 although the exact solution has not grown at all. We can
conrm this by direct calculation. Write the forward Euler approxima-
tion to (8.18) in the form x
k+1
= Ax
k
, where A is a 2 2 matrix that
depends on t. Calculate the eigenvalues of A up to second order in t:

1
= 1 + it + at
2
+ O(t
3
), and
2
= 1 it + bt
2
+ O(t
3
). Find
the constants a and b. Calculate
1
= ln(
1
) = it + ct
2
+ O(t
3
)
so that
1
= exp(it + ct
2
+ O(t
3
)). Conclude that for k = R/t
2
,

k
1
= exp(k
1
) = e
iR/t
e
R/2+O(t)
, which shows that the solution has
grown by a factor of nearly e
R/2
as predicted by (8.19). This s**t is good
for something!
3. Another two stage second order Runge Kutta method sometimes is called
the modied Euler method. The rst stage uses forward Euler to predict
the x value at the middle of the time step:
1
=
t
2
f(x
k
, t
k
) (so that
x(t
k
+ t/2) x
k
+
1
). The second stage uses the midpoint rule with
that estimate of x(t
k
+t/2) to step to time t
k+1
: x
k+1
= x
k
+tf(t
k
+
t
2
, x
k
+
1
). Show that this method is second order accurate.
4. Show that applying the four stage Runge Kutta method to the linear
system (8.30) is equivalent to approximating the fundamental solution
S(t) = exp(tA) by its Taylor series in t up to terms of order t
4
(see
Exercise ??). Use this to verify that it is fourth order for linear problems.
5. Write a C/C++ program that solves the initial value problem for (8.1),
with f independent of t, using a constant time step, t. The arguments
to the initial value problem solver should be T (the nal time), t (the
time step), f(x) (specifying the dierential equations), n (the number of
components of x), and x
0
(the initial condition). The output should be
the apaproximation to x(T). The code must do something to preserve
8.10. EXERCISES 179
the overall order of accuracy of the method in case T is not an integer
multiple of t. The code should be able to switch between the three
methods, forward Euler, second order Adams Bashforth, forth order four
state Runge Kutta, with a minimum of code editing. Hand in output for
each of the parts below showing that the code meets the specications.
(a) The procedure should return an error ag or notify the calling routine
in some way if the number of time steps called for is negative or
impossibly large.
(b) For each of the three methods, verify that the coding is correct by
testing that it gets the right answer, x(.5) = 2, for the scalar equation
x = x
2
, x(0) = 1.
(c) For each of the three methods and the test problem of part b, do a
convergence study to verify that the method achieves the theoretical
order of accuracy. For this to work, it is necessary that T should be
an integer multiple of t.
(d) Apply your code to problem (8.18) with initial data x
0
= (1, 0)

.
Repeat the convergence study with T = 10.
6. Verify that the recurrence relation (8.39) is unstable.
(a) Let z be a complex number. Show that the sequence x
k
= z
k
satises
(8.39) if and only if z satises 0 = p(z) = z
3
+
3
2
z
2
3z +
1
2
.
(b) Show that x
k
= 1 for all k is a solution of the recurrence relation.
Conclude that z = 1 satises p(1) = 0. Verify that this is true.
(c) Using polynomial division (or another method) to factor out the
known root z = 1 from p(z). That is, nd a quadratic polynomial,
q(z), so that p(z) = (z 1)q(z).
(d) Use the quadratic formula and a calculator to nd the roots of q as
z =
5
4

_
41
16
2.85, .351.
(e) Show that the general formula x
k
= az
k
1
+ bz
k
2
+ cz
k
3
is a solution to
(8.39) if z
1
, z
2
, and z
3
are three roots z
1
= 1, z
2
2.85, z
3
.351,
and, conversely, the general solution has this form. Hint: we can nd
a, b, c by solving a vanderMonde system (Section 7.4) using x
0
, x
1
,
and x
2
.
(f) Assume that |x
0
| 1, |x
1
| 1, and |x
2
| 1, and that b is on the
order of double precision oating point roundo (
mach
) relative to
a and c. Show that for k > 80, x
k
is within
mach
of bz
k
2
. Conclude
that for k > 80, the numerical solution has nothing in common with
the actual solution x(t).
7. Applying the implicit trapezoid rule (8.47) to the scalar model problem
(8.31) results in u
k+1
= m(t)u
k
. Find the formula for m and show that
|m| 1 if Re() 0, so that |u
k+1
| |u
k
|. What does this say about the
applicibility of the trapezoid rule to sti problems?
180 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS
8. Exercise violating time step constraint.
9. Write an adaptive code in C/C++ for the initial value problem (8.1) (8.2)
using the method described in Section 8.4 and the four stage fourth order
Runge Kutta method. The procedure that does the solving should have
arguments describing the problem, as in Exercise 5, and also the local
truncation error level, e. Apply the method to compute the trajectory
of a comet. In nondimensionalized variables, the equations of motion are
given by the inverse square law:
d
2
dt
2
_
r
1
r
2
_
=
1
(r
2
1
+ r
2
2
)
3/2
_
r
1
r
2
_
.
We always will suppose that the comet starts at t = 0 with r
1
= 10,
r
2
= 0, r
1
= 0, and r
2
= v
0
. If v
0
is not too large, the point r(t) traces
out an ellipse in the plane
15
. The shape of the ellipse depends on v
0
. The
period, P(v
0
), is the rst time for which r(P) = r(0) and r(P) = r(0).
The solution r(t) is periodic because it has a period.
(a) Verify the correctness of this code by comparing the results to those
from the xed time step code from Exercise 5 with T = 30 and
v
0
= .2.
(b) Use this program, with a small modication to compute P(v
0
) in
the range .01 v
0
.5. You will need a criterion for telling when
the comet has completed one orbit since it will not happen that
r(P) = r(0) exactly. Make sure your code tests for and reports
failure
16
.
(c) Choose a value of v
0
for which the orbit is rather but not extremely
elliptical (width about ten times height). Choose a value of e for
which the solution is rather but not extremely accurate the error is
small but shows up easily on a plot. If you set up your environment
correctly, it should be quick and easy to nd suitable paramaters
by trial and error using Matlab graphics. Make a plot of a single
period with two curves on one graph. One should be a solid line
representing a highly accurate solution (so accurate that the error is
smaller than the line width plotting accuracy), and the other being
the modestly accurate solution, plotted with a little o for each time
step. Comment on the distribution of the time step points.
(d) For the same parameters as part b, make a single plot of that contains
three curves, an accurate computation of r
1
(t) as a function of t (solid
line), a modestly accurate computation of r
1
as a function of t (o
for each time step), and t as a function of t. You will need to use a
15
Isaac Newton formulated these equations and found the explicit solution. Many aspects
of planetary motion elliptical orbits, sun at one focus, |r|

= const had beed discovered
observationally by Johannes Keppler. Newtons inverse square law theory t Kepplers data.
16
This is not a drill.
8.10. EXERCISES 181
dierent scale for t if for no other reason than that it has dierent
units. Matlab can put one scale in the left and a dierent scale on
the right. It may be necessary to plot t on a log scale if it varies
over too wide a range.
(e) Determine the number of adaptive time stages it takes to compute
P(.01) to .1% accuracy (error one part in a thousand) and how many
xed t time step stages it takes to do the same. The counting will
be easier if you do it within the function f.
10. The vibrations of a two dimensional crystal lattice may be modelled in a
crude way using the dierential equations
17
r
jk
= r
j1,k
+ r
j+1,k
+ r
j,k1
+ r
j,k+1
4r
jk
. (8.50)
Here r
jk
(t) represents the displacement (in the vertical direction) of an
atom at the (j, k) location of a square crystal lattice of atoms. Each atom
is bonded to its four neighbors and is pulled toward them with a linear
force. A lattice of size L has 1 j L and 1 k L. Apply reecting
boundary conditions along the four boundaries. For example, the equation
for r
1,k
should use r
0,k
= r
1,k
. This is easy to implement using a ghost
cell strategy. Create ghost cells along the boundary and copy appropriate
values from the actual cells to the ghost cells before each evaluation of
f. This allows you to use the formula (8.50) at every point in the lattice.
Start with initial data r
jk
(0) = 0 and r
jk
(0) = 0 for all j, k except r
1,1
= 1.
Compute for L = 100 and T = 300. Use the fourth order Runge Kutta
method with a xed time step t = .01. Write the solution to a le every
.5 time units then use Matlab to make a movie of the results, with a 2D
color plot of the solution in each frame. The movie should be a circular
wave moving out from the bottom left corner and bouncing o the top and
right boundaries. There should be some beautiful wave patterns inside the
circle that will be hard to see far beyond time t = 100. Hand in a few
of your favorite stills from the movie. If you have a web site, post your
movie for others to enjoy.
17
This is one of Einsteins contributions to science.
182 CHAPTER 8. DYNAMICS AND DIFFERENTIAL EQUATIONS