1 Introduction

1.1 Definitions

Section 1. Introduction and basic concepts

Section 2. Parabolic PDE’s (e.g. heat equation).

Section 3. Hyperbolic PDE’s (e.g. wave equation).

Section 4. Elliptic PDE’s (e.g. Poisson equation).

For all types of equations we consider the finite difference method. In addition, finite element
methods are considered in Section 4.
Definition A Partial Differential Equation (PDE) is one or more equations connecting partial
derivatives of one or more unknown functions (or dependent variables). In addition the equa-
tion(s) can contain known functions. All functions are functions of two or more independent
variables.
Examples

(1) u(x, t) : ut = 3uxx − 2u cos(x) + 5 exp(−3t) sin(2x)

(2) u(x, y) : uxx + (1 + ux )uyy = 0

In (1), u ≡ u(x, t) is the dependent variable, x and t are the independent variables, and we use
the standard notation that
∂2u
uxx = 2 , etc.
∂x
Exercise: show that one exact solution of (1) is u(x, t) = 5 exp(−3t) sin(x).
Definition The Order/Degree of a PDE is the highest order partial derivative occurring in the
equation. In the examples above, both PDEs are 2nd order.
Definition A PDE is linear if the dependent variable only occurs in linear combinations mul-
tiplied by known functions of the independent variables. Otherwise it is nonlinear. In the
examples above, (1) is linear and (2) is nonlinear.
The most general linear, first order PDE has the form

aux + but = c + d u,

here u = u(x, t) is the dependent variable, x and t are the independent variables, and a, b, c,
and d are constants or depend on x and t only. We will meet this type of equation in Section
3 since it is always of hyperbolic type. When we get to 2nd order equations we have more
possibilities:
 Page 2

Classification It is useful to classify general 2nd order linear PDEs of 2 independent variables
into elliptic, parabolic, or hyperbolic PDEs. If the PDE can be written as
auxx + buxy + cuyy + dux + euy + f u + g = 0
where a, b, etc. are functions only of x and y, then we have the following
• if b2 − 4ac < 0 then the equation is elliptic
• if b2 − 4ac = 0 then the equation is parabolic
• if b2 − 4ac > 0 then the equation is hyperbolic
(These definitions can be generalized to higher number of dimensions (independent variables)
and other orders).
Examples
• elliptic:
Poisson’s equation: ∇2 u = g(x, y)
Laplace’s equation: ∇2 u = 0.
where u ≡ u(x, y), ∇2 u = uxx + uyy .
Generally this type of equation is associated with equilibrium (steady-state) problems,
e.g. steady flow of an incompressible fluid.
• parabolic:
Heat or diffusion equation: ut = κuxx
where κ > 0. Here for example u(x, t) is the temperature in a thin insulated rod, or the
concentration of a certain chemical in a thin tube.

1
Black-Scholes equation: vt + rsvs + σ 2 s2 vss = rv
2
where v(s, t) is the value of a share option, s is the share price, r is the interest rate, and
σ is the share “volatility”.
• hyperbolic:
Wave equation: utt = uxx
Here u(x, t) could be for example the water height in a narrow channel or the air pressure
in a thin tube.
Exercise: Check that the examples given above of elliptic, parabolic, hyperbolic equations
satisfy the definitions of such types.
Note that equations with variable coefficients can change type in different regions of the domain
of the independent variables, for example
xuxx + utt = 0
Exercise: Use the definitions above to determine what type this equation is, in which regions
of the (x, t) plane.
 Page 3

1.2 Numerical methods for PDEs

We postpone until later a discussion of the extra information we need to solve the PDE, such
as initial conditions and boundary conditions.
Although we can sometimes write down some exact solutions for some PDE’s, in general this is
not possible. In this case we need to have a way of finding approximate numerical solutions for
a given equation. In this module we will generally stick to simple examples of PDEs which do
have exact solutions, which has the great advantage that we can check whether our numerical
methods and our computer programs are correct before we progress to more difficult cases.
We can group the main numerical techniques for PDEs as follows

1. Finite Difference (FD) Methods. We will use these throughout most of this course. In
this type of method we forget about trying to find the values of the dependent variable,
say u, for all values of the independent variables, say x and t. Instead we try to find
approximate solutions of the problem at a discrete set of points in the (x, t) plane, normally
a rectangular grid of points.

2. Finite Element (FE) Methods. We will touch on these briefly when looking at elliptic
problems. In this method we first divide up the (x, t) plane into small finite elements,
usually triangles or quadrilaterals, and then approximate solutions of the problem on each
of these elements by simple linear or polynomial functions of the independent variables.

3. Spectral Methods. We do not treat these methods in this course due to lack of time,
but they form an important class of techniques. Here we approximate the solutions of
the problems by a truncated expansion in the eigenfunctions of some linear operator, for
example a truncated Fourier Series.

The advantages and disadvantages of these three approaches can be briefly summarised as
follows.

1 FD Methods. Simple to construct and to analyse. Need some modification in regions

where the solution is changing rapidly. More difficult to apply when the solution region
is not rectangular.

2 FE Methods. More difficult to set up than the FD method, but much better in irregular
shaped domains. Not so easy to analyse at a simple level, but some rigorous results can
be found with the aid of variational calculus. Widely used in engineering.

3 Spectral Methods. These give the highest accuracy when applied to problems in rect-
angular domains with smooth solutions. Not so useful in irregular domains or when the
solutions has discontinuities such as shocks.

1.3 Introduction to the Finite Difference method

We will need to approximate partial derivatives such as

∂u(x, t) ∂u(x, t) ∂ 2 u(x, t)

ux ≡ , ut ≡ , uxx ≡ .
∂x ∂t ∂x2
 Page 4

The continuous variables x and t are discretized, so that we only consider u(x, t) evaluated
at the intersections (node points) of the grid lines parallel to the x and t axes. We use the

∆x
t4

t3

t2

t1
∆t
t
0
x x x x x
0 1 2 3 4

Figure 1: Finite difference mesh or grid

notation xj = x0 + j∆x, tn = t0 + n∆t, where (x0 , t0 ) is the origin for the grid (often (0, 0)),
and ∆x, ∆t are the (constant) grid spacings in the x and t directions respectively. We will
write unj = u(xj , tn ) for the exact solution of the PDE at the grid point (xj , tn ), and wjn for the
approximate solution generated by the FD method at the same point.
The next step is to introduce some Finite Difference operators. Consider a function f (x) of a
single variable, then by definition
 
df f (x + ∆x) − f (x)
= lim .
dx ∆x→0 ∆x

So for small ∆x,

df f (x + ∆x) − f (x)
≈ .
dx ∆x
We now define the Forward Difference operator Fx such that Fx f (x) = f (x+∆x)−f (x). Hence

df Fx f (x)
≈ .
dx ∆x
The same reasoning goes through if f depends on the variable t also, so

∂u(x, t) Fx u(x, t) u(x + ∆x, t) − u(x, t) ∂u(x, t) Ft u(x, t)

ux = ≈ = , ut = ≈ ,
∂x ∆x ∆x ∂t ∆t

where we define Ft in an analogous way, Ft f (x, t) = f (x, t + ∆t) − f (x, t).

Exercise: Take u(x, t) = exp(−t) sin(x). Find the exact value of ∂u(x, t)/∂x and ∂u(x, t)/∂t
at the point (x, t) = (0.5, 0.1) and calculate the approximate values given by the above using
∆x, ∆t = 0.1, 0.01, . . . . Define the error as the difference between the exact and approximate
results and consider what happened to the errors as ∆x, ∆t → 0.
 Page 5

We can similarly define the corresponding Backwards difference operators Bx u(x, t) = u(x, t) −
u(x − ∆x, t), Bt u(x, t) = u(x, t) − u(x, t − ∆t), so that

∂u(x, t) Bx u(x, t) u(x, t) − u(x − ∆x, t)

≈ = ,
∂x ∆x ∆x
∂u(x, t) Bt u(x, t) u(x, t) − u(x, t − ∆t)
≈ = .
∂t ∆t ∆t

(This is equivalent to replacing ∆x by −∆x in the definitions of Fx , etc.)

If we average the two approximations we have so far for ux and ut we introduce the Central
difference operators Dx u(x, t) = u(x+∆x, t)−u(x−∆x, t), Dt u(x, t) = u(x, t+∆t)−u(x, t−∆t)

∂u(x, t) Dx u(x, t) u(x + ∆x, t) − u(x − ∆x, t)

≈ = ,
∂x 2∆x 2∆x
∂u(x, t) Dt u(x, t) u(x, t + ∆t) − u(x, t − ∆t)
≈ = .
∂t 2∆t 2∆t

Going back to functions of one variable, these approximations have the simple geometric inter-
pretation as various approximations to the tangent to the curve y = f (x), see Fig. 2.

y=f(x)

C
F

x−∆ x x x+∆ x

Figure 2: Approximations to o- - -o, the tangent to y = f (x) at x, ·—· with D/F/B is Cen-
tral/Forward/Backward difference approx.

Exercise: which of these three approximations do you expect to be the best approximation
to the tangent? Check your guess by reworking the exercise above for u(x, t) = exp(−t) sin(x)
with Backward and Central Differences.
There is one further finite difference operator we need, a second central difference operator δ 2
to approximate second derivatives. (When there is danger of confusion we refer to D as the
first central difference operator). Derivation of this one requires some FD manipulations. First
 Page 6

define another FD operator

1 1
δf (x) = f (x + ∆x) − f (x − ∆x)
2 2
So δf (x)/∆x is like a central difference approximation to df (x)/dx but using a half step size.
   
df (x) δf (x) d2 f (x) d df (x) δ δf (x) δ 2 f (x)
so ≈ , and = ≈ = .
dx ∆x dx2 dx dx ∆x ∆x ∆x2
Now
 
2 1 1
δ f (x) = δ (δf (x)) = δ f (x + ∆x) − f (x − ∆x)
2 2
= (f (x + ∆x) − f (x)) − (f (x) − f (x − ∆x)) = f (x + ∆x) − 2f (x) + f (x − ∆x)
Combining these results we have
d2 f (x) f (x + ∆x) − 2f (x) + f (x − ∆x)
2
≈ .
dx ∆x2
Where the numerator can be written as δx2 f (x) = f (x + ∆x) − 2f (x) + f (x − ∆x). So for partial
derivatives we have
∂ 2 f (x, t) δx2 f (x, t) f (x + ∆x, t) − 2f (x, t) + f (x − ∆x, t)
2
≈ 2
= ,
∂x ∆x ∆x2
∂ 2 f (x, t) δt2 f (x, t) f (x, t + ∆t) − 2f (x, t) + f (x, t − ∆t)
2
≈ 2
= .
∂t ∆t ∆t2

Exercise: With u(x, t) = exp(−t) sin(x) again, find the exact value of ∂ 2 u(x, t)/∂x2 and
∂ 2 u(x, t)/∂t2 at the point (x, t) = (0.5, 0.1) and calculate the approximate values given by the
above 2nd central difference approximations using ∆x, ∆t = 0.1, 0.01, . . . .
These four Finite Difference operators will be the key to all our work for the next few weeks.
We will see a different justification for their use in the next section. Before leaving the topic
we mention some other related results
We can establish various relations between the Finite Difference operators with a bit of algebra,
for example Fx Bx = Bx Fx = δx2 . For example
Fx Bx f (x) = Fx (Bx f (x)) = Fx (f (x) − f (x − ∆x)) =
= (f (x + ∆x) − f (x)) − (f (x) − f (x − ∆x) = δx2 f (x)
In general the FD operators commute with each other.
If we need to approximate a mixed derivative, for example ∂ 2 u(x, t)/∂x∂t, we have seen that
Fx ∂ Ft ∂
≈ , ≈ ,
∆x ∂x ∆t ∂t
so some possibilities are
∂ 2 u(x, t) Fx Ft Bx Bt Bx Ft Dx Dt
≈ u(x, t), u(x, t), u(x, t), u(x, t), etc.
∂x∂t ∆x ∆t ∆x ∆t ∆x ∆t 2∆x 2∆t
Exercise: repeat the previous exercise for uxt .
 Page 7

1.4 Taylor series and difference operators

We can use Taylor series expansions in one of the variables to see how well Finite Difference
(FD) approximations work. Consider the approximation (u(x + ∆x, t) − u(x, t))/∆x to ∂u/∂x.
Fx u(xj , tn ) = u(xj + ∆x, tn ) − u(xj , tn )
 
1 2 1 3

= u + ∆xux + ∆x uxx + ∆x uxxx + . . .  − u(xj , tn )
2 3! (xj ,tn )
 
1 2 
= ∆xux + ∆x uxx + . . . 
2 (xj ,tn )

≈ ∆x ux (xj , tn ) when ∆x is sufficiently small

So
Fx
u(xj , tn ) = ux (xj , tn ) + O(∆x)
∆x
where O(∆x) means a quantity which → K∆x for some constant K, so → 0 as ∆x → 0.
Similarly
Dx u = u(xj + ∆x, tn ) − u(xj − ∆x, tn )
 
1 2 1 3 1 4 5

= u + ∆x ux + ∆x uxx + ∆x uxxx + ∆x uxxxx + O(∆x ) 
2 3! 4! (x ,t )
 j n
1 1 1 
− u − ∆x ux + ∆x2 uxx − ∆x3 uxxx + ∆x4 uxxxx + O(∆x5 ) 
2 3! 4! (xj ,tn )
 2

Dx u ∆x 
⇒ = ux + uxxx + O(∆x4 ) 
2∆x 6 (xj ,tn )

So
Dx
u(xj , tn ) = ux (xj , tn ) + O(∆x2 ).
2∆x
This is more accurate because ∆x2 → 0 faster than ∆x as ∆x → 0.
In a similar way we can show that
δx2 u(xj , tn ) = ∆x2 uxx (xj , tn ) + O(∆x4 )
so
δx2
u(xj , tn ) = uxx (xj , tn ) + O(∆x2 ) when ∆x is small.
∆x2
By expanding first in one variable and then in the other(s), we can analyse approximations
involving terms like u(xj + ∆x, tn + ∆t).
Exercise: Show that
Fx Ft
u(xj , tn ) = uxt (xj , tn ) + O(∆x, ∆t).
∆x∆t
and
Dx Dt
u(xj , tn ) = uxt (xj , tn ) + O(∆x2 , ∆t2 ).
4∆x∆t
so that again Central differences are better.
We now have enough background to start looking at PDEs, starting with parabolic problems.
 Page 8

2 Parabolic PDE’s
2.1 Introduction
The most practical approach is to study a concrete example. Hence we start by looking at the
heat or diffusion equation
∂U ∂2U
=K , (2.1)
∂T ∂X 2
where U = U(X, T ) is measured in some units (say ◦ C) and the physical time and space variables
T > 0, X ∈ (0, L) are also measured in appropriate units. K is a constant which depends on
the material, called the Thermal Conductivity. We specify Dirichlet boundary conditions (BCs)
at X = 0, L, i.e. U(0, T ) and U(L, T ) given for all T > 0, and initial conditions (ICs) U(X, 0)
given for X ∈ (0, L).

T
U(0,T) given

U(L,T) given

Need to find the solution in

UT=UXX the open box T > 0, X ∈
(0, L)

0 U(X) given L X

The heat equation models many things, for example

• the temperature U in a long thin insulated bar with fixed temperatures at each end

U(0,T) U(L,T)
given given
0 X L

• The diffusion of a chemical in a solid or stationary fluid (in this case U is the concentration
of the chemical).

Since the dimensions and units used in each application can be different, it is useful, before we
do anything else, to transform to a “standard problem” in dimensionless form. First define a
new variable x = X/L so that the equation becomes

∂U K ∂2U
= 2 2 , for x ∈ (0, 1).
∂T L ∂x
 Page 9

Now define a new variable t = KT /L2 so that the equation becomes

∂U ∂2U
= .
∂t ∂x2
Finally define u(x, t) = U(x, t)/U0 , where U0 is some typical constant in the problem (for
example the maximum temperature of the bar), to get finally

∂u ∂2u
= 2, (2.2)
∂t ∂x
where now u ∈ (0, 1). From now on we will usually work with the equation in the form
(2.2) with x ∈ (0, 1). We can always return to the physical version (2.1) by reversing the
transformations above. In these dimensionless variable we will define the initial conditions and
boundary conditions as

u(x, 0) = F (x), u(0, t) = α(t), u(1, t) = β(t),

for some given functions F (x), α(t), β(t).

2.2 A simple Finite Difference method for the heat equation

As described in Section 1.3, we divide up the (x, t) plane using a rectangular grid of mesh points
separated by ∆x, ∆t respectively.

∆x

t4

t3

t2

t1
∆t
t0
x0 x1 x2 x3 x4 xJ−1 xJ

We take ∆x = 1/J so that x0 = 0, xJ = 1, xj = x0 +j∆x. We wish to approximate the equation

(2.2) at the mesh points (nodes) so that

u(xj , tn ) = unj ≈ wjn

where wjn is our numerical approximation to the exact solution u(xj , tn ) = unj at (xj , tn ).
We consider in this section the simplest approximation to the PDE, using forward differences
for ut and central differences for uxx .

Ft n δx2 n
ut (xj , tn ) ≈ u , uxx (xj , tn ) ≈ u
∆t j ∆x2 j
 Page 10

So we have from (2.2) that

Ft n δ2
uj ≈ x 2 unj ,
∆t ∆x
at all interior points (xj , tn ), 0 < j < J, n > 0. We define wjn to satisfy this equation with
equality.
Ft n δ2
wj = x 2 wjn
∆t ∆x
so
wjn+1 − wjn n
wj−1 − 2wjn + wj+1
n
=
∆t ∆x2
Multiplying through by ∆t, defining r = ∆t/∆x2 , and re-arranging, we get

FTCS scheme: wjn+1 = rwj−1

n
+ (1 − 2r)wjn + rwj+1
n
(2.3)

Here FTCS stands for Forward Time, Central Space difference approximation. We also need
the following IC and BCs:

0 w0n = α(tn )
IC: wj = F (xj ), j = 0, . . . , J, BC: n>0
wJn = β(tn )

With these conditions we can use (2.3) to find the other wjn for 0 < j < J, n > 0 as described
below. The key to the process is to understand that the FTCS scheme (2.3) gives an equation
connecting one value of w at the time level n+1 with three values at the time level n. Graphically

we can represent this as a “computational molecule” which looks like this: .

n
If we know all the values of wj for 0 ≤ j ≤ J at a time level n, we can use (2.3) to calculate
wjn+1 for j = 1, 2, . . . , J − 1 in turn. The scheme is explicit - we don’t have to solve any coupled
algebraic equations to get from tn to tn+1 = tn + ∆t. The method will become clear once we
have worked through some examples, but we summarise the main steps here

1. Choose J and r then calculate ∆x = 1/J, ∆t = r∆x2 .

2. Calculate wj0 and set n = 0.

3. Use (2.3) to get wjn+1, j = 1 . . . J − 1, and get w0n+1 , wJn+1 from the boundary conditions.

4. Set n → n + 1 and repeat from Step 3 until tn is “big enough”.

Example

Use the FTCS scheme (2.3) to find the approximate solution to (2.2) with BCs u(0, t) =
u(1, t) = 0, IC u(x, 0) = sin(πx), when r = 0.4 and J = 2, 4 respectively. Calculate at least two
time steps. Using the fact that an exact solution of this problem is u(x, t) = exp(−tπ 2 ) sin(πx),
calculate the maximum error at each time step.
 Page 11

J=2

t n=3
3

t n=2
2

t n=1
1

t n=0
0
x x x
0 1 2
j=0 j=1 j=2

We have ∆x = 1/2 = 0.5, ∆t = r∆x2 = 0.1, and we need to find the solution at the points •
in the figure above. Along the bottom row t = t0 = 0 we have xj = 0, 0.5, 1, and from the ICs

w00 = sin(πx0 ) = 0, w10 = sin(πx1 ) = 1, w20 = sin(πx2 ) = 0,

Now take n = 0 in (2.3) with j = 1. We have

w11 = rw00 + (1 − 2r)w10 + rw20 = 0 + 0.2 × 1 + 0 = 0.2

The two boundary conditions give

w01 = 0, w21 = 0.

Now take n = 1 in (2.3) with j = 1. We have

w12 = rw01 + (1 − 2r)w11 + rw21 = 0 + 0.2 × 0.2 + 0 = 0.04

The two boundary conditions give

w02 = 0, w22 = 0.

So for t = 0.1, 0.2 we have the numerical values w11 = 0.2, w12 = 0.04. The corresponding exact
values are exp(−π 2 tn ) sin(πx1 ) = exp(−0.1π 2 ), exp(−0.2π 2 ) = 0.37, 0.14 to 2D. So the accuracy
is not too great with this large value of ∆x.
Exercise: carry out more time steps in this calculation. Can you guess at a general solution
for w1n ?
 Page 12

J=4

t n=3
3

t n=2
2

t n=1
1

t n=0
0
x x x x x
0 1 2 3 4
j=0 j=1 j=2 j=3 j=4

We have ∆x = 1/4 = 0.25, ∆t = r∆x2 = 0.025, and we need to find the solution at the points
• in the figure above. Along the bottom row t = t0 = 0 we have xj = 0, 0.25, 0.5, 0.75, 1, and
from the ICs
√
w00 = sin(πx0 ) = 0, w10 = sin(πx1 ) = 1/ 2, w20 = sin(πx2 ) = 1,
√
w30 = sin(πx3 ) = 1/ 2, w40 = sin(πx4 ) = 0.

Now take n = 0 in (2.3) with j = 1. We have

1
w11 = rw00 + (1 − 2r)w10 + rw20 = 0 + 0.2 × √ + 0.4 × 1 = 0.5414
2
1 1
w21 = rw10 + (1 − 2r)w20 + rw30 = 0.4 × √ + 0.2 × 1 + 0.4 × √ = 0.7657
2 2
1
w31 = rw20 + (1 − 2r)w30 + rw40 = 0.4 × 1 + 0.2 × √ + 0 = 0.5414
2
The two boundary conditions give

w01 = 0, w41 = 0.

Now take n = 1 in (2.3) with j = 1. We have

w12 = rw01 + (1 − 2r)w11 + rw21 = 0 + 0.2 × 0.5414 + 0.4 × 0.7657 = 0.4146

w22 = rw11 + (1 − 2r)w21 + rw31 = 0.4 × 0.5414 + 0.2 × 0.7657 + 0.4 × 0.5414 = 0.5863
w32 = rw21 + (1 − 2r)w31 + rw41 = 0.4 × 0.7657 + 0.2 × 0.5414 + 0 = 0.4146

The two boundary conditions give

w02 = 0, w42 = 0.

So for t = 0.025, 0.05 we have following numerical values for wj1 , wj2 to 4D

wj1 = [0, 0.5414, 0.7657, 0.5414, 0]

wj2 = [0, 0.4146, 0.5863, 0.4146, 0]
 Page 13

The corresponding exact values are u1j , u2j = exp(−π 2 tn ) sin(πxj ) = exp(−0.025π 2 ) sin(πxj ),
exp(−0.05π 2 ) sin(πxj ), given below to 4D
u1j = [0, 0.5525, 0.7813, 0.5525, 0]
u2j = [0, 0.4317, 0.6105, 0.4317, 0]
So the accuracy is better with this smaller value of ∆x (but note we are now working at much
smaller values of tn also).
We plot below the exact solution in the J = 4 case and the corresponding numerical approxi-
mation. It can be seen that qualitatively at least the agreement is good.

t t0
0

t t1
1
u(x,t)

wnj

t t2
2

x x x x x x0 x1 x2 x3 x4
0 1 2 3 4

j=0 j=1 j=2 j=3 j=4 j=0 j=1 j=2 j=3 j=4

Exercise: carry out more time steps in this calculation, and repeat the above calculations for
r = 1.0.
Another example we can look at, with J = 4, is the “triangular” initial conditions

2x, x ≤ 0.5
F (x) =
2(1 − x), x > 0.5
(This choice of an IC with a discontinuous derivative, rather than a sin(πx), results in problems
becoming apparent at smaller times. Similar things happen for sin(πx) but take longer to
manifest themselves). With r = 0.4 (left) and r = 0.6 (right) we get the two graphs shown
below. Note that when r = 0.4 the numerical solution decreases smoothly towards zero, in line
with what we would expect in the physical model. But when r = 0.6 the solution develops
spatial oscillations which are increasing in amplitude. If run for further times (try it!), these
oscillations become unbounded, i.e. wjn → ∞ as n → ∞, in contrast to the exact solution
which can be shown to satisfy → 0 as t → ∞. We would regard this as bad. Even at small
times we find the temperature is becoming negative at some points, physically contradicting
the laws of thermodynamics!
Technically if this sort of bad behaviour occurs we say the scheme unstable. We study one way
of analysing this behaviour over the next few lectures.
Exercise: repeat the above calculations for r = 1.0 and w0 = [0, 0.5, 1, 0.5, 0]. What goes
wrong? Why? We will answer this question in a few lectures’ time. First we consider something
called the Local Truncation Error (LTE) of the scheme (2.3).
 Page 14

t t
0 0

t
1
n

wn
wj

j
t1
t2
t3
t
2 t3
t t
4 4

t5

t
5

x0 x1 x2 x3 x4 x0 x1 x2 x3 x4

j=0 j=1 j=2 j=3 j=4 j=0 j=1 j=2 j=3 j=4

2.3 Local Truncation Error (LTE) analysis

Given an approximation for the PDE (2.2), how do we know it is any good? The first step
in analysing any scheme is to examine the Local Truncation Error (LTE). To do this we first
write the PDE in the operator form

∂ ∂2
Lu = 0, where L = − 2
∂t ∂x
We then replace the partial derivatives by the FD approximations we are using, to get a FD
approximation to L, which we will call L∆ . For the FTCS scheme we have

Ft δ2
L∆ = − x2
∆t ∆x
The numerical solution wjn will satisfy the equation

wjn+1 − wjn wj−1

n
− 2wjn + wj+1
n
L∆ wjn =0= − (2.4)
∆t ∆x2
Note that we do not multiply this equation by ∆t at this stage, in contrast to our derivation
of (2.3).
Definition: Local Truncation Error
Let L be a differential operator and let L∆ be the corresponding difference operator approxi-
mating L. The LTE of the approximation is given by the leading terms of the Taylor expansion
of L∆ u, where u satisfies Lu = 0. In other words, the LTE is found by substituting the exact
solution of the PDE into the finite difference equation, then Taylor expanding and cancelling
as many terms as possible.

Warning: do not multiply or divide L∆ by any factors

such as ∆t or ∆x when working out the LTE
 Page 15

Example: Calculate the LTE of the FTCS scheme (2.3).

In (2.4), we replace wjn by u(xj , tn ), etc., and expand all the terms in a Taylor series around
the point (xj , tn ).

LTE = L∆ u(xj , tn )
u(xj , tn + ∆t) − u(xj , tn ) u(xj − ∆x, tn ) − 2u(xj , tn ) + u(xj + ∆x, tn )
= −
 ∆t  ∆x2
1 1 
= u + ∆tut + ∆t2 utt + · · · − u  −
∆t 2 xj ,tk

1 1 1 1
− 2
u − ∆xux + ∆x2 uxx − ∆x3 uxxx + ∆x4 uxxxx − 2u+
∆x 2 3! 4!

1 2 1 3 1 4

+u + ∆xux + ∆x uxx + ∆x uxxx + ∆x uxxxx 
2 3! 4! xj ,tk
1 1
= ut + ∆tutt − uxx − ∆x2 uxxxx + . . .
2 12
where all the u and derivatives are evaluated at (xj , tn ). Since u satisfies the heat equation,
ut − uxx = 0, we are left with

∆t ∆x2
LTE = utt − uxxxx + O(∆t2 , ∆x4 )
2 12
By using the fact that

∂ ∂ ∂2 ∂2
utt = ut = uxx = 2 ut = 2 uxx = uxxxx
∂t ∂t ∂x ∂x
we can also write the LTE as
   
∆t ∆x2 2 4 ∆x2 1
LTE = − uxxxx + O(∆t , ∆x ) = r− uxxxx + O(∆t2 , ∆x4 )
2 12 2 6

where r = ∆t/∆x2 . From this form we see that the O(∆x2 ) terms vanish if r = 1/6, and the
LTE becomes O(∆t2 , ∆x4 ).
The first term on the right of the LTE for the FTCS scheme is referred to as the leading term
of the local truncation error (LTE): for this scheme it is first order accurate in time and 2nd
order in space.
Definition: consistency/order If the LTE of a scheme → 0 as ∆x, ∆t → 0, the scheme is
said to be consistent. This is the minimum requirement for any numerical scheme. Furthermore,
if the LTE is of order O(∆xp , ∆tq ), the scheme is said to be of order p in space and q in time. If
r is fixed so we can write the LTE as O(∆xp ) as in the above expression for the FTCS scheme,
we say more generally that the scheme is pth order. So the FTCS scheme for the heat equation
is 2nd order if r 6= 61 , and 4th order if r = 61 .
Exercise: Check that all the 3rd order terms in the LTE for the FTCS scheme vanish.
 Page 16

2.4 Matrix version of FTCS scheme

We have seen that if ∆x, ∆t are small, then the LTE for the FTCS scheme → 0 as ∆x, ∆t → 0.
This is saying that in some sense the error is small after one time step, starting with the exact
solution. However this is not enough to guarantee that the scheme gives good results after
a large number of time steps. The examples we looked at in previous lectures suggest that
problems occur in the FTCS scheme if r > 0.5. If we run a program which shows oscillations
becoming unbounded in magnitude, i.e. wjn → ∞ as n → ∞, in contrast to the exact solution
which can be shown to satisfy → 0 as t → ∞, we regard this as bad. Even at small times we
find the temperature is becoming negative at some points, physically contradicting the laws of
thermodynamics!
Technically if this sort of bad behaviour occurs we say the scheme unstable. We study one way
of analysing this behaviour in this section. To do this we write the FTCS scheme in matrix
form.
Set  
w1n
 wn 
2
wn =  ..  ,
 
 . 
n
wJ−1
the vector of values of the numerical solution at the internal spatial grid points at time level
tn . We know from the initial conditions that
 
F (x1 )
 F (x2 ) 
0 0  
w =u = .. ,
 . 
F (xJ−1 )

so w0 is known, also w0n = α(tn ), wJn = β(tn ). The scheme (2.3) is

wjn+1 = rwj−1
n n
+ (1 − 2r)wjn + rwj+1 , j = 1, . . . , J − 1.

This can be condensed to

wn+1 = Swn + bn , (2.5)
where
 
1 − 2r r 0 ... 0  
 .. ..  rα(tn )
 r 1 − 2r r 0 . .   0 
 ..   .. 
 0 r 1 − 2r r 0 .  
.

bn = 
   
S= .. .. .. .. .. .. , .. .
 . . . . . .   . 
   
 .. ..   0 
 . . 0 r 1 − 2r r 
.. .. rβ(tn )
0 . . 0 r 1 − 2r

Note that the matrix version of the FTCS scheme (2.5) includes both the PDE and the BCs.
However we have chosen to calculate and store in w only the internal points, in contrast to the
 Page 17

matlab program presented earlier. Hence j just goes from 1 to J − 1 instead of from 0 to J
(maths, includes boundary points) or from 1 to J+1 (Matlab, includes boundary points). This
form is useful for analysis but not so useful for a computer program since we would have to
add the boundary values before plotting the results.
Warning: Using the scheme in the form (2.5) for actual calculation will only be efficient in
Matlab if you declare the matrix S to be sparse. A sparse matrix is one which has most elements
zero. This way Matlab will not spend any time multiplying together terms which will give zero.
From (2.5) we have

wn+1 = Swn + bn
= S(Swn−1 + bn−1 ) + bn
= S 2 wn−1 + Sbn−1 + bn .

we repeat back to w0 or use induction to get

wn+1 = S n+1 w0 + S n b0 + S n−1 b1 + · · · + Sbn−1 + bn , (2.6)

where
Sn = S
| × S × S{z× · · · × S} .
n times

We want to find out what happens to the solution for large n. We first simplify (2.6) by setting
α(t) = β(t) = 0 (as in our numerical examples earlier) – this makes the calculations below
easier, but a similar result can be found in the more general case. Then we have bn = 0 for all
n so (2.6) becomes
wn+1 = S n+1 w0 .
Note that the superscript on the matrix S means S raised to the n + 1th power, but superscript
on the w refers to the time level. What does the solution wn+1 look like? We need to know
something about the eigenvalues of S to answer this.

2.4.1 Some facts about Eigenvalues

(i) If λ is an eigenvalue of S with |λ| > 1 and e is the corresponding eigenvector (Se = λe),
then
||S n e|| = ||λn e|| → ∞,
1
where || · || is the Euclidean norm ||x|| = (|x1 |2 + |x2 |2 + · · · + |xn |2 ) 2 .

(ii) If all the eigenvalues of S satisfy λ ≤ 1 in modulus, then ||S n z|| → 0, (λ < 1) or remain
bounded (some λ = 1) as n → ∞ for all vectors z. Note: for this we require S to be
symmetric.
In summary, the solution to (2.5) (and hence to (2.3)) is only well-behaved when all the
eigenvalues of S are ≤ 1 in modulus.
 Page 18

(iii) Because S is tri-diagonal with constant coefficients, it is possible to derive a nice formula
for its eigenvalues
 sπ 
λs = 1 − 4r sin2 , s = 1, . . . , J − 1,
 2J 
2 sπ
= 1 − 4r sin ∆x since J∆x = 1.
2

For stability we require

 sπ 
−1 ≤ 1 − 4r sin2 ≤1 as ∆x → 0.
2J
The right-hand inequality is always true, so we just need to satisfy the left-hand one. We have
 sπ 
1 − 4r sin2 ≥ −1
2J 
2 (J − 1)π
1 − 4r sin ≥ −1 (worst case)
2J
 
2 π
i.e. 1 − 4r sin (1 − ∆x) as ∆x → 0
π 2 
so 2r sin2 (1 − ∆x) ≤ 1
2
1
finally r ≤ 2 π


2 sin 2 (1 − ∆x)

Hence we need
1
r≤ ,
2
since ∆x can be arbitrary small. So we expect r ≤ 12 to give a “good” solution and r > 1
2
to
give a poor solution – which is what we find in practice.
Remarks

(i) This way of analysing the stability of a scheme is not easily generalized since it involves
finding the eigenvalues of the corresponding S-matrix.

(ii) The condition |λs | ≤ 1 only guarantees stability because S is symmetric (true in general
for parabolic equations but not for hyperbolics).

We now look at a different way of determining whether an approximation scheme is stable or

not - the Fourier method or von Neumann method.

2.5 Fourier or von Neumann method

We saw in the simple J = 2 case that we could find an exact solution of the numerical scheme
in the form
w1n = (1 − 2r)n .
Exercise: for what values of r does the above solution become unstable?
 Page 19

We would like to find a more general solution of the numerical scheme in the J > 2 case. We
turn back to the PDE for guidance. We have already seen that the function

u(x, t) = a exp(−π 2 t) sin(πx)

satisfies the heat equation with zero boundary conditions (α = β = 0). More generally, a
solution satisfying the same PDE and BCs is

u(x, t) = ak exp(−π 2 k 2 t) sin(kπx), k = 1, 2, . . . , J

for any choice of constant ak .

Exercise: Check this!
In fact the most general solution we can write down for the heat equation with zero boundary
conditions is a linear sum of such terms
J
X
u(x, t) = ak exp(−π 2 k 2 t) sin(kπx),
k=1

which you will recognise as the Fourier series solution for the heat equation.
It is easier to proceed if we write sin(kπx) = (exp(ikπx) − exp(−ikπx)) /2i, where i2 = −1,
and write the Fourier series in complex form
J
X
u(x, t) = ck exp(−π 2 k 2 t) exp(iπkx).
k=−J

Consider now a single term in this series at (xj , tn ) = (j∆x, n∆t).

exp(−π 2 k 2 tn ) exp(iπkxj ) = exp(−π 2 k 2 n∆t) exp(iπkj∆x).

We can write this as

exp(−π 2 k 2 ∆t)n exp(iπkj∆x) = ξ n exp(iωj)
where ξ = exp(−π 2 k 2 ∆t) and ω = πk∆x, where k can take any value between −J and J.
The first part of this looks like the solution of the numerical problem with J = 2 we discussed
earlier. The essential step in the von Neumann method is to assume a typical component of
the numerical solution has the form

wjn = ξ n exp(iωj) (2.7)

for some constants ξ and ω. The term ξ is called the amplification factor.
Lets try this in the FTCS scheme. If we insert this ansatz into (2.3) we get

ξ n+1 exp iωj = rξ n exp(iω(j − 1)) + (1 − 2r)ξ n exp(iωj) + rξ n exp(iω(j + 1))

cancelling through by ξ n exp(iωj) we get

ξ = r exp(−iω) + (1 − 2r) + r exp(iω)

= 1 − 2r + 2r cos(ω) = 1 − 2r(1 − cos(ω))
 Page 20

Now 1 − cos(ω) = 1 − (1 − 2 sin2 1
2
ω ) = 2 sin2 1
ω
, so we have finally
2
 
2 1
ξ = 1 − 4r sin ω (2.8)
2

If ξ satisfies (2.8) then wjn = ck ξ n exp(iωj) is a solution to the FTCS scheme (2.3). What does
this solution look like?
(i) If |ξ| > 1 then |wjn | = |ck ξ n | → ∞ as n → ∞, so the solution blows up and the scheme is
said to be unstable.
(ii) If |ξ| ≤ 1 then the solution doesn’t blow up and the scheme is said to be von Neumann
stable.
Let’s see what this means for the FTCS scheme. The requirement that |ξ| ≤ 1 for stability
becomes  
2 1
−1 ≤ 1 − 4r sin ω ≤ 1.
2
This is very similar to what we got from the matrix analysis - the r.h. inequality is always true
and the left-hand one leads to
1 1
r≤ 2 1

=
2 sin 2 ω 2
the last step following from the fact that the worst case for the inequality is ω = π. This is the
same result that we got from the matrix analysis, but the von Neumann method is much easier
to apply.
Note that the result r ≤ 12 corresponds to ∆t ≤ 12 ∆x2 , which fits in with the assumption
∆t = O(∆x2 ) which we made when looking at the LTE.

2.5.1 Summary - von Neumann stability analysis

• Substitute a solution of the form

wjn = ξ n exp(iωj)
into the difference scheme, simplify by cancelling common factors, and solve for ξ in terms
of ω, r, etc.
• Determine if the amplification factor ξ has modulus ≤ 1 for all values of |ω| ≤ π. If this
is so for all values of r we have unconditional stability.
• If |ξ| ≤ 1 for some range of r, we say the scheme is von Neumann stable for r in the stated
range, otherwise the scheme is unstable.

Notes:
• von Neumann stability is always necessary but may not be a sufficient condition for
stability (for example for difference schemes involving 3 or more time levels). It can’t
deal easily with nonzero boundary conditions. In practice the method often gives useful
results even when its application is not fully justified.
 Page 21

• If the exact solution of the PDE increases (exponentially) with time, the von Neumann
stability condition that we need is

|ξ| ≤ 1 + K∆t

for some positive K in the limit of small ∆t.

Now that we have established that the FTCS scheme is not useful for ∆t > 12 ∆x2 , we look at
other schemes which have better stability properties. We can group a whole class of schemes,
including the FTCS scheme, into the name “θ-method”.

2.6 The θ-method for ut = uxx

Recall that we derived the FTCS scheme for the heat equation by using a Forwards difference
approximation in time and a Central difference approximation in space applied at the point
(xj , tn ). If instead we work at the forward point (xj , tn+1 ) and use a Backwards difference
approximation in time and again a Central difference approximation in space, we get the BTCS
scheme:
wjn+1 − wjn n+1
wj−1 − 2wjn+1 + wj+1n+1
=
∆t ∆x2
If we multiply through by ∆t, define r = ∆t/∆x2 as before, and collect all n + 1 terms on the
left, we get
n+1
−rwj−1 + (1 + 2r)wjn+1 − rwj+1
n+1
= wjn
Note that in contrast to the FTCS scheme, we now have three unknowns in this equation, the
three values of w at the higher time level. This equation holds for j = 1, 2, . . . , J − 1, so in
total we get J − 1 equations for the J − 1 unknowns wjn+1, j = 1, 2, . . . , J − 1. In matrix form
this becomes
Swn+1 = wn + bn+1
where
     
1 + 2r −r 0 ... w1n rα(tn+1 )

 −r 1 + 2r −r 0 ... 


 w2n 


 0 

 0 −r 1 + 2r −r 0  n  ..  n+1  .. 
S= , w = . , b = . 
 .. .. .. .. ..   ..   
. . . . . 0
   .   
.. n rβ(tn+1 )
. 0 −r 1 + 2r wJ−1

We say this scheme is implicit because we need to solve a set of simultaneous equations for each
time level.
Example: Solve ut = uxx using the BTCS scheme with B.C. u(0, t) = u(1, t) = 0, t > 0, I.C.
u(x) = sin(πx), and r = 0.4, J = 4.
Solution: The I.C.s tell us that
h √ √ i
w0 = 0, 1/ 2, 1, 1/ 2, 0 .
 Page 22

At n = 1 the B.C.s tell us that w01 = w41 = 0. Setting n = 0 in the BTCS scheme we get
(j = 1, 2, 3)
(1 + 0.8)w11 − 0.4w21 =w10 + 0.4w01
− 0.4w11 + (1 + 0.8)w21 − 0.4w31 =w20
− 0.4w21 + (1 + 0.8)w31 =w30 + 0.4w41
or
1
1.8w11 − 0.4w21 = √ + 0.4 × 0
2
− 0.4w11 + 1.8w21 − 0.4w31 =1
1
− 0.4w21 + 1.8w31 = √ + 0.4 × 0
2
or   1   1 
1.8 −0.4 0 w1 √
2
 −0.4 1.8 −0.4   w21  =  1 
0 −0.4 1.8 w31 √1
2
Solving this by Gauss elimination gives
w1 = [0, 0.5729, 0.8102, 0.5729, 0]
to 4 significant figures. We now repeat this process to get w2 , w3 , etc. The exact result is
u1 = exp(−π 2 ∆t) sin(πxj ) = [0, 0.5525, 0.7813, 0.5525, 0]

A natural generalization of the FTCS and BTCS schemes is to take a weighted average of the
two
wjn+1 − wjn n
wj−1 − 2wjn + wj+1
n n+1
wj−1 − 2wjn+1 + wj+1
n+1
= (1 − θ) + θ .
∆t ∆x2 ∆x2
This is the so-called θ method for ut = uxx , with θ ∈ [0, 1]. Note that

(i) if θ = 0 we get the FTCS scheme.

(ii) if θ = 1 we get the BTCS scheme.
(iii) if θ > 0 the scheme is implicit

(iv) If θ ∈ (0, 1), the computational molecule is .

Another special value of θ is θ = 1/2, which gives the so-called Crank-Nicolson method:
wjn+1 − wjn n
1 wj−1 − 2wjn + wj+1
n n+1 n+1 n+1
1 wj−1 − 2wj + wj+1
= + .
∆t 2 ∆x2 2 ∆x2
Multiplying through by ∆t and re-arranging in the usual way we get
r n+1 r n+1 r n r n
− wj−1 + (1 + r)wjn+1 − wj+1 = wj−1 + (1 − r)wjn + wj+1 .
2 2 2 2
 Page 23

2.6.1 LTE analysis of the θ-method

The PDE ut = uxx is approximated on a uniform grid of size ∆x = 1/N in space and ∆t in
time, with approximate solution

wjn ≈ u(xj , tn ) , xj = x0 + j∆x , tn = n∆t,

where u(x, t) is an exact solution of the PDE. The θ-method scheme can be written as

wjn+1 − wjn δ2 δ2
= (1 − θ) x 2 wjn + θ x 2 wjn+1
∆t ∆x ∆x
where δx2 wjn = wj−1
n
− 2wjn + wj+1
n
. The scheme can be written as

Ft n δ2 δ2
L∆ wjn ≡ wj − (1 − θ) x 2 wjn − θ x 2 wjn+1 = 0. (2.9)
∆t ∆x ∆x

The LTE is found in the usual way by plugging in an exact solution u(xj , tn ) in place of the
approximate solution wjn (for all j, n) into L∆ wjn , Taylor expanding about (x, t) = (xj , tn ) and
eliminating terms using the PDE.
Remember, do not multiply or divide (2.9) by ∆t or ∆x when working out the LTE.

Recall that when applied to a smooth enough function u(x, t) we can write

Ft u(xj , tn ) = u(xj , tn +∆t) − u(xj , tn )

 
∂ ∆t2 ∂ 2 ∆t3 ∂ 3 4
= ∆t + + + O(∆t ) u(xj , tn )
∂t 2! ∂t2 3! ∂t3
 
∆t2 ∆t3
= ∆t ut + utt + uttt + O(∆t4 ).
2! 3! (xj ,tn )

At line 2 above we have expanded in Taylor series, using the operator form of the series, and
cancelled terms. Expanding the 2nd central space difference term δx2 u(xj , tn ) gives

δx2 u(xj , tn ) = u(xj +∆x, tn ) − 2u(xj , tn ) + u(xj −∆x, tn )

 2

2 ∂ 2∆x4 ∂ 4 6
= ∆x + + O(∆x ) u(xj , tn )
∂x2 4! ∂x4
 
2 ∆x4
= ∆x uxx + uxxxx + O(∆x6 ).
12 (xj ,tn )

(We have again missed out the details of the Taylor series expansion). The similar term
δx2 u(xj , tn+1 ) (time level n + 1) becomes

δx2 u(xj , tn +∆t) = u(xj +∆x, tn +∆t) − 2u(xj , tn +∆t) + u(xj −∆x, tn +∆t)
 
2 ∆x4 6
= ∆x uxx + uxxxx + O(∆x ) .
12 (xj ,tn+∆t)
 Page 24

Note that this term is evaluated at time t = tn + ∆t and so it must also be expanded in ∆t.
That is
 
2 2 ∆x4 6
δx u(xj , tn +∆t) = ∆x uxx + uxxxx + O(∆x )
12 (xj ,tn+∆t)
 2 2
 
∂ ∆t ∂ 2 ∆x4 6
= 1 + ∆t + + ... ∆x uxx + uxxxx + O(∆x )
∂t 2! ∂t2 12 (xj ,tn )
 4

∆x
= ∆x2 uxx + ∆t∆x2 uxxt + uxxxx + O(∆t∆x4 , ∆t2 ∆x2 , ∆x6 )
12 (xj ,tn )

We now substitute each of the three terms into (2.9) and collect terms together to get
 
1 ∆x2
LTE = (ut − uxx ) + ∆t utt − θutxx − uxxxx + O(∆t2 , ∆t∆x2 , ∆x4 ).
2 12

Since u is a solution of the PDE, this eliminates ut − uxx . Also, differentiating the PDE once
with respect to t gives
utt = utxx = uxxxx
and hence    
1 ∆x2
LTE = ∆t −θ − uxxxx + O(∆t2 , ∆t∆x2 , ∆x4 ).
2 12
If we do nothing special with the choice of θ, ∆t, ∆x then the LTE is O(∆t, ∆x2 ). This is often
described as first order accurate in time and second order in space. If we make the restriction
that ∆t = O(∆x2 ), then the LTE is O(∆x2 ) and the scheme is simply second order accurate.
When θ = 1/2 the O(∆t) term is eliminated and the scheme is second order accurate in time
and space.
Higher accuracy can be obtained by choosing parameters such that
 
1 ∆x2
∆t −θ − = 0.
2 12

This can be achieved by choosing ∆t, ∆x and θ to satisfy

1 1 ∆t
θ= − where r = .
2 12r ∆x2
In fact when we satisfy this condition and choose ∆t = O(∆x2 ), the LTE is O(∆x4 ) and so the
scheme is fourth order accurate. When θ = 0 we have the FTCS scheme and the choice r = 1/6
used before satisfies the above requirements.
 Page 25

2.6.2 Stability analysis of the θ method

We now carry out a von Neumann analysis of the θ method

n+1 n+1 n+1 n n n
−θrwm−1 + (1 + 2θr)wm − θrwm+1 = (1 − θ)rwm−1 + (1 − 2(1 − θ)r)wm + (1 − θ)rwm+1 .
n
Substitute wm = ξ n eimω and simplify in the usual way
− θrei(m−1)ω ξ n+1 + (1 + 2θr)eimω ξ n+1 − θrei(m+1)ω ξ n+1 =
(1 − θ)rei(m−1)ω ξ n + (1 − 2(1 − θ)r)eimω ξ n + (1 − θ)rei(m+1)ω ξ n
⇒ −θre−iω ξ + (1 + 2θr)ξ − θreiω ξ = (1 − θ)re−iω + (1 − 2(1 − θ)r) + (1 − θ)reiω
(taking out factors eimω and ξ n )
⇒ ξ − ξθr(eiω − 2 + e−iω ) = 1 + (1 − θ)r(eiω − 2 + e−iω )
Now
eiω − 2 + e−iω = −2(1 − cos(ω)) = −4 sin2 (ω/2)
so the above becomes
ξ + 4ξθ sin2 (ω/2)r = 1 − 4(1 − θ)r sin2 (ω/2)
1 − 4(1 − θ)r sin2 (ω/2)
⇒ξ=
1 + 4θr sin2 (ω/2)
We need |ξ| ≤ 1 for stability for all ω ∈ [−π, π]. Since ξ is clearly real in this case this means
we require −1 ≤ ξ ≤ 1. Now
1 + 4θr sin2 (ω/2) − 4r sin2 (ω/2) 4r sin2 (ω/2)
ξ= = 1 −
1 + 4θr sin2 (ω/2) 1 + 4θr sin2 (ω/2)
so ξ is clearly less than +1. Now consider the inequality ξ ≥ −1. This is (on multiplying
through by the denominator)
−1 − 4θr sin2 (ω/2) ≤ 1 − 4(1 − θ)r sin2 (ω/2)
⇒ 2(1 − 2θ)r sin2 (ω/2) ≤ 1
(i) If θ ≥ 1/2, this last inequality will clearly hold for all r.
(ii) If θ < 1/2, we need in the worst case (ω = π) that
1
r≤ .
2(1 − 2θ)
for stability.
We can check that when θ = 0, we recover the familiar r ≤ 1/2 result for the FTCS scheme.
In summary, the θ-scheme is stable
(i) For all r, if θ ≥ 1/2.
(ii) For
1
r≤ , if θ < 1/2.
2(1 − 2θ)
 Page 26

2.6.3 Matrix form of the θ method

We can write the θ method in much the same form as the BTCS scheme,

Swn+1 = Mwn + (1 − θ)bn + θbn+1

where
     
1 + 2θr −θr 0 ... w1n rα(tn )

 −θr 1 + 2θr −θr 0 ... 


 w2n 


 0 

 0 −θr 1 + 2θr −θr 0  n  ..  n  .. 
S= , w = . , b =  . ,
 .. .. .. .. ..   ..   
 . . . . .   .   0 
.. n rβ(tn )
. 0 −θr 1 + 2θr wj−1
 
1 − 2(1 − θ)r (1 − θ)r 0 ...

 (1 − θ)r 1 − 2(1 − θ)r (1 − θ)r 0 ... 


M = 0 (1 − θ)r 1 − 2(1 − θ)r (1 − θ)r 0 
.
 .. .. .. .. .. 
 . . . . . 
..
. 0 (1 − θ)r 1 − 2(1 − θ)r

with similar definitions for wn+1 and bn+1 . So supposing we know wn , then wn+1 is computed
by

(i) Set q = Mwn + (1 − θ)bn + θbn+1 .

(ii) Solve Sv = q for v.

(iii) Set wn+1 = v

The matrix S is tridiagonal (if θ > 0), so solving (ii) is fairly quick and easy. This is still more
work than solving the explicit FTCS scheme (θ = 0) but not much more.

2.6.4 Summary
Scheme Order type stability
θ=0 O(∆t, ∆x2 ) explicit r ≤ 1/2
0 < θ < 1/2 O(∆t, ∆x2 ) implicit r ≤ 1/2(1 − 2θ)
θ = 1/2 O(∆t2 , ∆x2 ) implicit stable for all r
1/2 < θ ≤ 1 O(∆t, ∆x2 ) implicit stable for all r

(Note the accuracy of some schemes can be increased by choosing a special value for r).
Although the FTCS scheme is easy to apply (because it is explicit), the time step is constrained
by stability requirements to be ∆t ≤ 1/2∆x2 , and since ∆x is small, this implies ∆t is very
small. The θ-method for θ > 1/2 allows a larger time step for stability (but not too large,
otherwise the LTE gets big), and hence can require less overall computing. The Crank-Nicolson
scheme has the added advantage of a higher order LTE.
 Page 27

2.6.5 Extension to more general PDEs

We can follow a similar approach to a more general PDE. For example consider the equation

ut = uxx + c1 ux + c2 u

The corresponding FTCS scheme is

Ft n δ2 Dx n
wj = x 2 wjn + c1 w + c2 wjn
∆t ∆x 2∆x j
⇒wjn+1 = (r − c1 p)wj−1
n
+ (1 − 2r + c2 ∆t)wjn + (r + c1 p)wj+1
n
,

where r = ∆t/∆x2 , p = ∆t/2∆x.

The corresponding θ-method is
 2   2 
Ft n δx n Dx n n δx n+1 Dx n+1 n+1
w = (1 − θ) w + c1 w + c2 w j + θ w + c1 w + c2 w j
∆t j ∆x2 j 2∆x j ∆x2 j 2∆x j
n+1
⇒ − θ(r − c1 p)wj−1 + (1 + 2θr − θ∆tc2 )wjn+1 − θ(r + c1 p)wj+1
n+1
=
n
(1 − θ)(r − c1 p)wj−1 + (1 + (∆tc2 − 2r)(1 − θ))wjn + (1 − θ)(r + c1 p)wj+1
n
.

A von Neumann stability analysis shows that the extra term makes no difference to the results
already obtained for the heat equation.

2.7 Other Boundary Conditions

So far we have dealt with Dirichlet BCs of the form

u(0, t) = α(t), u(1, t) = β(t),

where α(t), β(t), are known functions of t. Other possible boundary conditions are Neumann
(conditions on ux at the boundary) or mixed (a mixture of Dirichlet and Neumann).
Suppose for example we have the following condition at x = 0 (similar calculations result if
we have the condition at x = 1, although we will need to replace the Forwards Difference
approximation at x = 0 by a Backwards Difference approximation at x = 1.)

ux (0, t) = γ(t).

In the finite difference scheme, previously from u(0, tn ) = α(tn ) we knew that w0n = α(tn ). Now
we do not have a value for u(0, t) and need a way of calculating w0n .

2.7.1 Forward Difference approximation to ux

The simplest (but least accurate) way to do this is by approximating ux (0, t) by a Forward
Difference in space at (0, tn )
Fx
ux (0, t) ≈ u(0, tn )
∆x
 Page 28

so the BC ux (0, t) = γ(t) becomes

Fx n w1n − w0n
w = = γ(tn ) ≡ γn
∆x 0 ∆x
giving
w1n − w0n = ∆x γn (∗)
How we use this extra equation depends on out choice of scheme, in particular whether it is
explicit or implicit.
If we are using an explicit scheme such as the FTCS scheme, at j = 1 we have

w1n+1 = rw0n + (1 − 2r)w1n + rw2n .

As soon as we have w1n+1 we can use (*) to calculate w0n+1 .

w0n+1 = w1n+1 − ∆x γn+1 .

If we are working with an implicit scheme, there are two possible ways to proceed. The first is
to add (*) to the set of Finite Difference equations which approximate the PDE. The second is
to eliminate w0n directly by modifying the FD equations. For example, suppose we are using the
Crank-Nicolson scheme. We can add (*) at n + 1 to the set of equations, with a new unknown
w0n+1, so that  
n+1 ′
wn+1 = w0n+1, w1n+1 , . . . , wj−1 .
so S now has an extra row and column, and the first two rows of S now look like
 
−1 1 0
 − 21 r 1 + r − 12 r 0 . . . 
 
 ... ..
.
..
.
.. .. 
. .
 
.. .. .. .. ..
. . . . .

with
bn+1 = [∆xγn+1 , 0, . . . , ]
Alternatively we can eliminate w0n+1 from (*) and the approximation to the PDE at j = 1, but
this is not so useful since presumably we will still need to calculate w0n+1.
Although straightforward to apply, the FD method has the disadvantage that the approximation
is only O(∆x) accurate and hence spoils the O(∆x2 ) accuracy we are getting from the PDE
scheme. A better method is the following:

2.7.2 Central Difference approximation to ux

This method is also known as the fictitious point scheme, since the first step is to introduce a
fictitious point x = −∆x (j = −1) outside the region [0, 1] (see figure).
 Page 29

n=2

n=1

n=0
j= −1 0 1 2
x=0

We now approximate ux (0, t) by a Central Difference in space at (0, tn )

Dx
ux (0, t) ≈ u(0, tn )
2∆x
so the BC ux (0, t) = γ(t) becomes

Dx n w1n − w−1
n
w = = γn
2∆x 0 2∆x
with accuracy O(∆x2 ). So now we have

w1n − w−1
n
= 2∆x γn (∗∗)

The other essential step is to write down the PDE approximation at (0, tn ) and eliminate the
n
value w−1 between these two equations. For example, for the FTCS scheme, we get

w0n+1 = rw−1
n
+ (1 − 2r)w0n + rw1n
= r(w1n − 2∆x γn ) + (1 − 2r)w0n + rw1n
= (1 − 2r)w0n + 2rw1n − 2r∆x γn

We now have an extra explicit equation for w0n+1 involving only the points (0, tn ) and (∆x, tn ).
When using (**) with an implicit scheme like the θ-method, we need to apply it at both n
and n + 1 to get a scheme connecting the values of w at (0, tn ), (∆x, tn ), (0, tn+1 ), (∆x, tn+1 ).
For example, with the Crank-Nicolson Scheme we combine (**) at tn and at tn+1 with the
approximation to the PDE at (0, tn )
1 n+1 1 1 n 1
− rw−1 + (1 + r)w0n+1 − rw1n+1 = rw−1 + (1 − r)w0n + rw1n
2 2 2 2
to get
1 1 1 1
− r(w1n+1 − 2∆xγn+1 ) + (1 + r)w0n+1 − rw1n+1 = r(w1n − 2∆xγn ) + (1 − r)w0n + rw1n
2 2 2 2
n+1 n+1 n n
⇒ (1 + r)w0 − rw1 = (1 − r)w0 + rw1 − r∆x(γn + γn+1)
 Page 30

2.8 Multilevel schemes for ut = uxx

We now move to a class of schemes for the heat equation which involve more than 2 time levels.
These are called multilevel schemes. The first of these is:

2.8.1 The Richardson scheme

In this scheme we approximate ut by the Central Difference operator in time

Dt
ut ≈ u
2∆t
so with the usual approximation in space at tn we have
n+1 n−1 n
Dt n w j − w j δx2 n wj−1 − 2wjn + wj+1
n
w ≡ = w ≡
2∆t j 2∆t ∆x2 j ∆x2
giving the Richardson scheme

wjn+1 = wjn−1 + 2r(wj−1

n
− 2wjn + wj+1
n
), n ≥ 1, j = 1, . . . , J − 1 (2.10)

An examination of the LTE shows this scheme is in general O(∆t2 , ∆x2 ). The scheme gives the
solution at the n + 1th time level in terms of the solutions at the nth and n − 1th time levels.
In order to start this off, we will need to calculate the n = 1 time level using a different scheme
(for example the θ-method).
Stability Analysis of (2.10)
Set wjn = ξ n eiωj in the usual way to get eventually a quadratic for ξ
 
2 2 1
ξ + 8r sin ω ξ−1 =0
2
This will have two solutions ξ1 , ξ2 . For stability we need both of them to be less than 1 in
modulus, i.e (2.10) is only stable for values of r for which |ξ1 |, |ξ2| ≤ 1.
A useful trick is to note that if we write the quadratic as (ξ − ξ1 )(ξ − ξ2 ) = ξ 2 + bξ + c, then
b = −(ξ1 + ξ2 ), c = ξ1 ξ2 . So in this case, since |c| = 1, and |b| =
6 2, we must have |ξ1 ||ξ2 | = 1,
and either (i) both ξi are complex with |ξi | = 1, or (ii) both ξi are real and distinct with one
|ξi | < 1 and the other > 1. Since solving the quadratic gives
r
2 ω ω
ξ1,2 = ξ± = −4r sin ± 1 + 16r 2 sin4
2 2
then if both roots are real, one will have |ξ| > 1. In fact
p ω
ξ− = −p − 1 + p2 , p = 4r sin2 ≥ 0
2

so clearly |ξ− | > 1 when p > 0, and hence (2.10) is unstable for all r.
So as a practical scheme, the Richardson method is useless, but it can be modified to make the
following useful scheme:
 Page 31

2.8.2 The Du Fort-Frankel scheme

This is similar to Richardson’s method, but the term wjn on the right-hand side is replaced
by the average of its two nearest neighbours in time: (wjn+1 + wjn−1)/2. So the approximate
operator L∆ is
!
n+1 n−1 n n−1 n+1 n
w j − w j w j−1 − w j − w j + w j+1
L∆ wjn = − (2.11)
2∆t ∆x2
The scheme for ut = uxx can therefore be arranged as
1 − 2r n−1 2r

wjn+1 = wj + n
wj−1 n
+ wj+1 , (2.12)
1 + 2r 1 + 2r
where (as usual) r = ∆t/∆x2 . So this scheme is explicit but needs to have w 1 supplied by a
different scheme.
Exercise: Show that the LTE of (2.11) is
 
2 1 ∆t2
LTE = r − utt ∆x2 + uttt + O(∆t4 , ∆x4 , r 4 ∆x6 )
12 6
i.e. it is second order in time and space. However, note that we have implicitly assumed
that ∆t = O(∆x2 ) in order to deduce second order accuracy, because of the r 2 factor in the
first term of the LTE. If for example we set ∆t = ∆x, then r = 1/∆x and the leading term of
the LTE is utt . Hence the LTE doesn’t tend to zero as ∆t, ∆x → 0.
p
Note also that if we set ∆t = r∆x2 with r = 1/12 then the method is fourth order
accurate.
Exercise: Show that the amplification factor in the von Neumann stability analysis of the Du
Fort-Frankel scheme (2.12) satisfies the quadratic equation
(1 + 2r)ξ 2 − 4rξ cos ω + 2r − 1 = 0,
where ω is the frequency parameter.
The roots of the quadratic are
p
2r cos ω ± 1 − 4r 2 sin2 ω
ξ± = .
1 + 2r
We shall examine the scheme’s stability by considering two cases separately.
(i) 4r 2 sin2 ω ≤ 1, so both roots are real. It then follows that
p
2r cos ω + 1 − 4r 2 sin2 ω 2r cos ω + 1
ξ+ = ≤ ≤ 1,
1 + 2r 1 + 2r
since cos ω ≤ 1 for all ω, and we also have (since cos ω ≥ −1 for all ω)
p
−2r + 1 − 4r 2 sin2 ω −2r
ξ+ ≥ ≥ > −1,
1 + 2r 1 + 2r
i.e. −1 < ξ+ ≤ 1. It can similarly be shown that −1 ≤ ξ− < 1, so in this case both roots
satisfy |ξ| ≤ 1.
 Page 32

(ii) 4r 2 sin2 ω > 1, so both roots of the quadratic are complex, i.e. ξ± = α ± iβ, where
p
2r cos ω 4r 2 sin2 ω − 1
α= , β= .
1 + 2r 1 + 2r
This means that |ξ+ | = |ξ− |. But the product of the two roots is (2r − 1)/(1 + 2r) and so
both roots must satisfy
|2r − 1|
|ξ|2 = ≤1
1 + 2r
for all r > 0, and so |ξ| ≤ 1 for any r.

Thus we have shown in both cases, that if ξ is a root of the quadratic then |ξ| ≤ 1 for all r and
all ω ∈ [−π, π]. Hence the Du Fort-Frankel scheme (2.12) is unconditionally stable (stable
for all r > 0).

Comparison of methods:

Scheme Comments Stability limit LTE

Du Fort-Frankel explicit, but need to use a OK ∀ r O(r ∆x , ∆x2 , ∆t2 ). Sec-
2 2

different scheme to get w 1 ond order if ∆t = O(∆x2 ),

fourth√ order if ∆t =
∆x2 / 12
Crank-Nicolson implicit: need to solve a OK ∀ r O(∆x2 , ∆t2 ). Second order
(θ = 1/2) tridiagonal system (not too in space and time separately
bad)
FTCS (θ = 0) explicit OK if r ≤ 1/2 O(∆x2 , ∆t). Second order if
∆t = O(∆x2 ), fourth order
if ∆t = ∆x2 /6

2.9 Convergence
Suppose we use the difference operator L∆ to approximate L = ∂/∂t − ∂ 2 /∂x2 , and that u is
the exact solution (Lu = 0) and wjn the corresponding approximate solution (L∆ wjn = 0). The
approximation will only be useful if wjn tends to u as ∆x, ∆t → 0. Roughly speaking this is
what we mean by convergence.
Fix x∗ ∈ (0, 1) and t∗ > 0. The exact solution here is u(x∗ , t∗ ). The approximate solution is
wjn , where j∆x = x∗ , n∆t = t∗ . We want to see what happens to the approximate solution
when we let ∆x, ∆t → 0 and j, n → ∞ in such a way that j∆x and n∆t remain fixed at x∗
and t∗ respectively. i.e. we choose ∆x = x∗ /j, ∆t = t∗ /n and let j, n → ∞. We can then write
t∗ /∆t
wjn = wx∗ /∆x , and in this notation we have the following definition.
 Page 33

Definition (Convergence)
The approximate solution wjn converges to the exact solution u at (x∗ , t∗ ) if
 
t∗ /∆t 
u(x∗ , t∗ ) − wx∗ /∆x  → 0


as ∆x, ∆t → 0.
We have only space here to mention one result on convergence. We say that a scheme is
consistent if its LTE → 0 as ∆x, ∆t → 0, and is stable if wjn remains bounded as n → ∞ for
fixed ∆t, ∆x. There is a nice theorem that covers all the analysis.

Lax Equivalence Theorem

Given a properly posed linear initial value problem and a finite difference approximation of
it that satisfies the consistency condition (i.e. its LTE → 0 as ∆x, ∆t → 0) and the stability
condition, then the scheme converges.
In fact the full result is actually stronger than this: if the scheme is consistent then
stability ⇐⇒ convergence.
Therefore it is enough to establish stability and consistency to get convergence. On the other
hand, instability rules out convergence.

2.10 Approximations to more general parabolic PDEs

• Reaction-Diffusion equations
The general form is
ut = κuxx + f (x, t, u)
where the first term on the rhs is the diffusion term and the other is the reaction term.
We approximate the ut and the uxx term in the usual way (note that r becomes κr and
stability results apply to this new value, i.e. the FTCS scheme is unstable for κr > 1/2.
The reaction term is approximated by f (xj , tn , wjn ) at time level n. If f (x, t, u) is nonlinear
in u, and if we use an implicit scheme, then we will end up with a set of nonlinear equations
for wjn+1 at each time level.

• Linear equations with Varying coefficients

A typical equation is

ut = A(x, t)uxx + B(x, t)ux + C(x, t)u

Again we approximate the terms in the usual way, except that we replace A(x, t) by
Anj = A(xj , tn ), etc.
 Page 34

2.11 More space dimensions

So far we have looked at the PDE ut = uxx , plus variations, but always with time and one
space-like dimension such as x as the independent variables. When more than one space
dimensions are involved (usually the case in real life), we have to deal with equations such as
ut = uxx + uyy or ut = uxx + uyy + uzz with solutions u(x, y, t) or u(x, y, z, t) and approximate
n n
solutions wj,k ≈ u(xj , yk , tn ) and wj,k,ℓ ≈ u(xj , yk , zℓ , tn ). Now the space grid is 2D or 3D, but
we use the same principles to develop a numerical scheme.
For example, let us develop a FTCS scheme for the 2D heat equation ut = uxx + uyy defined
in a rectangle of size J∆x × K∆y. We approximate ut and uxx as before, and uyy as for uxx
using δy2 .
n+1 n  2 
Ft n wj,k − wj,k δx δy2 n
wj,k = = 2
+ 2
wj,k
∆t ∆t ∆x ∆y
n n n n n n
wj−1,k − 2wj,k + wj+1,k wj,k−1 − 2wj,k + wj,k+1
= +
∆x2 ∆y 2
giving the scheme
n+1 n n n n n n n
wj,k = wj,k + rx (wj−1,k − 2wj,k + wj+1,k ) + ry (wj,k−1 − 2wj,k + wj,k+1 ) (2.13)
with rx = ∆t/∆x2 , ry = ∆t/∆y 2 . At each time level n we have a (J + 1) × (K + 1) grid of
n
values wj,k , j = 0, . . . , J; k = 0, . . . , K. With the boundary values given on the sides of the
rectangle, applying (2.13) at each interior point gives the values at the next time level n + 1.

2.11.1 Numerical example: the FTCS scheme for the 2D heat equation

Suppose we wish to solve ut = uxx + uyy on the unit square with ∆x = ∆y = 1/3, initial
conditions u(x, y, 0) = sin(πx/2) sin(πy), and boundary conditions u(0, y, t) = u(x, 1, t) =
u(x, 0, t) = 0, u(1, y, t) = sin(πy). Carry out two steps of the scheme with r = 0.25.
The grid for this scheme at each time level looks like this:

j=0 j=1 j=2 j=3

k=3

k=2

k=1

k=0

0
We have the initial conditions wj,k given by the formula sin(πj/6) sin(πk/3), j = 0, . . . , 3; k =
0
0, . . . , 3. The BCs give w0,j = wj,0 = wj,3 = 0, w3,j = sin(πj/3), so the values of wj,k look like
this:
 Page 35

k\j 0 1 2 3
0 0 0 0 0
1 0 0.4330 0.7500 0.8660
2 0 0.4330 0.7500 0.8660
3 0 0 0 0
1
We now apply (2.13) for j = 1, 2; k = 1, 2, n = 0 to get the values of wj,k

k\j 0 1 2 3
0 0 0 0 0
1 0 0.2958 0.5123 0.8660
2 0 0.2958 0.5123 0.8660
3 0 0 0 0
2
We now apply (2.13) for j = 1, 2; k = 1, 2, n = 1 to get the values of wj,k

k\j 0 1 2 3
0 0 0 0 0
1 0 0.2020 0.4185 0.8660
2 0 0.2020 0.4185 0.8660
3 0 0 0 0

(all results to 4D). Note that the solution is symmetric about the line y = 1/2, which we could
expect since the PDE, the initial conditions and the boundary conditions satisfy this symmetry.
We now example the stability of this scheme.

2.11.2 Stability of the FTCS scheme for the 2D heat equation

We investigate the von Neumann stability of the scheme (2.13) by following the steps below.
n
1. Substitute wj,k = ξ n exp(ijα) exp(ikβ) into the scheme (2.13).

2. Divide through by ξ n exp(ijα) exp(ikβ) and rearrange to get an expression for ξ.

3. Find conditions on the mesh ratios that guarantee |ξ| ≤ 1 for all (α, β) ∈ [−π, π]2 .

Step 1 gives
 
(ξ n+1 − ξ n ) eijα eikβ = rx ξ n eikβ ei(j−1)α − 2eijα + ei(j+1)α
 
+ ry ξ n eijα ei(k−1)β − 2eikβ + ei(k+1)β ,

and simplifying (step 2) gives

ξ − 1 = rx e−iα − 2 + eiα + ry e−iβ − 2 + eiβ
α β
= −4rx sin2 − 4ry sin2 .
2 2
 Page 36

Note that the second central difference operators δx2 and δy2 become (2 cos α − 2) = −4 sin2 α2
and (2 cos β − 2) = −4 sin2 β2 after simplification.
For step 3 we first look at the case of a square spatial mesh, i.e. when ∆x = ∆y, so that ξ is
given by
α β
ξ = 1 − 4r sin2 − 4r sin2 where r = rx = ry .
2 2
The amplification factor ξ is real, and so |ξ| ≤ 1 is equivalent to −1 ≤ ξ ≤ 1. Hence we need
to find conditions on r that guarantee that the minimum value of ξ is greater than -1 and the
maximum value is less than 1 for all α, β ∈ [−π, π]. The max and min values of ξ occur at the
maximum and minimum values of the sine functions (because r ≥ 0), i.e. at (α, β) = (±π, ±π)
and (α, β) = (0, 0) respectively. So
1 − 8r ≤ ξ ≤ 1 for all α, β ∈ [−π, π].
For stability we therefore require 1 − 8r ≥ −1, i.e. the scheme is only stable when r ≤ 1/4.
If the space grid is not square then the stability analysis is almost the same but now
α β
ξ = 1 − 4rx sin2 − 4ry sin2
2 2
and
max ξ = 1, min ξ = 1 − 4rx − 4ry
α,β α,β

so that the scheme is stable if and only if

1
0 ≤ rx + ry ≤ .
2
Exercise: Verify this and show that it gives the square mesh limit when ∆x = ∆y.
When rx = ry , ∆x = ∆y, the stability limit is twice as bad as for the 1D case, and there is
much more work at each time level since we have M 2 equations to calculate at each time level.
It is thus even more important than before to develop schemes which are more efficient than
the FTCS scheme.
In 1D we saw that implicit schemes were more stable than the explicit FTCS scheme, and hence
could be used with a larger value of ∆t. This suggests trying to develop implicit schemes in
2D. Our first attempt will be a 2D version of the θ-method.

The 2D θ-method

It is straightforward to develop a 2D version of this scheme in the same way as for the FTCS
scheme. For simplicity we work with θ = 21 , but the same principles apply for general values of
θ > 0. Write
n+1 n
Ft n wj,k − wj,k 1 δx2 n n+1

1 δy2 n n+1


wj,k = = 2
w j,k + w j,k + 2
wj,k + wj,k
∆t ∆t 2 ∆x 2 ∆y
n n n n n n
wj−1,k − 2wj,k + wj+1,k wj,k−1 − 2wj,k + wj,k+1
= + +
2∆x2 2∆y 2
n+1 n+1 n+1 n+1 n+1 n+1
wj−1,k − 2wj,k + wj+1,k wj,k−1 − 2wj,k + wj,k+1
+ +
2∆x2 2∆y 2
 Page 37

n+1
taking all the unknown wj,k to the left gives the scheme

n+1 1 1 1 1
wj,k − rx δx2 wj,k
n+1
− ry δy2 wj,k
n+1 n
= wj,k + rx δx2 wj,k
n
+ ry δy2 wj,k
n
, (2.14)
2 2 2 2
or in full
n+1 1 n+1 n+1 n+1 1 n+1 n+1 n+1
wj,k − rx (wj−1,k − 2wj,k + wj+1,k ) − ry (wj,k−1 − 2wj,k + wj,k+1 )=
2 2
n 1 n n n 1 n n n
wj,k + rx (wj−1,k − 2wj,k + wj+1,k ) + ry (wj,k−1 − 2wj,k + wj,k+1 ).
2 2
The problem with (2.14) is that it has 5 unknowns, and if we write down (2.14) at each interior
point (xj , yk ), j = 1, . . . , J − 1; k = 1, . . . , K − 1, we end up with (J − 1) × (K − 1) equations
n+1
for the (J − 1) × (K − 1) unknowns wj,k , j = 1, . . . , J − 1; k = 1, . . . , K − 1. Although sparse,
the matrix of coefficients no longer has a simple tri-diagonal structure as in 1D.
Ignoring sparseness, and writing (J −1)×(K −1) ≈ MJ, we recall that a system of N equations
generally requires 31 N 3 floating point operations to solve it using Gaussian elimination. In this
case then we will have approx 13 J 3 K 3 operations, and even with small values for J and K like
10 we will have around 3 × 105 operations at each time step to carry out.
We need another idea to be able to develop a more practical scheme in 2D, the so-called ADI
method.

2.12 Alternating direction implicit (ADI) schemes

To help us here it is useful to develop a more compact notation, the Exponential operator
notation.
We start out by observing that when u is well enough behaved, the Taylor expansion of u(xj +
∆x, yk , tn ) can be written as
 
∂u ∆x2 ∂ 2 u ∆x3 ∂ 3 u
u(xj +∆x, yk , tn ) = u + ∆x + + + ...
∂x 2! ∂x2 3! ∂x3 (x ,y ,tn )
 2 2 3 3
 j k
∂ ∆x ∂ ∆x ∂
= 1 + ∆x + 2
+ + . . . u |(xj ,yk ,tn )
∂x 2! ∂x 3! ∂x3
 
∂
= exp ∆x u |(xj ,yk ,tn ) .
∂x

Similarly (missing out the steps in between) we can write

 
∂
u(xj , yk + ∆y, tn ) = exp ∆y u |(xj ,yk ,tn )
∂y
and  
∂
u(xj , yk , tn + ∆t) = exp ∆t u |(xj ,yk ,tn )
∂t
We now use the exponential operator notation to derive a different implicit scheme for
ut = uxx + uyy . Suppose that u(x, y, t) is a smooth solution of the PDE. Taylor expanding
 Page 38

u(x, y, tn + ∆t) about (x, y, tn ) gives

 
∂
u(x, y, tn +∆t) = exp ∆t u |(x,y,tn ) .
∂t

Use ea = ea/2 · ea/2 to rewrite this as

   
∆t ∂ ∆t ∂
u|t=tn+∆t = exp exp u|t=tn
2 ∂t 2 ∂t

and hence    
∆t ∂ ∆t ∂
exp − u|t=tn+∆t = exp u|t=tn . (2.15)
2 ∂t 2 ∂t
| {z } | {z }
new time-level old time-level
We now use the fact that u solves the PDE to write
    
∆t ∂ ∆t ∂ 2 ∂2
exp ± u = exp ± + u
2 ∂t 2 ∂x2 ∂y 2
   
∆t ∂ 2 ∆t ∂ 2
= exp ± exp ± u
2 ∂x2 2 ∂y 2

(using ea+b = ea · eb ). Plugging this into (2.15) gives

       
∆t ∂ 2 ∆t ∂ 2 ∆t ∂ 2 ∆t ∂ 2
exp − exp − u|t=tn+∆t = exp exp u|t=tn .
2 ∂x2 2 ∂y 2 2 ∂x2 2 ∂y 2

We now chop the exponentials at first order (e±q ≈ 1 ± q) to get

     
∆t ∂ 2 ∆t ∂ 2 ∆t ∂ 2 ∆t ∂ 2
1− 1− u|t=tn+∆t ≈ 1 + 1+ u|t=tn ,
2 ∂x2 2 ∂y 2 2 ∂x2 2 ∂y 2

and finally use second central differences to approximate the space derivatives at x = xj , y = yk
to end up with the scheme
 rx   ry  n+1  rx   ry  n
1 − δx2 1 − δy2 wj,k = 1 + δx2 1 + δy2 wj,k , (2.16)
2 2 2 2
where rx , ry defined as before.
This just looks like another (slow) implicit scheme. But it is actually much easier and
quicker to use than it looks. It splits into two stages involving an intermediate quantity
vj,k :  rx 2   ry 2  n 
Stage 1: 1 − δx vj,k = 1 + δy wj,k  
2 2 
 (2.17)
ry 2  n+1  rx 2  

Stage 2: 1 − δy wj,k = 1 + δx vj,k 
2 2
 Page 39

y
x x
Stage 1 (on left), Stage 2 (on right)

Stage 1 involves solving a tridiagonal system of equations along each row of the solution in
the x−direction (which can be done quickly); Stage 2 is similar, but in the y−direction (see
figure)
The name ADI comes from this idea of alternately solving along the x−direction and y−direction.
We need to
verify that the two stages of (2.17) are equivalent to the full scheme (2.16). Applying
1 − 21 rx δx2 to Stage 2 gives
     
1 2 1 2 n+1 1 2 1 2
1 − rx δx 1 − ry δy wj,k = 1 − rx δx 1 + rx δx vj,k
2 2 2 2
  
1 2 1 2
= 1 + rx δx 1 − rx δx vj,k (difference operators commute)
2 2
  
1 2 1 2 n
= 1 + rx δx 1 + ry δy wj,k by Stage 1.
2 2

So it does reproduce (2.16).

Why is all this faster than the 2D θ-method? Because the matrices at each step are triadiagonal,
and because they involve only J or K unknowns, they can be solved very efficiently in O(J)
or O(K) operations. So although we have K or J separate equations in J or K unknowns, the
total number of operations required for each time step are just of order O(JK). Compare this
with the O(J 3 K 3 ) estimate above, and you will see that the ADI method implies a large saving
in computer time.

Stability
Exercise: Verify that if the mesh is square (i.e ∆x = ∆y and so rx = ry = r) then the ADI
scheme is stable for all r.