Numerics: The analysis and implementation of

numerical methods for solving differential equations

Dr Hilary Weller, <[email protected]>

October 4, 2018

Timetable
Week Chapters to read Videos to watch Class Assignment Deadline Prop-
before class before class Date ortion
2 1-3 1-4 Wed 10 Oct Code review 24 Oct 5%
Introduction and linear advection schemes
3 4-5 5.0-5.4, 6.0-6.1 Wed 17 Oct Draft report 7 Nov 5%
Analysis of Linear Advection Schemes and dispersion for feedback
6 6-7 7.0-7.5, 8.0-8.5 Wed 7 Nov Final 21 Nov 15%
More Advection Schemes and Wave equations code
7 Wed 14 Nov Final report 28 Nov 25%

2
Further Resources (Not essential reading)
Python
• “A Hands-On Introduction to Using Python in the Atmospheric and Oceanic Sciences” by
Johnny Wei-Bing Lin. http://www.johnny-lin.com/pyintro
• “Think Python. How to Think Like a Computer Scientist” by Allen B. Downey.
http://www.greenteapress.com/thinkpython/thinkpython.html
• http://www.python.org
• http://docs.python.org/tutorial/
• http://matplotlib.sourceforge.net/users/index.html
• http://software-carpentry.org/
• http://www.diveintopython.net/
• http://learnpythonthehardway.org/
• http://www.ibiblio.org/g2swap/byteofpython/read/
• “Making Use of Phython” by Rashi Gupta
• “Python Essential Reference” by David M. Beazley (Addison Wesley)
• “Learning Python” Mark Lutz (O’Reilly Media)

Numerical Methods
• Weller, H. Draft of chapter for “Mathematics for Planet Earth, a Primer”
http://www.met.reading.ac.uk/~sws02hs/teaching/
PDEsNumerics/WellerPrimer.pdf
• Wikipedia http://en.wikipedia.org
• Durran, D. R. Numerical methods for fluid dynamics (Springer).
3

• LeVeque, R. Numerical Methods for Conservation Laws (Springer)

• Ortega, J.M. and Poole, W.G. An Introduction to Numerical Methods for Differential
Equations. 1981 (Pitman Publishing Inc)
• Ferziger, J. H. and Peric, M. Computational Methods for Fluid Dynamics (Springer).
• Numerical Recipes: The Art of Scientific Computing, Third Edition (2007), (Cambridge
University Press). http://www.nr.com/
• Lloyd N. Trefethen. Finite Difference and Spectral Methods for Ordinary and Partial
Differential Equations.
https://people.maths.ox.ac.uk/trefethen/pdetext.html

Other People’s lecture notes for similar courses

• https://people.maths.ox.ac.uk/trefethen/pdetext.html

Websites for this course:

http://mpecdt.bitbucket.org/
It contains notes, example programs and solutions to problems and practicals.
https://mpecdtnumerics.slack.com
For discussion cite to asking and answering questions about the course

4
Contents
1 The Navier Stokes Equations 10
1.1 The Potential Temperature Equation . . . . . . . . . . . . . . . . . . . . . . 11
1.2 Advection of Pollution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.1 Pure Linear Advection . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.2 Advection/Diffusion with Sources and Sinks . . . . . . . . . . . . . 13
1.3 The Momentum Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.3.1 Coriolis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.3.2 The Pressure Gradient Force . . . . . . . . . . . . . . . . . . . . . . 15
1.3.3 Gravitational Acceleration: A simulation of a dam break . . . . . . . 19
1.3.4 Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.3.5 The Complete Navier Stokes Equations . . . . . . . . . . . . . . . . 22

2 Linear Finite Difference Schemes for Advection 23

2.1 Forward in Time, Backward in Space (FTBS) . . . . . . . . . . . . . . . . . 24
2.1.1 Order of Accuracy of FTBS . . . . . . . . . . . . . . . . . . . . . . 26
2.2 Centred in Time, Centred in Space (CTCS) and FTCS . . . . . . . . . . . . . 27
2.2.1 Order of Accuracy of CTCS . . . . . . . . . . . . . . . . . . . . . . 28
2.3 Implicit and Explicit Schemes . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4 Backward in Time, Centred in Space (BTCS) . . . . . . . . . . . . . . . . . 30

3 Fourier Analysis 32
3.1 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5

3.4 Differentiation and Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 38

3.5 Spectral Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.6 Wave Power and Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.7 Recap Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.8 Analysing Power Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4 Numerical Analysis of Advection Schemes 43

4.1 Introduction: What is Numerical Analysis and why is it done . . . . . . . . . 43
4.2 Some Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3 Convergence and errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.4 Lax-Equivalence Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.5 Domain of Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.5.1 Courant-Friedrichs-Lewy (CFL) criterion: . . . . . . . . . . . . . . . 50
4.5.2 Domain of Dependence of FTBS . . . . . . . . . . . . . . . . . . . . 50
4.5.3 The Domain of Dependence of the CTCS Scheme . . . . . . . . . . 51
4.6 Von-Neumann Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . 52
4.6.1 Von-Neumann Stability Analysis of FTBS . . . . . . . . . . . . . . . 53
4.6.2 What should A be for real linear advection? . . . . . . . . . . . . . . 54
4.6.3 Von Neumann Stability Analysis of CTCS . . . . . . . . . . . . . . . 55
4.7 Conservation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.7.1 Conservation of higher moments . . . . . . . . . . . . . . . . . . . . 57
4.8 Exercises: Analysis of Advection Schemes (answers on blackboard) . . . . . 58

5 Waves, Dispersion and Dispersion Errors 59

5.1 Some Background on Waves . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2 Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6
 5.3 Some Excellent Animations and Videos which demonstrate Dispersion . . . . 67
5.4 Phase speed of linear advection . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.5 Phase speed and dispersion errors of CTCS . . . . . . . . . . . . . . . . . . 69
5.6 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6 Alternative Advection Schemes 73

6.1 Semi-Lagrangian Advection . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.1.1 Semi-Lagrangian Advection in One Dimension . . . . . . . . . . . . 75
6.2 Artificial Diffusion to Remove Spurious Oscillations . . . . . . . . . . . . . 76
6.3 The Finite Volume Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.3.1 The Finite Volume Method in one dimension, for constant u . . . . . 78
6.4 Lax-Wendroff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.5 Total Variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.6 Total Variation Diminishing (TVD) schemes . . . . . . . . . . . . . . . . . . 82
6.6.1 Limiter functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.6.1.1 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.6.2 The Sweby Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.6.3 The Van Leer Limiter . . . . . . . . . . . . . . . . . . . . . . . . . . 84

7 Modelling Wave Equations 85

7.1 Simulations of the SWE on the surface of a sphere . . . . . . . . . . . . . . . 86
7.2 Processes Represented by the SWE . . . . . . . . . . . . . . . . . . . . . . . 87
7.2.1 Linearised SWE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.3 Analytic Solultion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.4 Unstaggered Forward-Backward (1d A-grid FB) . . . . . . . . . . . . . . . . 90
7.4.1 Von-Neumann Stability Analysis . . . . . . . . . . . . . . . . . . . . 91
7

7.4.2 Dispersion of Unstaggered Forward-Backward (1d A-grid FB) . . . . 92

7.5 Problems with Co-location of h and u . . . . . . . . . . . . . . . . . . . . . 93
7.6 Staggered Forward-Backward (1d C-grid FB) . . . . . . . . . . . . . . . . . 94
7.7 Arakawa Grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.8 Linearised Shallow-Water Equations on Arakawa Grids . . . . . . . . . . . . 96
7.9 Discussion Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

8
Numerical Modelling of the Atmosphere and Ocean
Weather, ocean and climate forecasts predict the winds, temperature and pressure from the
Navier-Stokes equations:

Chapter 1: The Navier Stokes Equations

The Navier-Stokes Equations for a compressible, rotating atmosphere
DΨ ∂ Ψ
The Lagrangian derivative: = + u · ∇Ψ
Dt ∂t
 
Du ∇p 2 1
Momentum Ω×u−
= -2Ω + g + µu ∇ u + ∇(∇ · u)
Dt ρ 3
Dρ
Continuity + ρ∇ · u = 0
Dt
Dθ
Energy = Q + µθ ∇2 θ
Dt
An equation of state, eg perfect gas law, p = ρRT

u Wind vector g Gravity vector (downwards)

t Time θ Potential temperature, T (p0 /p)κ
Ω Rotation rate of planet κ heat capacity ratio ≈ 1.4
ρ Density of air Q Source of heat
p Atmospheric pressure µu , µθ Diffusion coefficients

We will learn how to solve simplified versions numerically.

You do not need to memorise the equations but you should be able to describe the meaning
and behaviour of the terms ...

10
1.1 The Potential Temperature Equation
Dθ ∂θ
= + u · ∇θ = Q + µθ ∇2 θ
Dt ∂t
Lagrangian Rate of change Advection Heat Diffusion
derivative at fixed point of θ source of θ

θ will be created and destroyed

by the heat source, Q , it will be
moved around by the wind field,
u, and θ will be diffused by a
diffusion coefficient, µθ

11

1.2 Advection of Pollution

1.2.1 Pure Linear Advection
Advection of concentration φ without diffusion or sources or sinks:
Dφ ∂φ
= + u · ∇φ = 0 (1.1)
Dt ∂t
Changes of φ are produced by the component of the wind in the same direction as gradients
of φ . In order to understand why the u · ∇φ term leads to changes in φ , consider a region of
polluted atmosphere where the pollutant has the concentration contours shown below:

Exercise: Draw on the figure the

φ = 0.1
directions of the gradients of φ and
φ = 0.2 thus mark with a +, − or 0 locations
φ = 0.3 where u · ∇φ is positive, negative and
φ = 0.4 zero. Thus deduce where φ is
increasing, decreasing or staying the
same based on equation 1.1. Hence
u overlay contours of φ at a later time.

12
1.2.2 Advection/Diffusion with Sources and Sinks
DΨ ∂Ψ
= + u · ∇Ψ = S + µΨ ∇2 Ψ
Dt ∂t
Lagrangian Rate of change Advection Sources Diffusion
derivative at fixed point of Ψ and sinks of Ψ

13

1.3 The Momentum Equation

 
Du ∇p 2 1
= Ω×u
-2Ω - + g + µu ∇ u + ∇(∇ · u)
Dt ρ 3
Lagrangian Coriolis Pressure Gravitational Diffusion
derivative gradient acceleration

1.3.1 Coriolis
Inertial Oscillations governed by part of the momentum equation:

∂u
Ω×u
= -2Ω
∂t

• A drifting buoy set in motion

by strong westerly winds in
the Baltic Sea in July 1969.
• Once the wind subsides, the
upper ocean follows inertia
circles

14
1.3.2 The Pressure Gradient Force

If the pressure gradient force is the only

large term in the momentum equation, then
together with the continuity equation and
perfect gas law, we get equations for
acoustic waves:
∂u 1
+ ∇p = 0
∂t ρ0
∂p
+ ρ0 c2 ∇ · u = 0
∂t
where ρ0 is a reference density and c is the
speed of sound.

15

Pressure Gradients lead to very fast acceleration - Acoustic Waves

16
Geostrophic Balance: Pressure Gradients versus Coriolis

17

Geostrophic turbulence: pressure gradients, Coriolis and the (non-linear) advection of

velocity by velocity

18
1.3.3 Gravitational Acceleration: A simulation of a dam break

19

Gravitational Acceleration: Explosive Comulonimbus

20
1.3.4 Diffusion
Diffusion of quantity φ with diffusion Diffusion of a noisy profile
coefficient µφ in arbitrary spatial (zero gradient boundary conditions)
dimensions
0.7
∂φ
= µφ ∇2 φ 0.6
∂t
0.5
And in 1d:
0.4
∂φ ∂ 2φ
= µφ 2

φ
∂t ∂x 0.3

The second derivative of φ is high at 0.2

troughs and low in peaks of φ . Therefore 0.1

diffusion tends to remove peaks and 0.00.0 0.2 0.4 0.6 0.8 1.0
troughs and make a profile more smooth: x

Discussion Questions:
• Which equations have a diffusion
coefficient?
• What causes diffusion?
• Is diffusion a large term of the equations
of atmospheric motion?

21

1.3.5 The Complete Navier Stokes Equations

With moisture, phase changes, radiation, ... from NUGAM (courtesy of Pier Luigi Vidale)

22
Chapter 2: Linear Finite Difference Schemes for Advection
In one spatial dimension, x, with constant wind, u, no diffusion and no sources or sinks, the
linear advection equation (eqn 1.1) for φ reduces to:

∂φ ∂φ
+u =0 or φt + uφx = 0 (2.1)
∂t ∂x
This equation has an analytic solution. If the initial condition is φ (x, 0) then the solution at
time t is φ (x,t) = φ (x − ut, 0).
Exercise:
Check that this is the analytic solution.
∂X ∂X ∂φ ∂φ ∂X
(Hint: define X = x − ut, calculate and and use the chain rule, = )
∂x ∂t ∂x ∂X ∂x

23

2.1 Forward in Time, Backward in Space (FTBS)

∂φ ∂φ φ φ j−1 φ j
To solve +u = 0 numerically: φ j+1
∂t ∂x
• divide space into N equal intervals of size
∆x so that x j = j∆x for j = 0, 1, ..., nx

• Divide time into time steps ∆t

(so that tn = n∆t, n = 0, 1, 2, ....) x0 x1 x1 x j−1 x j x j+1 x
• Define φ j
(n)
= φ (x j ,tn ). ∆x
(n) (n+1) (n)
∂φj φj −φj
• Approximate ∂ φ /∂t at x j , tn by a forward difference: =
∂t ∆t
(n) (n) (n)
∂φj φ j − φ j−1
• Approximate ∂ φ /∂ x at x j , tn by a backward difference: =
∂x ∆x
• Substituting these into eqn (2.1) gives FTBS:
(n+1) (n) (n) (n)
φj −φj φ j − φ j−1
+u =0 (2.2)
∆t ∆x
(n+1)
• This can be re-arranged to get φ j on the LHS and all other terms on the RHS so that
we can calculate φ at the new time step at all locations based on values at previous time
steps. Also in this equation, remove u, ∆t and ∆x by substituting in the Courant number,
c = u∆t/∆x:
(n+1) (n) (n) (n)

φj = φ j − c φ j − φ j−1 (2.3)

24
Questions
• Why did we go forwards in time?
(n+1)
In order to be able to re-arrange in order to create an equation for φ j based on values
that are all known from the previous time-step.
• Why did we go backwards in space? We will come back to this
• What order accuracy is FTBS in space and time? (ie, what is the order, n, such that the
error of the approximation is proportional to ∆xn or ∆t n )
• What influence will the errors have on the solution?...

25

2.1.1 Order of Accuracy of FTBS

We will use a Taylor series to derive the backward in space approximation in FTBS, find its
order of accuracy and describe how the errors will affect the solution. The Taylor series for
φ j−1 about φ j is:
∆x2 00

φ j−1 = φ j − ∆xφ j0 + φj + O ∆x3 .
2!
We can rearrange this to find the backward in space approximation for ∂ φ /∂ x = φ j0 :

φ j − φ j−1 ∆x 00

φ j0 = + φ j + O ∆x2 .
∆x 2

∆x 00
• The leading error term is φ . This tells us two things:
2
– The error is proportional to ∆x1 so the spatial derivative approximation is first-order
accurate.
– The error behaves like φ 00 which is the spatial term from the diffusion equation. Adding
this error to the advection equation makes the equation diffusive.
You will be writing code to solve the linear advection equation using FTBS in the
assignment. Is the solution diffusive in comparison to the analytic solution? Is the error
proportional to ∆x?

26
2.2 Centred in Time, Centred in Space (CTCS) and FTCS
∂φ ∂φ
To solve +u = 0 numerically:
∂t ∂x φ φ j−1 φ j
• Approximate ∂ φ /∂t at x j , tn by a centred φ j+1
(n−1) (n+1)
difference, using φ j and φ j :
(n) (n+1) (n−1)
∂φj φj −φj
=
∂t 2∆t
• Approximate ∂ φ /∂ x at x j , tn by a centred
(n)
∂φj
(n)
φ j+1 − φ j−1n x0 x1 x1 x j−1 x j x j+1 x
difference: = ∆x
∂x 2∆x
(n+1)
• Substituting these into eqn (2.1) and re-arrange to get φ j on the LHS and all other
terms on the RHS and substituting in the Courant number, c = u∆t/∆x gives CTCS:
(n+1) (n−1) (n) (n)

φj = φj − c φ j+1 − φ j−1 (2.4)
• This is a three-time-level formula (it involves values of φ at times tn−1 , tn and tn+1 . To
start the simulation, values of φ are needed at times t0 and t1 . However, only φ (x,t0 ) is
available. So another scheme (such as FTCS) must be used to obtain φ (1) = φ (x,t1 ):
(n+1) (n) c (n) (n)

FTCS: φ j = φ j − φ j+1 − φ j−1 (2.5)
2

27

2.2.1 Order of Accuracy of CTCS

Use Taylor series approximations for φ j−1 and φ j+1 about φ j to find a second-order
approximation for φ j0 and show that it is second-order accurate:
∆x2 00

φ j−1 = φ j −∆xφ j0 + φ j + O ∆x3
2!
∆x2 00

φ j+1 = φ j + ∆xφ j0 + φ j + O ∆x3
2!
Subtract these to eliminate φ j00 and rearrange to find φ j0 and find its order of accuracy:
φ j+1 − φ j−1

φ j0 = + O ∆x2
2∆x
Question to consider during the assignment:
Does CTCS do better than FTBS? Why or why not?

28
2.3 Implicit and Explicit Schemes
Explicit Values at the next time-step are determined from values at the current (and previous)
time-steps. Straightforward.
Implicit Values at the next time-step are determined from values at the next time step!
• This leads to a set of simultaneous equations
• How do we solve a set of linear simultaneous equations?
- create a matrix equation and then solve using, eg Gaussian elimination
• How do we solve a set of non-linear simultaneous equations?
- multi-dimensional generalisation of Newton-Raphson method - hard and
expensive
• Easier to solve a linear equation implicitly rather than a non-linear equations
• Why would we want to use an implicit method at all?...

29

2.4 Backward in Time, Centred in Space (BTCS)

Derive and equation for BTCS
(n+1) (n) c (n+1) (n+1)

φj = φj − φ j+1 − φ j−1 (2.6)
2
Is this an implicit or explicit scheme? Implicit
(n)
Exercise: Assume that all of the j values, φ j (for a particular n) can be represented as a
 
(n) (n) (n) (n) T
vector: φ = φ0 , φ1 , · · · , φ j , · · · , φN−1 . Find a matrix M such that BTCS is defined
(n)

as Mφφ (n+1) = φ (n) given x : 0 → 1 and periodic boundary conditions so that φ0 = φN .

30
• Will these advection schemes approximate the continuous PDE given sufficient
resolution?
• Will we be able to take large time-steps?

In order to answer these questions, we will need von-Neumann stability analysis.

von-Neumann stability analysis relied on Fourier analysis. Some revision of Fourier analysis
will be presented for 3 reasons:
1. It is needed for stability analysis
2. Some weather forecasting models use spectral decompositions (eg ECMWF) which are a
spherical version of Fourier decompositions
3. Fourier analysis is used to analyse climate data

This is revision material and so is not directly examinable

31

Chapter 3: Fourier Analysis

3.1 Fourier Series
Any periodic, integrable function, f (x) (defined on [−π, π]), can be expressed as a Fourier
series; an infinite sum of sines and cosines:
∞ ∞
a0
f (x) = + ∑ ak cos kx + ∑ bk sin kx (3.1)
2 k=1 k=1

• The ak and bk are the Fourier coefficients.

• The sines and cosines are the Fourier modes.
• k is the wavenumber - number of complete waves that fit in the interval [−π, π]
sin kx for different values of k
1.0
k =1
k =2
k =4
0.5

0.0

0.5

1.0−π −π/2 π/2 π

0
x

• The wavelength is λ = 2π/k

• The more Fourier modes that are included, the closer their sum will get to the function.

32
 Sum of First 4 Fourier Modes of a Periodic Function
1.0
Fourier Modes Original function
4 Sum of first 4 Fourier modes

0.5
2

0.0 0

2
0.5

4
1.0−π −π/2 π/2 π −π −π/2 π/2 π
0 0
x x

33

The first four Fourier modes of a square wave.

The additional oscillations are “spectral ringing”

Each mode can be represented by motion around a

circle. ↓
The motion around each circle has a speed and a
radius. These represent the wavenumber and the
Fourier coefficients. Which is which?

speed is wavenumber, radius is coefficient

34
Equivalently, equation (3.1) can be expressed
√ as an infinite sum of exponentials using the
relation eiθ = cos θ + i sin θ where i = −1:
∞ ∞ ∞
a0
f (x) = + ∑ ak cos kx + ∑ bk sin kx = ∑ Ak eikx . (3.2)
2 k=1 k=1 k=−∞

Exercise
Evaluate the Ak s in terms of the ak s and bk s.

35

3.2 Fourier Transform

• The Fourier Transform transforms a function f which is defined over space (or time) into
the frequency domain, so that it is defined in terms of Fourier coefficients.
• The Fourier transform calculates the Fourier coefficients as:
Z Z
1 π 1 π
ak = f (x) cos(kx)dx , bk = f (x) sin(kx)dx
π −π π −π
3.3 Discrete Fourier Transform
A discrete Fourier Transform converts a list of 2N + 1 equally spaced samples in [0, 2π) of a
complex valued, periodic function, fn , to the list of the first 2N + 1 complex valued Fourier
coefficients:
N
1
Ak =
2N + 1 n=−N ∑ fn e−iπnk/N . (3.3)

The truncated Fourier series:

N
f (x) ≈ ∑ Ak eikx (3.4)
k=−N

is an approximation to the function f which fits the sampled points, fn , exactly.

On a computer this is done with a Fast Fourier Transform (or fft). The inverse Fourier
transform (sometimes called ifft) transforms the Fourier coefficients back to the f values
(transforming from spectral back to real space):
fft
f0 , f1 , f2 , · · · f2N −−→ A0 , A1 , · · · A2N
ifft
A0 , A1 , · · · A2N −−−→ f0 , f1 , f2 , · · · f2N

36
Exercise
Check the formulae in equations (3.3) and (3.4) by defining your own function, f , at 2N + 1
equally space points in [0, 2π). Use these function values to calculate the Fourier
coefficients from equation (3.3) then plug them back into equation (3.4) to check that you
retrieve the same function values at the original points. Please contact me if you have
problems as there could be errors in equations (3.3) and (3.4).

37

3.4 Differentiation and Interpolation

If we know the Fourier coefficients, Ak , of a function f then we can calculate the gradient of
f at any point, x: If
∞ ∞ ∞
f (x) = ∑ ak cos kx + ∑ bk sin kx = ∑ Ak eikx (3.5)
k=0 k=0 k=−∞
then
∞ ∞ ∞
f 0 (x) = ∑ −kak sin kx + ∑ kbk cos kx = ∑ i k Ak eikx . (3.6)
k=0 k=0 k=−∞
and the second derivative:
∞ ∞ ∞
f 00 (x) = ∑ −k2 ak cos kx − ∑ k2 bk sin kx = ∑ −k2 Ak eikx . (3.7)
k=0 k=0 k=−∞
These have spectral accuracy; the order of accuracy is as high as the number of points.
Similarly equation 3.1 or 3.2 can be used directly to interpolate f onto an undefined point, x.
Again, the order of accuracy is spectral.
3.5 Spectral Models
• ECMWF use a spectral model.
• The prognostic variables are transformed between physical and spectral space using ffts
and iffts.
• Gradients are calculated very accurately in spectral space

3.6 Wave Power and Frequency

• If a function, f , has Fourier coefficients, ak and bk , then wavenumber k has power a2k + b2k .
• A plot of wave frequency versus power is referred to as the power spectrum. Before we
learn how power spectra are used, we will have some revision questions...
38
 3.7 Recap Questions
1. In the Fourier decomposition
∞ ∞
a0
f (x) = + ∑ ak cos kx + ∑ bk sin kx
2 k=1 k=1

what are:
(a) the Fourier coefficients (the ak and the bk )
(b) the Fourier modes (the sines and cosines)
(c) the wavenumbers (or frequencies) (the ks)
(d) the power of a given wavenumber (a2k + b2k )
2. How would you describe the operation:
Z π Z π
1 1
ak = f (x) cos(kx)dx , bk = f (x) sin(kx)dx
π −π π −π
(a Fourier transform)
3. Given a list of 2N + 1 equally spaced samples of a real valued, periodic function, fn , how
would you describe the following operation to convert this into a list of N + 1 values:
1 N
Ak = ∑ fn e−iπknx/N
N n=−N
(a discrete Fourier transform)
4. What is the wavelength of a wave described by sin 4x (2π/4)

39

3.8 Analysing Power Spectra

Daily rainfall at a station in the Middle East for 21 years
60
daily rainfall
Fourier filtered using 40 wave numbers Observations about the
50
Truncated Fourier
40 filtered rainfall:
rainfall (mm)

30
• very smooth (only low
20
wavenumbers included)
10
• includes negative values
0
– “spectral ringing”
0 5 10 15 20
years

Power Spectrum of Middle East Rainfall

100

10-1
Observations
10-2
• dominant frequency at one year
power

10-3

10-4
(annual cycle)
10-5 • power at high frequencies (ie
10-6
daily variability)
10-1 100 101 102
Number per year

number per year = wavenumber×365/total number of days

40
Time Series of the Nino 3 sea surface temperature (SST)
The SST in the Nino 3 region of the equatorial Pacific is a diagnostic of El Nino
30
raw

How were the dashed

2 years and slower
29 annual cycle

lines generated?
28
• Annual cycle is the
Fourier mode at 1 year
SST (deg C)

27

26 • The “two years and

slower” filtered data is
25
the sum of all the
24 Fourier modes of these
frequencies.
23
1990 1995 2000 2005 2010

Power Spectrum of Nino 3 SST

Observations
1

0.1

• Dominant frequency at 1 year (annual

cycle)
0.01
power

• Less power at high frequencies (SST

0.001

0.0001
varies slowly)
1e−05
• Power at 1-10 years (El Nino every 3-7
1e−06 years)
0.01 0.1 1 10
Frequency (per year)

41

Time Series of the Quasi-Biennial Oscillation (QBO)

The QBO is an oscillation of the equatorial zonal wind between easterlies and westerlies in
the tropical stratosphere which has a mean period of 28 to 29 months:
20
equatorial wind (m/s)

10

-10

-20

-30
1950 1960 1970 1980 1990 2000 2010

Power Spectrum of QBO

8
10 Observations
107
106 • Dominant frequency at
105 close to 2 years
power

104 • Less power at high

103 frequencies
102
• Less power at long
101
time-scales
10-2 10-1 100 101
frequency (per year)

... Fourier decompositions will be used to mathematically analyse the behaviour of the
numerical schemes...
42
Chapter 4: Numerical Analysis of Advection Schemes
4.1 Introduction: What is Numerical Analysis and why is it done
• Numerical analysis involves mathematically analysing numerical methods in order to
predict how they will behave when they are used.
• Numerical analysis is important because
– Model development by trial and error is very time consuming
– We cannot test our models for every possible situation. We need evidence that they will
work for all situations
– We gain insight into how numerical methods work and so how to design better ones
• This chapter describes various types of analysis that are done for advection schemes for
atmospheric models

43

4.2 Some Definitions

1. Convergence: A finite difference scheme is convergent if the solutions of the scheme
converge to the solutions of the PDE as ∆x and ∆t tend to zero.
2. Consistency: A finite difference scheme is consistent with a PDE if the errors in
approximating all of the terms tend to zero as ∆x and/or ∆t tend to zero. (Terms of the
finite difference scheme are typically analysed using Taylor series.)
3. Order of accuracy: Error ∝ ∆xn (error is O(∆xn )) means scheme is nth order accurate.
Errors of an nth order scheme converge to zero with order n.
4. Stability: Errors do not tend to infinity for any number of time steps. Stability is typically
proved using von-Neumann stability analysis.
(a) Conditionally stable - if stable only for a sufficiently small time-step
(b) Unconditionally stable - if stable for any time-step
(c) Unconditionally unstable - if unstable for any time-step
5. Conservation: If, eg mass, energy, potential vorticity are conserved by the PDEs, are
they conserved by the numerical scheme?
6. Boundedness: If the initial conditions are bounded between values a and b then a
bounded solution will remain bounded between a and b for all time.
7. Monotonicicity: Monotone schemes do not generate new extrema or amplify existing
extrema. If the initial conditions are monotonic then they will remain monotonoic after
the action of a monotonic numerical scheme.

44
Based on your knowledge of FTCS and based on the numerical solutions below, which of
properties 2-7 apply to the numerical solution of the linear advection equation using FTCS?

1.4 1.4
Initial conditions, mean = 0.5 Exact, mean = 0.5
1.2 1.2 Numerical after 20 time steps, mean = 0.5
1.0 1.0
0.8 0.8
0.6 0.6
φ

φ
0.4 0.4
0.2 0.2
0.0 0.0
0.20.0 0.2 0.4 0.6 0.8 1.0 0.20.0 0.2 0.4 0.6 0.8 1.0
x x

1.4 1.4
Exact, mean = 0.5 Exact, mean = 0.5
1.2 Numerical after 40 time steps, mean = 0.5 1.2 Numerical after 60 time steps, mean = 0.5
1.0 1.0
0.8 0.8
0.6 0.6
φ

0.4 0.4
0.2 0.2
0.0 0.0
0.20.0 0.2 0.4 0.6 0.8 1.0 0.20.0 0.2 0.4 0.6 0.8 1.0
x x
Consistent Yes Order of Accuracy 1st Stable No
Conservative Yes Bounded No Monotone No
45

How about when modifying all out of range values to be one or zero at every time-step?

1.4 1.4
Initial conditions, mean = 0.5 Exact, mean = 0.5
1.2 1.2 Numerical after 20 time steps, mean = 0.497688501049
1.0 1.0
0.8 0.8
0.6 0.6
φ

0.4 0.4
0.2 0.2
0.0 0.0
0.20.0 0.2 0.4 0.6 0.8 1.0 0.20.0 0.2 0.4 0.6 0.8 1.0
x x

1.4 1.4
Exact, mean = 0.5 Exact, mean = 0.5
1.2 Numerical after 40 time steps, mean = 0.497563296064 1.2 Numerical after 60 time steps, mean = 0.498957455923
1.0 1.0
0.8 0.8
0.6 0.6
φ

0.4 0.4
0.2 0.2
0.0 0.0
0.20.0 0.2 0.4 0.6 0.8 1.0 0.20.0 0.2 0.4 0.6 0.8 1.0
x x
Consistent Probably Order of Accuracy ? Stable Yes
Conservative No Bounded Yes Monotone No

46
4.3 Convergence and errors
If an analytic solution, f (x,t), to a PDE is known, then errors of a numerical solution,
φ (x,t), can be calculated as functions of space and time. Errors over space can be
summarised by integrating over space to calculate error norms:
∑ j ∆x|φ j − f (x j ) |
`1 (φ ) =
∑ j ∆x| f (x j ) |
q
2
∑ j ∆x (φ j − f (x j ))
`2 (φ ) = q
2
∑ j ∆x f (x j )
max |φ j − f (x j ) |
`∞ (φ ) = .
max | f (x j ) |
If a numerical scheme is described as nth order accurate, then these error metrics should
converge to zero at a rate proportional to ∆xn :
`i ∝ ∆xn .

47

4.4 Lax-Equivalence Theorem

The Lax equivalence theorem is the fundamental theorem in the analysis of finite difference
methods for the numerical solution of partial differential equations. It states that for a
consistent finite difference method for a well-posed linear initial value problem, the method
is convergent if and only if it is stable:
consistency + stability ⇐⇒ convergence
So if you can show that the finite difference approximations for each of the terms is at least
first-order accurate and you can show that the scheme is stable, then you know that solutions
to the finite difference scheme will converge to solutions of the PDE.
This is why we study order of accuracy and stability.

48
4.5 Domain of Dependence
The domain of dependence of the solution of a PDE at position x and at time t is the set of
points at a previous time that influence the solution at position x and at time t.
Let us consider the 1D linear advection equation for dependent variable φ and advecting
velocity u:
φt + uφx = 0

Remember the analytic solution:

φ (x,t) = φ (x − ut, 0)
Draw the domain of dependence of x = 12m, t = 8s on the graph, assuming u = 1.5m/s:
t

0
0 2 4 6 8 10 12 x
49

4.5.1 Courant-Friedrichs-Lewy (CFL) criterion:

The domain of dependence of the numerical solution should include the domain of
dependence of the original PDE.
• The CFL criterion is necessary but not sufficient
• For linear advection, the domain of dependence of the differential equation at ( j∆x, n∆t)
is the straight line of slope 1/u through ( j∆x, n∆t) for t ≤ n∆t in the (x,t) plane.

4.5.2 Domain of Dependence of FTBS

(n+1) (n) (n) (n)
FTBS: φj = φ j − c(φ j − φ j−1 )
(n) (n−1) (n−1) (n−2) (n−2) (n−2)
φj depends on φ j and φ j−1 . In turn, these depend on φ j , φ j−1 and φ j−2 .
(n+1)
Exercise: Mark the dots that make up the domain of dependence of φ j for FTBS. Draw
lines corresponding to the real (physical) domain of dependence for cases when
c = −1, 0, 1, 2. What can you deduce?
t

n+1

n−1

n−2

x
j−3 j−2 j−1 j j+1

The numerical domain of dependence contains the physical domain of dependence when
0 ≤ c ≤ 1 so FTBS is unstable for c > 1 and c < 0. But we cannot say if FTBS is ever stable.
50
4.5.3 The Domain of Dependence of the CTCS Scheme

(n+1) (n−1) (n) (n)

CTCS: φj = φj − c(φ j+1 − φ j−1 )

(n+1)
Draw the domain of dependence of φ j for CTCS.
t

n+1

n−1

n−2

x
j−3 j−2 j−1 j j+1

For what values of c will CTCS be unstable? c < −1 and c > 1

Except at the initial time, the solution is found on two sets of points that are not coupled.
The solution can oscillate between two unrelated solutions. This is a manifestation of the
computational mode of CTCS.
The domain of dependence and the CFL criterion can tell us when some schemes are
unstable. How about proving stability? ...

51

4.6 Von-Neumann Stability Analysis

• Assume that the solution at time level n + 1 can be represented as an amplification factor,
A, multiplied by the solution at time-level n:
φ (n+1) = Aφ (n)
• The amplification factor will tell us the following about the numerical method:
|A| < 1 ∀ k, ∆x stable and damping
|A| = 1 ∀ k, ∆x neutrally stable
|A| > 1 for any k, ∆x unstable (amplifying)
Where |A|2 = AA∗ (A multiplied by its complex conjugate).
• For linear advection A should be complex since the solution changes location every
time-step. So how are we going to find A? ...
• The solution of a PDE in one spatial dimension can be expressed as a sum of Fourier
modes:
∞
φ= ∑ Ak eikx (4.1)
k=−∞

each with wavenumber k.

• Consider the stability of a solution for individual wavenumbers
• For a uniform grid, x = j∆x
= An eik j∆x into the equation for a linear numerical scheme.
(n)
• Substitute φ j
• Rearrange to give an equation for the amplification factor, A(k, ∆x, ∆t)

52
4.6.1 Von-Neumann Stability Analysis of FTBS
FTBS for linear advection is:
(n+1) (n) (n) (n)

φj = φ j − c φ j − φ j−1 (4.2)

= An eik j∆x :
(n)
Substitute in φ j

An+1 eik j∆x = An eik j∆x − cAn (eik j∆x − eik( j−1)∆x ) (4.3)
Cancel powers of An eik j∆x and rearrange to find A in terms of c and k∆x:
A = 1 − c(1 − e−ik∆x ) (4.4)
We need to find the magnitude of A so we need to write it down in real and imaginary form.
So substitute e−ik∆x = cos k∆x − i sin k∆x:
A = 1 − c(1 − cos k∆x) − ic sin k∆x (4.5)
and calculate |A|2 = AA∗ (A multiplied by its complex conjugate):
|A|2 = 1 − 2c(1 − cos k∆x) + c2 (1 − 2 cos k∆x + cos2 k∆x) + c2 sin2 k∆x
=⇒ |A|2 = 1 − 2c(1 − c)(1 − cos k∆x)
We need to find for what value of ∆t or c is |A| ≤ 1 in order to find when FTBS is stable:
|A| ≤ 1 ⇐⇒ |A|2 − 1 ≤ 0
⇐⇒ −2c(1 − c)(1 − cos k∆x) ≤ 0
⇐⇒ c(1 − c)(1 − cos k∆x) ≥ 0

53

We know that 1 − cos k∆x ≥ 0 for all k∆x so FTBS is stable when
c(1 − c) ≥ 0 ⇐⇒ 0 ≤ c ≤ 1.
u∆t
We have proved that FTBS is unstable if u < 0 or if > 1. We will now define:
∆x
Upwind scheme FTBS when u ≥ 0
FTFS when u < 0
The upwind scheme is first order accurate in space and time, conditionally stable and
damping.
4.6.2 What should A be for real linear advection?
For initial conditions consisting of a single Fourier mode: φ (x, 0) = Ak eikx , the solution at
time t is:
φ (x,t) = Ak eik(x−ut) .
This can be represented as a factor times eikx :
φ (x,t) = Ak e−ikut eikx
So if t = n∆t we have An = e−ikut = e−ikun∆t . Therefore A = e−iku∆t which has |A| = 1 and
so, under the influence of the linear advection equation, waves should not amplify or decay.

54
4.6.3 Von Neumann Stability Analysis of CTCS
The amplification factor, A, is found by substituting φ j = An eik j∆x into the equation for
(n)

CTCS:
(n+1) (n−1) (n) (n)

φj = φj − c φ j+1 − φ j−1 . (4.6)
This gives:
An+1 eik j∆x − An−1 eik j∆x + c(An eik( j+1)∆x − An eik( j−1)∆x ) = 0
=⇒ A2 + (2ic sin k∆x)A − 1 = 0
p
=⇒ A = −ic sin k∆x ± 1 − c2 sin2 k∆x
There are two cases to consider:
(i) |c| ≤ 1 =⇒ c2 sin2 k∆x ≤ 1
=⇒ |A|2 = 1 − c2 sin2 k∆x + c2 sin2 k∆x = 1
=⇒ The solution is stable and not damping
(ii) |c| > 1 =⇒ c2 sin2 k∆x > 1 for some k∆x
p
=⇒ |A|2 = (c sin k∆x ± c2 sin2 k∆x − 1)2
=⇒ At least one of the roots has |A| > 1. The solution is unstable
So CTCS is conditionally stable: it is stable for |c| ≤ 1
Regardless of c, there are always two possible values of A. This means that there will always
be two possible solutions; a realistic solution (the physical mode) and the un-realistic,
oscilating solution; the spurious computational mode. This will contaminate the solution
and behave in an un-realistic way.
55

4.7 Conservation
For a property, φ , advected with velocity, u: (φt + uφx = 0) we can show that the total mass
of φ is conserved. Define the total mass to be
Z 1
M= φ dx
0
and assume that φ has periodic boundary conditions so that φ (0,t) = φ (1,t) ∀t. Then:
Z 1  Z1 Z 1 Z 1
dM d dφ dφ
= φ dx = dx = − u dx = −u dφ = −u [φ ]10 = 0.
dt dt 0 0 dt 0 dx 0
since φ (0,t) = φ (1,t). Therefore M is conserved.
Is M conserved by a numerical scheme? Consider FTBS advection:
(n+1) (n) (n) (n)

φj = φ j − c φ j − φ j−1 .

From this we can calculate M (n+1) as a function of M (n) : !

nx nx 
 nx nx
M (n+1) = ∑ ∆xφ j = ∆x ∑ φ j − c φ j − φ j−1 = M (n) − c∆x ∑ φ j − ∑ φ j−1
(n+1) (n) (n) (n) (n) (n)

j=1 j=1 ! j=1 j=1

nx nx −1  
= M (n) − c∆x ∑ φ j − ∑ φ j
(n) (n) (n) (n)
= M (n) − c∆x φnx − φ0 = M (n)
j=1 j=0
(n) (n)
due to the periodic boundaries since φnx = φ0 . Therefore M (n+1) = M (n) which means that
mass is conserved.

56
4.7.1 Conservation of higher moments
We can show that the variance of φ is also conserved under linear advection. The variance is:
Z 1
V= φ 2 dx − M 2 .
0
Then we can calculate the rate of change of variance under linear advection, assuming that
Mt = 0:
Z 1 Z 1 Z 1 Z 1
dV dφ 2 dφ dφ  1
= dx = 2φ dx = − 2φ u dx = −2u φ dφ = −u φ 2 0 = 0.
dt 0 dt 0 dt 0 dx 0
Is V conserved by a numerical scheme? Consider FTBS advection:
(n+1) (n) (n) (n)

φj = φ j − c φ j − φ j−1 .

From this we can calculate V (n+1) as a function of V (n) (ignoring M for brevity):
nx   nx  
(n+1) 2 (n)
2
V (n+1) = ∑ ∆x φ j = ∆x ∑ φ j − c φ j − φ j−1
(n) (n)

j=1 j=1
nx  nx nx nx  
(n) 2 (n) 2
= ∆x ∑ φ j − 2c∆x ∑ φ j φ j + 2c∆x ∑ φ j φ j−1 + c2 ∆x ∑ φ j − φ j−1
(n) (n) (n) (n) (n)

j=1 j=1 !j=1 j=1

nx
= V (n) − 2c(1 − c) V (n) − ∆x ∑ φ j φ j−1
(n) (n)

j=1
The term multiplying 2c(1 − c) is always greater than zero and so the variance always
decreases when using FTBS.

57

4.8 Exercises: Analysis of Advection Schemes (answers on blackboard)

1. Derive expressions for the amplification factors for the following advection schemes:
(a) FTCS
(b) BTCS
(c) CTBS
2. Which of the above schemes suffer from a spurious computational mode?
3. Sketch of the domain of dependence for these schemes
4. For FTCS and BTCS
(a) Find for what Courant numbers the schemes are stable.
(b) Are the schemes damping, amplifying or neutral?
5. Determine if mass is conserved by CTCS

58
Chapter 5: Waves, Dispersion and Dispersion Errors
Inaccurate dispersion of numerical methods is a common source of errors. Therefore we will
learn about dispersion and dispersion errors of numerical methods.
5.1 Some Background on Waves
A travelling wave can be described by the equation

y = a sin (kx − ωt) (5.1)

where y(x,t) is the height of the wave at position x, time t

a is the amplitude of the wave
k is the wavenumber (number of whole waves between 0 and 2π)
ω is the angular wave frequency – the number of complete oscillations in
time 2π at a fixed point
59

60
How do variations in k and ω influence the wave and its propagation according to the
equation
y = a sin (kx − ωt)

Exercise:
Write down an expression for the wave length, λ , and the wave speed, u in terms of k and ω
λ = 2π/k, u = ω/k

61

5.2 Dispersion
Dispersion occurs when waves of different frequencies propagate at different speeds.

ω1 ω2
• u= = so wave of both wavelengths propagate at the same speed
k1 k2
• No dispersion occurs – the function does not change shape

62
 ω1 ω2
• u1 = = 10 so the short wavelength waves are much slower than the long waves
k1 k2
• Dispersion occurs – the function changes shape

63

ω1 1 ω2
• u1 = = so the short wavelength waves travel more quickly
k1 3 k2
• Dispersion occurs – the function changes shape

64
For a function to propagate without changing shape, all of the Fourier modes must
propagate at the same speed:
This shows the propagation of a Gaussian and propagation of the first 9 Fourier modes.

65

Propagation of a Gaussian with high wavenumber waves propagating more slowly

The original Gaussian function changes shape

66
5.3 Some Excellent Animations and Videos which demonstrate
Dispersion
• Some description and animations from Dr. Dan Russell, Grad. Prog. Acoustics, Penn
State:
http:
//www.acs.psu.edu/drussell/Demos/Dispersion/dispersion.html
• A video of the wake of a motor-boat from JNHeyman (41 seconds):
https://www.youtube.com/watch?v=lWi_KpBy8kU
• A video of ripples in a pond with descriptions of the dispersion (2:28)
https://www.youtube.com/watch?v=dESm6VjfSNs

67

5.4 Phase speed of linear advection

We know:
• the one-dimensional linear advection equation:
φt + uφx = 0
• its analytic solution:
φ (x,t) = φ (x − ut, 0)
• under the action of the linear advection equation, waves propagate with phase speed u.
(linear advection is non-dispersive since u does not depend on k)
• If we multiply a single Fourier mode, eikx , by e−iku∆t then that mode will move a distance
. . . u∆t . . . around the x-axis
• The amplification factor for exact linear advection is A = e−iku∆t
• Therefore if a numerical method has amplification factor re−iθ for wavenumber k then the
phase speed of the numerical waves of wavenumber k will be ... θ /(k∆t)
• If the phase speed is dependent on k then we will get numerical dispersion

68
5.5 Phase speed and dispersion errors of CTCS
The amplification factor of CTCS is
p
A = −ic sin k∆x ± 1 − c2 sin2 k∆x
where c = u∆t/∆x. Therefore we can find the phase-speeds of the numerical waves relative
to the analytic waves:
!
un 1 −1 c sin k∆x
= tan p
u uk∆t ± 1 − c2 sin2 k∆x
This can be simplified by substituting in u∆t = c∆x and sin α = c sin k∆x:
un α
=±
u ck∆x
There are two possible phase-speeds for each mode. These can be plotted against k∆x to find
out how waves propagate when advected by CTCS. A plot of un /u against k∆x (or ω = ku
against k) is called a dispersion relation. The dispersion relation for CTCS for c = 0.4:

1.0

0.5

0.0
un /u

0.5

1.00 π/4 π/2 3π/4 π

k ∆x
69

The dispersion relation for CTCS for c = 0.4 plotted as ω against k∆x

1.5
numerical
1.0 true
0.5

0.0
ωn /u

0.5

1.0

1.50 π/4 π/2 3π/4 π

k ∆x
What does this tell us about CTCS?
• There are two possible solutions:
– the physical mode (u p /u ≥ 0)
– the computational mode (u p /u < 0)
• The computational mode propagates in the wrong direction
• All waves propagate too slowly
• For the physical mode, when waves are well resolved (small k or small ∆x), they
propagate at nearly the correct speed (this is why it is called the physical mode)
• Grid-scale waves (k∆x = π) do not propagate

70
5.6 Exercise
What can dispersion relations tell us about the behaviour of numerical methods? These are
the results of solving the linear advection equation using two finite volume methods. The
generic finite-volume method for linear advection in 1d for a uniform grid is define as:
(n+1) (n)
φj −φj φ j+1/2 − φ j−1/2
= −u
∆t ∆x
The two finite-volume methods are:
Lax-Wendroff Warming and Beam
1 1
φ j+1/2 = /2 (1 + c)φ j + /2 (1 − c)φ j+1 φ j+1/2 = 1/2 (3 − c)φ j − 1/2 (1 − c)φ j−1
Advection of a top-hat profile to the right using c = 0.2, 100 points

71

These are the dispersion relations for the two schemes. Which dispersion relation is related
to which scheme and why?

3.5
3.0 Scheme a
Scheme b
2.5 true
2.0
ω/u

1.5
1.0
0.5
0.00 π/4 π/2 3π/4 π
k ∆x

72
Chapter 6: Alternative Advection Schemes
Some problems we have encountered with advection schemes:
• First-order in space schemes are diffusive
• Second-order linear schemes suffer from dispersion errors which contaminate solutions
with grid-scale oscillations
• Explicit Eulerian schemes have time-step restrictions

These are consistent with Godunov’s theorem:

Linear numerical schemes for solving partial differential equations, having the property of
not generating new extrema (monotone scheme), can be at most first-order accurate.
We will therefore present some alternatives:
1. Semi-Lagrangian advection
2. Artificial diffusion to remove spurious oscillations
3. The Finite Volume Method
4. Lax-Wendroff and Warming and Beam
5. Total variation diminishing (TVD) schemes

73

6.1 Semi-Lagrangian Advection

• Explicit without time-step restrictions associated with c = u∆t/∆x
• Used by the UK Met Office and ECMWF

• To calculate a new value of a

dependent variable, look upwind
to where it has come from

74
6.1.1 Semi-Lagrangian Advection in One Dimension
CFL criterion: The domain of dependence of the numerical solution must contain the
domain of dependence of the original PDE.
• So to allow long time-steps, construct the numerical domain of dependence to contain the
xj
physical domain of dependence. t n+1
∂φ
• Recall the linear advection equation: ∂t + u ∂∂φx = 0
u
• and its analytic solution: φ (x,t) = φ (x − ut, 0)
• Semi-Lagrangian advection is defined from this:
φ (x j ,tn+1 ) = φ (x j − u∆t,tn ) = φ (x jd ,tn )
x jd = x j − u∆t is the departure point of point x j .
tn x jd
• So interpolate φ from known points onto x jd . xk x k+1 x

• First find k such that xk ≤ x jd ≤ xk+1 : (Use floor(x) meaning the integer below or at x)
k = floor((x j − u∆t)/∆x) = floor( j − c)
• Interpolate from xk−1 , xk , xk+1 , ... onto x jd (for example using cubic-Lagrange interpolation)

Exercise: Find β = (x jd − xk )/∆x as a function of only j, k and c given x j = j∆x and

c = u∆t/∆x. β = j − k − c
Advantage: Explicit scheme without Courant number restriction
Problem: The advected quantity is not conserved
Order of Accuracy: Ignoring errors in calculating the departure point, semi-Lagrangian
should achieve an order of accuracy of O[(∆x) p+1 /∆t] where p is the order of the
interpolation [1].

75

6.2 Artificial Diffusion to Remove Spurious Oscillations

Numerical schemes are designed not to produce spurious oscillations. However, once a
forecasting model is put together, often spurious oscillations are still generated. These can
be removed by adding an artificial diffusion term to the equations. For example, the linear
advection equation with diffusion:
∂φ
+ u • ∇φ − K∇2 φ = 0 (6.1)
∂t
or in one dimension:
∂φ ∂φ ∂ 2φ
+u −K 2 = 0 (6.2)
∂t ∂x ∂x

• This is dampens spurious oscillations and real features

• It is only (but frequently) used as a last resort.
• More scale-selective filtering can be achieved using ∇4 rather than ∇2 :
∂φ
+ u • ∇φ + K∇4 φ = 0 (6.3)
∂t
Stablity limits of the advection-diffusion scheme
The advection diffusion scheme:
(n+1) (n−1) (n) (n) (n−1) (n−1) (n−1)
φj −φj + c(φ j+1 − φ j−1 ) − 2d(φ j+1 − 2φ j + φ j−1 ) = 0 (6.4)
where c = u∆t/∆x and the diffusion number, d = K∆t/∆x2 has the stability constraint
c2 + 4d ≤ 1. The algebra is complicated and does not need to be reproduced.
Questions: How much diffusion is needed to control oscillations? How much does this
reduce accuracy?
76
6.3 The Finite Volume Method
The volume integrals of predicted variables (eg φ ) are calculated on small volumes (cells
with volume V ) by calculating the quantities entering and leaving the cell:
  4
d
V φ = ∑ fi
dt i=1
f4 Where fi is the flux of φ through edge i.
4 The finite volume method can also be derived by
1
Vφ
3
discretising the flux form of the advection
f1 f3 equation. Constant density flow is divergence free
2
(∇ · u = 0) and so two forms of the linear
f2 advection equation are equivalent:
∂ φ /∂t + u · ∇φ = 0 advective form
∂ φ /∂t + ∇ · (u φ ) = 0 flux form
The flux (or conservative) form can be discretised using Gauss’s divergence theorem:
Z Z
1 1 1
∇ · uφ ≈
V V
∇ · uφ dV =
V S
φ u · dS =
V ∑ fi
i

where volume V has surface S, dS is the outward pointing normal to surface S and
fi = φ u · dS is the flux over edge i.
Conservation: the finite volume method is conservative: however fi is calculated, the total
mass of φ is conserved because exactly what leaves one cell will enter another.
Finite volume is a family of methods. There are numerous approximations for estimating fi
from surrounding φ and u values.

77

6.3.1 The Finite Volume Method in one dimension, for constant u

For constant u: f j+1/2 = uφ j+1/2 .

φ j−2 φ j−1 φj φ j+1 φ j+2
So to solve: φt = −uφx we use
φ j−1/2 φ j+1/2
∂φj φ j+1/2 − φ j−1/2
= −u
∂t ∆x
It remains to estimate φ j±1/2 from
surrounding values of φ j .
Exercise Demonstrate the following:
∂φj
• If we choose centred in time for and
∂t
1

φ j+1/2 = φ j + φ j+1 we recover the
2
CTCS finite difference scheme
∂φj
• If we choose forward in time for and
∂t
φ j+1/2 = φ j we recover the FTBS finite
difference scheme
Solution on notes on blackboard.
Using the finite volume method, we can solve equations on arbitrary meshes.

78
6.4 Lax-Wendroff
We start with the finite volume method:
φj
(n+1) (n)
−φj φ j+1/2 − φ j−1/2 φ j+1
= −u .
∆t ∆x
For Lax-Wendroff, φ j+1/2 is calculated to be the
average value that passes through position x j+1/2 φj
between time-steps n and n + 1. So you could call it
(n+1/ )
φ j+1/ 2 . Therefore we need to calculate the average
2
value of φ between positions x j+1/2 and a distance
u∆t upstream of x j+1/2 . In order to calculate the xj x j+1 /2 x j+1
average, we assume that φ varies linearly between x j u∆t
and x j+1 . Then we can calculate φ at x j+1/2 − 1/2 u∆t .
 
(n+1/ )
This could be written φ j+1/ 2 = φ (n) x j+1/2 − 1/2 u∆t .
2

Exercise: Using the Courant number, c = u∆t/∆x, show that this gives:
(n+1/2 )
= 1/2 (1 + c)φ j + 1/2 (1 − c)φ j+1 (solution on blackboard notes)
(n) (n)
φ j+1/
2
Notes:
• Lax-Wendroff is second-order accurate in space and time.
• Lax-Wendroff is not monotonic or bounded
• Stable for |c| ≤ 1

79

Lax-Wendroff Warming and Beam

1 1
φ j+1/2 = /2 (1 + c)φ j + /2 (1 − c)φ j+1 φ j+1/2 = 1/2 (3 − c)φ j − 1/2 (1 − c)φ j−1
Advection of a top-hat profile to the right using c = 0.2, 100 points

Smooth ahead of the discontinuity Smooth behind the discontinuity

Remember Godunov’s theorem:
Linear numerical schemes for solving partial differential equations, having the property of
not generating new extrema (monotone scheme), can be at most first-order accurate.
• So if we want a monotone, second-order scheme it must be ... non-linear
• Warming and Beam is also second-order accurate in space and time
• So why not combine Lax-Wendroff and Warming and Beam based on local gradients?
• By making the scheme based on the gradient of φ , the scheme is non-linear.

80
6.5 Total Variation
• Linear, second-order advection schemes produce unbounded, unrealistic, grid-scale
oscillations. These can be measured by the total variation:
nx −1
TV = ∑ |φ j+1 − φ j |
j=0
Exercise: Calculate the total variation of these functions:
7 7 7
(a) (b) (c)
6 6 6
5 5 5
4 4 4
φ

φ
3 3 3
2 2 2
1 1 1
00 2 4 6 8 10 00 2 4 6 8 10 00 2 4 6 8 10

7 7 7
(d) (e) (f)
6 6 6
5 5 5
4 4 4
φ

φ
3 3 3
2 2 2
1 1 1
00 2 4 6 8 10 00 2 4 6 8 10 00 2 4 6 8 10

(a) 8, (b) 8, (c) 12, (d) 8, (e) 12, (f) 6

• A total variation diminishing (TVD) scheme has TV (n+1) ≤ TV (n) .

Question: Why is total variation used rather than boundedness to measure the generation of
spurious oscillations? ...Because spurious oscillations can be generated within the bounds of
the original function. Eg see function (e) above.
81

6.6 Total Variation Diminishing (TVD) schemes

• A total variation diminishing (TVD) scheme has TV (n+1) ≤ TV (n)
• First-order upwind is the only linear TVD scheme. Other TVD schemes are non-linear...
• We start with the linear advection equation discretised in conservative (flux) form:
(n+1) (n)
φj −φj φ j+1/2 − φ j−1/2
= −u
. (6.5)
∆t ∆x
• Each φ j+1/2 , is calculated as a weighted average of a high order flux (φH ) and a low order
flux (φL ):
φ j+1/2 = Ψ j+1/2 φH + (1 − Ψ j+1/2 ) φL
where Ψ is a limiter function.
• Use as much of φH as possible without introducing oscillations
• So Ψ should be close to one where the solution is smooth so that the solution is close to
second-order accurate and Ψ close to zero where the solution changes rapidly so as to use
the upwind flux which guarantees boundedness.
• The scheme is now non-linear since Ψ depends on φ .
• We can use:
1
– Lax-Wendroff as the high-order flux: φH = (/2 (1 + c)φ j + 1/2 (1 − c)φ j+1
φj if u ≥ 0
– First-order upwind as the low-order flux: φL =
φ j+1 otherwise
• So what should Ψ be?

82
6.6.1 Limiter functions
The limiter function, Ψ, is based on the ratio of the upwind gradient to the local gradient:
φ j − φ j−1
r j+1/2 =
φ j+1 − φ j
for u > 0.
6.6.1.1 Exercise
Warming and Beam (WB) is not a TVD scheme but the flux,
φ j+1/2 = 1/2 (3 − c)φ j − 1/2 (1 − c)φ j−1 can be expressed in the form:
φ j+1/2 = Ψ j+1/2 φH + (1 − Ψ j+1/2 )φL

where φH is Lax-Wendroff (φH = 1/2 (1 + c)φ j + 1/2 (1 − c)φ j+1 ) and φL is backward in space
(φL = φ j ). If WB is expressed in this form, what is Ψ j+1/2 as a function of r j+1/2 ? (Hint,
equate the two forms for φ j+1/2 and then equate coefficients of either c0 or c1 )

φ j − φ j−1
⇒ Ψ j+1/2 = = r j+1/2
φ j+1 − φ j
83

6.6.2 The Sweby Diagram

The Sweby diagram shows the limiter function, Ψ, as a function of the upwind to local
gradient, r. It is used to find limiter functions that give TVD advection schemes.

1. Draw lines of Ψ for Lax-Wendroff (LW)

and Warming and Beam (WB): 4
2. Shade the region outside the two lines.
The unshaded region shows the convex 3
combination of LW and WB. If Ψ takes a
value in the unshaded region then the 2
Ψ

scheme will be 2nd-order accurate

because LW and WB are 2nd-order 1
3. Draw lines of Ψ = 2 and Ψ = 2r. Sweby
(SIAM J. Numer. Anal. 1984) showed that 01 0 1 2 3 4 5
if Ψ ≤ 2 and Ψ ≤ 2r then the scheme is r
TVD
4. Shade the region where Ψ > 2 and Ψ > 2r.
The unshaded region now is the 2nd-order TVD region. If Ψ lies in this region then the
scheme will be 2nd-order TVD
6.6.3 The Van Leer Limiter
A popular limiter function is due to Van Leer: Ψ(r) = (r + |r|) / (1 + |r|)
Sketch this function on the Sweby diagram.
There are many other limiter functions that give TVD schemes. See, eg
https://en.wikipedia.org/wiki/Flux_limiter
84
Chapter 7: Modelling Wave Equations
Many of the processes in the atmosphere are represented by the shallow water equations
(SWE). The assumptions needed to derive the SWE are:
• Horizontal length scale >> vertical length scale
• Very small vertical velocities

Depth integrate the Navier-Stokes equations over orography to give the SWE:
Free surface
Du
= −2Ω × u − g∇(h + h0 ) + µu ∇2 u (7.1) u fluid
Dt depth, h
Dh
+ h∇ · u = 0 (7.2) h0
Dt
Solid surface
where

u Depth intetraged wind vector g Acceleration due to gravity

t Time ∇ Gradients in the horizontal
Ω Rotation rate of planet h0 Height of the bottom topography
h Fluid depth µu Diffusion of momentum

Exercise: Considering the meaning of the terms in the momentum equation in section 1.3,
what are the meaning of the terms of the momentum equation of the SWE?

85

7.1 Simulations of the SWE on the surface of a sphere

The shallow-water
equations can be
solved on the surface
of a sphere with u
being the horizontal
wind (ignoring
updrafts and
downdrafts) and h
being the depth of a
layer of atmosphere.
The results look
similar to large-scale
atmospheric
circulation. The
vectors show u, the
black contours show a
mountain (h0 ) and
colours show h + h0 .

5000 5100 5200 5300 5400 5500 5600 5700 5800 5900 6000

86
7.2 Processes Represented by the SWE
Which of these processes are represented by the SWE and which are only represented by the
full NS equations?
Horizontal advection SWE Acoustic waves NS
Vertical advection NS Coriolis SWE
Gravity waves SWE Diffusion SWE
Rossby waves SWE Heat transport NS
Adiabatic expansion NS Atmospheric convection NS
Geostrophic balance SWE Geostrophic turbulence SWE

87

Component Form of the SWE

Assuming u = (u, v, 0)T and 2Ω = (0, 0, f )T , equations (7.1) and (7.2) written in component
form are:
 2 
∂u ∂u ∂u ∂ (h + h0 ) ∂ u ∂ 2u
+u +v = fv−g + µu +
∂t ∂x ∂y ∂x ∂ x2 ∂ y2
 2 
∂v ∂v ∂v ∂ (h + h0 ) ∂ v ∂ 2v
+u +v = −fu−g + µu +
∂t ∂x ∂y ∂y ∂ x2 ∂ y2
 
∂h ∂h ∂h ∂u ∂v ∂ h ∂ hu ∂ hv
+u +v +h + = 0 or + + =0
∂t ∂x ∂y ∂x ∂y ∂t ∂x ∂y

88
7.2.1 Linearised SWE
In order to find analytic solutions and to analyse numerical methods, we linearise the SWE.
Assume:
• u = (u, v, 0)T is small
• 2Ω = (0, 0, f )T
• h = H + h0 where H is uniform in space and time and h0 is small
• the product of two small variables is ignored (even if one or both are inside a differential)
• h0 and µu are ignored

This gives the following equations for u,v and h0 expressed in terms of f (rather than Ω):
∂u ∂ h0
= fv−g (7.3)
∂t ∂x
∂v ∂ h0
= −fu−g (7.4)
∂t ∂y
 
∂ h0 ∂u ∂v
= −H + (7.5)
∂t ∂x ∂y
7.3 Analytic Solultion
Ignoring Coriolis, the linearised SWE have wave-like solutions – gravity waves. In 1d these
are: √
h0 = H eikx e±ikt gH (7.6)
p √
u = ± g/H H eikx e±ikt gH (7.7)
p
for any constant H. So waves
p with wavenumber k in space oscillate with frequency k gH
and the wave speed is ... gH (so gravity waves are non-dispersive).
89

7.4 Unstaggered Forward-Backward (1d A-grid FB)

As there are two equations that depend on each other, it is quite natural to solve them using
forward-backward time-stepping – forward for u and backward for h. We will also start by
assuming that h and u are defined at the same spatial positions (this is called co-located,
unstaggered or A-grid) and we will use centred spatial discretisation:
(n+1) (n) (n) (n)
∂u ∂h uj −uj h j+1 − h j−1
= −g → = −g (7.8)
∂t ∂x ∆t 2∆x
(n+1) (n) (n+1) (n+1)
∂h ∂u hj −hj u j+1 − u j−1
= −H → = −H (7.9)
∂t ∂x ∆t 2∆x
(n) (n)
where x j = j∆x, t (n) = n∆t, h j = h(x j ,t (n) ) and u j = u(x j ,t (n) ).

90
7.4.1 Von-Neumann Stability Analysis
We can find the stability limits and dispersion relation for the numerical scheme given in
section 7.4 (1d A-grid FB) using von-Neumann stability analysis.
To calculate an amplification factor, A, for each wavenumber, k, we assume wave-like
solutions for h and u:
h j = H An eik j∆x
(n)
(7.10)
u j = U An eik j∆x
(n)
(7.11)
for some constants
√ H and U. Substituting these into (7.8) and (7.9) and defining the Courant
gH∆t
number c = leads to:
∆x
c2 ic p
A = 1 − sin2 k∆x ± sin k∆x 4 − c2 sin2 k∆x (7.12)
2 2
There are two solutions for A but this is correct because there are also two analytic solutions
to the equations (because of the ± in the analytic solution). For |c| ≤ 2 this gives |A|2 = 1
so the scheme is stable and undamping for sufficiently small time steps. However for |c| > 2
we have:  2
c2 2 c p
2 2
|A| = 1 − sin k∆x ± sin k∆x c2 sin k∆x − 4
2 2
p
which can be greater than 1 and so the scheme is unstable for |c| > 2 where c = gH∆t/∆x.
So this scheme is conditionally stable. Stable for c ≤ 2.

91

7.4.2 Dispersion of Unstaggered Forward-Backward (1d A-grid FB)

A reminder of the amplification factor for this method:
c2 ic p
A = 1 − sin2 k∆x ± sin k∆x 4 − c2 sin2 k∆x.
2 2
The argument of A gives us the wave frequency as a function of wavenumber:
p
1 c
sin k∆x 4 − c2 sin2 k∆x
ω = ± tan−1 2 (7.13)
∆t 2
1 − c sin2 k∆x
2
c
This can be simplified by assuming that sin k∆x = sin α to give:
2
2α 2 c

ω =± = ± sin−1 sin k∆x (7.14)
∆t ∆t 2
This is the A-grid dispersion relation:
3.5
3.0 exact
A-grid
2.5
2.0
ω ∆x

Grid-scale gravity waves (k∆x/π = 1) have 1.5

zero frequency! This is highly unrealistic.
1.0
0.5
0.00.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5
k ∆x
92
7.5 Problems with Co-location of h and u
Consider the following initial conditions of the linearised non-rotating SWE:

h0 u u=0

x x

Questions:
1. How do you expect the real solution of the linearised SWE to evolve?
High-frequency waves will be generated that propagate in both directions. The solution
will oscillate between having non-zero h0 and non-zero u.

2. How will the solution of the 1d A-grid FB scheme evolve?

The solution will not change after initialisation. The grid-scale wave in h0 will remain. No
non-zero u will be generated.

93

7.6 Staggered Forward-Backward (1d C-grid FB)

So that gradients of h can be calculated where u is located and gradients of u can be
calculated where h is located, h and u can be staggered in space:
h j−2 h j−1 hj h j+1 h j+2
u j−3/2 u j−1/2 u j+1/2 u j+3/2

Using centered, 2-point spatial differences and forward-backward in time gives:

(n+1) (n) (n) (n)
∂u ∂h u j+1/ − u j+1/ h j+1 − h j
2 2
= −g → = −g (7.15)
∂t ∂x ∆t ∆x
(n+1) (n+1)
∂h ∂u hj
(n+1)
−hj
(n) u j+1/ − u j−1/
2 2
= −H → = −H (7.16)
∂t ∂x ∆t ∆x
Von-Neumann stability analysis gives: 3.5
( 3.0 exact
1 for |c| ≤ 1 A-grid
• |A| = 2.5
wave frequency, ω

> 1 for |c| > 1 for some k∆x

C-grid
2.0
∴ neutrally stable for |c| ≤ 1
1.5
• Dispersion relation:
k∆x
1.0
ω = ±2α = ±2 sin−1 c sin
2 0.5
• ∴ the C-grid is dispersive
0.00.0 0.2 0.4 0.6 0.8 1.0
• grid-scale waves propagate too slowly k∆x/π
• C-grid widely used in atmosphere and ocean models
• What about in 2d?
94
7.7 Arakawa Grids
In two dimensions, there are more possibilities for where the prognostic variables are
located:
A-grid B-grid C-grid

D-grid E-grid

95

7.8 Linearised Shallow-Water Equations on Arakawa Grids

The linearised SWE with rotation are:
∂ u/∂t = −2Ω × u − g∇h
∂ h0 /∂t + H∇ · u = 0

• The linearised SWE are solved numerically on Arakawa A, B and C grids, starting from
initial conditions consisting of zero velocity and zero h0 everywhere except a positive h0 in
one central grid-box
• The colours show h0 in the grid boxes. Red/yellow positive, blue negative, white zero

A-grid B-grid C-grid

7.9 Discussion Question

For solving the 2d, linearised rotating SWE (eqns 7.3-7.5) what are the advantages and
disadvantages of the different grids? Which terms or which balances between terms will be
represented accurately by different grids?
96
Bibliography
[1] D. Durran. Numerical Methods for Fluid Dynamics With Applications to Geophysics,
volume 32 of Texts in Applied Mathematics. Springer, 2nd edition, 2010. ISBN
1441964118.

97