An Introduction to Finite Difference Methods for Advection Problems Peter Duffy, Dep.

of Maths Physics, UCD Introduction

These 12 lectures form the introductory part of the course on Numerical Weather Prediction for the M.Sc.
in Meteorology at UCD. The aims of this part of the NWP course are to 1. introduce and discuss the analytic/exact solution of the linear advection equation


where  is given and we wish to solve for  starting from some initial condition (as we shall see this equation describes the advection of the function at speed ), 2. introduce the nite difference method for solving the advection equation numerically, 3. discuss the issue of numerical stability and the Courant Friedrich Lewy (CFL) condition, 4. extend the above methods to non-linear problems such as the inviscid Burgers equation



!

 
This is similar to the advection ! equation in appearance but has a crucial difference in that the advection speed is now equal to  . It is the left hand side of equation ?? which occurs in the primitive equations of gas dynamics and its numerical solution is critical in the study of NWP.

Finally we will also write programmes, and do exercises, for the methods introduced in these lectures. This
will be done using the MATLAB language.

Textbooks

Although the lecture notes and programmes will be self contained there are chapters in some well known
textbooks that provide useful further material and examples. 1. Atmospheric Modelling, Data Assimilation and Predictability by E. Kalnay. Chapter 3 discusses numerical discretisation. 2. Numerical Methods for Physics by A. Garcia (Prentice Hall). Good mix of numerical methods, applications and Matlab programmes. Chapter 1 is good for MATLAB and chapter 6 discusses the advection equation. 3. Numerical Solution of Partial Differential Equations by K.W. Morton and D.F. Mayers (Cambridge University Press). Very good on the numerical analysis of pdes. More formal and mathematical than the previous text, chapter 4 is concerned with advection problems.
Introduction to Finite Difference Methods Peter Duffy, Department of Mathematical Physics, UCD

4. Numerical Methods for Conservation Laws by R.J. LeVeque (Birkhauser). Not for the faint-hearted intended as further reading ! But accessible introductory chapters.

1. Basic Denitions

We begin with some basic denitions most of which you will have come across already in a different  ! context. Consider a gas moving in one dimension, along the axis, and let "  be the mass density at  !  point and time . If  is the bulk gas ow velocity then " is the mass ux at and . If matter is
neither created nor destroyed then the mass density at a point can only change of there is a gradient in the mass ux,

 " #   "

which is an example of a conservation law and the type of equation we are interested in solving. In Meteorology we are, of course, interested in three dimensional ow in which case this law takes the form

" %$'&( "0)

#

We can also write down two further equations describing momentum conservation and energy conservation.
The former case describes how the uid accelerates as a result of a pressure gradient (in the absence of viscosity) and can be written in the form

21 " 43  "



where 3 is the pressure. We can combine this with the mass conservation equation and re-write it as

5 8 3   76  "

(1)

which contains the Lagrangian derivative of and the pressure gradient term which you have already seen in the primitive equations. Finally, for completeness, the energy conservation equation,

9A@
where

B

DC

 9
, 3 and " .

is the energy density. An equation of state - the ideal gas law is also needed to relate

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

 All of the above partial differential equations (pdes) have the same conservative form, DST  Y BEGFIHQPIR  VUXW
We want to study methods of solving such equations when exact or analytic methods are not available.

Some numerical methods solve the equations in their pure conservative form (i.e. in terms of densities such as the momentum density " and so on). Once the densities are known they then return to the primitive variables such as " and . In these lectures we will concentrate on solving for the primitive variables
directly, for example equation 1.

1.2 Linear Advection Equation

Physically equation 1 says that as we follow a uid element (the Lagrangian time derivative), it will accelerate as a result of the local pressure gradient and this is one of the most important equations we will need to solve.

However, we will begin by looking at a far simpler equation which is known as the linear advection equation ! for the quantity  !   Y !
(2)

where  is known. We will also have to supplement this equation with an initial condition,  and, if necessary, boundary conditions (we will discuss these later).

 

Physically this equation says that if we start from any point and time and subsequently always move with a ! speed given by  then ! will never  change ! from  its initial value. To see this consider the change in 4` , in moving from point  to point  4`

`
Therefore, if we move along any curve  rate

 `  `
in the 

 

!

plane then, along that curve,

will change at the

` `

a `

 `

b 

There are two contributions to the right hand side. The rst is the explicit dependence on (what we would  see if we stood still) and the second is the change we see by our movement along the axis.
Introduction to Finite Difference Methods Peter Duffy, Department of Mathematical Physics, UCD

 For the problem given by equation 2 there are special curves called characteristics dened by ` `  #c !
(3)

along which

is constant,

` `

   

Characteristic Curves

u=constant

x
Therefore, once we have an initial condition       5d    fe
on the

we can nd the characteristic curve that originates from any point

axis, i.e. solve the problem

   ; i.e. and along that curve  we need to nd where  to nd the solution at  the charactersitic passing through that point hits the axis where the value of is simply set by the initial condition.
Therefore, the method of characteristics reduces the problem of solving a partial differential equation to the solution of an ordinary differential equation.
Introduction to Finite Difference Methods Peter Duffy, Department of Mathematical Physics, UCD

!

` c  et  

ghc

! 5d 2eu

iqprts!r

  2e   !

Examples 1. Solve the advection equation where is a constant

  6 with initial condition    , a Gaussian prole. ! A characteristic curve, along which  is constant, is the solution to ` `  h   2e   rwvyx  1t1

so that the characteristics are straight lines on the 

! 

plane. Therefore the solution is

  ! 2eI    

5d    6

which describes the the initial shape moving with velocity . For the initial shape of a Gaussian prole then at later times we have

  ! rwvyx   

a 6 

6 1

 1 b

the Gaussian at time is now centered at  . In the diagram below we plot the Gaussian for three successive time intervals, from left to right, and it can be seen that the shape of that prole is perfectly preserved.
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0.5

1.5

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

2. Solve the more difcult problem, again with a Gaussian initial prole,

5
Characteristic curve is solution to

8  1   1    8

` `

8  1   1  8

'

 6

2eu 8  1 

(check this !) which gives the solution to the pde in terms of the initial condition

  ! 2et     
Again if the initial shape is 

5dq

6 8

 1

 

rwvyx

6

 1t1

then

8   ! ruvTx  6   6 1 8

 1

In this case the initial shape, although it moves to the right, is not preserved (as can be seen below)  since the speed,  , is not constant.
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0.5

1.5

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

1.3 Finite difference methods for linear advection

How could we solve the linear advection equation if   were too complicated to use the analytic method
of characteristics, i.e.how do we solve the linear advection equation numerically ?

We begin by replacing the continuous  ! plane by a grid where have a uniform spacing, , along the  axis and timestep, d , on the axis. fe nm   5d g h Dd H d  8 I  jkjIjttl g # i 8 I  jIjIjwqp HoY

(x j,t n )

h
If we want to solve a partial differential equation for  on a computational grid, we only consider the m values of at discrete points. r 5d sDd utY e

!

m We say that a numerical scheme is convergent if e r

i.e.

em

= the numerical value for

hg

H d

at the g th spatial point and H th temporal point.

 ; i.e. if we make the grid progressively more rened we want the numerical solution to tend to the analytical one. Otherwise the numerical scheme is useless !!.
Peter Duffy, Department of Mathematical Physics, UCD

f  e!nm r  as

 and d

Introduction to Finite Difference Methods

 Several numerical methods can be constructed for equation 2 and we discuss these below. Although this
equation has a simple analytic solution the numerical solution is not trivial - it is actually one of the most difcult problems we will have to solve numerically ! Finite Differences

m How do we express the derivatives of  ! in terms of e ?

Before we answer that question we will consider a function of one variable, v   . We introduce a grid on  euwx fe yeyx the axis of points equally spaced , , ,... At e by the distance ; ..., Tekthese points the numerical approximations are denoted by v etc. and the analytic values are given by v  . With v2e z the rst derivative of v   w.r.t.  at point Te we can write down the Taylor expansions for v   to fe
either side of

v  v 

feyx   feqwx

v  v 

fet fet

5v z

1 1

e v z z 4{| }

e e 6 5v z v z z 6{G }

Subtracting these Taylor series gives,

v 

fe!yx

6 v 

feqwx

Y 5v z 4{| }

v z 

v 

fe!yx

6 v 

feuwx

4{G

1q

so that the rst derivative can be raplced by the difference between v to either side, divided by , which 1 produces an error proportional to . Computationally we can use this centered difference to approximate the rst derivative on our grid and this is called the centered difference approximation to the rst derivative

v z 
which is second order accurate in .

eyx

6 v

euwx
(4)

We could, of course, have used each original Taylor expansion separately to approximate the derivative,

v z   v ez

v v e

eyx

euwx 6 v

6 v

V~ V

sisTrks!rtkrI q0isGrksrktr

which are each rst order accurate in .

Introduction to Finite Difference Methods Peter Duffy, Department of Mathematical Physics, UCD

 We can also get an approximation for the second derivative by adding the Taylor series to get v eyx v eqwx e 1 e  v v z z {G

which gives the centered difference approximation to the second derivative which is second order accurate

v zz 

e!yx

e euwx 6 v v 1

(5)

This expression may become a little clearer if written in the form

v zz 

v

e!yx

6 v

ek

e euwx 6# v 6 v

How do we generalise these difference formulae to a function of two variables ? m ei fe!nmj Y r   We now have a two-dimensional grid,
t n+1

n1 x

j1

xj

j+1

At the H th time-step (i.e. at

Dm

we can use the formulae for 

 

a b D e m a 1  1 b D e m  

m e! y x 

`v

m e! y x  6

5 1 1 ` v  and  1 : and  1 to determine ` m euwx 1 {G m em eu wx 1u 4{| 1

[In each of these formulae the index H does not change: is constant]

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

 Similarly, at the g th spatial position we can determine and myx mwx a e e 6 8  b eD m  d mwx mIyx a 1 e em e 6 V 1 b e m  1 d  [Now the index g does not change: is constant]. Te

1 1 : {G d {G d 1u 1u

Forward Time Centered Space (FTCS) method

Using forward time (because we must start from some initial condition) and centered space (because its
accurate) differencing on equation 2 gives,

myx 6 d

em 

m m y x eu w x m e 6 e 6 mIyx
from

so that if we know all of the values at time

m 6

we can calculate all of the values at

e m e t fe!nm

mIyx 

m e!yx

m euwx

V~sA

rt!p 2 

(6)

 where and completely useless !!

m e  d . This is known as the FTCS method for the advection equation. It is

Sought t n+1

Known

x j1

x j

x j+1

tn

This method is applied in FTCS.m but before showing the results we need boundary conditions. 8 If g , the FTCS scheme gives mIyx m m m x x d d  6  1 6
but if gi

is the leftmost point then

md

doesnt exist !

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

We need to implement a boundary condition when g and g . In this case we impose periodic m advected off the grid to the right, we would like it to re-appear on the boundary conditions; as the md wave r is left. We do this by setting , i.e.

mIyx x

 8

m x

m m d 6  1 6

(i.e. we imagine that just to the left of gi

 Similarly, re-appear on the left

mIyx

, the wave amplitude is given by ). m m d x w xD 6 6 As the wave disappears off the right hand side of the grid it will

As an initial condition we take,

    

F

w0Cw 1n





This initial prole must be preserved in shape by an accurate numerical code. FTCS fails to do this for any choice of d and .
2.5

Final amplitude

1.5

Initial amplitude
0.5 a(x,t) 0 0.5 1 1.5 2 0.5

0.4

0.3

0.2

0.1

0 x

0.1

0.2

0.3

0.4

0.5

Amplitude versus x and t

3 2 1 0 1 2 50 40 30 20 10 Time 0 0 10 20 30 Position 40 50 60

Physical parameters: 

j8

FTCS parameters: d Y ( ,

, h

8 ,  j l  ,  ( ,  

Note: Wave should have returned to its initial position with identical prole!
Introduction to Finite Difference Methods Peter Duffy, Department of Mathematical Physics, UCD

 The stability of a numerical method can be studied (before writing a programme !) by the following method.
Fourier/Von Neumann Stability analysis

Von Neumann stability analysis consists of inserting a trial solution into our scheme and seeing whether it
will grow or not. The trial solution is of the form,

  !   s  iAp w s 

C  8  k F R 6 x0 x rk(iqpqp   wx Ir s!r rt !p  6

This is just one Fourier component of the entire solution and on the grid this becomes

e
where is the amplifaction factor;

m 

Fk
.

et

h Fk

ew

#(

mwx

von Neumann stability analysis (which isnt very rigorous but always works) consists of inserting the trial 8 8 solution and examining whether (the amplication factor) lies between 6 and or not.

If we now insert this into the FTCS method m m myx m e e d e! y x eu w x 6  6

Substitute in the trial solution:

myx

Fk

et   

k F 6 ew5 m 8 F ew5 m 8 Fk

ew

d 6 6

6 k F k d 6F F d &R( V w

Fk

eyxV

eqwxVf

d m Y where  8 8 6 d &R( V 

8 0 always (unless #H2 which gives  ). m e is amplied by the FTCS scheme and the method is unstable. The Lax Equivalence Theorem, which we will not prove, further states that the stability of a scheme is
necessary and sufcient for convergence once the differencing scheme is consistent.
Introduction to Finite Difference Methods Peter Duffy, Department of Mathematical Physics, UCD

7

d 1 1   8

m The simplest way to overcome the instability of the FTCS method is to replace the e term on the right
hand side by

Lax Method

8 e mIyx  8 g m e!yx m euwx

m euwx

m e!yxD

m m d eyx eq w xD 6 6

B

rt!p 5 

(7)

This is the Lax method for the advection problem. As an exercise, adpat the le FTCS.m to solve the advection problem by the Lax method and show that 1. For do

the Lax scheme is stable. (Well see shortly where this condition comes from).

2. But, for d|

 dt the pulse dies out!

The reason for this damping is that the Lax method can be written as myx m m m m a e! e em em eq yx euwx w x 8 a eyx 6 6 6 6 b b d d
which is the FTCS representation of the equation

5 g6 

1 1  1  d qrts 

The Lax method has added articial viscosity to the advection equation. If d , the articial viscosity ensures that the method is stable (it smooths out the instabilities). But, if d is too small then the viscosity
term dominates and pulse dies away (it smooths out everything !).

Let us apply the Von Neumann stability analysis to the Lax method which gives myx Fk et  
so that

e!yxV euwxV d m Fk 4 Fk 6 HfFk 6 Fk ew 8 w w d m 6 F  F V F BF 6 F m m

eyxV

euwxV

 
Introduction to Finite Difference Methods

V  1 

6R

d  1 1 5 d V
Peter Duffy, Department of Mathematical Physics, UCD

8q

Conclusion: The Lax method is numerically stable only if

d e

Numerically the value of

This is known as the Courant-Friedrich-Lewy mIyx m (CFL) m mstability criterion. depends on

euwx ! e  eyx

Mathematically in time d the solution just shifts to the right a distance d .

then scheme

myx

depends on the value of

do

do yeuwx
which is not included in the numerical

at some point to the left of

Instability Upwind scheme

Another way of obtaining a stable scheme for the advection equation is the upwind scheme which makes ! q(   use of the concept of characteristics. Consider the T simplest (assume enmyx case where   ) and draw a characteristic through the point  .

n+1

tn j

x Characteristic

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

 Forward time and backward space differencing gives (with 6 e myx 6 em   em

 d ) m eu w x m m euwx

m yx d I e

6 8  6

Adapt FTCS.m to solve the advection problem by the upwind scheme

Teqwxu!nm fenmyx mIyx myx so that   m goes through and the upwind d , a characteristic eqwx e em e   scheme gives the correct answer . Similarly when  we get . FormIall yx cases e 8 i  where , the upwind scheme gives a linear interpolation (see below) approximation to . For a general c ! this idea can be generalised to give the upwind scheme for the linear advection equation Note that  8  e myx 
where

em m e 6 6

m e! y x  em 6 6

em m eu w x

if if

e e m

 
(8)

m e d . We will write a programme for this method after the next section. m Fourier analysis of the schemes stability gives, when e  , t 1  1   
so that 5V

6 8  6 8  6 8 6 8 6

 6F 1  1 1 8 4  6 8 8  6  6  8 1  6 

 T V 

for all provided the Courant Friedrich Lewy (CFL) condition,

We also note the Leapfrog scheme mIyx mwx e e  6

, is satised.

m eyx 6

m eqwx

V

rkx s rtp  

(9)

which is centered time, centered space and is therefore not self-starting - we cant just use the initial condition to get it going. How could you get around this ? But it has the special property that if the CFL condition is satised then there is no damping, prove that this is the case and adapt FTCS.m to use the leapfrog methd.

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

MATLAB Programme linadvect.m

We now study some of the above methods in practice with different initial conditions and advection speeds. The choices allowed are summarised in the following table. Note that the choice of the exact solution.   j 8 j  ruvyx  8 8 1!    6 V 6 6 B 6
shape1.m shape2.m



! 8

Exact

adspeed1.m exact1.m

adspeed2.m exact2.m

  1  8  1 8  

We are also free to choose either of the methods upwind (upwind.m) and Lax (adlax.m). The code can also run the Lax Wendroff method which is beyond the scope of these lectures but is discussed in the appendix for those who would like to learn more about slightly more advanced techniques. N.B. These problems all have an analytic solution (solid curves on the plots) - we are testing the numerical methods and showing exactly how difcult it is to solve the advection problem on a computer.

Results are presented here for the non-uniform advection speed, with either initial condition and the upwind
or Lax schemes; i.e. four different sets T of the using characteristics is also j8 sresults, (f and rtsq t analytic solution 8 plotted. In each case  and the  d   . Note that since changes from point to point, on most of the x-axis we will thereofre have a local courant number which is less than 1. A full set of results should be generated by the student. The programme terminates before the wave prole runs off the grid.

N.B. The programme calculates the timestep on the basis of fullling the CFL condition at the start of the  programme. However,  decreases with time (in the variable case) and this suggests that it might be

a good idea to change the timestep after each iteration. Adapt the programme in order to this. Does this improve the results and why ?

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

 The results for the upwind method with an initial gaussian prole are shown below for  ,  8 , 8  and  and compared against the analytic solution (solid line); the same times apply to the
subsequent three examples. It is clear that, although the peak of the numerical results moves with the correct speed, the upwind method reduces the height of the initial shape, i.e. an amplitude error which is quite large.
1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

The same qualitative features can be seen with the upwind method for the top-hat initial condition. Again
the pulse moves at the correct speed but the amplitude is reduced and the edges of the pulse are smoothed out.
1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

 The Lax scheme for the gaussian prole shows the damping effect one gets with a local courant number
that is less than 1.
1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

The Lax method for the top-hat condition.

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

Appendix

 6 1. Consider the linear advection equation in the range  with the advection speed c   8      6 and initial condition . Sketch the initial condition Why do   ! 8 and the characteristics. we not need any extra conditions at the boundaries of Y and  ? Solve for  . What does r the solution look like for ? Y 6 with the advection speed c 2. Consider the linear advection equation   8 in  the range   6 and initial condition    . Sketch the initial condition and the characteristics. Why  8 do we need extra conditions at the boundaries of  and  ? With the boundary conditions ! 8 ! ! r V    solve for  . What does the solution look like for ?
3. Solve each of the previous problems computationally and compare with the analytic results. 4. Show that when the CFL condition is satised by the leapfrog scheme for the linear advection scheme 8 that there is no damping; i.e.  if d . Write a programme which demonstrates this property. 5. For the linear advection equation dened by





myx

, where is a constant, the generalised upwind scheme is

e  fe

 6 8

m w x 4

6  6 d and  . Apply Von Neumann stability analysis to this method and where show that stability places no constraint on the choice of d .

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

Further Reading

There is a well known technique that is more accurate than anything we derived so far although its derivation
is somewhat more complex. It is called the Lax Wendroff method and starts with a Taylor expansion of the analytic function

  !  d

   @ !   d 

8 d

1q @@   !  {G d }

The differential equation allows us to replace time derivatives by spatial derivatives which can then be
central differenced.

@ @C CnC

76

@@

6

@ C

5C @

Noting that

76B

gives

@@

76

@ C

B

C C

The Taylor expansion can then be written as   !  d    !  6 C  d  1 C C @ 5C ! d 6  

Applying central differences for the spatial derivatives give the Lax Wendroff scheme for the linear advection equation

myx 

m e yx m ! 6 e e 6 d m x m ey 1 @ e d 6B m

m eu w x 6 m eu w x m m m y x e e yxV e! m  1 6 e 6 1 m euwxV 1 em m eu w x 6 V v

rts u 

(10)

so that it is therefore second order accurate.

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD

 The Lax Wendroff scheme for the gaussian prole shows very good agreement between the numerical
results and the analytic solution and represents a considerable improvement on the upwind method.
1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

However, Lax Wendroff method for the top-hat condition shows how difcult it is to reproduce non-smooth
solutions; quite advanced schemes are needed !
1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5

Introduction to Finite Difference Methods

Peter Duffy, Department of Mathematical Physics, UCD